Memo management method and device, electronic device, and storage medium

Abstract

This application provides a memo management method, apparatus, electronic device, and storage medium, belonging to the field of semantic processing technology. The method includes: responding to receiving first voice information containing a first type of trigger word; performing semantic integrity verification on the first voice information and acquiring environmental noise intensity and user motion state data; determining a trigger probability based on the trigger word matching degree, environmental noise intensity, and user motion state data; if the trigger probability is greater than or equal to a preset probability threshold and the first voice information is semantically complete, generating memo information based on the first voice information; if the trigger probability is greater than or equal to the preset probability threshold and the first voice information is semantically incomplete, outputting a first prompt message to instruct the user to input supplementary information. The memo management method, apparatus, electronic device, and storage medium provided by this application can improve the accuracy of memo information generation.

Description

Technical Field

[0001] This application belongs to the field of semantic processing technology, and more specifically, relates to a memo management method and apparatus, electronic device, and storage medium. Background Technology

[0002] With the development of wearable devices, users can record to-do lists and other reminders anytime, anywhere through voice interaction, without using their hands, greatly improving the timeliness and efficiency of reminder recording. This anytime, anywhere interaction mode frees reminders from being limited to fixed scenarios and devices, truly integrating them into fragmented daily time, bringing convenience to people's work and life.

[0003] However, wearable devices are often used in complex and varied scenarios. Users may be in highly disruptive environments such as commuting or noisy public places. Existing memo recording methods rely solely on simple keyword matching for trigger determination, which can easily lead to false triggers or failure to recognize valid commands, thus affecting the user experience. Summary of the Invention

[0004] The purpose of this application is to provide a memo management method, device, electronic device, and storage medium to solve the problem that memo generation instructions are prone to being triggered erroneously or that valid instructions cannot be recognized in the prior art, thereby improving the accuracy of memo information generation.

[0005] A first aspect of this application provides a memo management method, including: In response to receiving first voice information containing a first type of trigger word, the semantic integrity of the first voice information is checked, and environmental noise intensity and user motion state data are obtained. The trigger probability is determined based on the trigger word matching degree, the ambient noise intensity, and the user's motion state data; the trigger word matching degree is the matching degree between the first type of trigger word and a preset keyword library; If the trigger probability is greater than or equal to a preset probability threshold, and the semantic integrity verification result shows that the semantic integrity of the first voice information is complete, then memo information is generated based on the first voice information. If the trigger probability is greater than or equal to a preset probability threshold, and the semantic integrity verification result shows that the semantics of the first voice information are incomplete, then the first prompt information is output to instruct the user to input supplementary information, and memo information is generated based on the supplementary information and the first voice information. The steps of semantic integrity verification include: Obtain key entity categories; these key entity categories are obtained by extracting entity categories from multiple historical memos of the user. For each entity category in the key entity categories, extract the entity information corresponding to that entity category from the first voice information; If each entity category in the key entity categories has corresponding entity information, then the semantics of the first voice information are determined to be complete; otherwise, the semantics of the first voice information are determined to be incomplete.

[0006] A second aspect of this application provides a memo management device, including: The semantic verification module is used to perform semantic integrity verification on the first voice information containing the first type of trigger words in response to receiving the first voice information, and to obtain the environmental noise intensity and the user's motion state data. The trigger probability calculation module is used to determine the trigger probability based on the trigger word matching degree, environmental noise intensity, and user motion state data; the trigger word matching degree is the matching degree between the first type of trigger word and the preset keyword library; Memo generation module, used for: When the trigger probability is greater than or equal to a preset probability threshold and the semantic integrity verification result shows that the semantic integrity of the first voice information is complete, memo information is generated based on the first voice information; When the trigger probability is greater than or equal to the preset probability threshold and the semantic integrity verification result shows that the semantics of the first voice information is incomplete, the first prompt information is output to instruct the user to input supplementary information, and the memo information is generated based on the supplementary information and the first voice information. Specifically, when performing semantic integrity verification on the first voice information, the semantic verification module is used for: Obtain key entity categories; key entity categories are obtained by extracting entity categories from multiple historical memos of the user. Extract all entity categories from the first voice information to obtain the first entity category set; If the first entity category set contains all the key entity categories, then the semantics of the first voice information are determined to be complete; otherwise, the semantics of the first voice information are determined to be incomplete.

[0007] A third aspect of this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the memo management method described above.

[0008] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the memo management method described above.

[0009] The beneficial effects of the memo management method and apparatus, electronic device, and storage medium provided in this application are as follows: This application's embodiments break through the limitations of traditional single trigger word matching. By comprehensively considering trigger word matching degree, environmental noise intensity, and user motion state data to calculate trigger probability, it can effectively filter out accidental triggering of memos caused by speech distortion in high-noise / motion scenarios. Simultaneously, it extracts all entity categories from the first speech information to obtain a first entity category set. Comparing the first entity category set with key entity categories and judging the semantic integrity of the first speech information based on the comparison results can avoid generating invalid memo information.

[0010] Among them, the key entity categories are extracted based on the user's historical memo information, which can make the key entity categories fit the user's habits and further ensure that the semantic integrity verification results are consistent with the user's true intentions. Figure 1 To.

[0011] Therefore, the method of this embodiment can improve the accuracy of memo information generation. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 A flowchart illustrating a memo management method provided in an embodiment of this application; Figure 2 A structural block diagram of a memo management device provided in an embodiment of this application; Figure 3 This is a schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0014] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0015] It is understood that in the embodiments of this application, data such as user voice information are involved. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with relevant laws, regulations and standards.

[0016] To make the objectives, technical solutions, and advantages of this application clearer, the following description will be provided in conjunction with the accompanying drawings and specific embodiments.

[0017] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a memo management method provided in an embodiment of this application, which can be executed by an electronic device. The method may include: S101: In response to receiving first voice information containing a first type of trigger word, perform semantic integrity verification on the first voice information and obtain environmental noise intensity and user motion state data.

[0018] The steps involved in semantic integrity verification include: Obtain key entity categories; key entity categories are obtained by extracting entity categories from multiple historical memos of the user. Extract all entity categories from the first speech information to obtain the first entity category set; If the first entity category set contains all the key entity categories, then the semantics of the first speech information are determined to be complete; otherwise, the semantics of the first speech information are determined to be incomplete.

[0019] In this embodiment, the first type of trigger word can be a preset trigger word such as "make a note", "remember", "remind", or "memorandum"; the ambient noise intensity can be obtained through the microphone built into the electronic device, and the user's motion state data can be obtained through the accelerometer built into the electronic device. The data detected by the accelerometer can determine whether the user is currently in a static, walking, or running motion state.

[0020] Environmental noise intensity and user motion state data can characterize the degree of interference with voice information in the user's environment. In high-noise / movement scenarios, user voice is greatly affected by environmental noise, and when users are in a non-focused state such as moving or socializing, mentioning the first type of trigger words is more likely to be casual conversation or a casual remark (such as saying "note the pace" while running). In low-noise / stationary scenarios, users have relatively stable interaction conditions, and when mentioning the first type of trigger words, it is more likely to be a genuine need for memorization.

[0021] The first voice information is the text information obtained after converting the user's input voice signal into text. Entity information in the first voice information refers to things or concepts with independent meaning, such as "3 o'clock" or "meeting." Entity information can be categorized according to semantic attributes, resulting in multiple entity categories, such as time, location, people, events, and objects. For memo information, the key entity category refers to the core semantic entity category that supports the semantic completeness of a memo message.

[0022] When the electronic device receives the first voice message containing the first type of trigger word, it initially determines that the user's intention is to generate a memo, and the first voice message is a memo generation command. At this time, the semantic integrity of the first voice message can be verified, and the ambient noise intensity and the user's motion state data can be obtained. Based on the semantic integrity verification result, the ambient noise intensity, and the user's motion state data, the validity of the memo generation command can be further determined.

[0023] Specifically, when performing semantic integrity verification on the first speech information, this can be achieved by checking whether the first speech information is missing necessary entity information. For example, a natural language processing algorithm (e.g., a pre-trained named entity recognition model, NER) can be used to extract all entity categories from the first speech information, resulting in a first entity category set. This first entity category set is then compared with key entity categories. If the first entity category set contains all key entity categories, it is determined that the first speech information does not lack necessary entity information, and the semantics of the first speech information are complete. Otherwise, if the first entity category set does not contain all key entity categories, it is determined that the first speech information lacks necessary entity information, and the semantics of the first speech information are incomplete. For example, the key entity category is: {time [sub-dimension 1: date, sub-dimension 2: moment], event}, and the first voice message is "Remember to have a meeting tomorrow". The total entity categories of the first voice message (i.e., the set of first entity categories) are: {time [sub-dimension 1: date], event}. Comparing the set of first entity categories with the key entity categories, it can be seen that the set of first entity categories does not contain all the key entity categories, and the semantics of the first voice message is determined to be incomplete.

[0024] Among them, the pre-trained named entity recognition (NER) model is a semantic analysis model for sequence labeling tasks. Its core function is to accurately locate and classify entities of preset categories from text. For the specific scenario of this embodiment, its training process is detailed in the following embodiment.

[0025] Furthermore, considering the different memo-taking habits of different users—for example, even with work meeting memos, some users prefer to include time, event, and participants, while others prefer only time and event—using the same key entity categories for semantic integrity verification could lead to some of the initial audio information containing the user's memo intent being judged as semantically incomplete, thus deviating from the user's true intention. To avoid this problem, this embodiment extracts entity categories based on multiple historical memos from the user to obtain key entity categories, ensuring that the key entity categories align with user habits, thereby making the semantic integrity verification results consistent with the user's actual needs.

[0026] S102: Determine the trigger probability based on the trigger word matching degree, environmental noise intensity, and user motion state data; the trigger word matching degree is the matching degree between the first type of trigger word and the preset keyword library.

[0027] In this embodiment, a keyword library can be preset, which includes keywords such as "remember", "remind", or "memo". When the electronic device receives the user's voice information, it can perform word segmentation on the voice information to obtain multiple words in the voice information. Based on the preset keyword library, it searches for whether there are any words in the multiple words that match the keywords in the keyword library (referred to as matching words). If there are, the voice information is determined to be the first voice information containing the first type of trigger word, and the matching words are used as the first type of trigger word. The similarity between the first type of trigger word and the corresponding keyword is calculated using string similarity algorithms such as edit distance and cosine similarity, which is used as the trigger word matching degree (value range [0,1]).

[0028] The trigger word matching degree characterizes the probability that the first voice information is a memo generation instruction. The higher the trigger word matching degree, the higher the probability that the first voice information is a memo generation instruction. The higher the ambient noise intensity and the more intense the movement, the greater the recognition error of the trigger word in the first voice information, and the higher the probability that the first voice information is misidentified as a memo generation instruction. Therefore, this embodiment determines the trigger probability based on the trigger word matching degree, ambient noise intensity, and user movement state data, so as to verify the effectiveness of the memo generation instruction based on the trigger probability.

[0029] For example, determining the trigger probability based on trigger word matching degree, ambient noise intensity, and user motion state data includes: A first probability value is determined based on the ambient noise intensity; the first probability value is negatively correlated with the ambient noise intensity. A second probability value is determined based on the user's motion state data; the second probability value is negatively correlated with the user's motion state data. The trigger probability is obtained by weighted summing of the trigger word matching degree, the first probability value, and the second probability value.

[0030] Specifically, the first probability value can be calculated based on the following formula: ; in, This represents the first probability value. Indicates the intensity of ambient noise. Indicates the first noise threshold. This represents the second noise threshold. Both the first and second noise thresholds are preset constants; for example, the first noise threshold could be 30 dB and the second noise threshold could be 70 dB.

[0031] Simultaneously, based on the user's motion state data, the following mapping relationship is found to obtain the second probability value: When the user's motion state data is stationary, the corresponding second probability value is 1; When the user's motion status data is walking, the corresponding second probability value is 0.7; When a user's activity status data is during running, the corresponding second probability value is 0.3.

[0032] Based on the first and second probability values, the trigger probability can be obtained by weighted summing of the trigger word matching degree, the first probability value, and the second probability value. Among them, the trigger word matching degree has the highest weight, which can be 0.6, while the weights of the first probability value and the second probability value are 0.2 and 0.2 respectively.

[0033] S103: If the trigger probability is greater than or equal to the preset probability threshold, and the semantic integrity verification result shows that the semantic integrity of the first voice information is complete, then generate memo information based on the first voice information.

[0034] In this embodiment, the validity of the memo generation instruction can be further verified based on the trigger probability and semantic integrity verification results. Specifically, if the trigger probability is greater than or equal to a preset probability threshold (e.g., 0.6) and the semantic integrity verification result is semantically complete, it indicates that the first voice information is highly credible as a memo generation instruction, the user's intent is clear, and the input information is complete. In this case, memo information can be directly generated based on the first voice information to improve interaction efficiency.

[0035] For example, generating memo information based on the first voice information includes: Extract entity information from the first voice information, and concatenate the entity information according to the order in which the entity information appears in the first voice information to obtain memo information. This can restore the user's expression habits.

[0036] S104: If the trigger probability is greater than or equal to the preset probability threshold and the semantic integrity verification result shows that the semantics of the first voice information is incomplete, then output the first prompt information to instruct the user to input supplementary information, and generate memo information based on the supplementary information and the first voice information.

[0037] In this embodiment, if the trigger probability is greater than or equal to the preset probability threshold and the semantic integrity verification result is semantically incomplete, it indicates that the first voice information is highly credible as a memo generation instruction, but the user's input information is incomplete, and effective memo information cannot be generated based on the current information.

[0038] At this point, the first prompt message can be output, instructing the user to input supplementary information. After the user completes the input, a memo message can be generated based on the supplementary information and the first voice message.

[0039] For example, the method for determining the first prompt message includes: For each entity category in the key entity categories, if the entity category is missing from the first entity category set, the entity category is marked as a missing entity category; The first dialogue template is filled in based on all missing entity categories to obtain the first prompt message.

[0040] Taking the first voice message "Remember the meeting tomorrow" as an example, the key entity category is: {Time[Sub-dimension 1: Date, Sub-dimension 2: Time], Event}, and the first entity category set is: {Time[Sub-dimension 1: Date], Event}. The first entity category set is missing the entity category (Sub-dimension 2: Time). Therefore, (Sub-dimension 2: Time) is taken as the missing entity category.

[0041] The preset first dialogue template can be: Information is incomplete, please supplement [ ] information. Fill the missing entity category above into the first dialogue template to get the first prompt message: Information is incomplete, please supplement [time] information.

[0042] For example, the missing entity categories include entity categories with missing information in all aspects and entity categories with missing sub-dimension information. Memo information is generated based on supplementary information and the first voice information, including: Initial memo information is generated based on the first voice information; If the missing entity category is an entity category for which all information is missing, the supplementary information will be appended to the end of the initial memo information to obtain the memo information; If the missing entity category is an entity category for which sub-dimension information is missing, the supplementary information is inserted after the target entity information in the initial memo information to obtain the memo information; wherein, the target entity information and the supplementary information have the same entity category.

[0043] Taking the first voice message "Remember to have a meeting tomorrow" as an example, the initial memo information can be generated based on the first voice message as "Meeting tomorrow". The missing entity category is (sub-dimension 2: time). The missing entity category is the entity category that is missing in the sub-dimension information. If the user inputs the supplementary information "3 pm", this supplementary information and "tomorrow" in the initial memo information both belong to the entity category of "time". Therefore, the supplementary information "3 pm" can be inserted after "tomorrow" to get the final memo information as "Meeting at 3 pm tomorrow".

[0044] As can be seen from the above, this embodiment breaks through the limitations of traditional single trigger word matching. By comprehensively considering the trigger word matching degree, environmental noise intensity, and user motion state data to calculate the trigger probability, it can effectively filter out erroneous memo triggers caused by speech distortion in high-noise / motion scenarios. Simultaneously, by extracting all entity categories from the first speech information to obtain a first entity category set, and comparing the first entity category set with key entity categories, the semantic integrity of the first speech information is determined based on the comparison results, thus avoiding the generation of invalid memo information.

[0045] Among them, the key entity categories are extracted based on the user's historical memo information, which can make the key entity categories fit the user's habits and further ensure that the semantic integrity verification results are consistent with the user's true intentions. Figure 1 To.

[0046] Therefore, the method of this embodiment can improve the accuracy of memo information generation.

[0047] In one embodiment of this application, the method for determining the key entity category includes: Retrieve multiple historical memos from the user; For each historical memo, extract the entity category of that historical memo to obtain a second set of entity categories; In the set of second entity categories corresponding to multiple historical memos, the sets of second entity categories that are the same are grouped together. The number of sets of second entity categories that are the same in each group is counted, and the set of second entity categories corresponding to the group with the largest number of sets is taken as the key entity category.

[0048] In this embodiment, multiple historical memos from different scenarios such as work and life can be obtained. For each historical memo, the entity category in the historical memo is extracted using a natural language processing algorithm (such as a pre-trained named entity recognition model, NER), resulting in a second entity category set. The second entity category sets corresponding to the multiple historical memos are grouped together if their elements are identical. The number of second entity category sets in each group is counted. The higher the number of sets in a group, the more frequently the user uses that set of entity categories when recording items. Therefore, the second entity category set corresponding to the group with the largest number of sets is taken as the user's personalized key entity category.

[0049] In one embodiment of this application, the memo management method further includes: Obtain a preset keyword library, as well as the corresponding historical first voice information for each of the multiple historical first voice information entries; For each historical first voice message, the historical first voice message is segmented to obtain multiple historical words. The first matching degree between each historical word and the preset keyword library is calculated. The historical word with the highest first matching degree is taken as the historical key word. The previous and next words of the historical key word are taken as historical auxiliary words. An auxiliary lexicon is constructed based on the historical auxiliary word segmentation corresponding to each of the multiple historical first speech information; The methods for determining the trigger word matching degree include: The first speech information is segmented into words to obtain multiple target words; Calculate the second matching degree between each target word segment and the preset keyword library; Select the target word with the highest second matching degree as the target key word; The word segment preceding and following the target keyword is used as the target auxiliary word segment, and the third matching degree between each target auxiliary word and the auxiliary vocabulary is calculated. The trigger word matching score is obtained by weighted summing the second matching score and the highest third matching score corresponding to the target keyword segmentation.

[0050] In this embodiment, the existing single keyword matching method is easily affected by irrelevant text when identifying trigger words. For example, if a user says "I remember," the voice information contains the keyword "remember," which may be misjudged as a trigger word.

[0051] To avoid the above problems, considering that there are often auxiliary words with fixed collocations before and after the matching participles. For example, if the first voice message is "Remember the meeting tomorrow", the matching participle is "Remember", and the auxiliary participle "一下" is collocated after the matching participle. These auxiliary words constitute the context features of the instruction and are important bases for determining the real memo generation instruction. Therefore, combining the matching participle and the auxiliary participle to judge the trigger word can reduce the probability of misjudgment.

[0052] At the same time, considering that most of the auxiliary participles are colloquial words, and different users have different speaking habits and different commonly used auxiliary participles. For example, User A prefers to use "Remember" as the trigger word for the memo generation instruction. His historical first voice messages are: "Remember the meeting at 9 am tomorrow", "Remember to go to the supermarket to buy laundry detergent at 7 pm the day after tomorrow". In User A's historical first voice messages, the matching participle is "Remember", and the commonly used auxiliary participle is "一下". Another example is that User B prefers to use "Please remember for me" as the trigger word for the memo instruction. His historical first voice messages are: "Please remember for me the itinerary of the business trip at 9 am tomorrow", "Please remember for me the customer visit at 3 pm on Friday". In User B's historical first voice messages, the matching participle is "Remember", and the commonly used auxiliary participle is "Please remember for me".

[0053] Therefore, in this embodiment, an auxiliary word library is constructed based on multiple historical first voice messages. Specifically, for each historical first voice message, perform word segmentation operation to obtain multiple historical participles; calculate the first matching degree between each historical participle and the preset keyword library, and select the historical participle with the highest first matching degree as the historical key participle; extract the previous participle and the next participle of the historical key participle as the historical auxiliary participles; finally, count the frequency of each auxiliary participle in multiple historical first voice messages, divide the frequency of each auxiliary participle by the total number of historical first voice messages to obtain the probability of each auxiliary participle appearing, and screen out the auxiliary participles whose appearance probability is greater than the preset probability (such as 0.6) to construct the auxiliary word library.

[0054] On this basis, for the first voice message, when calculating the matching degree of the trigger word, the first voice message can be first segmented to obtain multiple target participles, calculate the second matching degree between each target participle and the preset keyword library, select the target participle with the highest corresponding second matching degree as the target key participle, take the previous participle and the next participle of the target key participle as the target auxiliary participles, calculate the third matching degree between each target auxiliary participle and the auxiliary word library, and finally, perform a weighted sum of the highest second matching degree and the highest third matching degree to obtain the trigger word matching degree.

[0055] As can be seen from the above, this embodiment constructs an auxiliary word library based on the first historical voice information on the basis of the preset keyword library. Through the dual-layer judgment logic of core keyword matching and personalized auxiliary word segmentation verification, it can effectively filter irrelevant text interference and adapt to the spoken expression habits of different users, which is conducive to improving the accuracy of trigger word probability calculation, thereby improving the accuracy of trigger word recognition.

[0056] In one embodiment of this application, the memo management method further includes: If the trigger probability is less than the preset probability threshold and the semantic integrity verification result shows that the semantic integrity of the first voice information is complete, then a confirmation memo is generated based on the first voice information, and a second prompt is output to instruct the user to confirm the confirmation memo. If a confirmation message is received for the memo to be confirmed, the memo to be confirmed will be saved to the memo.

[0057] In this embodiment, when the trigger probability is less than a preset probability threshold and the semantics of the first voice information are incomplete, it indicates that the credibility of the first voice information as a memo generation instruction is low and the user's input information is incomplete. At this time, it can be determined that the first trigger word is a false trigger and there is no need to respond to the first voice information.

[0058] Correspondingly, when the trigger probability is less than a preset probability threshold and the semantics of the first voice information are complete, it indicates that the user's input information is complete. However, the reliability of the first voice information as a memo generation instruction is low. In this case, a memo to be confirmed can be generated based on the first voice information, and a second prompt can be output to guide the user to confirm. Only after receiving the user's confirmation instruction will the memo to be confirmed be saved to the memo; if the user does not confirm or chooses to cancel, the saving operation will not be performed.

[0059] The methods for determining the second prompt message include: The second prompt message is obtained by filling in the preset second dialogue template with the memo information to be confirmed.

[0060] For example, the preset second dialogue template could be: "Do you want to confirm saving the following memo? [ ]", and the memo information to be confirmed is "Meeting at 9 am on Wednesday". By filling the memo information to be confirmed into the above second dialogue template, the second prompt information can be obtained as: "Do you want to confirm saving the following memo? [Meeting at 9 am on Wednesday]".

[0061] As can be seen from the above, for scenarios where the trigger probability is low due to environmental noise, this embodiment generates the confirmation memo information based on semantically complete first voice information, and the user makes the final decision on whether to generate the memo information, which can effectively avoid the loss of useful information caused by misjudgment of trigger probability.

[0062] In one embodiment of this application, the memo management method further includes: In response to receiving second speech information containing a second type of trigger word, entity information is extracted from the second speech information to obtain the entity information of the second speech information; If the entity information of the second voice information contains entity information of a specified entity category, the entity information of the specified entity category is used as the target entity information; the memo is queried based on the target entity information to obtain the query results; If the entity information of the second voice message does not contain entity information of the specified entity category, all memo information in the memo will be used as the query result; Output the query results.

[0063] In this embodiment, the second type of trigger word can be words such as "check" or "check memos". When the user inputs second voice information containing the second type of trigger word, the second voice information input by the user is determined to be a memo check command. The user can also preset a specified entity category (e.g., time + event). The electronic device can extract all entity categories and the entity information corresponding to each entity category from the second voice information through a natural language processing algorithm (e.g., a pre-trained named entity recognition model NER). If the entity information of the second voice information contains entity information of the specified entity category, it indicates that the user has a clear query condition. At this time, the entity information of the specified entity category can be used as the query condition, and the memo can be queried based on the query condition to obtain the query result. If the entity information of the second voice information does not contain entity information of the specified entity category, it indicates that the user does not have a clear query condition. At this time, all memo information can be output.

[0064] In one embodiment of this application, before outputting the query results, the method further includes: If the query results contain first-person singular, replace the first-person singular with second-person singular.

[0065] In this embodiment, the text generated when the memo information is generated is based on the user's first-person expression (such as "My home Wi-Fi password is xxx"), and the first-person singular "I" in the query results is replaced with the second-person singular "you", which is more in line with the language habits of human-computer interaction.

[0066] For example, replacing "My Wi-Fi password is xxx" with "Your Wi-Fi password is xxx" makes the interaction process more natural.

[0067] Corresponding to the memo management method in the above embodiments, Figure 2 This is a structural block diagram of a memo management device provided according to an embodiment of this application. For ease of explanation, only the parts relevant to the embodiment of this application are shown. References Figure 2The memo management device 20 includes: a semantic verification module 21, a trigger probability calculation module 22, and a memo management module 23. The semantic verification module 21 is used to perform semantic integrity verification on the first voice information containing the first type of trigger word in response to receiving the first voice information, and to obtain environmental noise intensity and user motion state data. The trigger probability calculation module 22 is used to determine the trigger probability based on the trigger word matching degree, environmental noise intensity, and user motion state data; the trigger word matching degree is the matching degree between the first type of trigger word and the preset keyword library; Memo management module 23 is used for: When the trigger probability is greater than or equal to a preset probability threshold and the semantic integrity verification result shows that the semantic integrity of the first voice information is complete, memo information is generated based on the first voice information; When the trigger probability is greater than or equal to the preset probability threshold and the semantic integrity verification result shows that the semantics of the first voice information is incomplete, the first prompt information is output to instruct the user to input supplementary information, and the memo information is generated based on the supplementary information and the first voice information. Specifically, when performing semantic integrity verification on the first voice information, the semantic verification module is used for: Obtain key entity categories; key entity categories are obtained by extracting entity categories from multiple historical memos of the user. Extract all entity categories from the first speech information to obtain the first entity category set; If the first entity category set contains all the key entity categories, then the semantics of the first speech information are determined to be complete; otherwise, the semantics of the first speech information are determined to be incomplete.

[0068] In one embodiment of this application, the semantic verification module 21 is specifically used for: Retrieve multiple historical memos from the user; For each historical memo, extract the entity category of that historical memo to obtain a second set of entity categories; In the set of second entity categories corresponding to multiple historical memos, the sets of second entity categories that are the same are grouped together. The number of sets of second entity categories that are the same in each group is counted, and the set of second entity categories corresponding to the group with the largest number of sets is taken as the key entity category.

[0069] In one embodiment of this application, the semantic verification module 21 is specifically used for: Obtain a preset keyword library, as well as the corresponding historical first voice information for each of the multiple historical first voice information entries; For each historical first voice message, the historical first voice message is segmented to obtain multiple historical words. The first matching degree between each historical word and the preset keyword library is calculated. The historical word with the highest first matching degree is taken as the historical key word. The previous and next words of the historical key word are taken as historical auxiliary words. An auxiliary lexicon is constructed based on the historical auxiliary word segmentation corresponding to each of the multiple historical first speech information; The methods for determining the trigger word matching degree include: The first speech information is segmented into words to obtain multiple target words; Calculate the second matching degree between each target word segment and the preset keyword library; Select the target word with the highest second matching degree as the target key word; The word segment preceding and following the target keyword is used as the target auxiliary word segment, and the third matching degree between each target auxiliary word and the auxiliary vocabulary is calculated. The trigger word matching score is obtained by weighted summing the second matching score and the highest third matching score corresponding to the target keyword segmentation.

[0070] In one embodiment of this application, the trigger probability calculation module 22 is specifically used for: A first probability value is determined based on the ambient noise intensity; the first probability value is negatively correlated with the ambient noise intensity. A second probability value is determined based on the user's motion state data; the second probability value is negatively correlated with the user's motion state data. The trigger probability is obtained by weighted summing of the trigger word matching degree, the first probability value, and the second probability value.

[0071] In one embodiment of this application, the memo management module 23 is specifically used for: If the trigger probability is less than the preset probability threshold and the semantic integrity verification result shows that the semantic integrity of the first voice information is complete, then a confirmation memo is generated based on the first voice information, and a second prompt is output to instruct the user to confirm the confirmation memo. If a confirmation message is received for the memo to be confirmed, the memo to be confirmed will be saved to the memo.

[0072] In one embodiment of this application, the memo management module 23 is specifically used for: In response to receiving second speech information containing a second type of trigger word, entity information is extracted from the second speech information to obtain the entity information of the second speech information; If the entity information of the second voice information contains entity information of a specified entity category, the entity information of the specified entity category is used as the target entity information; the memo is queried based on the target entity information to obtain the query results; If the entity information of the second voice message does not contain entity information of the specified entity category, all memo information in the memo will be used as the query result; Output the query results.

[0073] In one embodiment of this application, before outputting the query results, the memo management module 23 is further used for: If the query results contain first-person singular, replace the first-person singular with second-person singular.

[0074] See Figure 3 , Figure 3 This is a schematic block diagram of an electronic device provided according to an embodiment of this application. Figure 3 The electronic device 300 in this embodiment may include one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The processors 301, input devices 302, output devices 303, and memories 304 communicate with each other via a communication bus 305. The memories 304 store computer programs, including program instructions. The processors 301 execute the program instructions stored in the memories 304. Specifically, the processors 301 are configured to invoke the program instructions to perform the functions of each module / unit in the above-described device embodiments, for example... Figure 2 The semantic verification module 21, trigger probability calculation module 22, and memo management module 23 are shown to have the following functions.

[0075] It should be understood that, in the embodiments of this application, the processor 301 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0076] Input device 302 may include a touchpad, a fingerprint sensor (for collecting the user's fingerprint information and fingerprint orientation information), a microphone, etc., and output device 303 may include a display (LCD, etc.), a speaker, etc.

[0077] The memory 304 may include read-only memory and random access memory, and provides instructions and data to the processor 301. A portion of the memory 304 may also include non-volatile random access memory. For example, the memory 304 may also store preset information such as a preset keyword library and preset probability thresholds.

[0078] In specific implementations, the processor 301, input device 302, and output device 303 described in the embodiments of this application can execute the implementation method described in the memo management method provided in the embodiments of this application, or they can execute the implementation method of the electronic device described in the embodiments of this application, which will not be repeated here.

[0079] In another embodiment of this application, a computer-readable storage medium is provided. This computer-readable storage medium stores a computer program, which includes program instructions. When executed by a processor, the program instructions implement all or part of the processes in the methods described above. Alternatively, the computer program can instruct related hardware to complete the process. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include any entity or device capable of carrying computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0080] The computer-readable storage medium can be an internal storage unit of the electronic device in any of the foregoing embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device. Furthermore, the computer-readable storage medium can include both internal and external storage units of the electronic device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0081] This application provides a computer program product, which includes computer-executable instructions or a computer program. The computer-executable instructions or computer program are stored in a computer-readable storage medium. The processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the memo management method described in this application.

[0082] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0083] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the electronic devices and units described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0084] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through some interfaces or units, or they may be electrical, mechanical, or other forms of connection.

[0085] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0086] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0087] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A memo management method, characterized in that, include: In response to receiving first voice information containing a first type of trigger word, the semantic integrity of the first voice information is checked, and environmental noise intensity and user motion state data are obtained. The trigger probability is determined based on the trigger word matching degree, the ambient noise intensity, and the user's motion state data; the trigger word matching degree is the matching degree between the first type of trigger word and a preset keyword library; If the trigger probability is greater than or equal to a preset probability threshold, and the semantic integrity verification result shows that the semantic integrity of the first voice information is complete, then memo information is generated based on the first voice information. If the trigger probability is greater than or equal to a preset probability threshold, and the semantic integrity verification result shows that the semantics of the first voice information are incomplete, then the first prompt information is output to instruct the user to input supplementary information, and memo information is generated based on the supplementary information and the first voice information. The steps of semantic integrity verification include: Obtain key entity categories; these key entity categories are obtained by extracting entity categories from multiple historical memos of the user. Extract all entity categories from the first voice information to obtain the first entity category set; If the first entity category set contains all the key entity categories, then the semantics of the first voice information are determined to be complete; otherwise, the semantics of the first voice information are determined to be incomplete. The determination of the trigger probability based on the trigger word matching degree, the environmental noise intensity, and the user's motion state data includes: A first probability value is determined based on the environmental noise intensity; the first probability value is negatively correlated with the environmental noise intensity. A second probability value is determined based on the user's motion state data; the second probability value is negatively correlated with the user's motion state data. The trigger probability is obtained by weighted summing of the trigger word matching degree, the first probability value, and the second probability value.

2. The memo management method as described in claim 1, characterized in that, The methods for determining the key entity categories include: Retrieve multiple historical memos from the user; For each historical memo, extract the entity category of that historical memo to obtain a second set of entity categories; In the second entity category set corresponding to each of the multiple historical memo messages, the same second entity category set is grouped together, the number of the same second entity category set in each group is counted, and the second entity category set corresponding to the group with the largest number is taken as the key entity category.

3. The memo management method as described in claim 1, characterized in that, Also includes: Obtain a preset keyword library and multiple historical first voice messages; For each historical first voice information, the historical first voice information is segmented to obtain multiple historical words. The first matching degree between each historical word and the preset keyword library is calculated. The historical word with the highest first matching degree is taken as the historical key word. The previous and next words of the historical key word are taken as historical auxiliary words. An auxiliary lexicon is constructed based on the historical auxiliary word segmentation corresponding to each of the multiple historical first speech information entries; The method for determining the matching degree of the trigger word includes: The first speech information is segmented into words to obtain multiple target words; Calculate the second matching degree between each target word segment and the preset keyword library; Select the target word with the highest second matching degree as the target key word; The first and second segments of the target keyword are used as target auxiliary segments, and the third matching degree between each target auxiliary segment and the auxiliary vocabulary is calculated. The matching degree of the trigger word is obtained by weighted summing the second matching degree corresponding to the target key word and the highest third matching degree.

4. The memo management method as described in claim 1, characterized in that, Also includes: If the trigger probability is less than a preset probability threshold and the semantic integrity verification result shows that the semantics of the first voice information are complete, then a confirmation memo is generated based on the first voice information, and a second prompt is output to instruct the user to confirm the confirmation memo. If a confirmation message is received for the memo to be confirmed, the memo to be confirmed will be saved to the memo.

5. The memo management method as described in claim 1, characterized in that, Also includes: In response to receiving second voice information containing a second type of trigger word, entity information is extracted from the second voice information to obtain the entity information of the second voice information; If the entity information of the second voice information contains entity information of a specified entity category, the entity information of the specified entity category shall be used as the target entity information; Based on the target entity information, the memo is queried to obtain the query results; If the entity information of the second voice information does not contain entity information of the specified entity category, all the memo information in the memo will be used as the query result; Output the query results.

6. The memo management method as described in claim 5, characterized in that, Before outputting the query results, the following is also included: If the query results contain a first-person singular, replace the first-person singular with a second-person singular.

7. A memo management device, characterized in that, include: The semantic verification module is used to perform semantic integrity verification on the first voice information containing the first type of trigger words in response to receiving the first voice information, and to obtain the environmental noise intensity and the user's motion state data. The trigger probability calculation module is used to determine the trigger probability based on the trigger word matching degree, environmental noise intensity, and user motion state data; the trigger word matching degree is the matching degree between the first type of trigger word and the preset keyword library; Memo generation module, used for: When the trigger probability is greater than or equal to a preset probability threshold and the semantic integrity verification result shows that the semantic integrity of the first voice information is complete, memo information is generated based on the first voice information; When the trigger probability is greater than or equal to the preset probability threshold and the semantic integrity verification result shows that the semantics of the first voice information is incomplete, the first prompt information is output to instruct the user to input supplementary information, and the memo information is generated based on the supplementary information and the first voice information. Specifically, when performing semantic integrity verification on the first voice information, the semantic verification module is used for: Obtain key entity categories; key entity categories are obtained by extracting entity categories from multiple historical memos of the user. For each entity category in the key entity categories, extract the entity information corresponding to that entity category from the first speech information; If each entity category in the key entity category has corresponding entity information, then the semantics of the first speech information is determined to be complete; otherwise, the semantics of the first speech information is determined to be incomplete. The trigger probability calculation module is specifically used for: A first probability value is determined based on the ambient noise intensity; the first probability value is negatively correlated with the ambient noise intensity. A second probability value is determined based on the user's motion state data; the second probability value is negatively correlated with the user's motion state data. The trigger probability is obtained by weighted summing of the trigger word matching degree, the first probability value, and the second probability value.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes a computer program, it implements the steps of the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When a computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 6.

Images