Content processing method, apparatus, and electronic device

Abstract

Embodiments of the present application disclose a content processing method and device and electronic equipment. In the method, when first input content is obtained, if it is determined that a control operation corresponding to the first input content cannot be executed, first content modification information corresponding to the first input content can be further obtained, wherein the first content modification information includes second input content input historically and modification of the second input content. When the second input content and the first input content satisfy a first similarity condition, third input content can be obtained through the first content modification information, so that the control intention of a user can be represented through the third input content, and a control operation corresponding to the third input content can be executed. Thus, the intelligent degree of the electronic equipment in executing the control operation is improved.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and more specifically, to a content processing method, apparatus, and electronic device. Background Technology

[0002] With the development of terminal technology, users can control electronic devices by triggering commands. For example, when a user wants to activate Bluetooth, they can use the voice command "Activate Bluetooth" to trigger the electronic device to activate Bluetooth. However, the electronic devices are not yet intelligent enough in executing user-triggered commands. Summary of the Invention

[0003] In view of the above problems, this application proposes a content processing method, apparatus, and electronic device to improve the above problems.

[0004] In a first aspect, this application provides a content processing method, the method comprising: upon obtaining first input content, if it is determined that a control operation corresponding to the first input content cannot be executed, then obtaining first content modification information corresponding to the first input content, the first content modification information including historically input second input content and the modification status of the second input content, wherein the second input content and the first input content satisfy a first similarity condition; obtaining third input content through the first content modification information; and executing a control operation corresponding to the third input content.

[0005] Secondly, this application provides a content processing apparatus, the apparatus comprising: a modification information acquisition unit, configured to, upon obtaining first input content, if it is determined that the control operation corresponding to the first input content cannot be executed, acquire first content modification information corresponding to the first input content, the first content modification information including historically input second input content and the modification status of the second input content, wherein the second input content and the first input content satisfy a first similarity condition; and a content processing unit, configured to obtain third input content through the first content modification information;

[0006] The control unit is used to perform control operations corresponding to the third input content.

[0007] Thirdly, this application provides an electronic device including a processor and a memory; one or more programs are stored in the memory and configured to be executed by the processor to implement the above-described method.

[0008] Fourthly, this application provides a computer-readable storage medium storing program code, wherein the above-described method is executed when the program code is run by a starter controller.

[0009] Fifthly, this application provides a computer program product comprising computer instructions stored in a computer-readable storage medium; the computer program product is used to implement the above-described method.

[0010] This application provides a content processing method, apparatus, and electronic device. In this method, upon obtaining first input content, if it is determined that the control operation corresponding to the first input content cannot be executed, first content modification information corresponding to the first input content can be further obtained. This first content modification information includes historically input second input content and its modification details. If the second input content and the first input content satisfy a first similarity condition, a third input content can be obtained through this first content modification information. This third input content can then represent the user's control intent, thereby executing the control operation corresponding to the third input content. Therefore, by using the above method, even when the user's current first input content cannot be executed, a third input content with the same control intent as the first input content can be obtained by utilizing historical inputs similar to the first input content and their corresponding modifications. This ensures that even when the first input content itself cannot be executed, the electronic device can still realize the user's control intent, improving the intelligence level of the electronic device in executing control operations. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A schematic diagram of an operating environment for the content processing method proposed in an embodiment of this application is shown;

[0013] Figure 2 A schematic diagram of another operating environment for the content processing method proposed in the embodiments of this application is shown;

[0014] Figure 3 A flowchart of a content processing method proposed in an embodiment of this application is shown;

[0015] Figure 4 A schematic diagram illustrating the recommended content shown in the embodiments of this application is provided.

[0016] Figure 5 A schematic diagram illustrating the execution of control operations corresponding to the third input content in an embodiment of this application is shown;

[0017] Figure 6 A flowchart of a content processing method according to another embodiment of this application is shown;

[0018] Figure 7 A flowchart of a content processing method according to another embodiment of this application is shown;

[0019] Figure 8 A flowchart of a content processing method according to another embodiment of this application is shown;

[0020] Figure 9 This paper shows a structural block diagram of a content processing system proposed in an embodiment of this application;

[0021] Figure 10 A structural block diagram of a call processing device according to an embodiment of this application is shown;

[0022] Figure 11 This invention illustrates a structural block diagram of an electronic device for performing a content processing method according to an embodiment of this invention;

[0023] Figure 12 This is a storage unit in this application embodiment for storing or carrying program code that implements the content processing method according to this application embodiment. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0025] With the rapid development of terminal technology, users can now control electronic devices through various interaction methods such as voice, gestures, and touch. Taking voice interaction as an example, users only need to say the command "Start Bluetooth," and the electronic device can use technologies such as voiceprint recognition and natural language processing to interpret the user's intent and trigger the Bluetooth module to start. However, the level of intelligence of related electronic devices is still insufficient.

[0026] Therefore, the inventors have proposed a content processing method, apparatus, and electronic device that can improve the above-mentioned problems. In this method, if it is determined that the control operation corresponding to the first input content cannot be executed after obtaining the first input content, first content modification information corresponding to the first input content can be further obtained. The first content modification information includes historically input second input content and its modification status. If the second input content and the first input content satisfy a first similarity condition, a third input content can be obtained through the first content modification information. This third input content can then represent the user's control intent, thereby executing the control operation corresponding to the third input content. Thus, by means of the above method, even when the user's current first input content cannot be executed, a third input content with the same control intent as the first input content can be obtained by using historical inputs similar to the first input content and their corresponding modifications. This ensures that even when the first input content itself cannot be executed, the electronic device can still realize the user's control intent, improving the intelligence level of the electronic device in the process of executing control operations.

[0027] The application scenarios involved in the embodiments of this application will be introduced below.

[0028] In this embodiment of the application, the provided content processing method can be executed by an electronic device. In this manner, all steps of the content processing method provided in this embodiment can be performed by the electronic device. For example, as... Figure 1 As shown, all steps in the content processing method provided in this application embodiment can be executed by the voice assistant in the electronic device 100.

[0029] Alternatively, the content processing method provided in this application embodiment can also be executed collaboratively by an electronic device and a server. In this method, some steps of the content processing method provided in this application embodiment are executed by the electronic device, while other steps are executed by the server. For example, as shown... Figure 2 As shown, the electronic device 100 can perform a content processing method including: obtaining first input content. Then, the electronic device 100 can transmit the first input content to the server 200. After receiving the first input content, the server 200 can perform subsequent steps to obtain third input content. After obtaining the third input content, the server 200 can return the third input content to the electronic device 100 so that the electronic device 100 can perform a control operation corresponding to the third input content.

[0030] It should be noted that in this method where electronic devices and servers work together, the steps performed by the electronic devices and servers are not limited to those described in the examples above. In practical applications, the steps performed by the electronic devices and servers can be dynamically adjusted according to the actual situation.

[0031] It should be noted that the electronic equipment 100, in addition to being for Figure 1 and Figure 2 Besides smartphones, the device can also be a tablet, wearable device, smart voice assistant, etc. Server 200 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as an artificial intelligence platform. In the case where the content processing method provided in this embodiment is executed by a server cluster or distributed system composed of multiple physical servers, different steps in the content processing method can be executed by different physical servers, or can be executed in a distributed manner by servers built on a distributed system.

[0032] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0033] Please see Figure 3 This application provides a content processing method, which includes:

[0034] S110: If, upon obtaining the first input content, it is determined that the control operation corresponding to the first input content cannot be executed, then the first content modification information corresponding to the first input content is obtained. The first content modification information includes the historically input second input content and the modification status of the second input content. The second input content and the first input content satisfy the first similarity condition.

[0035] In this embodiment, the first input content is content representing the user's control needs. That is, the first input content can also be understood as a control command input by the user. The first input content can be a word or a sentence based on natural language. Natural language sentences can be understood as sentences in everyday communication, which are composed of words and phrases combined according to grammatical rules to express thoughts, describe events, ask questions, or make statements. For example, the first input content could be "call Mom." The first input content can be entered by the user through an input control, or it can be selected by the user from recommended content provided by the electronic device.

[0036] In one approach, the input controls can be text input controls or voice input controls.

[0037] Optionally, the electronic device can directly use the content already entered in the text input control as the first input content, or the electronic device can semantically process the content in the text input control and use the semantically processed content as the first input content. The content in the text input control can be entered by the user using a virtual keyboard, or it can be input by the electronic device after converting the user's spoken speech.

[0038] Optionally, the electronic device can trigger voice acquisition by touching a voice input control, so that the content acquired during the voice acquisition process can be used as the first input. This voice input control can be a virtual control or a physical control of the electronic device.

[0039] Optionally, the recommended content can be generated by the electronic device based on content already entered in an input control (e.g., a text input control), or the recommended content can be generated by the electronic device based on the user's personal preferences or operating habits of the electronic device.

[0040] Displaying recommended content allows users to control electronic devices more quickly and conveniently. It should be noted that when recommended content is displayed, the electronic device can respond to touch operations (e.g., clicks) applied to the recommended content, thus identifying the touched content as the first input. For example, as shown... Figure 4 As shown, in Figure 4 The interface 10 shown in the image displays multiple recommended items in area 20.

[0041] In the embodiments of this application, there are multiple ways to obtain recommended content.

[0042] One approach is to obtain recommended content based on the input already provided.

[0043] Optionally, after obtaining the input content, the target entity words in the input content can be identified first. In the process of identifying target entity words, the input content can be broken down into meaningful lexical units to identify the part of speech of each word, such as noun, verb, adjective, etc. Based on the part-of-speech identification results, target entity words such as person names, place names, and organization names can be identified from the input content. Then, using the target entity words, multiple corresponding search terms are obtained from a candidate search term library; using multiple search terms and related information, recommended content is obtained, where the related information represents the degree of association between the search terms and the input content. Optionally, this related information may include statistical quantities such as co-occurrence frequency and mutual information.

[0044] Co-occurrence frequency (COF) refers to the number of times two words (in this example, the entered content and candidate search terms) appear together in the same document or the same set of documents (such as search logs). A higher COF indicates a stronger correlation between the two words, and a greater likelihood that they will appear in the same context. Mutual information is a statistic that measures the dependency between two random variables. In information theory, it represents the amount of information contained in one random variable about another. In text analysis, mutual information is used to measure the degree of correlation between two words, that is, the influence of the occurrence of one word on the probability of the occurrence of another word.

[0045] Optionally, recommended content is obtained through multiple search terms and related information, including obtaining multiple candidate recommended content through multiple search terms and related information. It should be noted that in this embodiment, the candidate recommended content is a sentence expressed in natural language. Obtaining multiple candidate recommended content through multiple search terms and related information can be understood as combining sentences based on related information and multiple search terms to obtain multiple candidate recommended content. Multiple search terms can be input into a prediction model to predict the word sequence that the user might input after the already input content, thus obtaining enriched search terms. The enriched search terms include not only the aforementioned multiple search terms but also search terms predicted by the prediction model. Then, the enriched search terms can be mapped to entities and relationships in a knowledge graph to construct semantically rich candidate recommended content.

[0046] After obtaining multiple candidate recommendations, the recommended content can be derived from these candidates based on their respective attribute information. These attributes include one or more of the following: click probability, search frequency, and semantic flow degree. Optionally, the multiple candidate recommendations can be ranked based on their respective attribute information, and the top N ranked candidates can be selected as the recommended content.

[0047] One approach is to recommend content based on user preferences.

[0048] Optionally, the electronic device can store the user's personalized data, which can characterize the user's preferences, allowing the electronic device to generate recommended content based on this personalized data. The personalized data stored locally on the electronic device can be personalized data related to multiple aspects. For example, it may include personalized data on diet, travel, sports, communication, and health.

[0049] As described above, the first input represents the user's control request. If the first input cannot be executed, it means that the electronic device cannot fulfill the user's control request. In this embodiment, it can be determined whether the control operation corresponding to the first input can be executed in various ways.

[0050] In one approach, if the target operation included in the first input content does not exist, then it is determined that the control operation corresponding to the first input content cannot be executed.

[0051] It's important to note that when a user controls an electronic device, there's usually an operation target. For example, if a user wants to make a call, the person they're calling can be considered an operation target. Similarly, if a user wants to activate Bluetooth on the electronic device, the Bluetooth module is the operation target. Therefore, after receiving the first input, the electronic device can first determine the corresponding operation target. If the operation target in the first input is not present or not recorded in the electronic device, it's determined that the operation target doesn't exist, and thus the control operation corresponding to the first input cannot be executed. For example, the first input might include "call Mom." In this case, the user's operation target could be "Mom." However, the electronic device's address book might not have a contact named "Mom," so the electronic device cannot execute "call Mom." Similarly, the first input might include "activate the infrared module." However, the electronic device might not have an infrared module (an operation target), so the electronic device cannot execute "activate the infrared module."

[0052] As one approach, if the modified historical input content contains input content corresponding to the first input content, then it is determined that the control operation corresponding to the first input content cannot be executed.

[0053] Optionally, each time the electronic device detects unexecutable input (e.g., the first input), it can issue a prompt to the user, suggesting clarification of the unexecutable input. This allows the electronic device to determine the actual operational intent of the unexecutable input based on the user's clarification. After receiving the user's clarification, the electronic device can also store the unexecutable input and simultaneously store any modifications made to it (based on the user's clarification). Storing the unexecutable input can be understood as storing the modified historical input.

[0054] For example, the user's input could be "Call Mom". If the electronic device detects that the contact "Mom" is not stored in the address book, it can further send a prompt message such as "No contact Mom found, please tell me her name in the address book". The user's further clarification input could be "Call Zhang San". In this case, the electronic device can store the input "Call Mom" ​​and store the corresponding modification information, which could include "Change the username <Mom> to <Zhang San>".

[0055] As one approach, if the intended operation corresponding to the first input is unclear, then the control operation corresponding to the first input cannot be executed. Specifically, when the first input is ambiguous, lacks information, or has a vague instruction, the electronic device cannot clearly determine the specific operation to be performed. For example, if the first input is "play music," the electronic device may not know the specific track the user wants to play. Similarly, if the first input is "order takeout," the electronic device may not know what kind of food the user wants to eat.

[0056] As mentioned above, the first input content that cannot be executed may have been entered by the user before and may have been clarified by the user, thus corresponding content modification information will be stored. In this case, if the first input content is detected as unexecutable, the first content modification information corresponding to the first input content can be further obtained.

[0057] In this embodiment of the application, the content modification information may include input content and the corresponding modification details. For example, a content modification message may be: Call Mom: Change username <Mom> to <Zhang San>; where "Call Mom" ​​is the input content in the content modification message, and "Change username <Mom> to <Zhang San>" is the modification details corresponding to the input content.

[0058] When multiple content modification messages exist, the input content (historical input content) included in each of the multiple content modification messages can be matched with the first input content for similarity. In this case, the second input content included in the determined first content modification message satisfies the first similarity condition with the first input content. Here, the second input content can be understood as the input content included in the first content modification message.

[0059] Optionally, the first similarity condition indicates that the second input content is identical to the content included in the first input content. For example, if the first input content is "call mom", then the second input content included in the determined first content modification information can also be "call mom". Optionally, the first similarity condition indicates that the second input content has the same semantics as the first input content. For example, if the first input content is "call mom", then the second input content included in the determined first content modification information can be "call mom".

[0060] S120: Modify the information based on the first content to obtain the third input content.

[0061] After obtaining the first content modification information, the third input content can be obtained through the second input content included in the first content modification information and the corresponding modification details of the second input content. Specifically, the second input content can be modified based on the modification details to obtain the third input content. For example, the obtained first content modification information could be: "Call Mom: Change username <Mom> to <Zhang San>". In this case, the obtained third input content could be: "Call Zhang San".

[0062] S130: Execute the control operation corresponding to the third input content.

[0063] After obtaining the third input content, the corresponding control operation can be executed to realize the user's control intention.

[0064] For example, such as Figure 5 As shown, the first input content entered by the user can be "call mom". However, if the electronic device determines that the first input content is unexecutable, it can obtain the corresponding first content modification information based on the method provided in this application. This first content modification information indicates that "mom" in the first input content has been changed to "Zhang San". In this case, as... Figure 5 As shown, the final operation performed by the electronic device could be to call Zhang San, where... Figure 5 The "Zhang San" in square brackets represents the actual recipient of this call.

[0065] This embodiment provides a content processing method that, in the above manner, when the first input content currently entered by the user cannot be executed, a third input content with the same control intent as the first input content can be obtained by using historical inputs similar to the first input content and the modification status corresponding to the historical inputs. This allows the electronic device to still realize the user's control intent even when the first input content itself cannot be executed, thereby improving the intelligence level of the electronic device in the process of executing control operations.

[0066] Please see Figure 6 This application provides a content processing method, which includes:

[0067] S210: If, upon obtaining the first input content, it is determined that the control operation corresponding to the first input content cannot be executed, it is determined whether the control intent of the first input content is the first target intent;

[0068] S220: If the control intent of the first input content is the first target intent, then obtain the first content modification information corresponding to the first input content. The first content modification information includes the second input content that was input in the past and the modification status of the second input content. The second input content and the first input content satisfy the first similarity condition.

[0069] In this embodiment, if the first input content cannot be executed, the corresponding control intent can be determined based on the first input content. Then, the determined control intent is used to determine how to obtain executable input content. It should be noted that in some cases, the electronic device may not store corresponding content modification information for each input content. In this case, the control intent can be used to determine whether the current first input content has corresponding first content modification information.

[0070] Specifically, if the control intent of the first input content is determined to be the first target intent, then it is determined that executable input content (e.g., the third input content) can be obtained through stored content modification information. Conversely, if the control intent of the first input content is determined not to be the first target intent, then the user can be prompted to re-enter the content, and the third input content can be obtained based on the re-entered content.

[0071] The first target intent is an intent that allows executable input content to be obtained through content modification information. This first target intent can be pre-configured by the developer. For example, the first target intent may include a phone call intent and a navigation intent.

[0072] S221: Modify the information using the first content to obtain the third input content.

[0073] S230: If the control intent of the first input content is not the first target intent, then a prompt message is sent to the user to prompt the user to re-enter the content, and the third input content is obtained based on the content re-entered by the user.

[0074] Obtaining the third input content based on the user's re-entered content can include determining the user's re-entered content as the third input content. Alternatively, it can involve modifying the first input content based on the user's re-entered content to obtain the third input content.

[0075] S240: Execute the control operation corresponding to the third input content.

[0076] This embodiment provides a content processing method that, when the user's current first input content cannot be executed, a third input content with the same control intent as the first input content can be obtained by utilizing historical inputs similar to the first input content and the corresponding modifications. This allows the electronic device to still realize the user's control intent even when the first input content itself cannot be executed, improving the intelligence level of the electronic device in the process of executing control operations. Furthermore, in this embodiment, the corresponding control intent can be determined based on the first input content. If the control intent is the first target intent, the third input content can be obtained through content modification information (e.g., first content modification information). If the control intent is not the first target intent, the user can be prompted to re-enter the content, and the third input content can be obtained based on the re-entered content. This allows the user's control intent to determine how to obtain the third input content that the device can successfully execute, avoiding resource waste caused by content clarification.

[0077] Please see Figure 7 This application provides a content processing method, which includes:

[0078] S310: If, upon obtaining the first input content, it is determined that the control operation corresponding to the first input content cannot be executed, then multiple second content modification information corresponding to the first input content is obtained, and the historical input content included in each of the multiple second content modification information satisfies the second similarity condition with the first input content.

[0079] In this embodiment, multiple second content modification pieces of information that are relatively similar to the first input content can be filtered from the stored content modification information. In this case, the process of obtaining multiple second content modification pieces of information can be understood as a coarse recall process. As a coarse recall process, the degree of similarity represented by the second similarity condition can be lower than the degree of similarity represented by the first similarity condition.

[0080] S320: Determine the first content modification information from multiple second content modification information, the first content modification information including the historical input second input content and the modification status of the second input content, the second input content and the first input content satisfy the first similarity condition.

[0081] After obtaining multiple sets of second content modification information, the first content modification information that best matches the first input content can be obtained from these multiple sets of second content modification information. In other words, compared to the process of obtaining the second content modification information, the process of obtaining the first content modification information can be understood as a more refined filtering process.

[0082] In one approach, the similarity between the historical input content and the first input content included in each of the multiple second content modification information can be sorted, so that the second content modification information with the highest similarity between the historical input content and the first input content is used as the first content modification information.

[0083] For example, the multiple second content modification information obtained through the aforementioned method can be shown in the following table:

[0084] Input content Modifications Input content S1 Change XXX to YYY Input content S2 Change XX to YY Input content S3 Change XXXX to YYYY

[0085] As shown in the table above, in one instance of second content modification information, the historical input content is "Input Content S1", and the corresponding modification is "Change XXX to YYY". In another instance, the historical input content is "Input Content S2", and the corresponding modification is "Change XX to YY". In yet another instance, the historical input content is "Input Content S3", and the corresponding modification is "Change XXXX to YYYY". If it is determined that input content S2 and the first input content satisfy the first similarity condition, then the second content modification information including input content S2 is determined to be the first content modification information.

[0086] S330: Modify the information based on the first content to obtain the third input content.

[0087] S340: Execute the control operation corresponding to the third input content.

[0088] This embodiment provides a content processing method that enhances the intelligence of electronic devices during control operations. Furthermore, in this embodiment, during the process of obtaining the first content modification information, multiple second content modification information pieces can be acquired first. Then, the most suitable first content modification information can be determined from these multiple second content modification information pieces, thereby facilitating the more accurate acquisition of content modification information that matches the first input content.

[0089] Please see Figure 8 This application provides a content processing method, which includes:

[0090] S410: If, upon receiving the first input content, it is determined that the control operation corresponding to the first input content cannot be executed, determine whether there is any first content modification information.

[0091] S420: If there is first content modification information, then obtain the first content modification information corresponding to the first input content. The first content modification information includes the historical second input content and the modification status of the second input content. The second input content and the first input content satisfy the first similarity condition.

[0092] S421: Modify the information using the first content to obtain the third input content.

[0093] S422: Execute the control operation corresponding to the third input content.

[0094] S430: If there is no first content modification information, a prompt message is sent to the user to prompt the user to clarify the first input content.

[0095] S440: Generate and store third content modification information based on the clarification content input by the user. The third content modification information includes the first input content and the modification status corresponding to the first input content. The modification status is obtained based on the clarification content.

[0096] S450: Obtain the fourth input content by modifying the information in the third content.

[0097] S460: Execute the control operation corresponding to the fourth input content.

[0098] As one approach, if, when generating third content modification information, there is stored fourth content modification information including the first input content, and the generation time of the fourth content modification information is earlier than the generation time of the third content modification information, the fourth content modification information is deleted.

[0099] As one approach, if there is no first content modification information, the control intent corresponding to the first input content can be detected first. If the control intent corresponding to the first input content is the second target intent, the user's preferences can be obtained so that the first input content can be modified according to the user's preferences to obtain the fifth input content, so that the electronic device can perform the control operation corresponding to the fifth input content.

[0100] The second target intent can be understood as a control intent that can be directly modified based on the user's preferences. For example, if the first input is "Order me some takeout," the electronic device may be unable to execute the corresponding control operation because the control intent of the first input is unclear. However, if the electronic device can obtain the user's food preferences, it can generate the fifth input directly based on the user's food preferences without requiring the user to re-enter the content. For example, if the electronic device can obtain information that the user likes steak, it can directly generate the fifth input based on this preference, such as "Order me some steak."

[0101] In this embodiment, user preferences can be determined using personalized user data. Personalized data can be understood as data representing a user's habits or preferences, or as data representing a user profile. Before acquiring personalized data, target users can be identified, and then their corresponding personalized data can be obtained. Target users can be understood as users referenced when acquiring personalized data. In this embodiment, target users can be understood as the user owning the electronic device or the user currently using the electronic device. The user owning the electronic device can be understood as the user corresponding to the identification information recorded in the electronic device. This identification information may include a user account or a current phone number.

[0102] In one approach, the personalized data of the user of an electronic device can be stored within the device itself. This locally stored personalized data can encompass multiple aspects, such as dietary, travel, sports, and health-related data. When a specific event is triggered, the personalized data corresponding to that event can be retrieved as the target personalized data. For example, if the specified event is related to diet (e.g., the specified event could include the user's recent preferences regarding food delivery), then dietary-related personalized data can be retrieved locally.

[0103] Alternatively, the electronic device can store the user's identification information locally. In this case, the user represented by the identification information can be used as the target user. Optionally, the identification information can be an account number or a phone number. Alternatively, all users' personalized data can be centrally stored in the cloud. In this way, when a specified event is triggered, the target personalized data can be retrieved from the cloud based on the stored identification information and the triggered specified event.

[0104] As another approach, the current user of an electronic device may not be the device's owner. For example, in some cases, the owner of an electronic device may lend it to another user. In this situation, directly obtaining personalized data based on the device's owner might result in personalized data that does not meet the current user's needs. For instance, if user U1 owns an electronic device and lends it to user U2, and user U2 triggers a specified event during use (e.g., inputting "order takeout for me"), but the electronic device still uses user U1's personalized data as the basis for subsequently modifying the first input, it would not only lead to a privacy breach for user U1 but also fail to meet user U2's operational needs. In this approach, when a specified event is triggered, the electronic device can first perform user identification to target the identified user.

[0105] In this embodiment, the method of user identification is not specifically limited. For example, as a user identification method, the electronic device can collect identity features through a functional device when a specified event is triggered, and then identify the user's identity by matching these identity features. Optionally, the functional device can be a camera or a structured light module. For example, the electronic device can collect images through a front-facing camera, and then perform facial recognition on the collected images to identify the identified user as the target user. Based on the aforementioned example of the electronic device, if user U1 lends the electronic device to user U2 for use, the electronic device can identify the current user as user U2 through user identification, and then the electronic device can obtain user U2's personalized data.

[0106] This embodiment provides a content processing method that enhances the intelligence of electronic devices during control operations. Furthermore, in this embodiment, if it is determined that the first input content cannot be executed, it can first check whether there is pre-stored first content modification information corresponding to the first input content. If so, the corresponding third input content can be obtained directly through the pre-stored first content modification information. If not, a prompt message can be issued to the user to prompt them to re-enter the content, and the third input content can be obtained based on the user's re-entered content. This allows the third input content to be obtained through multiple methods.

[0107] As one approach, if it is determined that the control operation corresponding to the first input content cannot be executed, then the first content modification information corresponding to the first input content is obtained, including: if it is determined that the control operation corresponding to the first input content cannot be executed, then the first content modification information corresponding to the first input content is obtained through the first functional module; the third input content is obtained through the first content modification information, including: the third input content is obtained through the second functional module and the first content modification information, wherein the first content modification information is transmitted from the first functional module to the second functional module in the form of a question-and-answer pair.

[0108] The first functional module can be understood as a coarse recall module, and the second functional module can be understood as a fine ranking module. The specific roles of the coarse recall and fine ranking modules will be discussed later. Figure 9 The content is described in the article.

[0109] Next, we will proceed through... Figure 9 The following describes a content processing system according to an embodiment of this application. The content processing system includes a dialogue rewriting module, a rewriting coarse recall module, an update and merging module, a main link memory module, a search intent recognition module, a coarse recall module, and a fine ranking module.

[0110] The dialogue rewriting module is responsible for semantically understanding the current multi-turn dialogue history and rewriting it into two parts: content a and content b. Content a is the retrieval key, which can be used by the subsequent coarse recall module to search for the rewritten content (i.e., to retrieve the first content modification information in this embodiment). Content b is the multi-turn understood dialogue, representing the user's final intention after multiple clarifications. For example, the user's original multi-turn dialogue is as follows:

[0111] Query: Call Mom;

[0112] Answer: I couldn't find the contact person's name. Please tell me her name in my contacts.

[0113] Query: Call Zhang San;

[0114] Answer: Okay.

[0115] For the example above, the rewritten content obtained after rewriting by the dialogue rewriting module is as follows:

[0116] Content a: Calling for Mom;

[0117] Content b: Change the username <Mom> to <Zhang San>.

[0118] Among them, content a and content b constitute a type of content modification information involved in this application embodiment. The rewritten content is transmitted through the central control unit to the rewrite coarse-recall retrieval module for processing.

[0119] The purpose of the rewrite coarse-recall module is to memorize the rewrite results and perform coarse-recall to prevent conflicts or duplications between newly added rewrite content and existing rewrite content in the memory. It primarily reads the rewrite results obtained from the upstream and retrieves the rewrite content from the main link memory based on content 'a' in the rewrite results. For example, the upstream outputs the rewrite content: {"Call Mom: Change username <Mom> to <Zhang San>", "Time": "October 2, 2024"}

[0120] The coarse-recall module has been rewritten based on semantic retrieval, returning the following similar or conflicting memories:

[0121] {“Call Mom: Change username <Mom> to <Xiaoqiang>”, “Time”: “October 1, 2024”}

[0122] {“Call Xiaoqiang: Change username <Xiaoqiang> to <Zhang Hong>”, “Time”: “October 1, 2024”}

[0123] By retrieving similar or conflicting memories, the information is passed to downstream modules (e.g., update and merge modules) for deduplication and resolution of similar or conflicting memories.

[0124] The update and merging module can combine the rewriting of similar or conflicting memories sent by the coarse recall module to deduplicate and resolve them. The overall deduplication and resolution module mainly considers the following three dimensions:

[0125] For similar memory header nodes that require modification, the most recent memory content is prioritized. For example, if two memories both involve modifying the username "Mom" during a phone call, the most recent memory content is used, and older memories should be deleted. For instance, in the aforementioned embodiment, if a third content modification information is obtained, the fourth content modification information will be deleted.

[0126] Connectivity: If memories are connected end-to-end, they should be merged, and intermediate memories should be deleted. For example, if historical memories contain two entries: "correct A to B" and "correct B to C", they should be merged into "correct A to C", and the intermediate memory of "correcting A to B" should be deleted.

[0127] Model understanding involves using LLM to summarize existing memories and obtain effective memories.

[0128] Based on the above three dimensions, the final updated memory content can be obtained. For example, based on the example mentioned above, the memory content after conflict resolution and deduplication is as follows: Added: {"Call Mom: Change username <Mom> to <Zhang San>", "Time": "October 2, 2024"} Deleted: {"Call Mom: Change username <Mom> to <Xiao Qiang>", "Time": "October 1, 2024"}.

[0129] Specifically, the memory content processed by the dialogue rewriting module, the coarse-recall rewriting module, and the update and merging module is uniformly stored in the main link memory bank in the form of vectors (embeddings). Each memory is assigned a unique memory ID to assist downstream modules in identifying the memory for retrieval. In other words, the aforementioned content modification information can be stored in the main link memory bank.

[0130] The intent recognition module, located upstream of the main dialogue decision module, is primarily used to recognize intents when a user triggers an ambiguous query again (e.g., when the first input cannot be executed) after valid information has been stored for a certain period. To reduce unnecessary resource waste, the intent recognition module can filter intents from historical dialogues and the current round of queries based on a classification model (e.g., the BERT model). In one approach, queries with control intents other than the target intent are filtered, effectively ensuring the effectiveness and accuracy of subsequent coarse recall and fine ranking modules. The target intent can include both making a phone call and navigation.

[0131] Similar to the rewritten coarse recall module, the coarse recall module can perform semantic retrieval (e.g., semantic retrieval based on vectors). The retrieved object is the query of the current round (i.e., the first input content obtained at the moment), and the retrieved candidate is the content stored in the main link memory. Optionally, the coarse recall module does not undergo semantic fine-tuning. It mainly uses a method of vector combined with threshold control to ensure that the most useful memory information enters the subsequent fine ranking module with high recall.

[0132] The fine-ranking module is obtained by semantic fine-tuning based on specific data, which enables the fine-ranking module to accurately process the memory information after coarse recall. For example, if the query of the current user is "Xiao Bu, call my mom", then the memory result after memory classification -> coarse recall -> fine-ranking is: "Call mom: correct the username <mom> to <Zhang San>".

[0133] The refined memory content will be transmitted through the central control session protocol, and as part of the session, it will be passed to the user's instruction module for instruction understanding and execution.

[0134] Memory enhancement rounds:

[0135] query: "Call Mom: Change username <Mom> to <Zhang San>";

[0136] Answer: "Okay".

[0137] This round transforms the control intention of "calling mom" into "calling Zhang San," ensuring the accuracy of machine execution while shortening the execution chain and achieving a "fast and accurate" execution effect.

[0138] Please see Figure 10 This application provides a content processing apparatus 500, which includes:

[0139] The modification information acquisition unit 510 is used to, when the first input content is obtained, if it is determined that the control operation corresponding to the first input content cannot be executed, acquire the first content modification information corresponding to the first input content. The first content modification information includes the historically input second input content and the modification status of the second input content. The second input content and the first input content satisfy a first similarity condition.

[0140] The content processing unit 520 is used to obtain the third input content by modifying the first content information.

[0141] The control unit 530 is used to perform control operations corresponding to the third input content.

[0142] In one approach, if the modified historical input content contains input content corresponding to the first input content, then it is determined that the control operation corresponding to the first input content cannot be executed; or if the operation target included in the first input content does not exist, then it is determined that the control operation corresponding to the first input content cannot be executed.

[0143] In one manner, the modification information acquisition unit 510 is specifically used to acquire first content modification information corresponding to the first input content if it is determined that the control operation corresponding to the first input content cannot be executed, and the control intent of the first input content is the first target intent. The modification information acquisition unit 510 is also specifically used to issue a prompt message to the user if it is determined that the control operation corresponding to the first input content cannot be executed, and the control intent of the first input content is not the first target intent, to prompt the user to re-enter the content.

[0144] As one method, the modification information acquisition unit 510 is specifically used to acquire multiple second content modification information corresponding to the first input content, each of the multiple second content modification information including historical input content, which satisfies a second similarity condition with the first input content; and to determine the first content modification information from the multiple second content modification information.

[0145] As one method, the modification information acquisition unit 510 is specifically used to acquire the first content modification information corresponding to the first input content if the first content modification information exists; if the first content modification information does not exist, issue a prompt message to the user to prompt the user to clarify the content of the first input content; and generate and store third content modification information based on the clarification content input by the user. The third content modification information includes the first input content and the modification status corresponding to the first input content, and the modification status is obtained based on the clarification content.

[0146] Optionally, the modification information acquisition unit 510 is further specifically used to delete the fourth content modification information if, when the third content modification information is generated, there is a fourth content modification information that includes the first input content and the generation time of the fourth content modification information is earlier than the generation time of the third content modification information.

[0147] This embodiment provides a content processing device that enables the electronic device to obtain a third input content with the same control intent as the first input content when the first input content currently entered by the user cannot be executed, by using historical inputs similar to the first input content and the modification status corresponding to the historical inputs. This allows the electronic device to still realize the user's control intent even when the first input content itself cannot be executed, thereby improving the intelligence level of the electronic device in the process of executing control operations.

[0148] It should be noted that the device embodiments in this application correspond to the aforementioned method embodiments. The specific principles in the device embodiments can be found in the content of the aforementioned method embodiments, and will not be repeated here.

[0149] The following will combine Figure 11 This application describes an electronic device.

[0150] Please see Figure 11 Based on the aforementioned content processing method and apparatus, this application embodiment also provides another electronic device 1000 capable of executing the aforementioned content processing method. The electronic device 1000 includes one or more (only one shown in the figure) processors 102, a memory 104, and a network module 106 coupled to each other. The memory 104 stores programs capable of executing the content described in the aforementioned embodiments, and the processors 102 can execute the programs stored in the memory 104.

[0151] The processor 102 may include one or more processing cores. The processor 102 connects to various parts within the electronic device 1000 using various interfaces and lines, and performs various functions and processes data of the electronic device 1000 by running or executing instructions, programs, code sets, or instruction sets stored in the memory 104, and by calling data stored in the memory 104. Optionally, the processor 102 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA), or the processor 102 may be an application processor. The processor 102 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 102, but may be implemented separately using a communication chip.

[0152] The memory 104 may include random access memory (RAM) or read-only memory (ROM). The memory 104 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 104 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), and instructions for implementing the various method embodiments described below.

[0153] The network module 106 is used to enable information interaction between the electronic device 1000 and other devices, such as transmitting device control commands, manipulation request commands, and status information acquisition commands. However, the network module 106 may differ depending on the specific device being used.

[0154] In another case, the electronic device 1000 may also include a screen, a sensor module, and an audio acquisition device.

[0155] The screen can be used for displaying images. For example, when a game program is running, it can be used to display the game screen corresponding to the game program. In one approach, the electronic device 1000 can take screenshots of multiple frames continuously displayed on the screen to obtain the multiple frames needed to detect whether the electronic device 100 is displaying an event animation.

[0156] The sensor module may include at least one sensor. Specifically, the sensor module may include, but is not limited to, light sensors, motion sensors, pressure sensors, infrared thermal sensors, distance sensors, accelerometers, and other sensors. Among them, the pressure sensor is a sensor that can detect pressure generated by pressing on the electronic device 1000. That is, the pressure sensor detects pressure generated by contact or pressing between the user and the audio playback device, such as pressure generated by contact or pressing between the user's ear and the mobile terminal. Therefore, the pressure sensor can be used to determine whether contact or pressing has occurred between the user and the electronic device 1000, and the magnitude of the pressure. The accelerometer can detect the magnitude of acceleration in various directions (generally three axes), and when stationary, it can detect the magnitude and direction of gravity. It can be used for applications that identify the posture of the electronic device 1000 (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometers, taps), etc. In addition, the electronic device 1000 may also be equipped with other sensors such as gyroscopes, barometers, hygrometers, and thermometers, which will not be elaborated here.

[0157] An audio acquisition device is used to acquire audio signals. Optionally, the audio acquisition device may include multiple audio acquisition components. These audio acquisition components may be microphones.

[0158] In one approach, the network module of the electronic device 1000 is a radio frequency (RF) module. This RF module is used to receive and transmit electromagnetic waves, converting electromagnetic waves into electrical signals, thereby enabling communication with a communication network or other devices. The RF module may include various existing circuit elements used to perform these functions, such as antennas, RF transceivers, digital signal processors, encryption / decryption chips, user identity module (SIM) cards, memory, etc. For example, the RF module can interact with external devices by transmitting or receiving electromagnetic waves, thereby receiving audio signals transmitted by external devices.

[0159] Please refer to Figure 12 This diagram illustrates a structural block diagram of a computer-readable storage medium provided in an embodiment of this application. The computer-readable medium 1100 stores program code that can be called by a processor to execute the methods described in the above method embodiments.

[0160] The computer-readable storage medium 1100 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 1100 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 1100 has storage space for program code 1110 that performs any of the method steps described above. This program code can be read from or written to one or more computer program products. The program code 1110 may, for example, be compressed in a suitable form.

[0161] In summary, the content processing method, apparatus, and electronic device provided in this application, when a first input content is obtained, if it is determined that the control operation corresponding to the first input content cannot be executed, then first content modification information corresponding to the first input content can be further obtained. This first content modification information includes historically input second input content and its modification status. If the second input content and the first input content satisfy a first similarity condition, a third input content can be obtained through this first content modification information. This third input content can then represent the user's control intent, thereby executing the control operation corresponding to the third input content. Thus, by using the above method, even when the user's current first input content cannot be executed, a third input content with the same control intent as the first input content can be obtained by utilizing historical inputs similar to the first input content and their corresponding modifications. This ensures that even when the first input content itself cannot be executed, the electronic device can still realize the user's control intent, improving the intelligence level of the electronic device in executing control operations.

[0162] In one implementation, the solution provided in this application can improve user experience satisfaction in voice assistant business scenarios. By autonomously learning the execution history of user commands (instructions input to electronic devices) and summarizing experience, it can assist users in improving the accuracy of subsequent command execution, shortening the execution path, and enhancing intelligence. Simultaneously, the solution provided in this application is designed to pass the fine-ranking results (output of the fine-ranking module) to downstream modules (e.g., dialogue decision modules) in the main link in a session context format. This can greatly alleviate the adaptation pressure on downstream modules to memory-enhanced knowledge, allowing them to directly treat it as context without requiring special model customization. Specifically, the solution provided in this application packages the fine-ranking results in the main link into a session QA format (question-answer format), where the query in the "QA" is the rewritten result retrieved from the memory bank, and the answer in the "QA" is a simulated "okay." This query and answer will be passed to downstream modules as context in the central control session, greatly alleviating the adaptation pressure on downstream modules to memory-enhanced knowledge content.

[0163] In one implementation, the solution provided in this application designs a module for coarse recall and memory conflict resolution, which works in conjunction with understanding, processing, and rewriting the historical memory information transmitted from the coarse recall retrieval module. First, the coarse recall retrieval module narrows down the scope of memory conflict resolution. Then, based on three dimensions—proximity, connectivity, and semantic post-processing—duplicate and conflicting memory content is merged and conflict-resolved, thereby generating new memories or deleting duplicate or conflicting memory information. This solution not only ensures the accuracy of information in the memory bank but also provides efficiency guarantees for the subsequent retrieval by the coarse recall module and fine ranking module in the main chain.

[0164] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical feature diagrams. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A content processing method, characterized in that, The method includes: If, upon obtaining the first input content, it is determined that the control operation corresponding to the first input content cannot be executed, then the first content modification information corresponding to the first input content is obtained. The first content modification information includes the historically input second input content and the modification status of the second input content. The second input content and the first input content satisfy a first similarity condition. The third input content is obtained by modifying the information based on the first content. Perform the control operation corresponding to the third input content.

2. The method according to claim 1, characterized in that, The method further includes: If there is input content in the modified historical input content that corresponds to the first input content, then it is determined that the control operation corresponding to the first input content cannot be executed; or If the operation target included in the first input content does not exist, then it is determined that the control operation corresponding to the first input content cannot be executed.

3. The method according to claim 1, characterized in that, The step of obtaining first content modification information corresponding to the first input content if it is determined that the control operation corresponding to the first input content cannot be executed includes: obtaining first content modification information corresponding to the first input content through the first functional module if it is determined that the control operation corresponding to the first input content cannot be executed. The process of obtaining the third input content by modifying the first content information includes: The third input content is obtained through the second functional module and the first content modification information, wherein the first content modification information is transmitted from the first functional module to the second functional module in the form of a question-and-answer pair.

4. The method according to claim 1, characterized in that, If it is determined that the control operation corresponding to the first input content cannot be executed, then the first content modification information corresponding to the first input content is obtained, including: If it is determined that the control operation corresponding to the first input content cannot be executed, and the control intent of the first input content is the first target intent, then the first content modification information corresponding to the first input content is obtained.

5. The method according to claim 4, characterized in that, The method further includes: If it is determined that the control operation corresponding to the first input content cannot be executed, and the control intent of the first input content is not the first target intent, then a prompt message is issued to the user to prompt the user to re-enter the content.

6. The method according to claim 1, characterized in that, The step of obtaining the first content modification information corresponding to the first input content includes: Obtain multiple second content modification information corresponding to the first input content, wherein each of the multiple second content modification information includes historical input content that satisfies a second similarity condition with the first input content; The first content modification information is determined from the plurality of second content modification information.

7. The method according to claim 1, characterized in that, The step of obtaining the first content modification information corresponding to the first input content includes: If the first content modification information exists, then obtain the first content modification information corresponding to the first input content; If the first content modification information does not exist, a prompt message is sent to the user to prompt the user to clarify the first input content; Based on the clarification content input by the user, third content modification information is generated and stored. The third content modification information includes the first input content and the modification status corresponding to the first input content. The modification status is obtained based on the clarification content.

8. The method according to claim 7, characterized in that, The method further includes: If, when the third content modification information is generated, a fourth content modification information including the first input content is stored, and the generation time of the fourth content modification information is earlier than the generation time of the third content modification information, the fourth content modification information is deleted.

9. A content processing apparatus, characterized in that, The device includes: The modification information acquisition unit is used to, when a first input content is obtained, if it is determined that the control operation corresponding to the first input content cannot be executed, acquire first content modification information corresponding to the first input content. The first content modification information includes the historically input second input content and the modification status of the second input content. The second input content and the first input content satisfy a first similarity condition. A content processing unit is used to obtain third input content by modifying information based on the first content; The control unit is used to perform control operations corresponding to the third input content.

10. An electronic device, characterized in that, Including the processor and memory; One or more programs are stored in the memory and configured to be executed by the processor to implement the method of any one of claims 1-8.

11. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium; the computer program product is used to implement the method as described in any one of claims 1-8.

Images