Information output method and device

Abstract

The invention discloses an information output method and device, and the method comprises the steps: obtaining a first feedback text according to query information of a user, the first feedback text at least comprises a first part and a second part, each part of the first feedback text corresponds to video and audio data, and the video and audio data comprises at least one of audio data, video data and image data; outputting first audio and video data corresponding to the first part, so that a user responds to the first audio and video data; and obtaining feature information of the user in response to the first audio and video data, and if the feature information of the user is matched with the first part, outputting second audio and video data corresponding to the second part of the first feedback text.

Description

Technical Field

[0001] The present application relates to the technical field of information output, and in particular to an information output method and device. Background Art

[0002] Currently, some electronic devices can obtain specific text according to user needs, and the user can browse the obtained text and respond according to the text. For example, the user can make corresponding body movements according to the instructions of the text.

[0003] The problem with this solution is that the text obtained according to the user's needs usually contains a large amount of information. Every time the user responds, it takes a certain amount of time to find the corresponding instructions from the text, which is less convenient. Summary of the invention

[0004] To this end, this application discloses the following technical solutions:

[0005] The first aspect of the present application provides an information output method, comprising:

[0006] Obtaining a first feedback text according to the query information of the user, the first feedback text at least including a first part and a second part, each part of the first feedback text corresponding to audio and video data, the audio and video data including at least one of audio data, video data and image data;

[0007] Outputting first audio-visual data corresponding to the first portion so that a user can respond to the first audio-visual data;

[0008] Acquire characteristic information of a user in response to the first audio-visual data, and if the characteristic information of the user matches the first part, output second audio-visual data corresponding to the second part of the first feedback text.

[0009] Optionally, obtaining the first feedback text according to the query information of the user includes:

[0010] A first feedback text is obtained according to the query information of the user and the interactive device information of the user, wherein the interactive device information indicates the device operated and controlled by the user.

[0011] Optionally, obtaining the first feedback text according to the query information of the user and the interactive device information of the user includes at least one of the following:

[0012] When the interactive device information indicates that the user is using the wearable device, obtaining a first feedback text related to an action of a target body part according to the query information of the user, wherein the target body part includes a body part where the user wears the wearable device;

[0013] When the interactive device information indicates that the user is using a handle device, a first feedback text related to a target application is obtained according to the user's query information, and the target application includes an application program operated by the user through the handle device.

[0014] Optionally, also include:

[0015] identifying a characteristic text included in the first feedback text;

[0016] identifying semantic relevance between feature texts contained in the first feedback text;

[0017] According to the semantic relevance between the feature texts, the first feedback text is divided into at least a first part and a second part, each part including at least one feature text with semantic relevance.

[0018] Optionally, also include:

[0019] When the user's characteristic information does not match the first part, outputting prompt information according to the first part to prompt the user to make a response to the first video data that matches the first part;

[0020] Alternatively, when the characteristic information of the user does not match the first part, a third part matching the characteristic information is determined from among the multiple parts included in the first feedback text;

[0021] Output the audio and video data corresponding to the third part.

[0022] Optionally, also include:

[0023] A third part matching the characteristic information is determined from among the multiple parts included in the first feedback text, and based on the position of the third part in the first feedback text, audio and video data corresponding to the text part after the position in the first feedback text is output.

[0024] Optionally, also include:

[0025] In the case that the multiple parts contained in the first feedback text do not match the characteristic information, obtaining a second feedback text;

[0026] Determining a fourth portion matching the feature information in the second feedback text;

[0027] Output the audio and video data corresponding to the fourth part.

[0028] Optionally, obtaining the second feedback text includes at least one of the following:

[0029] Obtaining a second feedback text according to the query information;

[0030] A second feedback text is obtained according to the characteristic information.

[0031] Optionally, each part of the first feedback text corresponds to audio and video data, including at least one of the following:

[0032] Generate corresponding audio and video data according to each part of the first feedback text;

[0033] According to each part of the first feedback text, the corresponding audio and video data is searched in the database.

[0034] A second aspect of the present application provides an information output device, comprising:

[0035] an obtaining unit, configured to obtain a first feedback text according to the query information of the user, wherein the first feedback text includes at least a first part and a second part, each part of the first feedback text corresponds to audio and video data, and the audio and video data includes at least one of audio data, video data and image data;

[0036] An interactive unit, configured to output first audio-visual data corresponding to the first part, so that a user can respond to the first audio-visual data;

[0037] The interaction unit is used to obtain characteristic information of a user in response to the first audio-visual data, and if the characteristic information of the user matches the first part, output the second audio-visual data corresponding to the second part of the first feedback text. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0039] Figure 1 is a flow chart of an information output method provided by an embodiment of the present application;

[0040] Figure 2 is a flow chart of another information output method provided in an embodiment of the present application;

[0041] Figure 3 is a flow chart of a method for dividing a first feedback text provided by an embodiment of the present application;

[0042] Figure 4 It is a schematic diagram of an information output device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0044] This application embodiment provides an information output method, see Figure 1 , is a flow chart of the method, and the method may include the following steps.

[0045] S101, obtaining a first feedback text according to the user's query information, the first feedback text at least including a first part and a second part, each part of the first feedback text corresponds to audio and video data, the audio and video data including at least one of audio data, video data and image data.

[0046] In step S101, the device for executing this embodiment can obtain query information in various ways. For example, the device can obtain query information in text form input by the user through a keyboard or touch screen, or can collect the user's voice and obtain the query information in the voice through voice recognition.

[0047] The query information may represent the type of text that the user wants to obtain. For example, the query information may include "perform boxing training", "practice singing a certain song", "how to play XX game", "please provide a yoga practice method. I am 25 years old and have been practicing yoga for 2 years. Please provide a content for practicing in the living room for 30 minutes", etc.

[0048] After obtaining the query information, the device can search the Internet based on the query information to obtain a first feedback text that matches the query information; alternatively, the device can input the query information into a data processing model (such as a large language model) with text generation capabilities, and use the data processing model to generate a first feedback text that matches the query information.

[0049] As an example, assuming that the query information "perform boxing training" is obtained, the first feedback text obtained based on the query information may be "First start training with the basic movements of boxing, and then start training, probing, probing, jabbing, side squatting..."

[0050] In some optional embodiments, when obtaining the first feedback text, the device may further combine the user attribute information and historical behavior information of the current user to obtain the first feedback text.

[0051] The device can obtain user attribute information and historical behavior information based on the user's input. For example, the user voice collected by the device can be: "Please provide a yoga practice method. I am 25 years old and have practiced yoga for 2 years. Please provide a content for practicing in the living room for 30 minutes." Through this voice, the device can recognize the query information "yoga practice method", user attribute information "25 years old this year" and "practice in the living room for 30 minutes", and historical behavior information "practice yoga for 2 years".

[0052] The device can also identify the identity of the user, and then obtain pre-stored user attribute information and historical behavior information based on the user's identity. For example, before obtaining the query information, the device can determine the identity information of the user who currently inputs the query information through password recognition, fingerprint recognition, or other identity recognition technologies, or when collecting the user's voice, the device can determine the identity information of the user who issued the voice through voiceprint recognition.

[0053] When user attribute information and historical behavior information are obtained, the device can search on the Internet in combination with the query information, user attribute information and historical behavior information to obtain a matching first feedback text, or it can input this information into a data processing model and use the model to generate a corresponding first feedback text.

[0054] In combination with the above example, when a voice message "Please provide a yoga practice method. I am 25 years old and have been practicing yoga for 2 years. Please provide a content for practicing in the living room for 30 minutes" is collected, the device can obtain the following first feedback text based on the information contained in the voice message:

[0055] "Okay, start with Mountain Pose. Stand with your feet hip-width apart. Lift your sternum while relaxing your lower ribs. Externally rotate your arms so your palms are facing forward and let your arms hang at your sides. Start in Mountain Pose. Inhale and straighten your arms. Exhale and fold forward, bending forward from the hips. Place your hands beside your feet or on the floor..."

[0056] The obtained first feedback text may be a text that has been divided into multiple parts, or may be a continuous text. In the latter case, the device may divide the first feedback text into multiple parts after obtaining the first feedback text.

[0057] The plurality of parts mentioned above may include two or more parts.

[0058] After obtaining multiple parts of the first feedback text, the device may obtain audio and video data corresponding to each part in at least one of the following ways.

[0059] First, the device can generate corresponding audio and video data according to each part of the first feedback text. In this acquisition method, the device can call a data processing model deployed locally and capable of generating audio and video data to process any part of the first feedback text, thereby generating audio and video data corresponding to the part, or can send any part of the first feedback text to the server, so that the server uses the data processing model capable of generating audio and video data to generate audio and video data corresponding to the part.

[0060] For example, a portion of the first feedback text may be "rotate your arm outward so that your palm faces forward", and the device may process this portion of text using a data processing model with text-to-speech capabilities to obtain the speech corresponding to this portion of text.

[0061] Secondly, the device can search for the corresponding audio and video data in the database according to each part of the first feedback text. In this acquisition method, the device or a server connected to the device in communication can pre-establish a database storing a large amount of audio and video data, wherein each audio and video data can correspond to one or more keywords. For any part of the first feedback text, the device can match the part with the keywords of multiple audio and video data in the database, and determine the audio and video data that successfully matches as the audio and video data corresponding to the part.

[0062] The first part in S101 may refer to any part included in the first feedback text, and the second part may refer to the latter part of the first part in the first feedback text.

[0063] S102, outputting first audio-visual data corresponding to the first portion, so that the user can respond to the first audio-visual data.

[0064] Each part of the first feedback text may include characteristic text that can instruct the user to make a corresponding response.

[0065] As some examples, the first feedback text may be text for instructing the user to make a specific body movement, and the characteristic text may be a verb contained in the first feedback text. As an example, one part of the first feedback text may be "stretch your arms", and the characteristic text may be "stretch".

[0066] The first feedback text may be text for instructing the user to control the target object. The target object may be an object displayed on the display by a specific application, such as a virtual character model, a virtual vehicle model, etc. displayed on the display. The feature text may be instructions for controlling the target object contained in the first feedback text. As an example, one part of the first feedback text may be "jump to the platform in the front left", and the feature text may be "jump".

[0067] The audio and video data corresponding to each part may include content corresponding to the characteristic text of the part. For example, the voice data generated based on the text of the first example may include the voice of "straighten", and the video data obtained based on the text of the second example may include several video frames representing the jumping of the target object.

[0068] Thus, the device can instruct the user to make a response corresponding to the feature text based on the first audio and video data by outputting the first audio and video data corresponding to the first part. For example, the audio and video data based on the first example instructs the user to stretch his arms, and the audio and video data based on the second example instructs the user to control the target object to jump to the left front.

[0069] S103, obtaining characteristic information of the user's response to the first audio and video data, and if the characteristic information of the user matches the first part, outputting second audio and video data corresponding to the second part of the first feedback text.

[0070] The characteristic information in response to the first audio and video data may include characteristic information collected from the start of outputting the first audio and video data to the completion of outputting the first audio and video data, or may include characteristic information of the user collected within a period of time after outputting the first audio and video data, or may include both.

[0071] The characteristic information may be in any form and may be information that can characterize the response made by the user in response to the first audio-visual data.

[0072] As some examples, the characteristic information in response to the first audio and video data may include at least one of the position data, velocity data and acceleration data of one or more body parts of the user collected within a corresponding time period; or, the characteristic information of the user may include a piece of video data obtained by filming the user within the corresponding time period; or, the characteristic information of the user may include a piece of audio data obtained by collecting the sound emitted by the user through a microphone within the corresponding time period; or, the characteristic information of the user may include a number of control instructions input by the user for controlling the target object within the corresponding time period.

[0073] For example, the characteristic information in response to the first audio and video data may include the acceleration data of the user's palm in multiple directions in space at the start time from the start of outputting the first audio and video data to the completion of outputting the first audio and video data, the acceleration data of the user's palm in multiple directions in space at the end time, and the acceleration data of the user's palm in multiple directions in space at a certain moment in the middle (for example, the moment of maximum acceleration).

[0074] Among them, the position data, speed data and acceleration data of specific body parts of the user can be obtained by wearable devices worn on the corresponding body parts. These wearable devices can have built-in sensors that can detect the above data, and these wearable devices can be communicated with the device executing this embodiment (for example, Bluetooth connection), so that the device can receive in real time at least one of the position data, speed data and acceleration data detected by these wearable devices through sensors.

[0075] After obtaining characteristic information in response to the first audio and video data, the device can determine whether the characteristic information matches the first part. If so, the device can continue to output the audio and video data corresponding to the latter part, that is, output the second audio and video data corresponding to the second part.

[0076] If, in response to the fact that the characteristic information of the first audio-visual data does not match the first part, the device may output corresponding information in at least one of the following output modes:

[0077] In the first way, when the characteristic information of the user does not match the first part, prompt information is output according to the first part to prompt the user to make a response to the first audio and video data that matches the first part.

[0078] The prompt information may be information for prompting the user to make a corresponding response according to the first video data. For example, the prompt information may be a voice message “Please perform the correct action according to the voice prompt just now”.

[0079] Alternatively, the prompt information may be consistent with the first audio and video data, that is, when the device determines that the characteristic information of the user's response to the first audio and video data does not match the first part, it may output the first audio and video data again until the characteristic information of the user's response to the first audio and video data matches the first part.

[0080] After outputting the prompt information, you can wait for a certain period of time so that the user can make a corresponding response, and obtain characteristic information representing the user response during this period of time. If the characteristic information obtained during this period of time matches the first part, you can continue to output the second audio and video data. If the characteristic information obtained during this period of time matches the first part, you can output the prompt information again, or you can continue to output the second audio and video data.

[0081] The second method is to determine the matching status of other parts of the output before the first part and the corresponding feature information. If the condition that N consecutive parts (including the first part) do not match the corresponding feature information is met, you can wait for a certain period of time and then continue to output the second audio and video data. During the waiting time, the user can be prompted that he has not made a response matching the corresponding part for N consecutive times; if the condition that N consecutive parts do not match the corresponding feature information is not met, you can continue to output the second audio and video data.

[0082] The third method is to continue to output the second video data after waiting for a certain period of time.

[0083] In some embodiments, each time the device outputs audio and video data corresponding to a part of the first feedback text, the device can regard the currently output audio and video data as the first audio and video data, and the current part as the first part, thereby outputting the audio and video data corresponding to each part of the first feedback text in sequence according to steps S102 and S103.

[0084] Exemplarily, assuming that the first feedback text includes 10 parts, which are recorded in sequence as part 1 to part 10, after the device obtains the first feedback text, it first outputs the audio and video data 1 corresponding to part 1, and then determines whether the characteristic information of the user's response to the audio and video data 1 matches part 1. If so, the device continues to output the audio and video data 2 corresponding to part 2. If the characteristic information of the user's response to the audio and video data 2 matches part 2, the device continues to output the audio and video data 3 corresponding to part 3, and so on, until the audio and video data 10 corresponding to part 10 is output.

[0085] As an example, see Figure 2 , assuming that the query information obtained in S101 is “boxing practice”, the first feedback text obtained based on the query information is: “First start training with the basic movements of boxing, and the training begins below, tentatively, tentatively, jab, side squat…”.

[0086] The first feedback text can be divided into four parts, namely "probe", "probe", "jab", and "side squat". The device can determine the first "probe" as the first part, output the corresponding first audio and video data, such as outputting the voice of "probe", and then determine whether the feature information of the user's response to the voice matches the "probe". If they match, it is determined that the user has made the probing action correctly.

[0087] At this time, the device can output the audio and video data corresponding to the second part of the first part, that is, output the next "probe" voice again, and obtain the feature information of the user's response to the voice output for the second time. If the feature information obtained for the second time also matches the "probe", it indicates that the user has correctly made the probing action again.

[0088] Then the device can output the voice of "jab". If the feature information of the voice of "jab" in response to the user matches "jab", it indicates that the user has correctly made the jab action. The device can then continue to output the voice of "side squat". If the feature information of the voice of "side squat" in response to the user matches "side squat", it indicates that the user has correctly made the side squat action. At this point, the boxing exercise based on the first feedback text ends.

[0089] The beneficial effects of this embodiment are:

[0090] After obtaining the feedback text, the audio and video data corresponding to a part of the feedback text is output first. When it is determined that the user's characteristic information matches the part, the audio and video data corresponding to the next part is output. In this way, the user can make corresponding responses in sequence according to the part output each time, without having to spend time finding the next step instructions from the text after each response, thereby improving the convenience of users responding based on the feedback text.

[0091] In some embodiments, the Figure 3 The illustrated manner divides the first feedback text into multiple parts.

[0092] S301: Identify a feature text included in a first feedback text.

[0093] S302: Identify semantic relevance between feature texts contained in the first feedback text.

[0094] S303: Divide the first feedback text into at least a first part and a second part according to the semantic relevance between the feature texts, each part including at least one feature text having semantic relevance.

[0095] In steps S301 and S302, the first feedback text obtained may be processed using a data processing model with semantic understanding capability, thereby identifying feature texts contained in the first feedback text and semantic relevance between the feature texts.

[0096] The semantic relevance between different feature texts may be different. For example, the semantic relevance between some feature texts may be: the response of one feature text occurs before the response of another feature text, that is, there is a sequential relationship between the responses corresponding to the two feature texts; the semantic relevance between some feature texts may be: the responses of two (or more) feature texts occur at the same time.

[0097] Taking the feature text being a verb as an example, assuming that the first feedback text includes "Start with Mountain Pose, inhale, straighten your arms, exhale, fold forward, and bend forward from the hips." Among them, "inhale", "straighten", "exhale", and "bend forward" all belong to feature texts. By identifying the semantic relevance of the feature texts, it can be determined that the semantic relevance of "inhale" and "straighten" is that the two actions of inhalation and straightening occur at the same time, the semantic relevance of "exhale" and "bend forward" is that the two actions of exhalation and bending forward occur at the same time, and the semantic relevance of "inhale" and "exhale" is that the inhale action occurs first and then the exhale action.

[0098] In step S303, the feature texts whose corresponding semantic association is that the responses occur simultaneously may be divided into the same part, and the feature texts whose corresponding semantic association is that the responses occur successively may be divided into different parts.

[0099] For the non-feature text other than the feature text in the first feedback text, the non-feature text may be divided into the part to which the feature text belongs according to the semantic relevance between the non-feature text and the feature text.

[0100] Combined with the above example, by analyzing the semantics of the text, we can find that "start mountain pose" and "inhale" are associated, and "fold forward" and "exhale" are associated. Therefore, "start mountain pose" and "inhale" can be divided into the same part, and "fold forward" and "exhale" can be divided into the same part. Therefore, the first feedback text of the above example can be divided into the first part "start mountain pose, inhale, stretch your arms" and the second part "exhale, fold forward, bend forward from the hips".

[0101] In some embodiments, for any part of the first feedback text, whether the part matches the characteristic information of the user may be determined as follows:

[0102] Obtain reference feature information of the feature text contained in the part;

[0103] Determine the similarity between the reference feature information of the feature text contained in the portion and the feature information of the audio and video data of the user's response to the portion;

[0104] If the similarity is greater than a preset matching threshold, it can be determined that the part matches the user's feature information;

[0105] If the similarity is less than or equal to the matching threshold, it can be determined that the part does not match the user's feature information.

[0106] The device may obtain a feature information library in advance, and the feature information library may include multiple feature texts and reference feature information corresponding to each feature text.

[0107] Based on this, after determining the first part, the device can find out the reference feature information corresponding to the feature text contained in the first part.

[0108] After the device outputs the first audio and video data and obtains the characteristic information of the user's response to the first audio and video data, the similarity between the characteristic information of the user and the reference characteristic information of the characteristic text contained in the first part can be determined, and whether the characteristic information of the user's response to the first audio and video data matches the first part is determined based on whether the similarity is greater than a matching threshold. If the characteristic information of the user's response to the first audio and video data matches the first part, it means that the user has made a correct response based on following the instructions of the first part. If the characteristic information of the user's response to the first audio and video data does not match the first part, it means that the user has not made a correct response based on following the instructions of the first part.

[0109] The similarity between the user's feature information and the reference feature information of the feature text contained in the first part can be determined by various methods for calculating similarity in the relevant technical field, which will not be described in detail.

[0110] The reference feature information corresponding to the feature text and the user's feature information may be in the same form.

[0111] For example, if the feature information obtained in response to the first audio and video data includes at least one of the position data, velocity data, and acceleration data of one or more body parts of the user over a period of time, then the reference feature information corresponding to the feature text may also include at least one of the position data, velocity data, and acceleration data of the corresponding body part of any person over a period of time.

[0112] The reference feature information corresponding to the feature text can be obtained as follows:

[0113] After an individual makes a correct response based on a feature text, the feature information acquired during the response is determined as reference feature information corresponding to the feature text.

[0114] As an example, assuming that the feature text is a verb, the feature information includes acceleration data of the wrist in the X, Y, and Z directions in the three-dimensional coordinate system when the response starts, when the response stops, and when the acceleration is maximum during the response. In this case, if it is necessary to obtain the reference feature information corresponding to the verb "jab", after the individual makes the correct jab action, the acceleration data of the wrist at the above three moments collected during the jab action can be determined as the reference feature information corresponding to "jab".

[0115] The reference feature information corresponding to the "jab" obtained in the above manner can be expressed in the following Table 1.

[0116] Table 1

[0117] Jab X-axis 0；60；-18 Y-axis 0；40；70 Z-axis 0；30；-55

[0118] Each row in Table 1 represents the acceleration data of a person's wrist at three moments on the corresponding coordinate axis during a correct jab action, in units of meters per second squared. The three moments here are the moment when the jab action starts, the moment when the acceleration on the corresponding coordinate axis is the largest during the jab action, and the moment when the jab action ends.

[0119] The reference feature information corresponding to the feature text can also be obtained as follows:

[0120] A plurality of similar feature texts are obtained, as well as a plurality of feature information when a plurality of users respond based on the feature texts, and then the feature information is clustered to determine the correspondence between the feature information and the feature texts, and finally the feature information corresponding to the feature texts is fused to obtain the reference feature information corresponding to the feature texts.

[0121] Similar feature texts may include feature texts having a portion of the same content.

[0122] As an example, multiple similar feature texts may include "open bottle cap", "open refrigerator" and "open school bag". For these three feature texts, feature information (such as the aforementioned acceleration data) when multiple users make bottle cap opening actions, feature information when multiple users make refrigerator opening actions, and feature information when multiple users make school bag opening actions may be obtained;

[0123] Then, the above-mentioned multiple feature information are clustered to determine which feature information corresponds to “opening the bottle cap”, which feature information corresponds to “opening the refrigerator”, and which feature information corresponds to “opening the schoolbag”. Finally, the multiple feature information corresponding to “opening the bottle cap” are fused to obtain the reference feature information corresponding to “opening the bottle cap”; the multiple feature information corresponding to “opening the refrigerator” are fused to obtain the reference feature information corresponding to “opening the refrigerator”; the multiple feature information corresponding to “opening the schoolbag” are fused to obtain the reference feature information corresponding to “opening the schoolbag”.

[0124] The reference feature information corresponding to the feature text can also be obtained as follows:

[0125] During the period when an individual continuously responds based on multiple continuous feature texts, multiple feature information within this period of time is obtained, known reference feature information is identified in the multiple feature information, and then according to the order of the multiple feature texts and the known reference feature information, reference feature information corresponding to the target feature text is determined in the multiple feature information. The target feature text refers to the feature text for which the reference feature information has not yet been determined in the multiple feature texts.

[0126] As an example, assume that three consecutive feature texts are "probe", "jab", and "side squat", where the reference feature information corresponding to "probe" and "jab" has been obtained through other means, and the target feature text is "side squat". After any person performs the three actions of probing, jabbing, and side squatting in sequence according to the three feature texts, the device can determine, based on the known reference feature information corresponding to "probe" and "jab", that the feature information obtained during the first two actions corresponds to "probe" and "jab" respectively, and then, based on the sorting of the above feature texts, determine that the feature information obtained during the last action is the reference feature information corresponding to "side squat".

[0127] Optionally, when obtaining the first feedback text according to the user's query information, the user's interactive device information can be further obtained, so that the first feedback text is obtained according to the user's query information and the user's interactive device information, and the interactive device information indicates the device controlled by the user. In this way, the matching degree between the obtained first feedback text and the user's usage requirements can be improved, which helps to obtain a first feedback text that better meets the user's requirements.

[0128] The interactive device information can be obtained based on the connection relationship between the device executing the method of this embodiment and other electronic devices. For example, when the device detects that it is currently connected to a wearable device (such as a wristband or smart watch), it can obtain interactive device information indicating that the user is using the wearable device; when the device detects that a handle device is currently connected, it can obtain interactive device information indicating that the user is using the handle device.

[0129] In some embodiments, when the interactive device information indicates that the user is using a wearable device, a first feedback text related to the action of a target body part can be obtained according to the user's query information, and the target body part includes the body part where the user wears the wearable device.

[0130] Exemplarily, if the wearable device used by the user is worn on the user's wrist or arm, a first feedback text matching the query information and related to the wrist movement or arm movement is obtained, for example, a first feedback text related to a boxing movement is obtained; if the wearable device used by the user is worn on the user's head, a first feedback text matching the query information and related to the head movement is obtained; if the wearable device used by the user is worn on the user's legs, a first feedback text matching the query information and related to the leg movement is obtained.

[0131] The body part where the wearable device is located can be determined by the type of the wearable device. For example, if the wearable device is a bracelet or smart watch type device, it is determined that the user wears it on the wrist. If the wearable device is a helmet type device or a glasses type device, it is determined that the user wears it on the head.

[0132] In some embodiments, when the interactive device information indicates that the user is using a handle device, a first feedback text related to a target application may be obtained according to the user's query information, and the target application includes an application program operated by the user through the handle device;

[0133] In this embodiment, one or more target applications that can be operated by the handle device can be determined from multiple applications that have been installed locally on the device, or one or more target applications that the user has operated with the handle device can be determined based on the user's historical operation data, and then a first feedback text that matches the query information and is related to the target application can be obtained.

[0134] For example, if the query information obtained is "boxing action practice", the interactive device information indicates that the user is using a handle device, and a boxing game that can be operated with the handle device is locally installed on the device, then the first feedback text obtained by the device may be an operation guide type text of the boxing game.

[0135] In some embodiments, when the interaction device information indicates that the user is using a headphone device and a microphone device, a first feedback text related to the audio may be obtained according to the query information.

[0136] Exemplarily, when the interactive device information indicates that the user is using a headphone device and a microphone device, a target music whose name matches the query information can be found from the local music on the device or from the music that the user has played on the device, and then a first feedback text related to the target music can be obtained, such as text for indicating how to sing the target music.

[0137] In some embodiments, when the interactive device information indicates that the user is using a handwriting device, such as a digital tablet, a stylus, or the like, a first feedback text related to drawing may be obtained according to the query information.

[0138] For example, when the interactive device information indicates that the user is using a digital tablet device, a painting matching the information can be searched and queried, and then a first feedback text related to the painting can be obtained, such as a first feedback text for guiding the user to copy the painting.

[0139] In some optional embodiments, when the device determines that the characteristic information of the user's response to the first audio and video data does not match the first part, the device may also perform at least one of the following processing methods.

[0140] Processing method 1: when the characteristic information of the user does not match the first part, a third part matching the characteristic information is determined from among the multiple parts included in the first feedback text;

[0141] Output the audio and video data corresponding to the third part.

[0142] Processing method 2 is to determine a third part that matches the feature information from among the multiple parts contained in the first feedback text, and based on the position of the third part in the first feedback text, output the audio and video data corresponding to the text part located after the position in the first feedback text.

[0143] Processing method three, when multiple parts and feature information contained in the first feedback text do not match, obtain the second feedback text;

[0144] Determining a fourth portion matching the feature information in the second feedback text;

[0145] Output the audio and video data corresponding to the fourth part.

[0146] In processing method one, the device can obtain reference feature information corresponding to each part of the first feedback text except the first part, and compare the reference feature information of each part with the feature information corresponding to the first audio and video data. If it is determined that the reference feature information of a certain part matches the feature information corresponding to the first audio and video data, then the part is determined to be the third part, and then the audio and video data corresponding to the third part is output.

[0147] In combination with the above example, assume that the first feedback text obtained is: "First, start training with the basic movements of boxing. Now let's start training: probe, probe, jab, side squat..." When the device outputs the voice corresponding to the first part "probe", the user makes a side squat movement in response to the voice. After the device obtains the feature information of the user's response to the first audio and video data, it can be determined that the feature information does not match "probe", and by comparing the feature information with the reference feature information of "side squat", it is found that the feature information of the user's response to the first audio and video data matches "side squat", so it can be determined that "side squat" is the third part, and the voice of "side squat" is output.

[0148] When executing processing method two, the device can start from the third part and output the audio and video data corresponding to each part after the third part in the first feedback text in sequence, and each time the audio and video data is output, it can be determined in the manner of the aforementioned embodiment whether the feature information of the user's response to the audio and video data matches the corresponding part to determine whether to continue to output the audio and video data corresponding to the next part.

[0149] When executing processing method 2, if it is determined that the third part in the first feedback text is before the first part, in this case, starting from the first part, the audio and video data corresponding to the part before the first part can be output one by one in reverse order until the audio and video data corresponding to the part after the third part is output.

[0150] Exemplarily, assuming that the first feedback text includes 10 parts, which are recorded in sequence as part 1 to part 10, and assuming that when the device outputs the audio and video data corresponding to part 7, it is determined that the feature information of the user's response to the audio and video data matches part 2, then part 7 can be regarded as the first part, and part 4 can be regarded as the third part. The device can output the audio and video data corresponding to part 6, the audio and video data corresponding to part 5, the audio and video data corresponding to part 4, and the audio and video data corresponding to part 3 in sequence.

[0151] When executing the second processing method, the device may also select several adjacent or non-adjacent parts from the multiple parts located after the third part in the first feedback text, and output the audio and video data corresponding to the selected parts.

[0152] In this embodiment, after determining the third part, if the first feedback text has been output multiple times, it is possible to combine previous output records to determine which parts of the audio and video data were output after the audio and video data corresponding to the third part each time the first feedback text was output in the past, and then select those parts that were frequently output after the third part in the previous output records, and output the audio and video data corresponding to these parts.

[0153] Exemplarily, the first feedback text also includes the fifth part, the sixth part and the seventh part after the third part. The first feedback text has been output 10 times in total, 5 of which are the times when the audio and video data corresponding to the sixth part is outputted after the audio and video data of the third part is outputted, and 3 of which are the times when the audio and video data corresponding to the seventh part is outputted after the audio and video data of the third part is outputted. In this case, the sixth part and the seventh part of the first feedback text can be selected, and the audio and video data corresponding to the sixth part and the audio and video data corresponding to the seventh part can be outputted in sequence.

[0154] By selectively outputting a specific portion of the audio and video data in the above manner, the output order of the multiple portions of the audio and video data in the first feedback text can be more consistent with the user's behavior habits.

[0155] In some optional embodiments, if it is determined that there are multiple parts in the first feedback text that match the characteristic information of the user's response to the first audio and video data, the part of the multiple parts that is closest to the first part can be determined as the third part.

[0156] In some embodiments, when the user responds that the characteristic information of the first audio and video data does not match the first part, the device may also execute processing method one and processing method two in sequence, that is, the device may first output the third audio and video data corresponding to the third part, and then output the audio and video data corresponding to each part of the first feedback text after the third part in sequence.

[0157] When processing method 3 is executed, the device may compare the reference feature information corresponding to each part contained in the first feedback text with the feature information of the user's response to the first audio and video data. If it is found after the comparison that each part of the first feedback text does not match the feature information of the user's response to the first audio and video data, a second feedback text may be obtained. The second feedback text may be a text that satisfies at least the following conditions:

[0158] The second feedback text includes at least one portion that matches the characteristic information of the user's response to the first audio-visual data.

[0159] After obtaining the second feedback text, the device can determine the fourth part in the second feedback text that matches the characteristic information of the user's response to the first audio and video data in the same manner as determining the third part in the first feedback text, and then output the audio and video data corresponding to the fourth part.

[0160] The method of dividing the second feedback text into multiple parts and obtaining the audio and video data corresponding to the multiple parts in the second feedback text can refer to the above-mentioned embodiment and will not be described in detail.

[0161] In some embodiments, the second feedback text may be obtained based on the query information. In this embodiment, when the query information is obtained, multiple feedback texts may be obtained based on the query information, and then a first feedback text may be determined from the multiple feedback texts in combination with information such as user attribute information, historical behavior information, and interactive device information, and the remaining feedback texts may be determined as the second feedback text;

[0162] Based on this, when it is found that all parts contained in the first feedback text do not match the characteristic information of the user's response to the first audio-visual data, a second feedback text can be selected from the remaining feedback texts.

[0163] In some embodiments, the second feedback text may be obtained based on the feature information. In this embodiment, the feature information of the user in response to the first audio-visual data may be compared with each reference feature information in the aforementioned feature information library, thereby determining the target reference feature information that matches the user's feature information, reading the feature text corresponding to the target reference feature information, and obtaining the second feedback text based on the query information and the feature text corresponding to the target reference feature information.

[0164] As some examples, the query information and the feature text corresponding to the target reference feature information can be used as search keywords to search the Internet to obtain the second feedback text. Alternatively, the query information and the feature text corresponding to the target reference feature information can be input into a data processing model with text generation capability, and the model generates the second feedback text based on the feature text corresponding to the query information and the target reference feature information.

[0165] The present application also provides an information output device, see Figure 4 , is a schematic diagram of the device, and the device may include the following units.

[0166] The obtaining unit 401 is used to obtain a first feedback text according to the query information of the user, the first feedback text at least including a first part and a second part, each part of the first feedback text corresponds to audio and video data, and the audio and video data includes at least one of audio data, video data and image data;

[0167] The interactive unit 402 is used to output the first audio-visual data corresponding to the first part, so that the user can respond to the first audio-visual data;

[0168] The interaction unit 402 is used to obtain characteristic information of the user in response to the first audio and video data, and output the second audio and video data corresponding to the second part of the first feedback text if the characteristic information of the user matches the first part.

[0169] Optionally, when the obtaining unit 401 obtains the first feedback text according to the query information of the user, it can be used to:

[0170] A first feedback text is obtained according to the query information of the user and the interactive device information of the user, where the interactive device information indicates the device operated and controlled by the user.

[0171] Optionally, when the obtaining unit 401 obtains the first feedback text according to the user's query information and the user's interactive device information, it can be used for at least one of the following:

[0172] When the interactive device information indicates that the user is using the wearable device, obtaining a first feedback text related to an action of a target body part according to the query information of the user, where the target body part includes a body part where the user wears the wearable device;

[0173] When the interactive device information indicates that the user is using a handle device, a first feedback text related to a target application is obtained according to the user's query information, and the target application includes an application program operated by the user through the handle device.

[0174] Optionally, the obtaining unit 401 is further used for:

[0175] Identifying feature text included in the first feedback text;

[0176] identifying semantic relevance between feature texts contained in the first feedback text;

[0177] According to the semantic relevance between the feature texts, the first feedback text is divided into at least a first part and a second part, each part including at least one feature text with semantic relevance.

[0178] Optionally, the interaction unit 402 may also be used to:

[0179] When the user's characteristic information does not match the first part, outputting prompt information according to the first part to prompt the user to make a response matching the first part with respect to the first audio-visual data;

[0180] Alternatively, when the characteristic information of the user does not match the first part, a third part matching the characteristic information is determined from among the multiple parts included in the first feedback text;

[0181] Output the audio and video data corresponding to the third part.

[0182] Optionally, the interaction unit 402 may also be used to:

[0183] A third part matching the characteristic information is determined from among the multiple parts included in the first feedback text, and based on the position of the third part in the first feedback text, audio and video data corresponding to the text part located after the position in the first feedback text is output.

[0184] Optionally, the interaction unit 402 may also be used to:

[0185] In the case that the multiple parts contained in the first feedback text and the characteristic information do not match, obtaining the second feedback text through the obtaining unit 401;

[0186] Determining a fourth portion matching the feature information in the second feedback text;

[0187] Output the audio and video data corresponding to the fourth part.

[0188] Optionally, when the obtaining unit 401 obtains the second feedback text, it can be used for at least one of the following:

[0189] Obtaining a second feedback text according to the query information;

[0190] A second feedback text is obtained according to the characteristic information.

[0191] Optionally, each part of the first feedback text corresponds to audio and video data, including at least one of the following:

[0192] Generate corresponding audio and video data according to each part of the first feedback text;

[0193] According to each part of the first feedback text, the corresponding audio and video data is searched in the database.

[0194] The working principle of the information output device provided in this embodiment can refer to the relevant steps of the information output method provided in any embodiment of the present application, and will not be repeated here.

[0195] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0196] For the convenience of description, the above system or device is described by dividing it into various modules or units according to its functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0197] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0198] Finally, it should be noted that, in this article, relational terms such as first, second, third and fourth are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0199] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. An information output method, comprising: Obtaining a first feedback text according to the query information of the user, the first feedback text at least including a first part and a second part, each part of the first feedback text corresponding to audio and video data, the audio and video data including at least one of audio data, video data and image data; Outputting first audio-visual data corresponding to the first portion so that a user can respond to the first audio-visual data; Acquire characteristic information of a user in response to the first audio-visual data, and if the characteristic information of the user matches the first part, output second audio-visual data corresponding to the second part of the first feedback text.

2. According to the method of claim 1, obtaining the first feedback text according to the user's query information comprises: A first feedback text is obtained according to the query information of the user and the interactive device information of the user, wherein the interactive device information indicates the device operated and controlled by the user.

3. The method according to claim 2, wherein obtaining the first feedback text according to the user's query information and the user's interactive device information comprises at least one of the following: When the interactive device information indicates that the user is using the wearable device, obtaining a first feedback text related to an action of a target body part according to the query information of the user, wherein the target body part includes a body part where the user wears the wearable device; When the interactive device information indicates that the user is using a handle device, a first feedback text related to a target application is obtained according to the user's query information, and the target application includes an application program operated by the user through the handle device.

4. The method according to claim 1, further comprising: identifying a characteristic text included in the first feedback text; identifying semantic relevance between feature texts contained in the first feedback text; According to the semantic relevance between the feature texts, the first feedback text is divided into at least a first part and a second part, each part including at least one feature text with semantic relevance.

5. The method according to claim 1, further comprising: When the user's characteristic information does not match the first part, outputting prompt information according to the first part to prompt the user to make a response to the first video data that matches the first part; Alternatively, when the characteristic information of the user does not match the first part, a third part matching the characteristic information is determined from among the multiple parts included in the first feedback text; Output the audio and video data corresponding to the third part.

6. The method according to claim 5, further comprising: A third part matching the characteristic information is determined from among the multiple parts included in the first feedback text, and based on the position of the third part in the first feedback text, audio and video data corresponding to the text part after the position in the first feedback text is output.

7. The method according to claim 1, further comprising: In the case that the multiple parts contained in the first feedback text do not match the characteristic information, obtaining a second feedback text; Determining a fourth portion matching the feature information in the second feedback text; Output the audio and video data corresponding to the fourth part.

8. The method according to claim 7, wherein obtaining the second feedback text comprises at least one of the following: Obtaining a second feedback text according to the query information; A second feedback text is obtained according to the characteristic information.

9. The method according to claim 1, wherein each part of the first feedback text corresponds to audio and video data, including at least one of the following: Generate corresponding audio and video data according to each part of the first feedback text; According to each part of the first feedback text, the corresponding audio and video data is searched in the database.

10. An information output device, comprising: an obtaining unit, configured to obtain a first feedback text according to the query information of the user, wherein the first feedback text includes at least a first part and a second part, each part of the first feedback text corresponds to audio and video data, and the audio and video data includes at least one of audio data, video data and image data; An interactive unit, configured to output first audio-visual data corresponding to the first part, so that a user can respond to the first audio-visual data; The interaction unit is used to obtain characteristic information of a user in response to the first audio-visual data, and output second audio-visual data corresponding to a second part of the first feedback text if the characteristic information of the user matches the first part.

Images