User interaction method and device for telling stories by looking at pictures

Abstract

The present invention discloses a user interaction method and device for storytelling through picture viewing, relating to the field of computer technology. A specific implementation of the method includes: receiving a user's voice request for story playback; parsing story keywords in the voice request, searching a collection of picture stories for a target picture story corresponding to the voice request; and playing the target picture story. This implementation eliminates reliance on mobile phones and other terminal devices, as well as other supporting equipment, allowing for more flexible expansion of content sources and enhanced ways to meet educational needs.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a user interaction method and device for telling stories by viewing pictures. Background Art

[0002] Picture storytelling helps improve children's language skills. Existing picture storytelling products often rely on mobile devices, raising concerns about entertainment abuse. Most content resources rely on well-produced audio and video resources, making them difficult to expand. Some point-to-read products require accompanying books and pens, making inspiration and retrieval more manual. Summary of the Invention

[0003] In view of this, an embodiment of the present invention provides a user interaction method and device for telling stories by looking at pictures, which can avoid dependence on terminal devices such as mobile phones and other supporting devices, more flexibly expand content sources, and enhance the way of meeting educational needs.

[0004] To achieve one or more of the above objectives, according to one aspect of an embodiment of the present invention, a user interaction method for storytelling through pictures is provided, comprising:

[0005] Receive user voice requests to trigger story-on-demand;

[0006] Parsing story keywords in the voice request and searching for a target picture story corresponding to the voice request from a picture story collection;

[0007] Play the target picture story.

[0008] Optionally, before searching for a target voice story corresponding to the voice request from a picture story collection, the method further includes:

[0009] Keywords are extracted from each picture story in the picture story collection to generate story tags for the picture stories; and a tag index for the picture story collection is generated based on the story tags of each picture story in the picture story collection.

[0010] Optionally, before generating the tag index of the picture story collection, the method further includes: filtering out story tags in a preset tag library from among the story tags of the picture stories.

[0011] Optionally, the picture story collection includes a system picture story collection and / or a user picture story collection; and before searching the picture story collection for a target picture story corresponding to the voice request based on the story keyword, the method further includes:

[0012] Extracting text information from each preset picture material, generating a picture story corresponding to the preset picture material according to the text information, and obtaining the system picture story set; and / or,

[0013] Receive various personal materials uploaded by users, generate user picture stories according to each personal material, and obtain the user picture story collection.

[0014] Optionally, before playing the target picture story, the method further includes:

[0015] The voice style of the voice request is analyzed, and the audio stream of the target picture story is adjusted according to the voice style.

[0016] Optionally, searching for a target picture story corresponding to the voice request from a picture story collection according to the story keyword includes:

[0017] The picture stories corresponding to the story keywords are searched from the picture story collection, and the picture stories with higher story scores are selected from the picture stories as the target picture stories.

[0018] Optionally, before screening the picture stories with higher story scores from the picture stories, the method further includes: setting a story score for each picture story in the picture story set;

[0019] After selecting a target picture story with a higher story score from the various picture stories, the method further includes: adjusting the story score of the target picture story according to the business scenario of the voice request and / or according to user feedback data during or after the playback.

[0020] According to another aspect of an embodiment of the present invention, a user interaction device for telling stories by looking at pictures is provided, comprising:

[0021] Request receiving module, receiving user's voice request to trigger story on demand;

[0022] A story query module parses story keywords in the voice request and searches for a target picture story corresponding to the voice request from a picture story collection;

[0023] The story playing module plays the target picture story.

[0024] Optionally, the user interaction device of an embodiment of the present invention also includes a story creation module, which is used to extract keywords from each picture story in the picture story collection and generate story tags for the picture story before querying the target voice story corresponding to the voice request from the picture story collection; and generate a tag index for the picture story collection based on the story tags of each picture story in the picture story collection.

[0025] Optionally, the story creation module is further configured to: before generating the tag index of the picture story collection, filter out story tags in the various story tags of the picture stories that are present in a preset tag library.

[0026] Optionally, the picture story collection includes a system picture story collection and / or a user picture story collection; the story creation module is further configured to: before searching the picture story collection for a target picture story corresponding to the voice request based on the story keywords,

[0027] Extracting text information from each preset picture material, generating a picture story corresponding to the preset picture material according to the text information, and obtaining the system picture story set; and / or,

[0028] Receive various personal materials uploaded by users, generate user picture stories according to each personal material, and obtain the user picture story collection.

[0029] Optionally, the playback module is further configured to: before playing the target picture story, analyze the voice style of the voice request, and adjust the audio stream of the target picture story according to the voice style.

[0030] Optionally, searching for a target picture story corresponding to the voice request from a picture story collection according to the story keyword includes:

[0031] The picture stories corresponding to the story keywords are searched from the picture story collection, and the picture stories with higher story scores are selected from the picture stories as the target picture stories.

[0032] Optionally, the story query module is further configured to: set a story score for each picture story in the picture story set before screening picture stories with higher story scores from the picture stories; and

[0033] After selecting a target picture story with a higher story score from the various picture stories, the story score of the target picture story is adjusted according to the business scenario of the voice request and / or according to user feedback data during or after the playback.

[0034] According to another aspect of an embodiment of the present invention, a user interactive electronic device for telling stories by looking at pictures is provided, comprising:

[0035] one or more processors;

[0036] a storage device for storing one or more programs,

[0037] When the one or more programs are executed by the one or more processors, the one or more processors implement the user interaction method for telling stories by looking at pictures provided by the present invention.

[0038] According to another aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by a processor, the user interaction method for telling stories by viewing pictures provided by the present invention is implemented.

[0039] One embodiment of the above invention has the following advantages or beneficial effects: by querying and playing the corresponding picture story according to voice request, it is possible to avoid dependence on terminal devices such as mobile phones and other supporting devices; by generating a picture story collection based on preset picture materials or personal materials uploaded by users, it is possible to expand the content source more flexibly; by extracting keywords from each picture story in the picture story collection to generate a label index for the picture story collection, it is convenient for retrieval inspiration.

[0040] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings are provided for a better understanding of the present invention and are not intended to limit the present invention.

[0042] Figure 1 This is an exemplary system architecture diagram suitable for use in a user interaction method for telling stories by looking at pictures or a user interaction device for telling stories by looking at pictures according to an embodiment of the present invention;

[0043] Figure 2 is a schematic diagram of the main process of the user interaction method for telling stories by looking at pictures according to an embodiment of the present invention;

[0044] Figure 3 This is a schematic diagram of a user interaction solution based on a speaker with a screen in an optional embodiment of the present invention;

[0045] Figure 4 This is a schematic diagram of generating a picture story collection based on preset picture materials in an optional embodiment of the present invention;

[0046] Figure 5 This is a schematic diagram of a process for a user to request a story on demand in an optional embodiment of the present invention;

[0047] Figure 6 This is a schematic diagram of a story screening strategy in an optional embodiment of the present invention;

[0048] Figure 7 is a schematic diagram of generating a picture story collection based on a user's personal material in an optional embodiment of the present invention;

[0049] Figure 8is a schematic diagram of main modules of a user interaction device for telling stories by looking at pictures according to an embodiment of the present invention;

[0050] Figure 9 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The following description of exemplary embodiments of the present invention is made in conjunction with the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding. These details should be considered as merely exemplary. Therefore, it should be appreciated by those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0052] Figure 1 An exemplary system architecture diagram of a user interaction method for telling stories by looking at pictures or a user interaction device for telling stories by looking at pictures suitable for application in an embodiment of the present invention is shown. Figure 1 As shown, an exemplary system architecture of a user interaction method for telling stories by looking at pictures or a user interaction device for telling stories by looking at pictures according to an embodiment of the present invention includes:

[0053] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. Network 104 is a medium for providing communication links between terminal devices 101, 102, 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0054] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0055] The terminal devices 101, 102, and 103 may be various electronic devices with display screens and supporting web browsing, including but not limited to screen-equipped speaker devices, smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0056] Server 105 may be a server that provides various services, such as a backend management server (for example only) that supports shopping websites browsed by users using terminal devices 101, 102, and 103. The backend management server may analyze and process received data such as on-demand requests, and feed back the processing results (for example, picture stories - for example only) to terminal devices 101, 102, and 103.

[0057] It should be noted that the user interaction method for telling stories by looking at pictures provided in the embodiment of the present invention is generally executed by the server 105 , and accordingly, the user interaction device for telling stories by looking at pictures is generally set in the server 105 .

[0058] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0059] Figure 2 FIG. 1 is a schematic diagram of the main process of the user interaction method for telling stories by looking at pictures according to an embodiment of the present invention. Figure 2 As shown, the user interaction method for telling stories through pictures includes:

[0060] Step S201: receiving a user's voice request to trigger story-on-demand;

[0061] Step S202: parsing story keywords in the voice request, and searching for a target picture story corresponding to the voice request from a picture story collection;

[0062] Step S203: Play the target picture story.

[0063] The process of playing a story is the process of playing the story description (i.e., the story text). In actual application, the picture corresponding to the target picture story can be displayed on the screen. Users can rely on speakers or smart voice devices such as speakers with screens, projection devices with voice recognition functions, etc. to play stories on demand. For example, the user relies on speakers with screens to play stories on demand, and sends a voice request through the speakers with screens to "play a story about deer". After the corresponding picture story is found, the description of the picture story is played by voice, and the corresponding picture and subtitles are displayed on the screen, such as Figure 3 shown.

[0064] By querying and playing the corresponding picture story based on the voice request, the embodiment of the present invention can avoid dependence on terminal devices such as mobile phones and other supporting devices. It should be emphasized that although the method of the embodiment of the present invention can avoid dependence on terminal devices such as mobile phones, it can also be implemented on terminal devices such as mobile phones.

[0065] Optionally, before searching for a target voice story corresponding to the voice request from the picture story collection, the method further includes: extracting keywords from each picture story in the picture story collection to generate a story tag for the picture story; and generating a tag index for the picture story collection based on the story tag for each picture story in the picture story collection. By extracting keywords from each picture story in the picture story collection to generate a tag index for the picture story collection, an inverted index is established based on the tag dimension to facilitate retrieval.

[0066] For example, Figure 4 As shown, character recognition technology is used to identify the text data on each picture in the picture file storage system (such as Figure 4 Today's weather in the image), and uses image description generation technology to generate a text data describing a story for the image (such as Figure 4 Based on the first two text data, the label of the picture story is obtained by keyword extraction (such as Figure 4 “deer” and “grassland” in the poem). Figure 4 The OCR in it is the abbreviation of Optical Character Recognition, which refers to optical character recognition.

[0067] Optionally, before generating the tag index for the picture story collection, the process further includes filtering out story tags from the individual story tags in the picture story that are present in a preset tag library. This filtering process can filter out data that the application is not interested in. For example, for children's picture stories, tags that are only applicable to adults can be filtered out.

[0068] In some optional embodiments, the picture story collection includes a system picture story collection. Before searching the picture story collection for a target picture story corresponding to the voice request based on the story keywords, the method further includes: extracting text information on each preset picture material, generating a picture story corresponding to the preset picture material based on the text information, and obtaining the system picture story collection. Figure 4 In the illustrated embodiment, picture stories generated based on images in a picture file storage system are stored in a picture story storage system. The process of generating picture stories based on preset picture materials is essentially the process of generating story descriptions. The implementation method can be selectively set based on actual circumstances, such as using a recurrent neural network to generate image text. By generating a collection of picture stories based on preset picture materials, content sources can be expanded more flexibly.

[0069] In other optional embodiments, the picture story collection includes a user picture story collection. According to the story keywords, before searching the picture story collection for the target picture story corresponding to the voice request, it also includes: receiving various personal materials uploaded by the user, generating a user picture story according to each personal material, and obtaining the user picture story collection. Users can provide personal materials such as family albums, photos of commercial picture books, etc. Users can independently circle the content source, thereby getting rid of the rhythm restrictions of the producer. The process of generating picture stories based on personal materials is essentially the process of generating story descriptions, and its implementation method can be selectively set according to actual conditions, such as using a recurrent neural network to generate image text.

[0070] like Figure 7 As shown, user photos or in-app photo albums are used as personal materials to automatically trigger the generation of content (such as user-uploaded voice content), original text on the image (such as user-edited text), and tags. Then, based on the user dimension, the corresponding picture story is saved to the user-level story storage system. When the user requests a story on demand, the corresponding picture story is retrieved from the user-level story storage system or stories generated by the content provider and the system.

[0071] By generating picture story collections based on personal materials uploaded by users, we can more flexibly expand the content source and meet the personalized needs of users.

[0072] Optionally, before playing the target picture story, the system further includes analyzing the voice style of the voice request and adjusting the audio stream of the target picture story based on the voice style. The method for switching voice styles can be selectively set based on actual circumstances, such as by converting prosodic features or spectral features. Personalized speech synthesis can provide parental-style voices, bringing listeners closer.

[0073] Optionally, based on the story keyword, searching for a target picture story corresponding to the voice request from a picture story collection includes searching for individual picture stories corresponding to the story keyword from the picture story collection, and selecting a picture story with a higher story score from the individual picture stories as the target picture story. The story score of each picture story can be pre-set. Selecting picture stories for playback based on the story score can make the selected picture stories more suitable for user needs and improve the accuracy of the selection results.

[0074] Optionally, before selecting a picture story with a higher story score from the picture stories, the method further includes: setting a story score for each picture story in the picture story set. After selecting a target picture story with a higher story score from the picture stories, the method further includes: adjusting the story score of the target picture story based on the business scenario of the voice request and / or based on user feedback data during or after playback.

[0075] For example, during playback of a target picture story, if user feedback is received indicating that the user dislikes the target picture story, the story score of the target picture story is lowered; if user feedback is received indicating that the user likes the target picture story, the story score of the target picture story is raised. For example, when determining the score based on the business scenario, in an animal scenario, picture stories containing more animals receive a higher story score.

[0076] By adjusting the story ratings of picture stories based on user feedback data and / or business scenarios, the screened picture stories can be made more in line with user needs, further improving the accuracy of the screening results.

[0077] The following combination Figure 5 and 6 The user interaction method of the embodiment of the present invention is exemplified. Figure 5 As shown, the audio stream sent by the user is received, and the user's voice request "Tell a story about deer" is recognized through speech recognition technology. Through semantic understanding and analysis, it is determined that the task of the voice request is "telling a story" and the keyword is "deer". Then, picture stories are retrieved from the content provider's story collection and the self-generated story collection as candidate picture stories. The story skill PK strategy is used to select candidate picture stories with high story scores as target picture stories. The target picture stories are displayed directly or after personalized speech synthesis processing. See Story Skill PK Strategy for details. Figure 6 For ease of presentation, we denote the candidate image story from the content provider's story collection as A, and the candidate image story from the self-generated story collection as B. We determine whether the score of story B is greater than that of story A. If so, story B is chosen as the target image story. Otherwise, we determine whether story A exists. If so, story A is chosen as the target image story; otherwise, story B is chosen as the target image story. During the cold start phase, the PK strategy can simply be set as a fallback for content provider stories, while the operations backend manually assigns scores to self-generated material to select content. Later, the scoring system for self-generated material can be adjusted based on user feedback.

[0078] This embodiment can provide an interactive solution for telling stories by looking at pictures on smart voice devices such as screen speakers. In addition to standard content sources, it can more flexibly expand the search for content sources and enhance the way to meet educational needs.

[0079] According to another aspect of the embodiments of the present invention, a device for implementing the above method is provided.

[0080] like Figure 8 As shown, the user interaction device 800 for telling stories by looking at pictures includes:

[0081] Request receiving module 801 receives a user's voice request to trigger story on demand;

[0082] A story query module 802 parses story keywords in the voice request and searches for a target picture story corresponding to the voice request from a picture story collection;

[0083] The story playing module 803 plays the target picture story.

[0084] Optionally, the user interaction device of an embodiment of the present invention also includes a story creation module, which is used to extract keywords from each picture story in the picture story collection and generate story tags for the picture story before querying the target voice story corresponding to the voice request from the picture story collection; and generate a tag index for the picture story collection based on the story tags of each picture story in the picture story collection.

[0085] Optionally, the story creation module is further configured to: before generating the tag index of the picture story collection, filter out story tags in the various story tags of the picture stories that are present in a preset tag library.

[0086] Optionally, the picture story collection includes a system picture story collection and / or a user picture story collection; the story creation module is further configured to: before searching the picture story collection for a target picture story corresponding to the voice request based on the story keywords,

[0087] Extracting text information from each preset picture material, generating a picture story corresponding to the preset picture material according to the text information, and obtaining the system picture story set; and / or,

[0088] Receive various personal materials uploaded by users, generate user picture stories according to each personal material, and obtain the user picture story collection.

[0089] Optionally, the playback module is further configured to: before playing the target picture story, analyze the voice style of the voice request, and adjust the audio stream of the target picture story according to the voice style.

[0090] Optionally, searching for a target picture story corresponding to the voice request from a picture story collection according to the story keyword includes:

[0091] The picture stories corresponding to the story keywords are searched from the picture story collection, and the picture stories with higher story scores are selected from the picture stories as the target picture stories.

[0092] Optionally, the story query module is further configured to: set a story score for each picture story in the picture story set before screening picture stories with higher story scores from the picture stories; and

[0093] After selecting a target picture story with a higher story score from the various picture stories, the story score of the target picture story is adjusted according to the business scenario of the voice request and / or according to user feedback data during or after the playback.

[0094] Figure 9 FIG. 1 is a schematic diagram of a computer system suitable for implementing a terminal device according to an embodiment of the present invention. Figure 9 As shown, the computer system 900 of the terminal device according to the embodiment of the present invention includes:

[0095] The system 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage unit 908 into a random access memory (RAM) 903. Various programs and data required for the operation of the system 900 are also stored in the RAM 903. The CPU 901, ROM 902, and RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0096] The following components are connected to the I / O interface 905: an input section 906 including a keyboard, a mouse, and the like; an output section 907 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 908 including a hard disk and the like; and a communication section 909 including a network interface card such as a LAN card or a modem. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 910 as needed, so that computer programs read therefrom can be installed into the storage section 908 as needed.

[0097] In particular, according to the embodiments disclosed in the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 909, and / or installed from a removable medium 911. When the computer program is executed by the central processing unit (CPU) 901, the above-mentioned functions defined in the system of the present invention are performed.

[0098] It should be noted that the computer-readable medium described in the present invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media can include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical fiber cable, RF, or any suitable combination thereof.

[0099] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0100] The modules involved in the embodiments of the present invention may be implemented in software or hardware. The modules described may also be provided in a processor. For example, they may be described as follows: a processor including a request receiving module, a story query module, and a story playing module. The names of these modules do not, in some cases, limit the modules themselves. For example, the story query module may also be described as a "module for playing the target picture story."

[0101] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiments, or may exist independently and not incorporated into the device. The computer-readable medium carries one or more programs. When executed by the device, the device performs the following operations: receiving a user's voice request to trigger story-on-demand; parsing story keywords in the voice request, searching a collection of picture stories for a target picture story corresponding to the voice request; and playing the target picture story.

[0102] According to the technical solution of the embodiments of the present invention, by querying and playing the corresponding picture story according to the voice request, it is possible to avoid dependence on terminal devices such as mobile phones and other supporting devices; by generating a picture story collection based on preset picture materials or personal materials uploaded by users, it is possible to expand the content source more flexibly; by extracting keywords from each picture story in the picture story collection to generate a label index for the picture story collection, it is convenient for retrieval inspiration.

[0103] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A user interaction method for telling stories by looking at pictures, characterized in that: include: Receive user voice requests to trigger story-on-demand; Parsing story keywords in the voice request and searching for a target picture story corresponding to the voice request from a picture story collection; Play the target picture story and display the corresponding picture and subtitles on the screen; Among them, the target picture story corresponding to the voice request is queried from the picture story collection according to the label index of the picture story collection; the label index of the picture story collection is generated in the following manner: character recognition technology is used to identify the text data on each picture in the picture story collection, image description generation technology is used to generate text data of a story description for the picture, keywords corresponding to the picture story are extracted on the basis of the first two types of text data, story labels corresponding to the picture story are generated, and the label index of the picture story collection is generated according to the story label of each picture story in the picture story collection.

2. The user interaction method according to claim 1, wherein: Before generating the tag index of the picture story collection, the method further includes: filtering out story tags in the preset tag library from among the story tags of the picture stories.

3. The method according to claim 1, wherein The picture story collection includes a system picture story collection and / or a user picture story collection; Before searching for a target picture story corresponding to the voice request from a picture story collection based on the story keyword, the method further includes: Extracting text information on each preset picture material, generating a picture story corresponding to the preset picture material according to the text information, and obtaining the system picture story set; and / or, Receive various personal materials uploaded by users, generate user picture stories according to each personal material, and obtain the user picture story collection.

4. The method according to claim 1, wherein Before playing the target picture story, it also includes: The voice style of the voice request is analyzed, and the audio stream of the target picture story is adjusted according to the voice style.

5. The method according to claim 1, wherein Querying a target picture story corresponding to the voice request from a picture story collection according to the story keyword includes: The picture stories corresponding to the story keywords are searched from the picture story collection, and the picture stories with higher story scores are selected from the picture stories as the target picture stories.

6. The method according to claim 5, wherein Before screening the picture stories with higher story scores from the picture stories, the method further includes: setting a story score for each picture story in the picture story set; After selecting a target picture story with a higher story score from the various picture stories, the method further includes: adjusting the story score of the target picture story according to the business scenario of the voice request and / or according to user feedback data during or after the playback.

7. A user interaction device for telling stories by looking at pictures, characterized in that: include: Request receiving module, receiving user's voice request to trigger story on demand; A story query module parses story keywords in the voice request and searches for a target picture story corresponding to the voice request from a picture story collection; A story playing module plays the target picture story and displays the corresponding picture and subtitles on the screen; Among them, the target picture story corresponding to the voice request is queried from the picture story collection according to the label index of the picture story collection; the device also includes a story creation module, which is used to generate the label index of the picture story collection in the following way: using character recognition technology to identify the text data on each picture in the picture story collection, using image description generation technology to generate a piece of story description text data for the picture, extracting the keywords corresponding to the picture story on the basis of the first two types of text data, generating the story label of the corresponding picture story, and generating the label index of the picture story collection according to the story label of each picture story in the picture story collection.

8. A user interactive electronic device for telling stories by looking at pictures, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer-readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images