Information display method and device, electronic equipment, storage medium and program product

Abstract

An information display method, apparatus, electronic device, storage medium, and program product are disclosed. The information display method includes: first, playing audio data on a first page, the audio data corresponding to a story and used to represent first content within the story; then, displaying a fragment of a first visual object on the first page, the first visual object corresponding to the aforementioned story, the content of which is related to the first content. This method assists users in more efficiently obtaining the desired visual objects, improving the efficiency of user retrieval.

Description

Technical Field

[0001] This document relates to the field of computer technology, and in particular to an information display method, device, electronic device, storage medium, and program product. Background Technology

[0002] Currently, users can browse stories or listen to audiobooks on the reader. For popular stories, corresponding visuals such as comics or short dramas can also be provided for users to browse. With the diversification of the types and content of visuals, how to help users obtain the visuals they need more efficiently is one of the problems that needs to be solved. Summary of the Invention

[0003] An information display method, apparatus, electronic device, storage medium, and program product are provided to assist users in obtaining the visual objects they need more efficiently and improve the efficiency of users in obtaining the visual objects they need.

[0004] In a first aspect, an information display method is provided, comprising: playing audio data on a first page; the audio data corresponding to a story; the audio data being used to represent first content in the story; displaying a fragment of a first visual object on the first page; the first visual object corresponding to the story; and the content of the fragment being related to the first content.

[0005] Secondly, an information display device is provided, comprising: a first playback unit for playing audio data on a first page; the audio data corresponding to a story; the audio data representing first content in the story; and a first display unit for displaying a fragment of a first visual object on the first page; the first visual object corresponding to the story; and the content of the fragment being related to the first content.

[0006] Thirdly, an electronic device is provided, comprising: a processor; and a memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the method described in the first aspect.

[0007] Fourthly, a computer-readable storage medium is provided for storing computer-executable instructions that, when executed by a processor, implement the method described in the first aspect.

[0008] Fifthly, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the method described in the first aspect.

[0009] First, audio data is played on the first page. This audio data corresponds to the story and represents the first content within the story. Then, a segment from the first visual object is displayed on the first page. This first visual object also corresponds to the story, and the content of the segment is related to the first content. Therefore, while the user is listening to the audio data of the story, a segment from the first visual object corresponding to the story can be displayed to the user, thereby helping the user obtain the required visual object more efficiently and improving the efficiency of the user's acquisition of the desired visual object. Attached Figure Description

[0010] To more clearly illustrate the technical solution, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings in the following description only record some implementation situations. For those skilled in the art, other drawings can be obtained from these drawings without creative effort. Figure 1 This is a schematic diagram illustrating an application scenario of an information display method provided under certain conditions. Figure 2 A flowchart illustrating a method for displaying information in a given scenario; Figure 3a This is a schematic diagram illustrating the presentation of a first visual object in one scenario. Figure 3b This is an illustration of a first visual object presented in another scenario; Figure 3c This is an illustration of a first visual object presented in yet another scenario; Figure 3d This is an illustration of a first visual object presented in yet another scenario; Figure 4 A schematic diagram of the structure of an information display device provided in one scenario; Figure 5 This is a schematic diagram of the structure of an electronic device provided in one scenario. Detailed Implementation

[0011] To enable those skilled in the art to better understand the technical solutions in one or more of the scenarios described below, the technical solutions in one or more of the scenarios described below will be clearly and completely described below in conjunction with one or more accompanying drawings. Obviously, the scenarios described are only some scenarios, not all scenarios. Based on one or more of the scenarios described below, all other implementations obtained by those skilled in the art without inventive effort should fall within the protection scope of this document.

[0012] It is understood that before using any of the technical solutions described below, relevant parties should be informed of the type, scope of use, and usage scenarios of the information involved and their authorization obtained in an appropriate manner in accordance with relevant laws and regulations.

[0013] For example, upon receiving a user's proactive request, a prompt message can be sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware such as electronic devices, applications, servers, or storage media that perform the technical solution, based on the prompt message.

[0014] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0015] It is understood that the above notification and user authorization process is merely illustrative and does not limit the implementation of the various scenarios described below. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the various scenarios described below.

[0016] An information display method, apparatus, electronic device, storage medium, and program product are disclosed, which can assist users in obtaining desired visual objects more efficiently and improve the efficiency of users in obtaining the desired visual objects. The information display method can be applied to and implemented by a terminal device, which includes, but is not limited to, laptops, tablets, desktop computers, set-top boxes, mobile devices (e.g., mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices), smartphones, smart speakers, smartwatches, smart TVs, in-vehicle terminals, and various other types of terminal devices.

[0017] Figure 1 This is a schematic diagram illustrating an application scenario of an information display method provided in a particular situation, such as... Figure 1 As shown, the scenario includes terminal device 101. Figure 1 In this context, terminal device 101 can be any user's terminal device. Figure 1 In the middle, the terminal device 101 runs an application. Figure 1 This example illustrates how an application can function as an e-book reader, with stories presented as e-books. Figure 1As shown, users can read e-books or listen to audio data of e-books in the e-book reader. While the user is listening to the audio data of the e-book, the terminal device 101 can also display fragments of the first visual object associated with the e-book to the user, thereby helping the user to obtain the required visual object more efficiently and improving the efficiency of the user in obtaining the required visual object.

[0018] Figure 2 This is a flowchart illustrating a method for displaying information in a given situation, such as... Figure 2 As shown, the method includes: Step S202: Play audio data on the first page; the audio data corresponds to the story; the audio data is used to represent the first content in the story; Step S204: Display a fragment from the first visual object on the first page; the first visual object corresponds to the story; the content of the fragment is related to the first content.

[0019] First, audio data is played on the first page. This audio data corresponds to the story and represents the first content within the story. Then, a segment from the first visual object is displayed on the first page. This first visual object also corresponds to the story, and the content of the segment is related to the first content. Therefore, while the user is listening to the audio data of the story, a segment from the first visual object corresponding to the story can be displayed to the user, thereby helping the user obtain the required visual object more efficiently and improving the efficiency of the user's acquisition of the desired visual object.

[0020] In one scenario, audio data from an ebook can be played on the first page. This audio data represents the first content of the ebook, such as a chapter. Additionally, a segment from a short drama corresponding to the ebook can be displayed on the first page. This segment can represent the content of the aforementioned chapter or the content of a related chapter in the plot of the aforementioned chapter.

[0021] In one scenario, a radio drama can be played on the first page. This radio drama corresponds to a story and represents the first element of that story, such as a chapter. Additionally, a segment from a comic book adaptation of the story can be displayed on the first page. This segment could represent the content of the aforementioned chapter or content from a related chapter's plot. The radio drama may or may not have corresponding text content, such as an ebook.

[0022] In step S202 above, the terminal device plays audio data on the first page. This audio data corresponds to a story and is used to represent the first content in the story. The first page can be any audio playback page; there is no limitation here.

[0023] In one scenario, audio data is obtained by converting the story's text to audio. In this case, TTS (Text-to-Speech) technology can be used to convert the story's text to audio, resulting in audio data that is then played on the first page. In this scenario, the audio data can be from an ebook. This approach allows the user to listen to the audio data obtained from the text-to-audio conversion of the story's text while simultaneously displaying segments from the first visual object, improving the efficiency of the user obtaining the desired video content.

[0024] In another scenario, audio data is obtained by having a live narrator read the story's text aloud. This could involve inviting a live narrator to read the story's text aloud, generating audio data, which is then played on the first page. In this case, the audio data could be from an ebook, an audiobook, or a radio drama. This approach allows users to listen to the audio data obtained from the live reading of the story's text while simultaneously displaying segments from the first visual element, improving the efficiency of users accessing the desired video content.

[0025] The aforementioned story may or may not have a corresponding story text, such as an ebook. Regarding the source of the story text, in one scenario, the story text includes an ebook describing the story. Specifically, the terminal device runs an ebook reader, allowing users to listen to the audio data of the ebook. The terminal device can play the ebook's audio data in either intelligent text-to-speech mode or human text-to-speech mode. Intelligent text-to-speech includes, but is not limited to, using TTS technology to process the ebook's text data to obtain corresponding audio data and play it. Human text-to-speech can involve inviting a live narrator to read the ebook's text data aloud, obtaining corresponding audio data and playing it. Therefore, in this scenario, while the user is listening to the ebook's audio data, segments from the first visual object can be simultaneously displayed to the user, improving the efficiency of the user obtaining the desired video object.

[0026] Of course, the audio data played in step S202 can also be audio data of any story, and the story can also be of any type; there are no restrictions here.

[0027] Audio data can be used to represent all or part of a story. Here, the content represented by the audio data is referred to as the first content, which is used to represent the first part of the story. The first content may include plot and / or characters.

[0028] In step S204 above, a fragment from a first visual object is displayed on the first page. The first visual object corresponds to the story mentioned above, and the content of the fragment is related to the first content.

[0029] The aforementioned story corresponds to a first visual object. A first visual object includes, but is not limited to, video data or image sequences generated based on the story content of the aforementioned story. Furthermore, the first visual object has a subject, which can be a character within the first visual object. The subject type in the first visual object includes at least one of real humans and virtual objects. Specifically, "real humans" refers to the subject in the first visual object being played by a real person, while "virtual objects" refers to the subject in the first visual object being played by a cartoon animal or other virtual object. For example, a first visual object includes, but is not limited to, one of the following generated based on the story content of the aforementioned story: a short drama, a comic series, a TV series, a movie, a comic, a character image sequence, or a plot image sequence. Specifically, a short drama refers to video data with a relatively large number of episodes, such as more than 20 episodes, and each episode being relatively short, such as less than 5 minutes. The subject in a short drama includes the main characters in the short drama, and the subject type in a short drama is entirely real humans or a combination of real humans and virtual objects. A comic series refers to video data with a relatively large number of episodes, such as more than 20 episodes, and each episode being relatively short, such as less than 5 minutes. The subject in a comic series includes the main characters in the comic series, and the subject type in a comic series is entirely virtual objects. A comic book series can include multiple animated video frames. Comic books can be generated through animation production methods, with or without the use of AI (Artificial Intelligence) technology.

[0030] The character image sequence comprises multiple character images generated for one or more characters in the aforementioned story. These images form the character image sequence, and the subject of each image includes the character itself. The subject type of the character image can include at least one of real humans and virtual objects. The plot image sequence comprises multiple plot images generated for one or more plots in the aforementioned story. These images form the plot image sequence, and the subject of each plot image includes the characters themselves. The subject type of the plot image can include at least one of real humans and virtual objects. The character image sequence or plot image sequence can be generated using AI-generated images. Both the character image sequence and the plot image sequence can be generated by readers of the aforementioned story or by the official operators of the aforementioned story.

[0031] In one example, the character image sequence may include posters of the various characters in the short drama. The plot image sequence may include a collection of plot images from the short drama. In another example, the character images in the character image sequence may include images generated by users or operations personnel for the characters in the aforementioned story using AI-generated images, and the plot images in the plot image sequence may include images generated by users or operations personnel for the plot of the aforementioned story using AI-generated images.

[0032] Among them, short dramas, comics, TV series, movies, comics, character image sequences, and plot image sequences are all generated based on the story content of the aforementioned stories and are used to represent the story content. For example, the story content of the aforementioned stories can be adapted into at least one of short dramas, comics, TV series, and movies. Alternatively, comics corresponding to the aforementioned stories can be generated based on the story content of the aforementioned stories, with or without AI image generation technology. Alternatively, character image sequences and plot image sequences corresponding to the aforementioned stories can be generated based on the story content of the aforementioned stories, with or without AI image generation technology.

[0033] In one scenario, the subject type of the first visual object matches the character type in the aforementioned story. The subject type includes at least one of real humans and virtual objects, and the character type includes at least one of real human characters and virtual characters.

[0034] The characters in the above story have corresponding character types, including at least one of real-human characters and virtual characters. Real-human characters refer to characters in the story who are real people, while virtual characters refer to characters in the story who are virtual. As mentioned earlier, the first visual object has a subject, which can be a character in the first visual object. The subject type in the first visual object includes at least one of real-human and virtual object types.

[0035] Based on this, the subject type of the first visual object matches the character type in the aforementioned story. For example, if the subject type of the first visual object includes real humans, and all characters in the first visual object are played by real people, then the character type in the aforementioned story includes real human characters, and all characters in the aforementioned story are real people. Similarly, if the subject type of the first visual object includes virtual objects, and all characters in the first visual object are played by virtual objects, then the character type in the aforementioned story includes virtual characters, and all characters in the aforementioned story are virtual characters. Furthermore, if the subject type of the first visual object includes both real humans and virtual objects, and some characters in the first visual object are played by real people while others are played by virtual objects, then the character type in the aforementioned story includes both real human characters and virtual characters, and some characters in the aforementioned story are real people while others are virtual characters.

[0036] Since the type of the main subject of the first visual object matches the type of the character in the story, it allows users to better understand the story content through the first visual object, thereby improving the accuracy of the visual object presented to the user.

[0037] In one scenario, the above story corresponds to multiple second visual objects; the first visual object is determined within each of the second visual objects in the following way: Select at least one object from each of the second visual objects as the third visual object; the subject type of the third visual object matches the character type of the story above; the subject type includes at least one of real human and virtual object types, and the character type includes at least one of real human character and virtual character; Select a first visual object from among the various third visual objects; the genre of the first visual object includes either an image sequence or video data.

[0038] In this scenario, the aforementioned story can be associated with multiple alternative visual objects, also known as second visual objects. Each second visual object is generated based on the story content of the aforementioned story. Each second visual object includes, but is not limited to, video data or image sequences generated based on the story content of the aforementioned story, and the subject in each second visual object includes at least one of real people and virtual objects. Each second visual object includes, but is not limited to, various types of short dramas, comics, TV series, movies, comics, character image sequences, and plot image sequences generated based on the story content of the aforementioned story. For example, second visual objects include short dramas, comics, character image sequences, and plot image sequences.

[0039] Based on this, the subject type of each second visual object can be determined. The subject type includes at least one of real humans and virtual objects. The inclusion of at least one of real humans and virtual objects in the subject type refers to whether the pre-defined main characters in the second visual object are all played by real people, all by cartoon animals or other virtual objects, or partly by real people and partly by cartoon animals or other virtual objects. If all the pre-defined main characters in the second visual object are played by real people, then the subject type includes real humans; if all the pre-defined main characters in the second visual object are played by cartoon animals or other virtual objects, then the subject type includes virtual objects; if the pre-defined main characters in the second visual object are partly played by real people and partly by cartoon animals or other virtual objects, then the subject type includes both real humans and virtual objects. In a typical example, the subject type of a short drama includes real humans or includes both real humans and virtual objects, while the subject type of a comic book includes virtual objects.

[0040] Next, determine the character types for the aforementioned story. Character types include at least one of real-human characters and virtual characters. Real-human characters refer to characters in the story that can be played by real people, while virtual characters refer to characters in the story that need to be played by virtual objects. If all the main characters in the aforementioned story are real-human characters that can be played by real people, then the character type of the aforementioned story includes real-human characters. If all the main characters in the ebook are virtual characters that need to be played by virtual objects, then the character type of the aforementioned story includes virtual characters. If some of the main characters in the aforementioned story are real-human characters that can be played by real people and others are virtual characters that need to be played by virtual objects, then the character type of the aforementioned story includes both real-human characters and virtual characters. Of course, for the same story, there may be multiple character types. For example, the characters in story A can be played by both real people and virtual objects, so the character type of story A can be either real-human characters or virtual characters. In this case, a character type can be determined for the story, and the first visual object can be determined based on that character type.

[0041] Furthermore, at least one object is selected from each of the second visual objects as the third visual object, and the subject type of the third visual object matches the character type of the story. For example, if the character type of the story includes real human characters, it means that the main characters in the story can be played by real people, so a third visual object whose subject type includes real humans is selected. Similarly, if the character type of the story includes virtual characters, it means that the main characters in the story can be played by virtual objects, so a third visual object whose subject type includes virtual objects is selected. Furthermore, if the character type of the story includes both real human characters and virtual characters, it means that the main characters in the story can be partially played by real people and partially by virtual objects, so a third visual object whose subject type includes both real human characters and virtual objects is selected.

[0042] Finally, a first visual object is selected from among the various third visual objects; the genre of the first visual object includes either image sequences or video data. The priority of image sequences can be set to be lower than that of video data. Video data types are preferentially selected as the first visual object.

[0043] In one example, assuming that the character types in the above story include real human characters, it means that the main characters in the above story can be played by real people. Then, among the various second visual objects, the second visual objects whose main type includes real human characters are selected, resulting in a short drama and a character image sequence. The character image sequence includes posters of each character in the short drama. Then, based on the fact that the image sequence has a lower priority than the video data, the above short drama is determined as the first visual object.

[0044] In one example, assuming the story above includes both real-life and virtual characters, and the main characters in the ebook can be partially played by real people and partially by virtual objects, then among the various second visual objects, those whose main types include both real-life and virtual objects are selected. This results in two short dramas, A and B. Then, based on the fact that short drama A is more popular than short drama B, short drama A is determined as the first visual object. Alternatively, a priority order can be preset among the third visual objects of the same genre to determine the first visual object. For example, if the priority of a comic book is determined to be higher than that of a short drama, then the comic book is determined as the first visual object.

[0045] Based on the above description, at least one third visual object can be selected from the various second visual objects corresponding to the story. The main type of the third visual object matches the character type of the story. Then, a first visual object is selected from the various third visual objects, which achieves the accurate determination of the first visual object. This allows the first visual object to accurately match the story content of the story, enabling the first visual object to better represent the content of the story and improve the user's experience of browsing related visual objects while listening to audio.

[0046] In step S204 above, the process of determining the first visual object can be performed by the terminal device or by the backend server corresponding to the application playing audio data on the terminal device. After determining the first visual object, it is also necessary to determine the segment to be displayed in the first visual object and display that segment on the first page. Similarly, the process of determining the segment can be performed by the terminal device or by the backend server corresponding to the application playing audio data on the terminal device.

[0047] The segments displayed in the first visual object are used to aid in understanding the story content. Specifically, the segments are obtained from the first visual object based on the first content represented by the audio data, and these segments can help users better understand the story content while listening to the audio data.

[0048] Based on this, in one scenario, displaying a fragment of a first visual object on a first page includes: displaying a first fragment of the first visual object on the first page; the content of the first fragment includes second content in the story; the second content includes key content related to the first content.

[0049] In this scenario, key content related to the first content in the aforementioned story can be identified. This key content can be highlight content following or preceding the first content, content with a plot connection to the first content, or content with a character connection to the first content. This key content related to the first content in the aforementioned story is designated as the second content. Within the first visual object, a segment representing this second content is identified, and this segment is displayed as the first segment. Highlight content refers to the most engaging content in the aforementioned story. Based on the number of times a user reads or listens to each segment of the aforementioned story, segments in the story with a listening frequency exceeding a threshold or a reading frequency exceeding a threshold are identified as highlight segments, and the content represented by these highlight segments is identified as the highlight content.

[0050] Another example involves determining whether the first content represented by the audio data currently playing on the first page is highlight content. If the first content represented by the audio data is highlight content, then the key content related to the first content is identified as the first content itself. Therefore, the second content includes the first content. In the first visual object, a segment representing the content of the currently playing audio data is identified and displayed as the first segment. If the first content represented by the audio data is not highlight content, then in the story content of the aforementioned story, highlight content following the first content, highlight content preceding the first content, content related to the plot of the first content, or content related to a character of the first content is identified. This identified content is then used as the second content. In the first visual object, a segment representing this second content is identified and displayed as the first segment.

[0051] In one example, a user is listening to audio data from an e-book. When the user hears the phrase "the female protagonist goes to university," and determines that the highlight content related to this content is the scene of the female protagonist's graduation ceremony, then the first clip shown represents the scene of the female protagonist's graduation ceremony.

[0052] By using the above methods, key content related to the first content can be presented to the user while the user is listening to the audio of the first content, thereby attracting the user to continue listening to the audio data or directly viewing the first visual object.

[0053] In another scenario, displaying a fragment of a first visual object on a first page includes: displaying a second fragment of the first visual object on the first page; the content of the second fragment matching the first content.

[0054] In this scenario, the first content represented by the currently playing audio data is determined, and the segment in the first visual object used to represent that first content is designated as the second segment for display. For example, if the audio data representing Chapter 5 of an e-book is determined to be playing, then the segment in the first visual object representing Chapter 5 of the e-book is designated as the second segment for display. This designated segment can include episodes or image sequences. In this scenario, it can be guaranteed that the visuals of the displayed second segment match the content of the currently playing audio data. For example, if the user hears the phrase "the female lead goes to university," the visuals of the displayed second segment will also represent the content "the female lead goes to university," thus achieving the effect of the user listening to audio while viewing visuals.

[0055] In another scenario, displaying a fragment of the first visual object on the first page includes: displaying a third fragment of the first visual object on the first page; the content of the third fragment includes the third content of the story; the third content includes subsequent content of the first content.

[0056] In this scenario, the subsequent content represented by the audio data currently playing in the aforementioned story is determined. The segment in the first visual object used to represent this subsequent content is designated as the third segment and displayed. For example, if the audio data is determined to be playing Chapter 5 of an ebook, then the segment in the first visual object used to represent the plot after Chapter 5 of the ebook is designated as the third segment and displayed. This designated segment can include episodes or image sequences. In this scenario, it can be ensured that the displayed third segment represents the plot following the currently playing audio data. For example, if the user hears the phrase "the female lead goes to university," the displayed third segment represents the plot after "the female lead goes to university," thus achieving a "spoiler" effect on the user and increasing the likelihood that the user will continue listening to the audiobook or directly jump to the first visual object.

[0057] In another scenario, displaying a fragment of a first visual object on the first page includes: displaying a fourth fragment of the first visual object on the first page; the content of the fourth fragment includes a fourth element of the story; the fourth element includes at least one highlight element of the first element.

[0058] Highlight content refers to the most engaging parts of the story. Based on the number of times a user reads or listens to different segments of the story, segments that have been listened to or read more than a certain number of times are identified as highlight segments, and the content represented by these highlight segments is defined as highlight content.

[0059] One or more segments within the first visual object are identified to represent highlight content. These segments are then combined into a fourth segment and displayed on the first page. Therefore, when the fourth segment represents one highlight element of the story, it can be called a highlight segment of the first visual object. When the fourth segment represents multiple highlight elements of the story, it can be called a collection of highlight segments of the first visual object, obtained by stitching together the individual highlight segments.

[0060] Taking the fourth segment as an example, representing a highlight of the story, the fourth segment can be obtained by clipping from the first visual object or by extracting keyframes. In one example, for a certain e-book, users repeatedly listen to the audio data of Chapter 5. The content represented by the audio data of Chapter 5 is the highlight. If the determined first visual object includes a short drama, then the episode in the short drama that represents the content of Chapter 5, such as episode 8, is determined as the fourth segment, and episode 8 is played on the first page. In another example, for a certain e-book, users repeatedly listen to the audio data of Chapters 5 to 10. The content represented by Chapters 5 to 10 is the highlight. If the determined first visual object includes a short drama, then the episode in the short drama that represents the content of Chapters 5 to 10 is determined, resulting in episodes 8 to 20. The set of keyframes from episodes 8 to 20 is determined as the fourth segment, and the fourth segment is played on the first page.

[0061] When the fourth segment is used to represent multiple highlights in the story, the image corresponding to each highlight can be obtained by clipping the first visual object or extracting keyframes in the manner described above. The images corresponding to each highlight are then spliced ​​together to obtain the fourth segment.

[0062] It's worth noting that the users in the above examples could be some or all of the users who have listened to or read the story, such as all users on the internet; this is not a limitation. When the primary visual object includes a sequence of images, the displayed segment can be a portion of the images in the sequence. For example, if the image sequence includes 100 images, images numbered 20 to 30 are the segments to be displayed.

[0063] In this way, while the user is listening to the audio data, the fourth segment of the first visual object can be displayed on the first page. The fourth segment is used to showcase the highlights of the story, thereby helping the user understand the exciting content of the story and increasing the likelihood that the user will continue to listen to the audio data or be triggered to browse the first visual object.

[0064] Figure 3a This is a schematic diagram illustrating the first visual object in a given situation, such as... Figure 3aAs shown, taking the story presented in an e-book as an example, with the audio data being the audio data of the e-book, the audio data of e-book A is played on the first page. In response to the first visual object associated with e-book A, a segment from the first visual object is displayed on the first page. This segment can be any one of the first segment, second segment, third segment, and fourth segment determined above. Figure 3a The first visual object includes video data, and video frame 1 in the first visual object is being displayed as an example.

[0065] like Figure 3a As shown, a playback card can be displayed on the first page, showcasing segments from the first visual object. If the first visual object includes video data, the playback card can play a specific video segment from that video data. If the first visual object includes an image sequence, the playback card can dynamically play each specific image from that image sequence sequentially. It is understandable that as the user continues listening to the book, the first page will gradually play the audio data of the ebook, and the first visual object will also gradually be displayed, for example, playing video frames in frame order or displaying images in image order.

[0066] In one scenario, the aforementioned story may correspond to a single audio data point, which is the audio data from step S202. In another scenario, the aforementioned story corresponds to multiple audio data points; the audio data played on the first page is not the first audio data point among the multiple audio data points; the method further includes: playing the first audio data point among the multiple audio data points on the first page, and displaying the story information of the story on the first page; correspondingly, playing audio data on the first page includes: playing non-first audio data points after playing the first audio data point on the first page; and displaying a segment from a first visual object on the first page includes: canceling the display of story information on the first page, and displaying a segment from the first visual object.

[0067] In this scenario, the story corresponds to multiple audio data, such as multiple chapters of audio data, which are arranged and played sequentially. The audio data played in step S202 is a non-first audio data corresponding to the story; it can be any non-first audio data other than the first audio data corresponding to the story. When playing a non-first audio data, a segment from the first visual object is displayed. Also, when the first page plays the first audio data, story information such as a story synopsis is displayed. When the first page plays from the first audio data to any non-first audio data, the story information is canceled, and a segment from the first visual object is displayed. The first page playing from the first audio data to any non-first audio data can be done automatically according to the playback order of the audio data, or it can be triggered by the user to play from the first audio data to a non-first audio data specified by the user.

[0068] Taking the story presented as an ebook, with audio data from the ebook itself, as an example, in this scenario, while playing the audio data of the first chapter of the ebook on the first page, the story information, including but not limited to the story title and synopsis, can be displayed on the first page. Alternatively, when playing audio data from other chapters of the ebook on the first page, the story information can be removed from the first page, and fragments from the first visual element can be displayed using the same method. This allows users to simultaneously obtain the story information while listening to the first audio data, improving the efficiency of obtaining the information they need.

[0069] In another scenario, taking the story presented as an ebook with audio data from the ebook as an example, if the first chapter's audio data is played on the first page, the fourth segment of the aforementioned first visual object can be displayed on the first page. If audio data from a non-first chapter of the ebook is played on the first page, the first, second, or third segment of the aforementioned first visual object can be displayed on the first page. Thus, depending on the chapters of the ebook the user is listening to, different segments from the first visual object are provided to the user accordingly.

[0070] In one scenario, the method further includes: displaying relevant information about the first visual object on the first page; the relevant information includes at least a description of the first visual object. This description includes, but is not limited to, at least one of the following: the name of the first visual object, a plot summary, and a character introduction. The relevant information about the first visual object also includes comment information about it.

[0071] Figure 3b This is an illustration of the first visual object provided in another context, such as... Figure 3b As shown, the primary visual object is displayed in a playback card on the first page. This playback card can display relevant information about the primary visual object. The image uses a comic book as the primary visual object, displaying a comment such as "From User A: The comic book adapted from this book is fantastic, highly recommended!" By displaying relevant information about the primary visual object, users can access more content about it, thereby increasing the likelihood of them clicking to view the primary visual object.

[0072] In one scenario, the method further includes: in response to a first trigger on the displayed fragment, displaying a second page, in which the first visual object is displayed.

[0073] In this scenario, a second page can be displayed in response to a first trigger targeting the first visual object, and the first visual object can be displayed on the second page. In one example, the first visual object can be displayed in a playback card on the first page, and a second page can be displayed in response to a trigger targeting the playback card, displaying the first visual object on the second page. The second page includes, but is not limited to, a video playback page or an image playback page. This scenario facilitates users jumping from the first page to the second page to view the first visual object.

[0074] Figure 3c This is an illustration of the first visual object provided in yet another scenario, such as... Figure 3a and Figure 3c As shown, the primary visual object is displayed in the playback card on the first page. Users can trigger... Figure 3a In response to the trigger, a second page is displayed at a preset or arbitrary position within the playback card in the first page. This second page displays the first visual object. The second page can continue displaying the first visual object based on its playback progress on the first page. For example, if the first visual object includes video data, the second page can continue playing that video data based on its playback progress on the first page. Similarly, if the first visual object includes an image sequence, the second page can continue displaying the images within that image sequence based on its playback progress on the first page. Figure 3c The first visual object in the middle includes video data, and the second page is based on... Figure 3a The playback progress continues to play video frame 2 in the first visual object as an example.

[0075] In one scenario, the method further includes: canceling the display of the segment in response to a second trigger on the displayed segment, and displaying story information of the story on the first page.

[0076] Furthermore, as mentioned earlier, the first visual object is displayed in the playback card on the first page. Based on this, in response to a trigger on the playback card, the playback card can be canceled, and a story card for the story can be displayed on the first page, displaying the story's story information. The story's story information includes at least one of the following: story review information, story content description information, and story cover image. In one example, the story is presented as an ebook, and the story card for the story can include the ebook's cover card, displaying the story's story information. The story's story information includes at least one of the following: ebook review information, ebook content description information, and ebook cover image. In this way, even if the user is not interested in the first visual object, it is convenient for the user to learn more about the story.

[0077] Figure 3d This is an illustration of the first visual object provided in yet another scenario, such as... Figure 3a and Figure 3d As shown, taking the story presented through an e-book as an example, users can trigger... Figure 3a The toggle component displayed in the playback card, in response to this trigger, can stop displaying the playback card on the first page, achieving the effect of stopping the display of the first visual object on the first page. Additionally, the ebook cover card is displayed on the first page, showing relevant information about the ebook. The image illustrates this by showing the ebook's content description and cover image.

[0078] Of course, in Figure 3d In the scenario shown, the cover card can also include a toggle component. The toggle component in the playback card and the toggle component in the cover card can be the same component or different components. The toggle component in the playback card can be called the first toggle component, and the toggle component in the cover card can be called the second toggle component. Furthermore, in response to a user's triggering of the toggle component in the cover card, the cover card can be canceled and the playback card can be redisplayed. When redisplaying the playback card, it can be done through... Figure 2 The method flow determines the first visual object to be displayed and the fragment to be displayed in the first visual object, so as to display the fragment in the first visual object on the first page.

[0079] In one scenario, the method further includes: adjusting display parameters of the segment in response to a third trigger on the segment being displayed; the display parameters include at least one of volume, speed, and progress.

[0080] The first page also displays adjustment components for the primary visual object, such as at least one of a volume adjustment component, a speed adjustment component, and a progress adjustment component. Therefore, a user can adjust the volume of a displayed segment, such as a video, using the volume adjustment component, and the terminal device will adjust the volume of that segment in response to the user's volume adjustment operation. Similarly, a user can adjust the speed of a displayed segment, such as a video, using the speed adjustment component, and the terminal device will adjust the display speed of that segment in response to the user's speed adjustment operation, such as switching to 0.8x speed, switching to 2.0x speed, etc. Furthermore, a user can adjust the progress of a displayed segment using the progress adjustment component, and the terminal device will adjust the display progress of that segment in response to the user's progress adjustment operation, such as switching to previous episode, next episode, previous image, next image, etc.

[0081] In this scenario, users can also independently adjust at least one attribute of the first visual object, such as display volume, display speed, and display progress, improving the convenience of adjusting the first visual object.

[0082] In one example, when the first visual object includes video data, the video data can be muted by default. When the first visual object includes a sequence of images, it can respond to user triggers such as landscape swiping to display the sequence of images, showing the previous or next image in the sequence.

[0083] In one scenario, the story text of the aforementioned story can also be generated by a generative model based on story-generating prompts. For example, the user inputs story-generating prompts into the generative model, and the model generates the story text based on these prompts. The generative model can also generate audio data and a first visual object corresponding to the story, and determine the segments within the visual object associated with the progress of the audio data. Furthermore, the generative model can directly output a link to a first page to the user. After the user triggers the link, the terminal device displays the first page, plays the audio data generated by the generative model, and obtains a configuration file generated by the generative model. This configuration file records segment information within the visual object associated with the progress of the audio data. Based on this configuration file, the terminal device determines the segment associated with the progress of the audio data played on the first page and displays that segment on the first page.

[0084] In this scenario, the generative model can output one or more audio data points. The progress of the audio data includes the playback progress within the same audio data point, as well as the order of the currently played audio data point within the various audio data points. For example, the progress of the audio data can be the timestamp of the currently played audio data point, or it can be the sequence number of the currently played audio data point within the various audio data points.

[0085] As a result, users can directly generate story text, audio data, and first visual objects through generative models, and directly click on the links output by the generative model to jump to the first page, enriching the interactive ways for users to listen to audio data and browse visual objects, and expanding the sources of audio data and visual objects that users can listen to and browse.

[0086] In summary, through the above processes, fragments of the first visual object corresponding to the story can be displayed to the user while the user is listening to the audio data of the story, thereby helping the user to obtain the required visual object more efficiently and improving the efficiency of the user in obtaining the required visual object.

[0087] Figure 4 This is a structural diagram of an information display device provided in one scenario, such as... Figure 4 As shown, the device includes: The first playback unit 41 is used to play audio data on the first page; the audio data corresponds to a story; the audio data is used to represent the first content in the story. The first display unit 42 is used to display a fragment of a first visual object on the first page; the first visual object corresponds to the story; the content of the fragment is related to the first content.

[0088] Optionally, the subject type of the first visual object matches the character type in the story; the subject type includes at least one of real humans and virtual object types; the character type includes at least one of real human characters and virtual characters.

[0089] Optionally, the content of the fragment is used to aid in understanding the story.

[0090] Optionally, the first display unit 42 is specifically used to: display a first fragment of the first visual object on the first page; the content of the first fragment includes a second piece of content in the story; the second content includes key content related to the first content.

[0091] Optionally, the first display unit 42 is specifically used to: display a second segment of the first visual object on the first page; the content of the second segment matches the first content.

[0092] Optionally, the first display unit 42 is specifically used to: display a third segment of the first visual object on the first page; the content of the third segment includes the third content of the story; the third content includes the subsequent content of the first content.

[0093] Optionally, the story corresponds to multiple audio data; the audio data played on the first page is not the first audio data among the multiple audio data; the device further includes: a second playback unit, configured to play the first audio data among the multiple audio data on the first page, and to display story information of the story on the first page; the first playback unit 41 is specifically configured to: play the non-first audio data after playing the first audio data on the first page; the first display unit 42 is specifically configured to: cancel the display of the story information on the first page, and to display a segment in the first visual object.

[0094] Optionally, the above-described apparatus further includes: a second display unit, configured to: display a second page in response to a first trigger on the segment, wherein the first visual object is displayed on the second page.

[0095] Optionally, the above-described apparatus further includes: a third display unit, configured to: cancel the display of the segment in response to a second trigger on the segment, and display story information of the story on the first page.

[0096] Optionally, the above device further includes: an adjustment unit for adjusting display parameters of the segment in response to a third trigger on the segment; the display parameters include at least one of volume, speed, and progress.

[0097] Optionally, the above device further includes: a fourth display unit, configured to: display relevant information about the first visual object on the first page; the relevant information includes at least the content description information of the first visual object.

[0098] Optionally, the audio data is obtained by converting the story text of the story into audio.

[0099] Optionally, the audio data is obtained by having a real person read the story text aloud.

[0100] Optionally, the story text of the story includes an ebook describing the story.

[0101] Optionally, the story text of the story is generated by a generative model based on story-generating prompts.

[0102] Optionally, the story corresponds to multiple second visual objects; the first visual object is determined among the various second visual objects by: selecting at least one object among the various second visual objects as a third visual object; the subject type of the third visual object is matched with the character type; the first visual object is selected among the various third visual objects; the genre type of the first visual object includes one of image sequences and video data.

[0103] Through the above processes, fragments of the first visual object corresponding to the story can be displayed to the user while the user is listening to the audio data of the story, thereby helping the user to obtain the required visual object more efficiently and improving the efficiency of the user in obtaining the required visual object.

[0104] The aforementioned information display device can realize each process of the aforementioned information display method and achieve the same effect and function, which will not be repeated here.

[0105] Figure 5 This is a schematic diagram of the structure of an electronic device provided in one scenario, such as... Figure 5 As shown, electronic devices can vary considerably due to differences in configuration or performance. They may include one or more processors 501 and memories 502, with the memory 502 storing one or more application programs or data. The memory 502 can be temporary or persistent storage. The application programs stored in the memory 502 may include one or more modules (not shown), each module including a series of computer-executable instructions from the electronic device. Furthermore, the processor 501 may be configured to communicate with the memory 502 and execute the series of computer-executable instructions stored in the memory 502 on the electronic device. The electronic device may also include one or more power supplies 503, one or more wired or wireless network interfaces 504, one or more input or output interfaces 505, one or more keyboards 506, etc.

[0106] In one specific scenario, the electronic device includes a processor; and a memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the following process: Audio data is played on the first page; the audio data corresponds to a story; the audio data is used to represent the first content in the story; A fragment from a first visual object is displayed on the first page; the first visual object corresponds to the story; the content of the fragment is related to the first content.

[0107] Through the above processes, fragments of the first visual object corresponding to the story can be displayed to the user while the user is listening to the audio data of the story, thereby helping the user to obtain the required visual object more efficiently and improving the efficiency of the user in obtaining the required visual object.

[0108] The aforementioned electronic devices can implement each process of the above information display method and achieve the same effect and function, which will not be repeated here.

[0109] In one embodiment, a computer-readable storage medium is also provided for storing computer-executable instructions that, when executed by a processor, implement the following process: Audio data is played on the first page; the audio data corresponds to a story; the audio data is used to represent the first content in the story; A fragment from a first visual object is displayed on the first page; the first visual object corresponds to the story; the content of the fragment is related to the first content.

[0110] Through the above processes, while users are listening to the audio data of an e-book, fragments of the first visual object associated with the e-book can be displayed to the user, thereby helping the user to obtain the required visual object more efficiently and improving the efficiency of the user in obtaining visual objects generated based on the e-book.

[0111] The aforementioned computer-readable storage medium can implement each process of the above information display method and achieve the same effect and function, which will not be repeated here.

[0112] In one embodiment, a computer program product is also provided, the computer program product comprising a computer program that, when executed by a processor, implements the following process: Audio data is played on the first page; the audio data corresponds to a story; the audio data is used to represent the first content in the story; A fragment from a first visual object is displayed on the first page; the first visual object corresponds to the story; the content of the fragment is related to the first content.

[0113] Through the above processes, fragments of the first visual object corresponding to the story can be displayed to the user while the user is listening to the audio data of the story, thereby helping the user to obtain the required visual object more efficiently and improving the efficiency of the user in obtaining the required visual object.

[0114] The aforementioned computer program products can implement each process of the above information display method and achieve the same effect and function, which will not be repeated here.

[0115] In all the above scenarios, the computer-readable storage medium includes read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, etc.

[0116] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0117] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0118] The systems, devices, modules, or units described above can be implemented by computer chips or physical entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0119] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing the above scenarios, the functions of each unit can be implemented in one or more software and / or hardware components.

[0120] Those skilled in the art will understand that one or more of the above-described embodiments can be provided as methods, systems, or computer program products. Therefore, one or more of the above-described embodiments can take the form of entirely hardware, entirely software, or a combination of software and hardware aspects. Furthermore, one or more of the above-described embodiments can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0121] The above description refers to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to the above descriptions. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0122] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0123] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0124] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0125] One or more of the above scenarios can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. One or more of the above scenarios can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0126] The above scenarios are described in a progressive manner. Similar or identical parts between scenarios can be referred to interchangeably. Each scenario focuses on its differences from the others. In particular, the system is described more simply because it is fundamentally similar to the method; relevant parts can be found in the method section.

[0127] The above description is only a partial description of this document and is not intended to limit the scope of this document. Various modifications and variations can be made to this document by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this document should be included within the scope of the claims of this document.

Claims

1. An information display method, comprising: Audio data is played on the first page; the audio data corresponds to the story. The audio data is used to represent the first content in the story; A fragment from a first visual object is displayed on the first page; the first visual object corresponds to the story; the content of the fragment is related to the first content.

2. The method according to claim 1, wherein the subject type of the first visual object matches the character type in the story; the subject type includes at least one of real humans and virtual object types; the character type includes at least one of real human characters and virtual character types.

3. The method according to claim 1, wherein the content of the fragment is used to aid in understanding the story.

4. The method according to claim 3, wherein displaying a fragment of the first visual object on the first page comprises: The first segment of the first visual object is displayed on the first page; The first segment includes the second content of the story; the second content includes key content related to the first content.

5. The method according to claim 3, wherein displaying a fragment of the first visual object on the first page comprises: The second segment of the first visual object is displayed on the first page; The content of the second segment matches the content of the first segment.

6. The method according to claim 3, wherein displaying a fragment of the first visual object on the first page comprises: The third segment of the first visual object is displayed on the first page; The content of the third segment includes the third part of the story; The third content includes the subsequent content of the first content.

7. The method of claim 1, the story corresponding to a plurality of audio data; The audio data played on the first page is not the first audio data among the plurality of audio data; The method further includes: Play the first audio data among the plurality of audio data on the first page, and display the story information of the story on the first page; The playback of audio data on the first page includes: After the first audio data is played on the first page, the non-first audio data is played. The step of displaying a fragment of the first visual object on the first page includes: The story information is removed from the first page, and a fragment from the first visual object is displayed instead.

8. The method according to claim 1, further comprising: In response to a first trigger on the fragment, a second page is displayed, in which the first visual object is displayed.

9. The method according to claim 1, further comprising: In response to a second trigger on the segment, the segment is canceled from display, and the story information of the story is displayed on the first page.

10. The method according to claim 1, further comprising: In response to a third trigger on the segment, adjust the display parameters of the segment; The display parameters include at least one of volume, speed, and progress.

11. The method according to claim 1, further comprising: The relevant information of the first visual object is displayed on the first page; The relevant information includes at least the content description information of the first visual object.

12. The method according to claim 1, wherein the audio data is obtained by converting the story text of the story into audio.

13. The method according to claim 1, wherein the audio data is obtained by having a real person read the story text of the story aloud.

14. The method of claim 12 or 13, wherein the story text of the story includes an ebook describing the story.

15. The method according to claim 12 or 13, wherein the story text of the story is generated by a generative model based on story-generating prompts.

16. The method of claim 2, wherein the story corresponds to a plurality of second visual objects; the first visual object is determined in each of the second visual objects in the following manner: At least one object is selected from each of the second visual objects as a third visual object; the subject type of the third visual object matches the role type. Select the first visual object from among the various third visual objects; The genre of the first visual object includes either an image sequence or video data.

17. An information display device, comprising: The first playback unit is used to play audio data on the first page; The audio data corresponds to the story; The audio data is used to represent the first content in the story; The first display unit is used to display a fragment of a first visual object on the first page; the first visual object corresponds to the story; the content of the fragment is related to the first content.

18. An electronic device comprising: processor; as well as, A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the method described in any one of claims 1-16.

19. A computer-readable storage medium for storing computer-executable instructions that, when executed by a processor, implement the method of any one of claims 1-16.

20. A computer program product comprising a computer program that, when executed by a processor, implements the method described in any one of claims 1-16.

Images