Content delivery device and program

Abstract

This invention provides a content provisioning device and program capable of providing appropriate character-related information for each character in accordance with the progress of the electronic content. [Solution] In the anime delivery system, the user terminal includes a character-related information update unit 38 that updates at least a portion of the character-related information, which is information related to a recognized character, based on the portion of the electronic content that has been delivered, while the electronic content is being delivered, and a character-related information presentation unit 40 that presents the character-related information of a recognized character while the electronic content is being delivered. The presented character-related information is based on the character-related information that has been updated by the character-related information update unit 38 up to the time the character-related information is presented.

Description

Technical Field

[0001] The present invention relates to a content providing apparatus and a program for providing electronic content.

Background Art

[0002] When a user is enjoying electronic content, there may be some unclear points about the characters appearing in the electronic content. In order to resolve such unclear points, during the provision of electronic content, character-related information related to the characters appearing in the electronic content is provided. As such character-related information, character-related information prepared in advance as metadata of the electronic content is provided (see, for example, Patent Documents 1 and 2).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] When providing character-related information prepared in advance as metadata of electronic content, it is virtually impossible to prepare character-related information for each character at each progress level of the electronic content, and it is difficult to provide appropriate character-related information according to the progress of the electronic content for each character.

[0005] An object of the present invention is to provide a content providing apparatus and a program capable of providing appropriate character-related information according to the progress of electronic content for each character.

Means for Solving the Problems

[0006] One aspect of the present invention is a content provisioning device comprising: a content provisioning unit that provides electronic content for a user to enjoy; a character recognition unit that recognizes characters appearing in the electronic content while the content provisioning unit is providing the electronic content; a character-related information updating unit that updates at least a portion of character-related information, which is information related to a character, based on the portion of the electronic content that has been provided, for the character recognized by the character recognition unit while the content provisioning unit is providing the electronic content; and a character-related information presentation unit that presents character-related information for the character recognized by the character recognition unit while the content provisioning unit is providing the electronic content, wherein the presented character-related information is based on character-related information updated by the character-related information updating unit up to the time the character-related information is presented.

[0007] Another aspect of the present invention is a content provision program that enables a computer to implement: a content provision function that provides electronic content for a user to enjoy; a character recognition function that recognizes characters appearing in the electronic content while the content provision function is providing the content; a character-related information update function that updates at least a portion of the character-related information, which is information related to a character, based on the portion of the electronic content that has been provided, for the character recognized by the character recognition function while the electronic content is providing the content provision function; and a character-related information presentation function that presents the character-related information of a character recognized by the character recognition function while the electronic content is providing the content provision function, wherein the presented character-related information is based on the character-related information that has been updated by the character-related information update function up to the time the character-related information is presented. [Effects of the Invention]

[0008] This invention makes it possible to provide appropriate character-related information for each character in accordance with the progress of the electronic content. [Brief explanation of the drawing]

[0009] [Figure 1] A block diagram showing the hardware configuration of the animation provisioning system according to the first embodiment of the present invention. [Figure 2] A block diagram showing the functional configuration of the animation provision system according to the first embodiment of the present invention. [Figure 3] A flowchart illustrating the animation provision method according to the first embodiment of the present invention. [Figure 4] A schematic diagram illustrating the updating of character-related information according to the first embodiment of the present invention. [Figure 5] A schematic diagram showing an example of displaying character-related information during animation playback according to the first embodiment of the present invention. [Figure 6] A schematic diagram showing an example of displaying character-related information for multiple characters during animation playback according to the first embodiment of the present invention. [Figure 7] A schematic diagram showing an example of displaying character-related information during animation pause according to the first embodiment of the present invention. [Figure 8] A schematic diagram showing an example of displaying character-related information for multiple characters during the pause of an animation according to the first embodiment of the present invention. [Figure 9] A block diagram showing a comic book distribution system according to a second embodiment of the present invention. [Figure 10] A flowchart illustrating a method for providing comics according to a second embodiment of the present invention. [Modes for carrying out the invention]

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0011] A first embodiment of the present invention will be described with reference to Figures 1 to 5. Regarding this embodiment, during the provision of the anime, the characters appearing in the anime are recognized, and based on the provided part of the anime, the character-related information of the character is updated. For the recognized characters, the character-related information is presented, and the presented character-related information is based on the character-related information updated until the time when the character-related information is presented.

[0012] Referring to FIGS. 1 and 2, the anime providing system of this embodiment will be described. Referring to FIG. 1, the hardware configuration of the anime providing system of this embodiment will be described. Note that the hardware configuration of this embodiment itself is common to those skilled in the art, and the characteristic configuration of this embodiment will be described by referring to FIG. 2 and below.

[0013] As shown in FIG. 1, physically, the server device 10 is configured as a computer including a processor 101, a memory 102, a storage 103, a communication device 104, an input device 105, an output device 106, and a bus or the like connecting these. Each of these devices operates by power supplied from a battery not shown in the figure. In the following description, the term "device" can be read as a circuit, a device, a unit, etc. The hardware configuration of the server device 10 may be configured to include one or more of the devices shown in FIG. 1, or may be configured without including some of the devices. Also, a plurality of devices with different casings may be communicatively connected to constitute the server device 10.

[0014] Each function in the server device 10 is realized by causing the processor 101 to load a predetermined software (program) onto hardware such as the processor 101 and the memory 102, so that the processor 101 performs calculations, controls communication by the communication device 104, and controls at least one of reading and writing data in the memory 102 and the storage 103.

[0015] Processor 101 controls the entire computer by operating, for example, an operating system. Processor 101 may be constituted by a central processing unit (CPU: Central Processing Unit) including an interface with peripheral devices, a control device, an arithmetic device, registers, and the like. Also, for example, a baseband signal processing unit, a call processing unit, etc. may be realized by Processor 101.

[0016] Processor 101 reads a program (program code), software module, data, etc. from at least one of storage 103 and communication device 104 into memory 102, and executes various processes according to these. As the program, a program for causing a computer to execute at least a part of the operations described later is used. The functional blocks of server device 10 may be stored in memory 102 and realized by a control program operating in Processor 101. Various processes may be executed by one Processor 101, or may be executed simultaneously or sequentially by two or more Processors 101. Processor 101 may be implemented by one or more chips. Note that the program may be transmitted to server device 10 via a telecommunication line.

[0017] Memory 102 is a computer-readable recording medium and may be constituted by, for example, at least one of ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), RAM (Random Access Memory), etc. Memory 102 may be referred to as a register, a cache, a main memory (main storage device), etc. Memory 102 can store a program (program code), software module, etc. executable for implementing the method according to this embodiment.

[0018] The storage 103 is a computer-readable recording medium and may consist of at least one of the following: an optical disc such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disc, a digital multipurpose disc, a Blu-ray® disc), a smart card, flash memory (e.g., a card, a stick, a key drive), a floppy® disk, a magnetic strip, etc. The storage 103 may also be called an auxiliary storage device.

[0019] The communication device 104 is hardware (transceiver / receiver device) for communicating between computers via at least one of a wired network and a wireless network, and is also referred to as a network device, network controller, network card, communication module, etc.

[0020] The processor 101, memory 102, and other devices are connected by a bus for communicating information. The bus may be configured using a single bus, or different buses may be configured for each device.

[0021] The server device 10 may be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array), and some or all of each functional block may be realized by such hardware. For example, the processor 101 may be implemented using at least one of these hardware components.

[0022] The user terminal 20 is a computer such as a smartphone, mobile phone, tablet, or wearable device. Physically, the user terminal 20 is configured as a computer device including a processor 201, memory 202, storage 203, communication device 204, input device 205, output device 206, and a bus connecting them. The processor 201, memory 202, and storage 203 are hardware similar to the processor 101, memory 102, and storage 103 of the server device 10.

[0023] The communication device 204 may include, for example, a high-frequency switch, duplexer, filter, frequency synthesizer, etc., to implement at least one of frequency division duplex (FDD) and time division duplex (TDD). For example, the transmitting and receiving antennas, amplifier section, transmitting and receiving section, transmission path interface, etc., may be implemented by the communication device 204. The transmitting and receiving section may be physically or logically separated into a transmitting section and a receiving section.

[0024] The input device 205 is an input device that accepts input from an external source (e.g., a key, microphone, switch, button, sensor, etc.). The output device 206 is an output device that outputs to an external source (e.g., a display, speaker, LED lamp, etc.). The input device 205 and the output device 206 may be configured as an integrated unit (e.g., a touchscreen).

[0025] Referring to Figure 2, the functional configuration of the animation provisioning system of this embodiment will be described. As shown in Figure 2, the anime distribution system consists of a server device 10 and a user terminal 20. In the server device 10, the anime distribution unit 12 distributes anime as electronic content and video content to the user terminal 20 upon request from the user terminal 20. In this embodiment, anime is distributed by on-demand streaming. Alternatively, anime may be distributed by live streaming or by download.

[0026] When the anime distribution unit 12 first distributes an anime, it also distributes metadata related to that anime. The metadata related to the anime includes identification information for each character to uniquely identify the character, and as reference information, it includes reference image information that serves as a basis for recognizing the character by image recognition from the anime video, and reference audio information that serves as a basis for recognizing the character by speech recognition from the anime audio. Furthermore, the metadata related to the anime includes item information that indicates the items of character-related information, which is information related to the character. The items of character-related information include name information regarding the character's name, appearance information regarding the part of the anime in which the character appears, status information regarding their status, and relationship information regarding their relationship with other characters. The status information includes affiliation information regarding the character's affiliation in the anime and action information regarding their actions.

[0027] On the user terminal 20, the user inputs an operation to the operation input unit 22. The animation provision unit 24, acting as a content provider, requests the server device 10 to distribute the animation in response to the user's request for animation distribution to the operation input unit 22, and saves the animation distributed from the server device 10. Then, in response to the user's request for animation provision to the operation input unit 22, the animation provision unit 24 outputs the animation from the output unit 26 and allows the user to view it. The provision status detection unit 28 detects the animation provision status by the animation provision unit 24, and the animation provision status by the animation provision unit 24 includes playback, pause, etc.

[0028] When an anime is first delivered to the user terminal 20, the character database 32 is configured based on metadata related to the anime that is delivered along with it. In the character database 32, reference image information and reference sound information are set for each character's identification information, and further, character-related information items are set, and initial information is set for each item.

[0029] While the animation is being provided by the animation provision unit 24, the character recognition unit 34 recognizes the characters appearing in the animation provided by the animation provision unit 24. Specifically, the character recognition unit 34 performs image recognition and sound recognition on the animation video and animation audio provided by the animation provision unit 24, respectively, and recognizes the characters appearing in the animation based on the reference image information and reference sound information for each character set in the character database 32.

[0030] The anime content recognition unit 36 ​​recognizes the content of character conversations by using speech recognition on the anime audio provided by the anime provision unit 24, and also uses image recognition on the anime video. If the anime video includes subtitles or other text overlays, it recognizes the content of the subtitles through character recognition.

[0031] While the animation is being provided by the animation provision unit 24, the character-related information update unit 38 updates the character-related information in the character database 32 based on the character recognition results by the character recognition unit 34 and the recognition results of the character's dialogue and subtitle content by the animation content recognition unit 36. Specifically, when a character is recognized by the character recognition unit 34, the character-related information update unit 38 determines that the character has appeared and updates the character's appearance information. In addition, the character-related information update unit 38 extracts the character's name, status, and relationship with other characters from the character's dialogue and subtitle content recognized by the animation content recognition unit 36, using LLM (Large Language Models) technology, and updates the character's name information, status information, and relationship information with other characters.

[0032] While the animation is being provided by the animation provision unit 24, the character-related information presentation unit 40, in response to a user's request for character-related information from the operation input unit 22, outputs the character-related information for the character recognized by the character recognition unit 34, which is currently set in the character database 32, from the output unit 26 and presents it to the user. For this reason, the character-related information presented by the character-related information presentation unit 40 is based on the character-related information updated by the character-related information update unit 38 up to the time the character-related information is presented. In other words, the character-related information presented by the character-related information presentation unit 40 is based only on the parts of the animation that have been provided by the animation provision unit 24 up to the time the character-related information is presented, and not on the parts of the animation that have not yet been provided. In this embodiment, the character-related information presentation unit 40 presents the character's name information, appearance information, and status information as character-related information.

[0033] Furthermore, the character-related information presentation unit 40 presents character-related information according to the animation provision status detected by the animation provision unit 24, as detected by the provision status detection unit 28. In this embodiment, if the provision status detection unit 28 detects that the animation is being played by the animation provision unit 24, the character-related information presentation unit 40 presents the latest character-related information, and if it detects that the animation is paused, it also presents the history of character-related information. In addition, for character-related information history that has not changed, the latest character-related information may be presented.

[0034] Furthermore, if there are multiple characters recognized by the character recognition unit 34, the character-related information display unit 40 displays, as character-related information, the name information, appearance information, status information of each character, as well as relationship information between those multiple characters. Also, if there are multiple characters recognized by the character recognition unit 34, the character-related information display unit 40 displays character-related information only for characters whose recognition accuracy by the character recognition unit 34 is above a certain level.

[0035] The animation provision method of this embodiment will be described with reference to Figures 3 to 5. In this embodiment of the method for providing animation, an animation series that continues over multiple episodes and is based on a high school club activity team sport will be used as an example for explanation.

[0036] As shown in Figure 3, the animation delivery method involves the following steps:

[0037] Anime Streaming Step S12 In the anime distribution step S12, in response to a user requesting the distribution of an anime to the user terminal 20, the server device 10 receives a request from the user terminal 20 to distribute the anime to the user terminal 20 and saves it to the user terminal 20. When the anime is first distributed from the server device 10 to the user terminal 20, metadata related to the anime is also distributed along with the anime.

[0038] As mentioned above, the metadata related to the anime includes identification information for each character, reference image information and reference sound information as reference information, and further includes item information for character-related information. The items for character-related information include the character's name information, appearance information, status information, and relationship information with other characters, and the status information includes affiliation information and action information.

[0039] For example, in the case of an anime series that continues over multiple episodes and is based on a high school club's team sport, metadata related to the anime will be distributed when the first episode is released. As shown in Figure 4(a), the following will be used for character-related information: "episode number" as appearance information, "name" as name information, "affiliation" such as the high school, class, and position as status information, "match record" in external matches as status information, and "relationship" such as senior / junior, teammate, classmate, rival, friend, old friend as relationship information.

[0040] Character database setup step S14 In the character database setup step S14, the character database 32 is set up based on the metadata related to the anime distributed in the anime distribution step S12. In the character database 32, reference image information and reference sound information are set for each character's identification information, and further, character-related information items are set, and initial information is set for each item.

[0041] For example, as shown in Figure 4(b), for the character with identification information "Y", the following items are set as character-related information: "Number of appearances", "Name", "Affiliation", "Battle record", and "Relationship", and "Unknown" is set as the initial information.

[0042] Anime Sponsorship Step S16 In the animation provision step S16, the animation that was distributed to and saved on the user terminal 20 in the animation distribution step S12 is output on the user terminal 20 in response to the user's request for animation provision to the user terminal 20, and the animation is made available for viewing by the user.

[0043] Service status detection step S18 In the provision status detection step S18, the provision status of the anime in the anime provision step S16 is detected. The provision status of the anime in the anime provision step S16 includes playback, pause, etc.

[0044] Character recognition step S20 In the character recognition step S20, during the animation provision step S16, the characters appearing in the provided animation are recognized. Specifically, the provided animation video and animation audio are recognized as images and audio, respectively, and the characters appearing in the animation are recognized based on the reference image information and reference audio information for each character set in the character database 32.

[0045] Anime content recognition step S22 In the animation content recognition step S22, the content of the animation being provided is recognized during the animation provision step S16. Specifically, the provided animation audio is recognized using speech recognition to understand the characters' dialogue, and the animation video is recognized using image recognition. If the animation video includes subtitles or other text overlays, the content of these subtitles is recognized using character recognition.

[0046] Character-related information update step S24 In the character-related information update step S24, the character-related information in the character database 32 is updated based on the character recognition results in the character recognition step S20 and the anime content recognition results in the anime content recognition step S22. Specifically, if a character is recognized, it is determined that the character has appeared, and the information regarding the character's appearance is updated. Furthermore, the character's name, status, and relationship with other characters are extracted from the recognized character's dialogue and subtitle content using LLM technology, and the character's name information, status information, and relationship information with other characters are updated.

[0047] For example, as shown in Figures 4(c) and 4(d), character-related information is recognized and set for each episode, and the character-related information is updated by rewriting it with the latest character-related information.

[0048] Assuming the main character, identified as "X," has the name "A" and belongs to "P High School 2-5," if the character identified as "Y" is recognized for the first time during the credits of episode 2, as shown in Figure 4(c), it is determined that this was their first appearance in episode 2, and "First Appearance" is set for "Episode 2" and "Appearance Time." If, from the dialogue and subtitles in episode 2, it is recognized that the character's name is "B," their affiliation is "Q High School 2-1," and their relationship is "A's old friend," then "B" is set for "Name," "Q High School 2-1," and "X's old friend" is set for "Episode 2." Then, as the latest information, "Appearance Time," "Name," "Affiliation," and "Relationship" are set to "Episode 2," "B," "Q High School 2-1," and "X's old friend." "Battle Record" remains "Unknown."

[0049] As shown in Figure 4(d), similarly, "Appearance" is set to "Appearance" and "Record" to "Withdrew midway through the match against R High School" for "Episode 4," and "Appearance" is set to "Appearance" for "Episode 5," "Record" to "Scored a large number of points against P High School" and "Relationship" to "Rival of X." At the end of the sponsorship for episode 5 of the anime, the latest information is as follows: "Appearance" is "Episode 5," "Name" is "B," "Affiliation" is "Q High School 2-1," "Record" is "Scored a large number of points against P High School," and "Relationship" is "Rival of X."

[0050] Character-related information presentation step S26 In the character-related information presentation step S26, during the animation provision step S16, in response to a user's request for character-related information to be presented to the user terminal 20, the character-related information of the character recognized in the character recognition step S20, as set in the character database 32 at that time, is output from the user terminal 20 and presented to the user. For this reason, the character-related information presented in the character-related information presentation step S26 is based on the character-related information updated in the character-related information update step S24 up to the time the character-related information is presented. In other words, the character-related information presented in the character-related information presentation step S26 is based only on the parts of the animation that have been provided by the animation provision step S16 up to the time the character-related information is presented, and not on the parts of the animation that have not yet been provided. In this embodiment, the character-related information presented includes the character's name information, appearance information, and status information.

[0051] Furthermore, in the character-related information presentation step S26, character-related information is presented based on the animation provision status in animation provision step S16, which was detected in the provision status detection step S18. In this embodiment, if the provision status detection step S18 detects that the animation is playing in animation provision step S16, the latest character-related information is presented, and if the animation is paused, the history of character-related information is also presented. For character-related information whose history has not changed, the latest character-related information may be presented.

[0052] Furthermore, in the character-related information presentation step S26, if multiple characters are recognized in the character recognition step S20, the character-related information displayed includes the name information, appearance information, and status information of each character, as well as relationship information between those multiple characters. Also, if multiple characters are recognized by the character recognition step S20, character-related information is presented only for characters whose recognition accuracy in the character recognition step S20 is above a certain level.

[0053] As described above, the character-related information presented is based only on the parts of the anime that have already been released, and not on the parts that have not yet been released. This helps prevent spoilers, which would mean unintentionally learning about the plot of parts of the anime that haven't been watched yet.

[0054] For example, regarding the character with identification information Y shown in Figure 4(d), if it appears in episode 5 and then disappears for a long period of time, and then reappears in episode 23, users may have questions about whether the character has already appeared, and if so, in which episode it appeared and what kind of character it was.

[0055] If the user has any questions about the character displayed on the screen, they can request character-related information from the user terminal 20. As shown in Figure 5(a), on a smartphone or personal computer, if the user taps the bottom right of the screen or moves the cursor to the bottom right of the screen on the screen where the character is displayed, a downward-pointing triangle mark T will appear in the bottom right of the screen, and the user can then tap or double-click the triangle mark T again.

[0056] As shown in Figure 5(b), when the character with identification information Y is displayed on the screen and the anime is playing, the character-related information for the displayed character with identification information Y is displayed as follows: first appearance episode "First appearance episode: Episode 2", latest name "B", affiliation "Q High School 2-1", and record "Scored a large number of points against P High School". By referring to the displayed first appearance episode, latest name, affiliation, and record, the user can immediately understand that the character has already appeared, that the character's name is "B", that the character belongs to "Q High School 2-1", and that the character "Scored a large number of points against P High School" is the character of protagonist A, and any questions about the character can be resolved immediately. As a result, the user can fully enjoy the anime without their immersion being hindered. In addition, by referring to the first appearance episode, it is possible to easily rewatch "Episode 2", which is the character's first appearance, at a later date.

[0057] Furthermore, as shown in Figure 6, if the character with identification information Y is displayed on the screen along with the character with identification information X, then, for the character with identification information Y, "A and rival" is displayed based on the latest relationship "X and rival" with the character with identification information X. By referring to the latest relationship, the user can immediately understand that the character in question is in a rival relationship with the main character "A," another character displayed on the screen.

[0058] On the other hand, as shown in Figure 7, when the anime is paused, the character-related information displays the first appearance episode "First Appearance: Episode 2," the latest name "B" and affiliation "Q High School 2-1" as there are no changes in the history regarding the name and affiliation, and the history of appearance episodes and battle record as there are changes in the history, displaying the appearance episode history "Appearance Episodes: Episode 2, Episode 4, Episode 5" and the battle record history "Episode 4: Withdrew midway against R High School" and "Episode 5: Scored a large number of points against P High School." By referring to the displayed first appearance episode, latest name and affiliation, and battle record history, the user can understand that the character has already appeared, that their name is "B," that they belong to "Q High School 2-1," that they "withdrew midway against R High School" in "Episode 4," but that they "scored a large number of points against protagonist A's P High School" in "Episode 5," thus fully resolving any unclear points about the character. As a result, users can deepen their understanding of the character and enjoy the anime more deeply. Furthermore, by referring to the history of appearances, it is possible to easily rewatch episodes 2, 4, and 5 in which the character appears.

[0059] Furthermore, as shown in Figure 8, if the character with identification information Y is displayed on the screen along with the character with identification information X, then for the character with identification information Y, the history of the relationship with the character with identification information X, "Episode 2: X and an Old Friend" and "Episode 5: X and a Rival," will be displayed, along with "Episode 2: X and an Old Friend" and "Episode 5: X and a Rival." By referring to the relationship history, the user can understand that in "Episode 2," the character in question had an "old friend" relationship with the protagonist "A," another character displayed on the screen, but in "Episode 5," after "scoring a large number of points against protagonist A's P High School," they became "A's rival." Note that in Figure 5(e), the display of character-related information for the character with identification information X is omitted.

[0060] Furthermore, as shown in Figures 5(b) and 6, if the animation is paused while the latest character-related information is being displayed during animation playback, the history of the character-related information may be immediately displayed, as shown in Figures 7 and 8.

[0061] Regarding the presentation of character-related information, the most appropriate method can be selected depending on the situation. For example, if a character makes their first appearance in an episode of the anime provided, that episode may simply be labeled as their first appearance. Additionally, for items in the character-related information that are "unknown," the display may be omitted or replaced with "-" or "unknown."

[0062] Furthermore, if one or a few characters are displayed prominently on the screen, while many other characters are displayed smaller, the accuracy of character recognition for the many smaller characters will be reduced, and therefore character-related information will not be displayed. Since these many smaller characters are assumed not to be important characters in that scene, the absence of character-related information will not inconvenience the user; rather, it avoids the situation where a large amount of unnecessary character-related information is displayed.

[0063] The embodiment described above provides the following effects. In this embodiment, for an anime series that continues over multiple episodes, character-related information can be presented to each character appearing in the anime during viewing, without having to prepare character-related information in advance as metadata for each character according to the progress of the anime episodes. The character-related information includes appearance information, including the episode in which the character first appeared, name information, affiliation information as status information, and battle record information, as well as relationship information with other characters. If multiple characters appear in the anime during viewing, relationship information between those multiple characters will be presented. Therefore, even if the user has questions about a character appearing in the anime, such as the episode in which the character first appeared, name, affiliation and battle record as status, or relationship with other characters, they can appropriately resolve these questions by referring to the presented character-related information, allowing them to fully enjoy the anime. In addition, by referring to the appearance information, including the episode in which the character first appeared, it is possible to easily review the episode in which the character first appeared. In particular, during anime playback, the system focuses on displaying only the latest character-related information, allowing viewers to immediately resolve any questions they may have about the characters and fully enjoy the anime without disrupting their immersion. On the other hand, when the anime is paused, the system displays a history of character-related information, allowing viewers to thoroughly resolve any questions they may have about the characters, deepen their understanding of them, and enjoy the anime more deeply.

[0064] The first modified example of the first embodiment of the present invention will be described below. The first modification applies to cases where nicknames, abbreviations, etc., are used for anime characters, affiliations, etc. Metadata related to anime includes correspondence information that shows the relationship between nicknames, abbreviations, etc., and official names. When nicknames, abbreviations, etc., are extracted during anime content recognition, the correspondence information is used to determine the official name corresponding to the nickname, abbreviation, etc. For character-related information, nicknames, abbreviations, etc., and official names may be used in combination, or only the official name may be used.

[0065] For example, in anime, if a character is referred to by their surname only, given name only, nickname, etc., the correspondence information used is the relationship between the character's surname, given name, nickname, etc. and their full name. Specifically, in the dialogue of the anime, if the character with identification information Y is referred to only by the nickname "b" and not their actual full name, then in the anime's content recognition, it is recognized that the character with identification information Y is being called "b". If the correspondence information contains the formal full name "B" that corresponds to the nickname "b", then the name information may be set to "b / B", "B", etc.

[0066] A second modified example of the first embodiment of the present invention will be described below. The second modification applies to cases where there are no changes to the character's name, affiliation, etc., throughout the anime. Name information, affiliation information, etc., are prepared in advance as metadata related to the anime, associated with each character's identification information. This name information, affiliation information, etc., is not subject to updates and remains constant; when the character corresponding to the identification information is recognized, the same name, affiliation, etc., is always displayed. Furthermore, if the character-related information to be displayed is limited to the parts of the episode in addition to this name information, affiliation information, etc., then anime content recognition becomes unnecessary, and the configuration and steps required for anime content recognition can be omitted.

[0067] A second embodiment of the present invention will be described with reference to Figure 9. This embodiment is applied to digital comics instead of anime. In the following, the same configuration and steps as those of the anime provision system and anime provision method in the first embodiment will be simplified or omitted.

[0068] Referring to Figure 9, the functional configuration of the electronic comic distribution system of this embodiment will be described. As shown in Figure 9, in the electronic comic distribution system, a comic distribution unit 52 is used in the server device 10. When the comic distribution unit 52 first distributes an electronic comic, it also distributes metadata related to the electronic comic along with the electronic comic. The metadata related to the electronic comic includes reference image information as reference information, which serves as a basis for recognizing characters from the comic images through image recognition.

[0069] On the user terminal 20, the comic distribution unit 54 outputs electronic comics distributed and stored from the server device 10 from the output unit 26 in response to a user's request for electronic comics from the operation input unit 22, allowing the user to read them. The comic distribution unit 54 also advances the electronic comic in response to user operations such as swiping on the electronic comic and other operations on the operation input unit 22.

[0070] While the comic distribution unit 54 is providing the electronic comic, the character recognition unit 34 recognizes the characters appearing in the electronic comic provided by the comic distribution unit 54. Specifically, the character recognition unit 34 performs image recognition on the comic images provided by the comic distribution unit 54 and recognizes the characters appearing in the electronic comic panel by panel based on the reference image information of each character set in the character database 32.

[0071] The comic content recognition unit 56 performs image recognition on the comic images provided by the comic provision unit 54, and for each panel, it performs character recognition on the text within speech bubbles to recognize the content of the characters' conversations, and performs character recognition on the narrative text to recognize the content of the narrative text.

[0072] While the comic distribution unit 54 is providing the electronic comic, the character-related information update unit 38 updates the character-related information in the character database 32 based on the character recognition results from the character recognition unit 34 and the recognition results of the character's dialogue content and narrative content from the comic content recognition unit 56.

[0073] While the comic provision unit 54 is providing the electronic comic, the character-related information presentation unit 40, in response to a user's request for character-related information via the operation input unit 22, outputs the character-related information for the character recognized by the character recognition unit 34, which is currently set in the character database 32, from the output unit 26 and presents it to the user. Therefore, the character-related information presented by the character-related information presentation unit 40 is based on the character-related information updated by the character-related information update unit 38 up to the time the character-related information is presented. In other words, the character-related information presented by the character-related information presentation unit 40 is based only on the parts of the comic that have been provided by the comic provision unit 54 up to the time the character-related information is presented, and not on the parts of the comic that have not yet been provided.

[0074] The electronic comic provision method of this embodiment will be described below with reference to Figure 10. As shown in Figure 10, the following steps are performed in the electronic comic delivery method.

[0075] Comic distribution step S32 In comic distribution step S32, the server device 10 distributes the electronic comic to the user terminal 20. When the electronic comic is first distributed from the server device 10 to the user terminal 20, metadata related to the electronic comic is also distributed along with the electronic comic. The metadata related to the electronic comic includes the aforementioned reference image information as reference information.

[0076] Character database setup step S34 The character database setup step S34 is the same as the character database setup step S14 in the first embodiment.

[0077] Comic provided step S36 In the comic provision step S36, the electronic comic that was delivered to and saved on the user terminal 20 in the comic distribution step S32 is output on the user terminal 20 in response to the user's request operation to provide the electronic comic to the user terminal 20, and the user is allowed to read it. In addition, in the comic provision step S36, the electronic comic is advanced in response to the user's operation to advance the electronic comic on the user terminal 20, such as a swipe operation.

[0078] Character recognition step S40 In the character recognition step S40, while the electronic comic is being provided in the comic provision step S36, the characters appearing in the provided electronic comic are recognized. That is, the provided comic image is recognized, and based on the reference image information of each character set in the character database 32, the characters appearing in the electronic comic are recognized frame by frame.

[0079] Comic content recognition step S42 In the comic content recognition step S42, the content of the provided electronic comic is recognized while the electronic comic is being provided in the comic provision step S36. Specifically, the provided comic image is recognized using image recognition, and for each panel, the text within the speech bubbles is recognized using character recognition to recognize the content of the characters' conversations, and the narrative text is recognized using character recognition to recognize the content of the narrative text.

[0080] Character-related information update step S44 In the character-related information update step S44, the character-related information in the character database 32 is updated based on the character recognition results in the character recognition step S40 and the recognition results of the content of the electronic comic in the comic content recognition step S42.

[0081] Character-related information presentation step S46 In the character-related information presentation step S46, during the provision of the electronic comic in the comic provision step S36, in response to the user's request for character-related information to be presented to the user terminal 20, the character-related information of the character recognized in the character recognition step S40, which is set in the character database 32 at that time, is output from the user terminal 20 and presented to the user. For this reason, the character-related information presented in the character-related information presentation step S46 is based on the character-related information updated in the character-related information update step S44 up to the time the character-related information is presented. In other words, the character-related information presented in the character-related information presentation step S46 is based only on the portion of the comic that has been provided in the comic provision step S36 up to the time the character-related information is presented, and not on the portion of the comic that has not yet been provided.

[0082] The embodiment described above provides the following effects. In this embodiment, character-related information can be presented to each character appearing in the digital comic while the digital comic is being read, without having to prepare character-related information for each character in the digital comic in advance as metadata related to the digital comic. Therefore, even if the user has questions about a character appearing in the digital comic while reading it, they can appropriately resolve those questions by referring to the presented character-related information, and fully enjoy the digital comic.

[0083] The present invention also includes programs that implement each function of the animation provision system and other features of each embodiment and each modified example described above, particularly programs that implement each function of the user terminal 20, and programs that execute each step of the animation provision method and other features, particularly programs that execute each step on the user terminal 20.

[0084] The characteristic features of each embodiment described above can be summarized as follows:

[0085] The first embodiment is a content providing device comprising: a content providing unit that provides electronic content for a user to enjoy; a character recognition unit that recognizes characters appearing in the electronic content while the content providing unit is providing the electronic content; a character related information updating unit that updates at least a portion of character related information, which is information related to a character, based on the portion of the electronic content that has been provided, for the character recognized by the character recognition unit while the content providing unit is providing the electronic content; and a character related information presentation unit that presents character related information for the character recognized by the character recognition unit while the content providing unit is providing the electronic content, and the presented character related information is based on character related information updated by the character related information updating unit up to the time the character related information is presented.

[0086] In this embodiment, character-related information can be presented to each character appearing in the electronic content while the user is enjoying the electronic content, without having to prepare character-related information in advance as metadata for each character in the electronic content according to the progress of the electronic content. Therefore, even if the user has questions about the characters appearing in the electronic content while enjoying the electronic content, they can appropriately resolve those questions by referring to the presented character-related information, and fully enjoy the electronic content.

[0087] The second embodiment is a content providing device of the first embodiment, wherein the character-related information includes appearance information relating to the part in which the character appears, the character-related information update unit updates the appearance information, and the character-related information presentation unit presents the appearance information.

[0088] In this embodiment, any unclear points regarding the appearance of a character can be resolved, and it becomes possible to easily review the appearance of that character.

[0089] The third embodiment is a content providing device according to the first or second embodiment, wherein the character-related information includes at least one name information relating to the character's name and status information relating to the character's status, the character-related information update unit updates at least one of the name information and status information, and the character-related information presentation unit presents at least one of the name information and status information.

[0090] In this embodiment, it is possible to resolve any ambiguities regarding the name and at least one of the stats of the characters that appear.

[0091] The fourth embodiment is a content providing device according to any one embodiment of the first to third embodiments, wherein the character-related information includes relationship information relating to the relationships between multiple characters, the character-related information update unit updates the relationship information, and the character-related information presentation unit presents the relationship information between multiple characters when multiple characters are recognized by the character recognition unit.

[0092] In this embodiment, when multiple characters appear, it is possible to resolve any ambiguities regarding the relationships between those multiple characters.

[0093] The fifth embodiment is a content providing device of the second embodiment, wherein the electronic content is a series of electronic content that continues over multiple installments, and the appearance information is first appearance information relating to the first appearance of a character.

[0094] In this embodiment, in a series of electronic content that continues over multiple installments, it is possible to check the first appearance of a character that appears in the electronic content, and to easily rewatch the first appearance of that character.

[0095] The sixth embodiment is a content providing device according to any one embodiment of the first to fifth embodiments, wherein the electronic content is video content, and the character-related information display unit displays the latest character-related information updated by the character-related information update unit while the video content is being played back by the content providing unit.

[0096] In this embodiment, during video content playback, only the latest character-related information is displayed as character-related information. This allows users to immediately resolve any questions they have about the characters, ensuring that their immersion in the video content is not hindered, and enabling them to fully enjoy the anime.

[0097] The seventh embodiment is a content providing device according to any one embodiment of the first to sixth embodiments, wherein the electronic content is video content, and the character-related information display unit displays a history of character-related information updated by the character-related information update unit while the video content is paused by the content providing unit.

[0098] In this embodiment, when the video content is paused, the history of character-related information is displayed as character-related information. This allows users to fully resolve any uncertainties about the characters, deepen their understanding of them, and enjoy the video content more deeply.

[0099] The eighth embodiment is a content provision program that enables a computer to implement: a content provision function that provides electronic content for a user to enjoy; a character recognition function that recognizes characters appearing in the electronic content while the content provision function is providing content; a character-related information update function that updates at least a portion of the character-related information, which is information related to a character, based on the portion of the electronic content that has been provided, for the character recognized by the character recognition function while the electronic content is providing content by the content provision function; and a character-related information presentation function that presents the character-related information of a character recognized by the character recognition function while the electronic content is providing content by the content provision function, wherein the presented character-related information is based on the character-related information that has been updated by the character-related information update function up to the time the character-related information is presented. This embodiment provides the same effects and advantages as the content provision device of the first embodiment. [Explanation of symbols]

[0100] 10…Server device 12…Anime distribution unit 20…User terminal 22…Operation input unit 24...Anime provision unit 26...Output unit 28...Provision status detection unit 32…Character Database 34…Character Recognition Unit 36…Anime Content Recognition Unit 38...Character-related information update section 40...Character-related information presentation section 54...Comic Distribution Department 56...Comic Content Recognition Department Triangle mark...T

Claims

1. The content provision department provides electronic content for users to enjoy, During the provision of electronic content by the content provision unit, a character recognition unit recognizes characters appearing in the electronic content provided by the content provision unit, During the provision of electronic content by the content provision unit, a content content recognition unit recognizes the content of the electronic content being provided by the content provision unit, During the provision of electronic content by the Content Provision Unit, the Character Related Information Update Unit extracts at least a portion of the character-related information, which is information related to the character, from the recognition results of the Character Recognition Unit and the Content Content Recognition Unit for the portion of the electronic content that has been provided, and updates at least a portion of the character-related information, for characters recognized by the Character Recognition Unit, and updates at least a portion of the character-related information. During the provision of electronic content by the content provision unit, the character-related information presentation unit presents character-related information for characters recognized by the character recognition unit, and the presented character-related information is based on character-related information updated by the character-related information update unit up to the time the character-related information is presented. A content provisioning device equipped with the following features.

2. The character-related information includes at least one of name information relating to the character's name and status information relating to the character's status, The aforementioned character-related information update unit updates at least one of the name information and status information. The character-related information display unit displays at least one of name information and status information. The content provision device according to claim 1.

3. A content provision unit that provides electronic content for users to enjoy, During the provision of electronic content by the aforementioned content provision unit, a character recognition unit recognizes characters appearing in the electronic content, During the provision of electronic content by the content provision unit, the character-related information update unit updates at least a portion of the character-related information, which is information related to a character, based on the portion of the electronic content that has been provided, for a character recognized by the character recognition unit. During the provision of electronic content by the content provision unit, the character-related information presentation unit presents character-related information for characters recognized by the character recognition unit, and the presented character-related information is based on character-related information updated by the character-related information update unit up to the time the character-related information is presented. It is equipped with, The aforementioned character-related information includes relationship information regarding the relationships between multiple characters, The aforementioned character-related information update unit updates the relationship information, The character-related information presentation unit presents relationship information between multiple characters when multiple characters are recognized by the character recognition unit. Content delivery device.

4. A content provision unit that provides electronic content for users to enjoy, During the provision of electronic content by the aforementioned content provision unit, a character recognition unit recognizes characters appearing in the electronic content, During the provision of electronic content by the content provision unit, the character-related information update unit updates at least a portion of the character-related information, which is information related to a character, based on the portion of the electronic content that has been provided, for a character recognized by the character recognition unit. During the provision of electronic content by the content provision unit, the character-related information presentation unit presents character-related information for characters recognized by the character recognition unit, and the presented character-related information is based on character-related information updated by the character-related information update unit up to the time the character-related information is presented. It is equipped with, The aforementioned character-related information includes information about the parts in which the character appears, The aforementioned character-related information update unit updates the information on the parts in which the character appears. The aforementioned character-related information display unit displays information about the parts in which the character appears. The aforementioned electronic content is part of a series of electronic content that continues over multiple installments. The aforementioned appearance information is information about the character's first appearance. Content delivery device.

5. The electronic content is video content, The character-related information display unit displays the latest character-related information updated by the character-related information update unit while the video content is being played by the content provision unit. The content provision device according to claim 1.

6. A content provision unit that provides electronic content for users to enjoy, During the provision of electronic content by the aforementioned content provision unit, a character recognition unit recognizes characters appearing in the electronic content, During the provision of electronic content by the content provision unit, the character-related information update unit updates at least a portion of the character-related information, which is information related to a character, based on the portion of the electronic content that has been provided, for a character recognized by the character recognition unit. During the provision of electronic content by the content provision unit, the character-related information presentation unit presents character-related information for characters recognized by the character recognition unit, and the presented character-related information is based on character-related information updated by the character-related information update unit up to the time the character-related information is presented. It is equipped with, The aforementioned electronic content is video content, The character-related information display unit displays the history of character-related information updated by the character-related information update unit while the video content provided by the content provision unit is paused. Content delivery device.

7. A computer, Content provision function that provides electronic content for users to enjoy, During the provision of content by the aforementioned content provision function, a character recognition function is provided to recognize characters appearing in the electronic content provided by the aforementioned content provision function. During the provision of electronic content by the aforementioned content provision function, a content content recognition function recognizes the content of the electronic content being provided by the aforementioned content provision function, During the provision of electronic content by the content provision function, a character-related information update function extracts at least a portion of character-related information, which is information related to a character, from the recognition results of the character recognition function and the content content recognition function for the portion of the electronic content that has been provided, and updates at least a portion of the character-related information, for characters recognized by the character recognition function, and updates at least a portion of the character-related information. During the provision of electronic content by the content provision function, the character-related information of a character recognized by the character recognition function is presented, and the presented character-related information is based on the character-related information updated by the character-related information update function up to the time the character-related information is presented, and the character-related information presentation function is provided, A content provision program that makes this a reality.

Images