Information processing device

Abstract

An information processing device (10) comprises: an extraction unit (11) that extracts, for each comic processing unit, text information representing a sound from an image published in a comic; a comparison unit (12) that compares the extracted text information of each unit of comic processing to text information representing a sound in an animation prepared in advance, and calculates a degree of similarity; and a correspondence derivation unit (13) that determines a correspondence relationship between the animation and the unit of comic processing on the basis of the calculated degree of similarity.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus for obtaining a correspondence relationship between an anime scene and a comic processing unit. In this specification, "comic" broadly refers to a manga that is a story expressed in pictures and text, and its form may be a printed and bound manga book, or an electronic comic that can be displayed and referred to on a mobile terminal, computer, etc. Also, "animation" means a video created by animating a comic and inserting the lines, sound effects, etc. of the characters. Hereinafter, "animation" will be abbreviated as "anime".

Background Art

[0002] Popular comics are often sold as comics and also animated and provided as anime movies or TV shows. A famous scene of an anime that moves the emotions of users who watch such popular anime comics is said to be a chance to promote the sale of the comic including the comic scene corresponding to that famous scene, and it is very important to grasp the correspondence relationship between the anime scene and the comic scene.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] As a related technique, Patent Document 1 discloses a technique for obtaining transition information between pages where data is similar by comparing the data of each page obtained by analyzing the character strings (text), images, etc. within each page of one electronic comic, but the correspondence relationship between the anime scene and the comic scene is not targeted.

[0005] Furthermore, in reality, the art styles of anime and comics often differ, and these images are not always similar. In comics in particular, speech bubbles may be separated, meaning that the same character's dialogue may not be found in a single location. Therefore, it was difficult to determine the correspondence between anime scenes and comic scenes accurately and with minimal effort by hand.

[0006] This disclosure aims to address the aforementioned challenges by providing a way to determine the correspondence between anime scenes and comic scenes appropriately and with minimal burden. [Means for solving the problem]

[0007] The applicant assumed that memorable scenes in anime are accompanied by memorable lines of dialogue, and conceived of a method to compare pre-prepared text information of each line of dialogue in anime with text information extracted for each comic processing unit, and to determine the correspondence between anime scenes and comic processing units from the comparison result (similarity). Specifically, the information processing device according to this disclosure comprises: an extraction unit that extracts text information representing sound from images published in a comic for each comic processing unit; a comparison unit that compares the extracted text information for each comic processing unit with pre-prepared text information representing sound in anime scenes and calculates a similarity; and a correspondence derivation unit that determines the correspondence between anime scenes and comic processing units based on the calculated similarity.

[0008] In the above-described information processing device, the extraction unit extracts text information representing sounds (e.g., dialogue, sound effects, etc.) from images in the comic, for each comic processing unit; the comparison unit compares the extracted text information for each comic processing unit with pre-prepared text information representing sounds in anime scenes to calculate similarity; and the correspondence derivation unit determines the correspondence between anime scenes and comic processing units based on the calculated similarity. In this way, the correspondence between anime scenes and comic scenes can be determined appropriately and with minimal effort. [Effects of the Invention]

[0009] According to this disclosure, the correspondence between anime scenes and comic scenes can be determined appropriately and with minimal burden. [Brief explanation of the drawing]

[0010] [Figure 1] This is a diagram illustrating the outline of an embodiment. [Figure 2] This is a functional block diagram of the information processing device in the embodiment. [Figure 3] This is a flowchart illustrating the process in the embodiment. [Figure 4] This is a diagram illustrating the process of extracting text information from comics. [Figure 5] This is a diagram to explain how the order of dialogue is estimated. [Figure 6] This is a diagram to explain the estimation of the same speaker when speaking lines of dialogue. [Figure 7] This is a diagram to explain the calculation of dialogue similarity. [Figure 8] This is a diagram to explain how to identify the corresponding page. [Figure 9] This figure shows an example of displaying corresponding anime scenes on a comic purchase screen. [Figure 10] This figure shows an example of an anime introduction page that displays the corresponding volume of the manga. [Figure 11] This figure shows an example of the hardware configuration of an information processing device. [Modes for carrying out the invention]

[0011] The embodiments of this disclosure will be described below with reference to the drawings. In the embodiments described below, as shown in Figure 1, it is assumed that famous scenes in anime are associated with dialogue, and the correspondence between anime scenes and comic scenes is determined by matching (i.e., comparing and calculating the similarity) text information representing dialogue from various scenes in anime, which has been extracted in advance, with text information representing dialogue obtained from a predetermined comic processing unit in the comic. The text information representing dialogue from various scenes in anime, which has been extracted in advance (hereinafter referred to as "anime dialogue information"), is assumed to have been obtained in advance from dialogue (speech) by speech recognition or the like. On the other hand, the text information representing dialogue written in the comic is obtained by character recognition or the like, which will be described later. The "comic processing unit" above may be each page of the comic, or a two-page spread of the comic may be used. If the latter is used, it means that the two pages of the spread are treated as a single picture (image). In the embodiments described below, the case in which each page of the comic is used as the "comic processing unit" will be described, and each page that is used as a comic processing unit will be referred to as a "comic page".

[0012] As shown in Figure 2, the information processing device 10 in this embodiment includes, as functional blocks for realizing the functions related to this disclosure, a dialogue information storage unit 15 that stores previously acquired anime dialogue information, an extraction unit 11 that extracts text information representing sound (in this embodiment, "dialogue") from images published in the comic for each comic page, a comparison unit 12 that compares the extracted text information for each comic page with previously acquired anime dialogue information (text information representing dialogue) and calculates a similarity, a correspondence derivation unit 13 that determines the correspondence between anime scenes and comic pages based on the calculated similarity, and an information presentation unit 14 that, based on the obtained correspondence, presents video information of the corresponding anime scene on the comic sales page and presents link information on the anime introduction page for transitioning to the comic sales page containing the corresponding comic page. Details of the functions of each unit will be explained below along with the processing content shown in Figure 3.

[0013] Next, with reference to FIGS. 4 to 10, the processing executed in the information processing apparatus 10 will be described along the flowchart of FIG. 3.

[0014] First, the extraction unit 11 estimates the order of dialogues and the identity of the speakers of the dialogues in the comic image data, and combines a plurality of dialogues based on the estimation results, thereby extracting text information representing the dialogues for each comic page (step S1 in FIG. 3).

[0015] As shown in FIG. 4, in step S1, the extraction unit 11 identifies a character region from each page of the comic using an existing text detection technique, performs character recognition by optical character recognition (OCR) or the like, extracts text information representing the dialogues, and acquires frame division information and speech balloon information using an existing object detection technique. Thereafter, the extraction unit 11 performs the following "dialogue order estimation" and "speaker identity estimation of dialogues", and combines a plurality of dialogues based on the estimation results, thereby extracting text information representing the dialogues for each comic page.

[0016] Here, as "dialogue order estimation", as shown in FIG. 5 for example, the extraction unit 11 estimates the frame order and the dialogue order from the coordinate information of the frames and the coordinate information of the dialogue (speech bubble & character area) using machine learning. For example, the extraction unit 11 estimates the dialogue order as the movement of the reader's line of sight that is normally assumed based on the coordinate information of the frames grasped from the comic page and the coordinate information of the dialogue. In addition to the coordinate information of the frames and the coordinate information of the dialogue, other metadata that can be grasped from the comic page, such as the movement line of the line of sight of the speaker (character) of the dialogue, may be used as the basic information to estimate the dialogue order. The obtained dialogue order is used as the basic information in the combination of a plurality of dialogues described later. The method for estimating the frame order and the dialogue order using the above machine learning is not limited to a specific method. For example, it can use the existing technology titled "Estimation of Object Ranking in Comics Based on Coordinate Information" in the Research Report of the Information Processing Society Vol. 2019-NL-241 No. 26 (Internet address: https: / / ipsj.ixsq.nii.ac.jp / ej / index.php?active_action=repository_view_main_item_detail&page_id=13&block_id=8&item_id=198867&item_no=1) to estimate the frame order and the dialogue order. The basic idea of the above estimation technology is that "there are certain rules for the movement lines of the line of sight such as the frame order and the dialogue order. By learning this from the coordinate information and the coordinate transition, it is estimated." As a result of the above estimation process, for example, as shown at the right end of FIG. 5, a processing result that the dialogue order is in the order of the arrows P1, P2, P3, P4, P5 is obtained.

[0017] Furthermore, as part of "estimating the identity of the speaker of the dialogue," the extraction unit 11 estimates the identity of the speaker of the dialogue based on either or both of the positional information of the speech bubble and speaker containing the dialogue, and the semantic connections between the lines of dialogue, as grasped from the comic page. For example, as shown in Figure 6, the extraction unit 11 uses machine learning to estimate the identity of the speaker, using the coordinate information of the panel, the coordinate information and text information of the dialogue (speech bubble and text area), as well as the positional relationship with the speaker (character) of the dialogue, the type and orientation of the speech bubble, etc. The method of estimating the same speaker using machine learning described here is not limited to a specific method. For example, one may use the paper "A Study on Correspondence Methods between Dialogue and Speakers in Manga" by Kazuki Abe (address: https: / / dl.nkmr-lab.org / papers / 227) or the paper "Speaker Estimation of Speech Bubbles in Manga Images" by Kazuyoshi Yamamoto (address: https: / / repository.dl.itc.u-tokyo.ac.jp / record / 51209 / files / 48166452.pdf) to perform same speaker estimation. The basic idea behind the estimation techniques described above is that "estimation is mainly achieved by learning the tendency that 'characters closer to the dialogue are more likely to be the speaker.' When using text information, the judgment is made based on the validity of the semantic connections between texts." As a result of the estimation process described above, for example, as shown on the right side of Figure 6, lines A1 "Yeah, sure," A2 "I will!", and A3 "I'll take them all down!" are estimated to be spoken by the same person, but lines B "This year's tournament has begun." and C "Interesting" are not estimated to be spoken by the same person. In step S1 of Figure 3 as described above, the extraction unit 11 estimates the order of the lines and the identity of the speakers of the lines, and based on these estimation results, combines multiple lines to extract text information representing the lines from the comic image data for each comic page.For example, if the sequence estimation result is A1→A2→A3→B→C, and the same speaker estimation result is obtained that lines A1 to A3 are spoken by the same person, then as shown in Figure 7, lines A1 to A3 are selectively combined from the above lines A1 to A3, B and C to form "Line A", and the sequence "Line A in panel 2" → "Line B in panel 3" → "Line C in panel 4" is estimated.

[0018] Although the above shows a simple example, the combination of dialogue between different frames may be adjusted so that the combined text length falls within the range of (text length L - predetermined allowable width ΔL) to (text length L + predetermined allowable width ΔL), depending on the length of the dialogue in the pre-prepared animation scene (hereinafter referred to as "text length") L.

[0019] Next, the comparison unit 12 compares the text information for each comic page extracted in step S1 with the dialogue information (text information representing dialogue) of the anime stored in the dialogue information storage unit 15, and calculates the similarity of these text (string) information (step S2 in Figure 3). Here, the comparison unit 12 uses, for example, the edit distance or the Jaccard coefficient as the similarity, as shown in Figure 7, and calculates the similarity S(W). q ,W n,i ) is calculated. Note that W q This is pre-prepared anime dialogue information and is used as a query to identify the page. n,i This refers to the set of strings that appear on page n of the comic.

[0020] Next, the correspondence derivation unit 13 determines the correspondence between anime scenes and comic pages based on the calculated similarity (step S3 in Figure 3). For example, as shown in Figure 8, the correspondence derivation unit 13 identifies the comic pages corresponding to the dialogue and determines the correspondence between anime scenes and comic pages by matching the prepared strings using the similarity between strings as described below. Here, we will explain an example in which "edit distance" is used as the similarity. The problem of identifying the comic pages n* corresponding to the dialogue can be formulated as follows.

number

number

number

[0021] Here's an example of edit distance. Assuming the same dialogue is being transcribed, the edit distance won't increase even if there are slight variations in wording. Example of edit distance = 2: The distance between the anime line "Yeah, that's fine. I'll take them all down!" and the text on the comic page "Yeah, that's fine. I'll! Take them all down!" Example of edit distance = 7: The distance between the anime line "Yeah, that's fine. I'll take them all down!" and the text on the comic page "Yeah, that's fine. I'll take them all on!" Example of edit distance = 14: The distance between the anime line "Yeah, that's fine. I'll take them all down!" and the text on the comic page "Come at me all at once!" Furthermore, when calculating the edit distance as described above, it is assumed that preprocessing such as standardizing variations in notation and removing symbols (punctuation marks, etc.), which are commonly done beforehand, will be performed. Note that in the example above, the "edit distance," which increases in similarity as the value decreases, was used as the similarity measure. However, if the "Jaccard coefficient," which increases in similarity as the value increases, is used as the similarity measure, the formula for identifying the comic page n* corresponding to the dialogue will need to be appropriately transformed.

[0022] Next, the information display unit 14 displays video information of the corresponding anime scene on the comic sales page based on the correspondence obtained in step S3 (step S4 in Figure 3). For example, as shown in Figure 9, the information display unit 14 displays a famous anime scene corresponding to the corresponding volume of the original comic on the detailed display screen of the original comic. Figure 9 shows an example where the sales page for "Work xxxx Comic Volume 7" displays a famous anime scene corresponding to volume 7 of the comic and the comic page corresponding to the anime scene. In this way, by displaying the corresponding famous anime scene on the detailed display screen of the original comic, it is possible to promote sales of the corresponding volume of the original comic.

[0023] Furthermore, based on the correspondence obtained in step S3, the information display unit 14 presents link information on the anime introduction page to transition to the sales page of the comic that includes the corresponding comic page (step S5 in Figure 3). For example, as shown in Figure 10, the information display unit 14 displays a famous scene on the anime introduction page and presents link information to transition to the sales page of the corresponding volume of the comic that corresponds to the displayed famous scene. Figure 10 shows an example where, on the introduction page for "Work xxxx Anime," a famous scene from the anime is presented along with a link (banner on the screen) to transition to the sales page of the comic that corresponds to the anime scene, and to the sales page of the corresponding volume of the comic (comic volume 7) that includes the comic page, allowing the user to transition to the sales page and make a purchase smoothly. By presenting a link to the sales page of the corresponding volume of the comic (comic volume 7) that includes the comic page corresponding to the anime scene on the anime introduction page in this way, it is possible to promote sales of the comic volume that corresponds to the anime scene.

[0024] According to the embodiment described above, the correspondence between anime scenes and comic scenes can be determined appropriately and with minimal effort. In determining the correspondence, the extraction unit 11 performs dialogue order estimation and same-speaker estimation, and by combining multiple lines of dialogue based on these estimation results, text information for each comic page can be extracted more accurately and appropriately. Note that it is not mandatory to perform both dialogue order estimation and same-speaker estimation; only one of them may be performed.

[0025] Furthermore, by using the correspondence between anime scenes and comic scenes, it is possible to promote sales of the corresponding comic volumes by displaying the corresponding famous anime scenes on the detailed display screen of the original comic, as shown in Figure 9, and by displaying links on the anime introduction page to the sales pages of the corresponding comic volumes that include the comic pages corresponding to the anime scenes, as shown in Figure 10. Note that it is not mandatory to implement both the information presentation shown in Figure 9 and the information presentation shown in Figure 10; either one may be implemented. Moreover, this is not limited to the "sale" of the corresponding comic volumes, but can also be applied to other commercial transaction forms such as "rental".

[0026] In the above embodiment, we showed that the text information of the comic pages and anime scenes being compared represents the "dialogue" of the characters. However, it is not limited to dialogue, and can be broadly applied to text information representing sounds, background sounds (wind sounds, rain sounds, etc.), animal sounds, etc.

[0027] The gist of this disclosure is found in the following [1] to [5]. [1] An extraction unit that extracts text information representing sound from images published in the comic, for each comic processing unit, A comparison unit compares the extracted text information for each comic processing unit with pre-prepared text information representing sounds in anime scenes and calculates the similarity. A correspondence derivation unit that determines the correspondence between anime scenes and comic processing units based on the calculated similarity, An information processing device equipped with the following features. [2] The text information for each comic processing unit and the text information representing sound in the animation scene are text information representing dialogue. [1] The information processing device described above. [3] The extraction unit estimates the order of the dialogue based on information including the position information of the panels and dialogue obtained from the comic processing unit, and extracts text information for each comic processing unit by combining multiple lines of dialogue based on the obtained order of the dialogue. [2] The information processing device described above. [4] The extraction unit estimates the identity of the speaker of the dialogue based on either or both of the speech bubble containing the dialogue and the speaker's position information obtained from the comic processing unit, and the semantic connections between the dialogues, and extracts text information for each comic processing unit by combining multiple dialogues based on the obtained speaker identity. The information processing device described in [2] or [3]. [5] Based on the correspondence between the animation scene and the comic processing unit derived by the correspondence derivation unit, The presentation of corresponding animated video information of the aforementioned anime scenes on a webpage for the commercial sale of the aforementioned comic, and, To provide a link to a webpage introducing the aforementioned anime scene, to provide link information to a webpage for commercializing comics that include the corresponding comic processing unit. An information display unit that performs at least one of the following: An information processing device described in any one of the following [1] to [4], further comprising:

[0028] (Explanation of terms, explanation of hardware configuration (Figure 11), etc.) The block diagrams used in the description of the above embodiments show functional units. These functional blocks (components) are realized by any combination of at least one of hardware and software. Furthermore, the method of realizing each functional block is not particularly limited. That is, each functional block may be realized using one device that is physically or logically coupled, or it may be realized using two or more physically or logically separated devices that are directly or indirectly connected (for example, using wired or wireless connections). A functional block may also be realized by combining the above one device or the above multiple devices with software.

[0029] Functions include, but are not limited to, judgment, decision, judgment, calculation, calculation, processing, derivation, investigation, exploration, confirmation, reception, transmission, output, access, resolution, selection, selection, establishment, comparison, assumption, expectation, assumption, broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating (mapping), and assigning. For example, a functional block (configuration part) that enables transmission is called a transmitting unit or transmitter. As mentioned above, the method of implementation is not particularly limited.

[0030] For example, the information processing device 10 in this embodiment may function as a computer that performs the processing of the disclosure. Figure 11 is a diagram showing an example of the hardware configuration of the information processing device 10. The information processing device 10 described above may be physically configured as a computer device including a processor 1001, memory 1002, storage 1003, communication device 1004, input device 1005, output device 1006, bus 1007, etc.

[0031] In the following explanation, the term "device" can be replaced with "circuit," "device," "unit," etc. The hardware configuration of the information processing device 10 may include one or more of the devices shown in the figure, or it may be configured to omit some of the devices.

[0032] Each function in the information processing device 10 is realized by loading predetermined software (programs) onto hardware such as the processor 1001 and memory 1002, which allows the processor 1001 to perform calculations, control communication by the communication device 1004, and control at least one of data reading and writing in the memory 1002 and storage 1003.

[0033] The processor 1001 controls the entire computer, for example, by running an operating system. The processor 1001 may consist of a central processing unit (CPU) that includes interfaces with peripheral devices, control units, arithmetic units, registers, etc.

[0034] Furthermore, the processor 1001 reads programs (program code), software modules, data, etc., from at least one of the storage 1003 and the communication device 1004 into the memory 1002 and executes various processes accordingly. The program used is one that causes the computer to execute at least a part of the operations described in the above embodiment. Although the above processes have been described as being executed by one processor 1001, they may be executed simultaneously or sequentially by two or more processors 1001. The processor 1001 may be implemented by one or more chips. The program may also be transmitted from a network via a telecommunications line.

[0035] Memory 1002 is a computer-readable recording medium and may consist of at least one of the following: ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), RAM (Random Access Memory), etc. Memory 1002 may also be called a register, cache, main memory, etc. Memory 1002 can store executable programs (program code), software modules, etc., for carrying out a wireless communication method according to one embodiment of the present disclosure.

[0036] Storage 1003 is a computer-readable recording medium and may consist of at least one of the following: an optical disc such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disc, a digital multipurpose disc, a Blu-ray® disc), a smart card, flash memory (e.g., a card, a stick, a key drive), a floppy® disk, a magnetic strip, etc. Storage 1003 may also be called an auxiliary storage device. The above-mentioned storage medium may be, for example, a database, server, or other suitable medium including at least one of memory 1002 and storage 1003.

[0037] The communication device 1004 is hardware (transceiver / receiver device) for communicating between computers via at least one of a wired network and a wireless network, and is also referred to as a network device, network controller, network card, communication module, etc. The communication device 1004 may be configured to include, for example, a high-frequency switch, duplexer, filter, frequency synthesizer, etc., in order to implement at least one of frequency division duplex (FDD) and time division duplex (TDD).

[0038] The input device 1005 is an input device that accepts input from an external source (e.g., a keyboard, mouse, microphone, switch, button, sensor, etc.). The output device 1006 is an output device that outputs to an external source (e.g., a display, speaker, LED lamp, etc.). The input device 1005 and the output device 1006 may be configured as an integrated unit (e.g., a touch panel).

[0039] Furthermore, each device, such as the processor 1001 and memory 1002, is connected by a bus 1007 for communicating information. The bus 1007 may be configured using a single bus, or different buses may be configured for each device.

[0040] Furthermore, the information processing device 10 may be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array), and some or all of each functional block may be realized by such hardware. For example, the processor 1001 may be implemented using at least one of these hardware components.

[0041] Information notification is not limited to the embodiments described herein and may be carried out by other means. For example, information notification may be carried out by physical layer signaling (e.g., DCI (Downlink Control Information), UCI (Uplink Control Information)), upper layer signaling (e.g., RRC (Radio Resource Control) signaling, MAC (Medium Access Control) signaling, broadcast information (MIB (Master Information Block), SIB (System Information Block))), other signals, or combinations thereof. RRC signaling may also be called RRC messages, and may be, for example, RRC Connection Setup messages, RRC Connection Reconfiguration messages, etc.

[0042] Each aspect / embodiment described in this disclosure includes LTE (Long Term Evolution), LTE-A (LTE-Advanced), SUPER 3G, IMT-Advanced, 4G (4th generation mobile communication system), 5G (5th generation mobile communication system), 6th generation mobile communication system (6G), xth generation mobile communication system (xG) (xG (where x is, for example, an integer or decimal)), FRA (Future Radio Access), NR (new Radio), New radio access (NX), Future generation radio access (FX), W-CDMA (registered trademark), GSM (registered trademark), CDMA2000, UMB (Ultra Mobile Broadband), IEEE 802.11 (Wi-Fi (registered trademark)), IEEE 802.16 (WiMAX (registered trademark)), and IEEE This may apply to at least one system utilizing 802.20, UWB (Ultra-WideBand), Bluetooth®, or other appropriate systems, and to next-generation systems extended, modified, created, or defined based thereon. It may also apply to a combination of multiple systems (for example, a combination of at least one of LTE and LTE-A with 5G).

[0043] The processing procedures, sequences, flowcharts, etc., of each aspect / embodiment described herein may be reordered, provided they are consistent with each other. For example, the methods described herein present various step elements in an exemplary order and are not limited to that specific order.

[0044] Input and output information may be stored in a specific location (e.g., memory) or managed using a management table. Input and output information may be overwritten, updated, or appended to. Output information may be deleted. Input information may be transmitted to other devices.

[0045] The determination may be made by a value represented by 1 bit (0 or 1), by a boolean value (true or false), or by a numerical comparison (for example, a comparison with a predetermined value).

[0046] Each aspect / embodiment described herein may be used individually, in combination, or switched between as needed during implementation. Furthermore, notification of specific information (e.g., notification that "X is") is not limited to explicit notification, but may also be implicit (e.g., by not providing such notification).

[0047] Although the present disclosure has been described in detail above, it will be clear to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure can be implemented in modified and altered forms without departing from the intent and scope of the present disclosure as defined by the claims. Therefore, the descriptions in the present disclosure are illustrative and not intended to be restrictive in any way.

[0048] Software should be broadly interpreted to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, functions, and so on, whether they are called software, firmware, middleware, microcode, hardware description languages, or by any other name.

[0049] Furthermore, software, instructions, information, etc., may be transmitted and received via a transmission medium. For example, if software is transmitted from a website, server, or other remote source using at least one of wired technology (such as coaxial cable, fiber optic cable, twisted pair, or digital subscriber line (DSL)) and wireless technology (such as infrared or microwave), then at least one of these wired and wireless technologies is included in the definition of a transmission medium.

[0050] The information, signals, etc. described in this disclosure may be represented using any of the various different techniques. For example, the data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.

[0051] In addition, terms used in this disclosure and terms necessary for understanding this disclosure may be replaced with terms having the same or similar meanings. For example, at least one of the communication channel and the symbol may be a signal (signaling). Also, the signal may be a message. Furthermore, the component carrier (CC) may be called a carrier frequency, cell, frequency carrier, etc.

[0052] The terms “system” and “network” as used in this disclosure are interchangeable.

[0053] Furthermore, the information, parameters, etc., described in this disclosure may be expressed using absolute values, relative values ​​from a given value, or other corresponding information. For example, wireless resources may be indicated by an index.

[0054] The names used for the parameters described above are not restrictive in any way. Furthermore, the formulas and other expressions using these parameters may differ from those expressly disclosed in this disclosure. Various communication channels (e.g., PUCCH, PDCCH, etc.) and information elements can be identified by any suitable name, and therefore, the various names assigned to these various communication channels and information elements are not restrictive in any way.

[0055] As used in this disclosure, the terms “determining” and “determining” may encompass a wide variety of actions. “Determining” may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up, searching, inquiry (e.g., searching in a table, database, or other data structure), and ascertaining. “Determining” may also include, for example, receiving (e.g., receiving information), transmitting (e.g., sending information), input, output, and accessing (e.g., accessing data in memory). Furthermore, "judgment" and "decision" can include considering something as having been "judged" or "decided" after resolving, selecting, choosing, establishing, comparing, etc. In other words, "judgment" and "decision" can include considering something as having been "judged" or "decided" after some action. Also, "judgment (decision)" can be reinterpreted as "assuming," "expecting," or "considering."

[0056] In this disclosure, the phrase "based on" does not mean "based solely on" unless otherwise specified. In other words, the phrase "based on" means both "based solely on" and "based at least on."

[0057] Any reference to elements using the designations “first,” “second,” etc., as used in this disclosure does not generally limit the quantity or order of those elements. These designations may be used in this disclosure as a convenient way to distinguish between two or more elements. Accordingly, references to the first and second elements do not imply that only two elements may be employed, or that the first element must precede the second element in any way.

[0058] Where the terms “include,” “including,” and variations thereof are used in this disclosure, these terms are intended to be inclusive, as is the term “comprising.” Furthermore, the term “or” as used in this disclosure is not intended to mean exclusive OR.

[0059] In this disclosure, if articles are added through translation, such as a, an, and the in English, this disclosure may include the fact that the noun following these articles is plural.

[0060] In this disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "combine" may be interpreted similarly to "different." [Explanation of Symbols]

[0061] 10... Information processing unit, 11... Extraction unit, 12... Comparison unit, 13... Correspondence derivation unit, 14... Information presentation unit, 15... Dialogue information storage unit, 1001... Processor, 1002... Memory, 1003... Storage, 1004... Communication device, 1005... Input device, 1006... Output device, 1007... Bus.

Claims

1. An extraction unit that extracts text information representing sound from images published in a comic, for each predetermined comic processing unit, A comparison unit compares the extracted text information for each comic processing unit with pre-prepared text information representing sounds in anime scenes and calculates the similarity. A correspondence derivation unit that determines the correspondence between anime scenes and comic processing units based on the calculated similarity, Equipped with, The text information for each comic processing unit and the text information representing sound in the animation scene are text information representing dialogue. The extraction unit estimates the order of dialogue based on information including the position information of panels and dialogue obtained from the comic processing unit, and extracts text information for each comic processing unit by combining multiple lines of dialogue based on the obtained order of dialogue. Information processing device.

2. An extraction unit that extracts text information representing sound from images published in a comic, for each predetermined comic processing unit, A comparison unit compares the extracted text information for each comic processing unit with pre-prepared text information representing sounds in anime scenes and calculates the similarity. A correspondence derivation unit that determines the correspondence between anime scenes and comic processing units based on the calculated similarity, Equipped with, The text information for each comic processing unit and the text information representing sound in the animation scene are text information representing dialogue. The extraction unit estimates the identity of the speaker of the dialogue based on either or both of the speech bubble containing the dialogue and the speaker's position information obtained from the comic processing unit, and the semantic connections between the dialogues. Based on the obtained speaker identity, it combines multiple dialogues to extract text information for each comic processing unit. Information processing device.

3. Based on the correspondence between the anime scene and the comic processing unit derived by the aforementioned correspondence derivation unit, The presentation of corresponding animated video information of the aforementioned anime scenes on a webpage for the commercial sale of the aforementioned comic, and, To provide a link to a webpage introducing the aforementioned anime scene, to provide link information to a webpage for commercializing comics that include the corresponding comic processing unit. An information display unit that performs at least one of the following: The information processing apparatus according to claim 1 or 2, further comprising:

Images