Information processing apparatus and information processing method

Abstract

Provided are an information processing apparatus and an information processing method that appropriately support the division of content such as manga and reduce man-hours. 【Solution means】The information processing apparatus 10 includes an input unit 11 that receives an input of manga, a model storage unit 13 that stores a learned model obtained by pre-learning, as teacher data, the feature amounts of each page constituting the manga and the pages into which the manga is divided, and an inference unit 12 that infers a score at which the manga is divided on each page based on the feature amounts of each page constituting the manga and the learned model, and an output unit 15 that presents information on division candidate pages based on the score.

Description

Technical Field

[0001] The present invention relates to a technique for assisting in the division of manga content.

Background Art

[0002] Conventionally, a system for dividing and selling e-books into microcontents consisting of unit articles has been known (Patent Document 1). In the invention described in Patent Document 1, microcontents are generated by dividing them into units such as chapters and sections. In addition, for electronic manga, sales are actively carried out in units of the story.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the case of electronic manga, there is a need to divide it into units smaller than the units of each story separated by the author. In response to this need, for example, it is also conceivable to divide it quantitatively, such as every 10 pages. However, with quantitative division, the story may be interrupted at uncomfortable places, and there is a risk that the purchasing desire for the work cannot be evoked.

[0005] Due to the above circumstances, the division of content such as manga has been carried out manually, but there has been a problem of high man-hours. Recently, since a huge number of works are published, it has been difficult to manually divide such works.

[0006] The present invention has been made in view of the above background, and an object thereof is to appropriately assist in the division of manga content and reduce man-hours.

Means for Solving the Problems

[0007] The information processing apparatus of the present invention includes an input unit that receives input of a manga, a storage unit that stores a learned model that has previously learned, as teacher data, feature amounts of each page constituting the manga and the pages into which the manga is divided, and an inference unit that infers, based on the feature amounts of each page constituting the manga and the learned model, a score at which the manga is divided on each page, and an output unit that presents information on candidate divided pages based on the score.

Effect of the Invention

[0008] According to the present invention, it is possible to appropriately support the division of a manga.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Modes for Carrying Out the Invention

[0010] Hereinafter, the information processing apparatus according to the present embodiment will be described with reference to the drawings. Note that the following description shows only an example of a preferred embodiment and is not intended to limit the invention described in the claims. The information processing apparatus according to the present embodiment is an apparatus that supports the microcontent conversion of manga, and presents candidates for locations where the content is to be divided. Here, "manga" is a story in the form of visual storytelling having frames with illustrations drawn. Although the term "manga" is used in this specification, the term "manga" includes content having a similar form (for example, comics, comic books, etc.).

[0011] (First Embodiment) FIG. 1 is a diagram showing the functional configuration of the information processing apparatus 10 according to the first embodiment, and FIG. 2 is a diagram showing the hardware configuration of the information processing apparatus 10. First, the hardware configuration of the information processing apparatus 10 will be described with reference to FIG. 2. Physically, the information processing apparatus 10 is configured as a computer including a processor 101, a memory 102, a storage 103, a communication device 104, an input device 105, an output device 106, and a bus connecting these. Each of these devices operates by power supplied from a power source (not shown). In the following description, the term "device" can be read as a circuit, a device, a unit, etc. The hardware configuration of the information processing apparatus 10 may be configured to include one or more of each device shown in FIG. 2, or may be configured without including some devices. Further, a plurality of devices having different housings may be communicatively connected to constitute the information processing apparatus 10.

[0012] Each function in the information processing apparatus 10 is realized by causing the processor 101 to read a predetermined software (program) onto hardware such as the processor 101 and the memory 102, so that the processor 101 performs calculations, controls communication by the communication device 104, and controls at least one of reading and writing of data in the memory 102 and the storage 103.

[0013] Processor 101 controls the entire computer by operating, for example, an operating system. Processor 101 may be constituted by a central processing unit (CPU: Central Processing Unit) including an interface with peripheral devices, a control device, an arithmetic device, registers, and the like. Also, for example, a baseband signal processing unit, a call processing unit, etc. may be realized by Processor 101.

[0014] Processor 101 reads programs (program codes), software modules, data, etc. from at least one of storage 103 and communication device 104 into memory 102, and executes various processes according to these. As the program, a program for causing a computer to execute at least a part of the operations described later is used. The functional blocks of information processing apparatus 10 may be stored in memory 102 and realized by a control program operating in Processor 101. Various processes may be executed by one Processor 101, or may be executed simultaneously or sequentially by two or more Processors 101. Processor 101 may be implemented by one or more chips. Note that the program may be transmitted to information processing apparatus 10 via a telecommunication line.

[0015] Memory 102 is a computer-readable recording medium and may be constituted by, for example, at least one of ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), RAM (Random Access Memory), etc. Memory 102 may be referred to as a register, a cache, a main memory (main storage device), etc. Memory 102 can store programs (program codes), software modules, etc. executable for implementing the method according to this embodiment.

[0016] Storage 103 is a computer-readable recording medium and may be composed of, for example, at least one of an optical disk such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disc, a digital versatile disc, a Blu-ray (registered trademark) disc), a smart card, a flash memory (e.g., a card, a stick, a key drive), a floppy (registered trademark) disk, a magnetic strip, etc. Storage 103 may also be referred to as an auxiliary storage device.

[0017] Communication device 104 is hardware (a transceiver device) for performing communication between computers via at least one of a wired network and a wireless network, and is also referred to as, for example, a network device, a network controller, a network card, a communication module, etc. Each device such as processor 101 and memory 102 is connected by a bus for communicating information. The bus may be configured using a single bus or may be configured using different buses for each device.

[0018] Information processing device 10 may be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), a field programmable gate array (FPGA), etc., and some or all of each functional block may be realized by the hardware. For example, processor 101 may be implemented using at least one of these hardware components.

[0019] Next, the functional configuration of the information processing apparatus 10 will be described. As shown in FIG. 1, the information processing apparatus 10 includes an input unit 11, an inference unit 12, a model storage unit 13, a division candidate search unit 14, and an output unit 15. The input unit 11 has a function of receiving the input of the content data of the manga. The input unit 11 also receives data specifying the range of the number of pages of the divided manga. For example, when it is desired to divide into micro contents consisting of 10 to 15 pages, the user inputs data of a lower limit value of 10 pages and an upper limit value of 15 pages. Note that the data specifying the page number range received by the input unit 11 is stored in the memory 102 or the storage 103. Thereby, the user does not need to input the data specifying the page number range every time. The user only needs to input data from the input unit 11 only when it is desired to change the range specification.

[0020] FIG. 3 is a diagram showing an outline of the processing performed by the inference unit 12. The inference unit 12 has a function of calculating a score for the input manga to be divided into each page by inference using the learned model stored in the model storage unit 13. The score is data obtained by quantifying the likelihood of the manga being divided into each page. The score can also be said to be a score indicating the appropriateness of being divided at that page. The inference unit 12 obtains a score for each page. The learned model is a model generated using manga that has been made into micro contents in the past as teacher data. As the teacher data, it is desirable to use excellent micro contents that have been read a lot in the past.

[0021] FIG. 4 is a diagram showing the configuration of the learned model stored in the model storage unit 13. The learned model is a model that takes the feature amount of a page as input and outputs a score for the manga to be divided at that page. Examples of the feature amount include the expressions of the characters appearing on the corresponding page, the content of the dialogue, the amount of dialogue, information about the page, the number of frames, and the like. It is not necessary to use all of these data, but at least one or more of them are used.

[0022] When using the expression of a character as a feature quantity, as an example, an emotion class (such as joy, sadness, surprise, anger, etc.) can be classified from an image of the character's face using a pre-trained emotion recognition model, and the emotion class can be used as a feature quantity. When using the content of the dialogue as a feature quantity, as an example, data vectorized using techniques such as Bag of Words or TF-IDF can be used as a feature quantity. The quantity of dialogue is data on the number of characters of the dialogue within a page. Information regarding a page is data on the total number of pages of the input content data and which page number the corresponding page is. The number of frames is the number of frames included in the corresponding page.

[0023] Here, the method of generating the learned model will be explained. As teacher data, use the content data of the manga before being micro-contentified and the data of the pages obtained by dividing the manga. Calculate the feature quantity of each page of the content data before division, apply the feature quantity of each page to the learned model for inference, and calculate the score of each page. On the other hand, using the correct score where the score of the actually divided page is "1" and the score of other pages is "0", the model is learned by backpropagating the error so as to minimize the error between the score obtained by inference and the correct score.

[0024] The inference unit 12 obtains the feature quantity of each page of the input manga content data, and sequentially obtains the score of each page by applying the feature quantity of the page to the learned model one page at a time. Note that the inference unit 12 does not perform inference from (i) the first page of the manga to the page advanced by the number of pages of the lower limit specified by range designation and (ii) from the page retrogressed by the number of pages of the lower limit specified by range designation from the last page of the manga to the last page. Since these pages do not become candidates for division due to the range designation conditions, by excluding them from the inference process, the computational load on the information processing apparatus 10 is reduced.

[0025] The split candidate search unit 14 has a function of searching for split candidate pages based on the score of each page obtained by the inference unit 12 and the range specification data of the number of pages. The split candidate search unit 14 searches for pages that have a score equal to or higher than a predetermined value and that satisfy the set range specification as split candidate pages.

[0026] The output unit 15 outputs data indicating the split candidate pages searched by the split candidate search unit 14 from the output device 105. As a result, the user can select the page to be split from the output split candidate pages. Since the user does not need to read all the manga to find the split location, the work efficiency of converting manga into micro content can be improved. Note that the output unit 15 may output the score of the page in addition to the data indicating the split candidate pages. The score helps the user select the split location.

[0027] In the case of a manga in a page view format similar to a paper manga, a frame may be represented using a double-page spread. In such a case, it is not preferable for the manga to be split at the double-page spread. Therefore, it is desirable for the split location to be an even page, and odd pages can also be excluded from the split candidates in advance. Specifically, odd pages may be excluded from the inference target in the inference by the inference unit 12, or odd pages may be excluded from the split candidates when the split candidate search unit 14 searches for split candidate pages.

[0028] FIG. 5 is a flowchart showing the operation of the information processing apparatus 10 according to the first embodiment. First, the information processing apparatus 10 receives an input of setting the range of the number of pages of the content after splitting (S10). Data on the upper limit value and the lower limit value of the page range of the content is received. The information processing apparatus 10 stores the input data in a memory or a storage. If range specification data is already stored in the memory or the storage, the data is overwritten. This step S10 may be performed when the page range needs to be changed, and does not necessarily need to be performed every time.

[0029] The information processing apparatus 10 receives the input of the content data of the manga to be segmented (S11), and calculates the feature amount of each page (S12). As described above, as the feature amount, the facial expression of the characters on the corresponding page, the content of the dialogue, the amount of dialogue, the information regarding the page, the number of frames, etc. are used. Which feature amount to use is determined by which feature amount the learned model was trained with, and the same feature amount as the input of the learned model is calculated.

[0030] The information processing apparatus 10 sequentially applies the feature amount of each page to the learned model, and calculates the score of each page by inference (S13). Subsequently, the information processing apparatus 10 searches for the candidate pages for segmentation based on the calculated scores of each page and the data specifying the range of the number of pages after segmentation (S14), and outputs the data of the candidate pages for segmentation (S15).

[0031] According to the information processing apparatus 10 of the present embodiment, since the pages that are candidates for segmentation are output based on various feature amounts of the manga to be segmented, the user only needs to select the pages to be segmented from among the candidates, so that the manga can be segmented efficiently.

[0032] (Second Embodiment) FIG. 6 is a diagram showing the functional configuration of the information processing apparatus according to the second embodiment. The information processing apparatus according to the second embodiment is different in that, in addition to the configuration of the information processing apparatus 10 according to the first embodiment, it further includes a preprocessing unit 16. Further, the input unit 11 of the information processing apparatus according to the second embodiment receives the input of the metadata of the manga in addition to the content data of the manga.

[0033] Metadata includes various types of data such as titles, authors, genres, basic data such as serial magazines, and formal data such as volume numbers and page numbers. In this embodiment, the metadata includes data related to the pages of the manga. Data related to pages refers to data indicating whether a page is a title page, a character introduction page, a color page, an extra page, or a page of another work. Here, another work refers to, for example, a teaser of another work by the same author that may be included in the remaining pages when there are extra pages in a single-volume book or the like that cannot be filled by the main story of the manga alone. All of the pages exemplified here are pages that are preferably not divided when the manga is converted into microcontents.

[0034] The preprocessing unit 16 has a function of identifying pages that should not be divided based on the input metadata. That is, if a page is any of a title page, a character introduction page, a color page, an extra page, or a page of another work, it is determined that the page cannot be divided. The pages listed as non-divisible here are examples, and pages other than those listed here may also be determined as non-divisible. The preprocessing unit 16 inputs data indicating non-divisible pages to the inference unit 12. As a result, the inference unit 12 can omit the inference process for non-divisible pages and reduce the computational load.

[0035] FIG. 7 is a flowchart showing the operation of the information processing apparatus according to the second embodiment. The information processing apparatus first accepts an input for setting the range of the number of pages of the content after division. It accepts data on the upper limit value and the lower limit value of the page range of the content (S20). The information processing apparatus stores the input data in the memory 102 or the storage 103. If range specification data is already stored in the memory 102 or the storage 103, the data is overwritten.

[0036] The information processing apparatus receives input of content data and metadata of the manga to be divided (S21). The information processing apparatus identifies pages that cannot be divided based on the metadata (S22). Subsequently, the information processing apparatus calculates the feature amounts of each page excluding the pages that cannot be divided (S23), sequentially applies the calculated feature amounts of each page to the learned model, and calculates the score of each page by inference (S24). The information processing apparatus searches for candidate pages for division based on the calculated scores of each page and the data specifying the range of the number of pages after division (S25), and outputs the data of the candidate pages for division (S26).

[0037] Similar to the information processing apparatus of the first embodiment, the information processing apparatus of the second embodiment can efficiently divide the manga. Also, since pages that cannot be divided are specified in advance based on the metadata in the information processing apparatus of the second embodiment, pages that cannot be divided are not output as candidates for division.

[0038] (Third Embodiment) FIG. 8 is a diagram showing the functional configuration of the information processing apparatus of the third embodiment. After outputting the data of the candidate pages for division, the information processing apparatus of the third embodiment receives input of the data of the division pages selected by the user, and retrains the learned model stored in the model storage unit 13. The information processing apparatus of the third embodiment further includes a learning unit 17 in addition to the configuration of the information processing apparatus 10 of the first embodiment.

[0039] The learning unit 17 performs learning of the learned model stored in the model storage unit 13. The learning of the model is performed by backpropagation of error that minimizes the error between the inference result and the correct data. In the information processing apparatus of the third embodiment, since the scores for each page of the content data have been calculated in order to search for candidate pages for division, learning can be performed efficiently.

[0040] As described above, the embodiments of the information processing apparatus of the present invention have been described in detail with examples, but the information processing apparatus of the present invention is not limited to the above-described embodiments.

[0041] In the above-described embodiment, an information processing apparatus that outputs data of a page to be a splitting candidate was described as an example. However, the page with the highest score within the range satisfying the range specification of the number of pages may be presented as a splitting location.

[0042] In the above-described embodiment, an example was given in which the inference unit applies the data of the feature amount page by page to the learned model to obtain the score of each page. However, when the inference unit obtains the score of a predetermined page, it may infer the score using the data of several pages before and after. For example, when obtaining the score of the Nth page, the feature amounts from the (N - 2)th page to the (N + 2)th page may be calculated, and the feature amounts may be applied to the learned model to calculate the score of the Nth page. Thereby, the score of the Nth page can be appropriately calculated in consideration of the context of the preceding and succeeding pages.

[0043] Regarding the embodiment described above, the following additional remarks are further described. [Supplementary Note 1] The information processing apparatus of Supplementary Note 1 is an information processing apparatus that supports the micro-content conversion of manga, and includes an input unit that receives an input of manga, a storage unit that stores a learned model that has been previously learned using the feature amounts of each page constituting the manga and the pages into which the manga has been split as teacher data, an inference unit that infers the score at which the manga is split on each page based on the feature amounts of each page constituting the manga and the learned model, and an output unit that presents information on the splitting candidate pages based on the score. With this configuration, pages of splitting candidates suitable for splitting are output, so that the work of converting manga into micro-content can be appropriately supported.

[0044] [Supplementary Note 2] In the information processing apparatus of Supplementary Note 1, the feature amount may be at least one or more pieces of information among the expression of a character, the content of a dialogue, the amount of dialogue, information regarding a page, and the number of frames.

[0045] [Supplementary Note 3] In the information processing apparatus according to Supplementary Note 1 or 2, the input unit may receive a specification of the range of the number of pages of the divided manga, and the output unit may output information on the candidate pages for division that satisfy the range specification. With this configuration, the manga can be divided into an appropriate number of pages. For example, although the number of pages may be set according to the offering price of the manga, this configuration can assist in generating microcontents of an appropriate length according to the price.

[0046] [Supplementary Note 4] In the information processing apparatus according to Supplementary Note 3, the inference unit may be configured not to perform inference from the first page of the manga to the page advanced by the number of pages of the lower limit specified by the range specification, or from the page retrogressed by the number of pages of the lower limit specified by the range specification from the last page of the manga to the last page. By excluding in advance the pages that are not subject to division due to the relationship of the range specification of the number of pages, the computational load on the information processing apparatus can be reduced.

[0047] [Supplementary Note 5] The information processing apparatus according to any one of Supplementary Notes 1 to 4 may include a preprocessing unit that includes metadata regarding each page of the content received by the input unit and identifies pages that should not be divided based on the metadata. When pages that are not candidates for division are known as prior knowledge, the computational load on the information processing apparatus can be reduced by excluding such pages from the division candidates in advance.

[0048] [Supplementary Note 6] In the information processing apparatus according to any one of Supplementary Notes 1 to 5, the output unit may present even-numbered pages as candidate pages for division. With this configuration, it is possible to prevent the inconvenience that the manga is divided on facing pages.

[0049] [Supplementary Note 7] In any of the information processing apparatuses according to Supplementary Note 1 to Supplementary Note 6, the input unit may include a learning unit that receives an input of a divided page from among the presented divided candidate pages and retrains the learned model using the data of the divided page. With this configuration, the accuracy of the learned model can be improved.

[0050] [Supplementary Note 8] The information processing method according to Supplementary Note 8 is an information processing method for assisting in the micro-content conversion of manga, and includes a step in which an information processing apparatus receives an input of manga, and a step in which the information processing apparatus infers a score for dividing the manga on each page based on a learned model that has been pre-trained using, as teacher data, the feature amounts of the pages constituting the manga and the feature amounts of the pages constituting the manga and the pages into which the manga has been divided, and a step in which the information processing apparatus presents information on divided candidate pages based on the score. With this configuration, pages of divided candidates suitable for division are output, so that the work of converting manga into micro-content can be appropriately assisted.

Explanation of Signs

[0051] 10 Information processing apparatus 11 Input unit 12 Inference unit 13 Model storage unit 14 Divided candidate search unit 15 Output unit 16 Preprocessing unit 17 Learning unit

Claims

1. An input unit that accepts the input of a manga and the specification of the range of the number of pages of the manga after division; A storage unit that stores a learned model that has been pre-learned using, as teacher data, the feature amounts of the respective pages constituting the manga and the pages into which the manga has been divided; An inference unit that infers, based on the feature amounts of the respective pages constituting the manga and the learned model, the score at which the manga is divided on each page; An output unit that presents information on division candidate pages that satisfy the range specification based on the score; An information processing apparatus comprising the above.

2. The information processing apparatus according to claim 1, wherein the feature amounts are at least one or more pieces of information among the expression of a character, the content of a dialogue, the amount of dialogue, information regarding a page, and the number of frames.

3. The information processing apparatus according to claim 1, wherein the inference unit does not perform inference from the first page of the manga to the page advanced by the number of pages of the lower limit specified by the range specification, or from the page retrogressed by the number of pages of the lower limit specified by the range specification from the last page of the manga to the last page.

4. The content received by the input unit includes metadata regarding each page of the content, The information processing apparatus according to claim 1, further comprising a preprocessing unit that identifies pages that should not be divided based on the metadata.

5. The information processing apparatus according to claim 1, wherein the output unit presents even-numbered pages as division candidate pages.

6. The input unit accepts the input of a division page from among the presented division candidate pages, The information processing apparatus according to claim 1, further comprising a learning unit that re-learns the learned model using the data of the division page.

7. A step in which an information processing apparatus accepts the input of a manga and the specification of the range of the number of pages of the manga after division; A step in which the information processing apparatus infers, based on the feature amounts of the pages constituting the manga and a learned model that has been pre-learned using, as teacher data, the feature amounts of the pages constituting the manga and the pages into which the manga has been divided, the score at which the manga is divided on each page; A step in which the information processing apparatus presents information on division candidate pages that satisfy the range specification based on the score; An information processing method comprising the above.

Images