Coloring video display system

Abstract

We provide a coloring video display system that allows users to enjoy coloring. [Solution] In the coloring video display system 1, the server device 10 generates a character image including a coloring pattern image based on image data obtained when the user terminal 20 images the sheet 30 so that an outline defining the area to be colored is drawn and a marker containing pattern identification information that can identify the coloring pattern corresponding to the outline is drawn, and a character image including a user image based on image data obtained when the user terminal 20 images the user. The user terminal 20 displays a video using the character images generated by the server device 10.

Description

Technical Field

[0001] The present invention relates to a coloring video display system.

Background Art

[0002] Conventionally, a system for displaying a user's handwritten picture while animating it on a screen is known.

[0003] For example, in Patent Document 1, a card having a handwritten part (object) is imaged by an imaging unit for display processing, and an object image corresponding to the object is displayed on the display unit of the display device in a predetermined size. It is described that the object image is synthesized with a diorama, illustration, or image installed on a support base using a half mirror for expression. In this display system, the object image is operated on the display unit based on the attribute information included in the card and a predetermined area ratio of the object handwritten on the card.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] An object of the present invention is to provide a coloring video display system that can enjoy coloring.

Means for Solving the Problems

[0006] According to one aspect of the present invention, a coloring video display system is a coloring video display system that displays a video using coloring, comprising: a sheet on which an outline defining the area to be colored is drawn, and a marker containing pattern identification information that can identify a coloring pattern corresponding to the outline is attached around the outline; an imaging data storage unit that stores imaging data obtained by imaging the sheet including the marker; a correction unit that corrects the plane of the imaging data obtained by imaging the sheet based on the marker; an identification information identification unit that identifies the pattern identification information contained in the marker based on the imaging data obtained by imaging the sheet; and a coloring pattern identification unit that identifies the coloring pattern based on the pattern identification information identified by the identification information identification unit. The system includes: a pattern information identification unit that identifies the contour and the position of the contour corresponding to the coloring pattern identified by the coloring pattern identification unit; a pattern image generation unit that generates a coloring pattern image by trimming the contour to include the inside of the contour based on the contour and the position of the contour identified by the pattern information identification unit; a character image generation unit that generates a character image including the coloring pattern image generated by the pattern image generation unit; a video data generation unit that generates video data for playing a video that uses the character image including the coloring pattern image; and a video display unit that displays a video using the character image including the coloring pattern image based on the video data generated by the video data generation unit. [Effects of the Invention]

[0007] According to the present invention, a coloring video display system that allows users to enjoy coloring can be provided. [Brief explanation of the drawing]

[0008] [Figure 1] This is an example of a system configuration diagram relating to one embodiment of the present invention. [Figure 2] This is an example of a hardware configuration diagram relating to one embodiment of the present invention. [Figure 3] This is an example of a sheet relating to one embodiment of the present invention. [Figure 4] This is an example of a functional block diagram relating to one embodiment of the present invention. [Figure 5] This is an example of data in a data table relating to one embodiment of the present invention. [Figure 6] This is an example of a processing flowchart related to one embodiment of the present invention. [Figure 7] This is an example of a processing flowchart related to one embodiment of the present invention. [Figure 8] This is an example of a screen display of a user terminal according to one embodiment of the present invention. [Figure 9] This is an example of a screen display of a user terminal according to one embodiment of the present invention. [Modes for carrying out the invention]

[0009] The following will be a detailed description with reference to the attached drawings. The drawings show preferred embodiments. However, it is possible to carry out the invention in many different forms and is not limited to the embodiments described herein.

[0010] For example, this embodiment describes the configuration and operation of a coloring video display system, but similar configurations, devices, computer programs, etc., can achieve similar effects. Furthermore, the program may be provided as a non-transitory computer-readable medium, provided for download from an external server, or provided so that the program is launched on an external computer and its functions are realized on a client terminal (so-called cloud computing).

[0011] Furthermore, in this embodiment, "part" may include, for example, hardware resources implemented by circuits in a broad sense, and the information processing of software that can be specifically realized by these hardware resources. In addition, various types of information are handled in this embodiment, and these types of information can be represented, for example, by the physical values ​​of signal values ​​representing voltage and current, the high or low values ​​of signal values ​​as a set of binary bits composed of 0s or 1s, or by quantum superposition (so-called qubits), and communication and calculations can be performed on circuits in a broad sense.

[0012] Furthermore, a circuit in a broad sense is a circuit realized by combining at least a suitable combination of circuits, circuits, processors, and memory. In other words, it includes application-specific integrated circuits (ASICs), programmable logic devices (for example, simple programmable logic devices (SPLDs), complex programmable logic devices (CPLDs), and field programmable gate arrays (FPGAs)), etc.

[0013] <System Configuration> Figure 1 is a block diagram showing the configuration of a system according to one embodiment. The coloring video display system 1 of this embodiment is configured to allow users to enjoy coloring.

[0014] As shown in Figure 1, the coloring video display system 1 is a system that creates videos using coloring pages. The coloring video display system 1 comprises a server device 10, a user terminal 20, and a sheet 30.

[0015] The server device 10 and the user terminal 20 are configured to be communicable via the communication network 2. In this embodiment, the communication network 2 is an IP (Internet Protocol) network, but there is no limitation on the type of communication protocol, and further, there is no limitation on the type and scale of the network.

[0016] On the sheet 30, a contour defining the range for coloring is drawn, and a marker including pattern identification information capable of identifying the coloring pattern corresponding to the contour is attached around the contour. In this embodiment, paper is taken as an example of the sheet 30, but other members such as plastic and wood can be used as the sheet 30.

[0017] In this embodiment, the user terminal 20 is used to image the sheet 30 so as to include the marker. Further, the user terminal 20 is preferably used to image the user. As the user terminal 20, general computers such as smartphones, tablet terminals, personal computers, and wearable devices can be used.

[0018] In this embodiment, the server device 10 generates a character image including a coloring pattern image based on the imaging data obtained by imaging the sheet 30 so that the user terminal 20 includes the marker. Further, the server device 10 preferably generates a character image including a user image based on the imaging data obtained by imaging the user by the user terminal 20. The user terminal 20 displays a video using the character image generated by the server device 10.

[0019] Note that in this embodiment, by installing a video creation program in the server device 10 and the user terminal 20, the server device 10 and the user terminal 20 execute the following processes. However, by installing a video creation program in either the server device 10 or the user terminal 20, the server device 10 or the user terminal 20 may execute the processes performed by the server device 10 and the user terminal 20.

[0020] <Hardware Configuration> Figure 2 shows the hardware configuration in this embodiment.

[0021] [Server device 10] Figure 2(a) shows an example of the hardware configuration of the server device 10. The server device 10 comprises a communication bus 100, a control unit 101, a storage unit 102, and a communication unit 103. The control unit 101, the storage unit 102, and the communication unit 103 are electrically connected within the server device 10 via the communication bus 100.

[0022] The server device 10 may be on-premises or in a cloud environment. In the case of a cloud-based server device 10, for example, it may provide the above-mentioned functions and processing in the form of SaaS (Software as a Service) or cloud computing.

[0023] The control unit 101 performs processing and control of the overall operation related to the server device 10. The control unit 101 is, for example, a Central Processing Unit (CPU). The control unit 101 realizes various functions related to the server device 10 by reading predetermined programs stored in the memory unit 102. That is, information processing by software stored in the memory unit 102 is concretely realized by the control unit 101, which is an example of hardware, so that each functional unit included in the control unit 101 can be executed. Note that the control unit 101 is not limited to being a single unit, and there may be multiple control units 101 for each function. There may also be a combination of such units.

[0024] The storage unit 102 stores various programs, variables, etc., related to the server device 10 executed by the control unit 101. As the storage unit 102, for example, a storage device such as a solid-state drive (SSD), hard disk drive (HDD), or read-only memory (ROM) can be used to store various programs, etc., related to the server device 10 executed by the control unit 101, or a memory such as random access memory (RAM) can be used to store temporarily necessary information (arguments, arrays, etc.) related to program calculations.

[0025] The communication unit 103 is used for the server device 10 to communicate various types of information with the outside world via the communication network 2. The communication unit 103 can utilize wired communication methods such as USB, IEEE1394, Thunderbolt®, and wired LAN network communication, as well as wireless communication methods such as wireless LAN network communication, 3G / LTE / 5G mobile communication, and Bluetooth® communication. In other words, it is more preferable to implement this as a collection of multiple communication methods. That is, the server device 10 may communicate various types of information from the outside world via the communication unit 13 and the network.

[0026] [User terminal 20] Figure 2(b) is a block diagram showing the hardware configuration of the user terminal 20. The user terminal 20 comprises a communication bus 200, a control unit 201, a storage unit 202, a communication unit 203, an input unit 204, an output unit 205, and an imaging unit 206. The control unit 201, storage unit 202, communication unit 203, input unit 204, output unit 205, and imaging unit 206 are electrically connected within the user terminal 20 via the communication bus 200. The explanation of the control unit 201, storage unit 202, and communication unit 203 is the same as the explanation of each part in the server device 10, so it is omitted. As will be described later, the communication unit 203 of the user terminal 20 transmits the user image captured by the imaging unit 206 to the server device 10.

[0027] The input unit 204 receives input operations performed by the user. The input operations are transmitted as command signals to the control unit 201 via the communication bus 200. The control unit 201 can perform predetermined controls or calculations based on the transmitted command signals as needed. The input unit 204 may be included in the casing of the user terminal 20 or it may be an external component. For example, the input unit 204 may be integrated with the output unit 205 to form a touch panel. If the input unit 204 is a touch panel, the user can input tap operations, swipe operations, etc. to the input unit 204. Instead of a touch panel, the input unit 204 can be a switch button, mouse, trackpad, QWERTY keyboard, etc.

[0028] The output unit 205 displays a graphical user interface (GUI) screen that can be operated by the user. The output unit 205 may be included in the casing of the user terminal 20 or it may be an external device. A display device such as a CRT display, liquid crystal display, organic EL display, or plasma display can be used as the output unit 205. It is preferable to use these display devices according to the type of user terminal 20. The output unit 205 also includes a speaker for outputting sound such as background music.

[0029] The imaging unit 206 is an imaging device (camera) configured to capture any visible image. The imaging unit 206 includes an in-camera and an out-camera, and the in-camera and out-camera are switchable. In this embodiment, the imaging unit 206 can capture images of the sheet 30 or user images. The imaging unit 206 may be included in the housing of the user terminal 20 or it may be an external attachment.

[0030] <Sheet 30> Figure 3 shows an example of a coloring book 300 including sheet 30. In Figure 3, the front and back covers of the coloring book 300 are omitted. The first page of the coloring book 300 contains the title 301 and a two-dimensional code 302 such as a QR code (registered trademark). When the user terminal 20 reads the two-dimensional code 302 and accesses the server device 10, the user terminal 20 displays instructions on how to display a video using the coloring book 300, and the user terminal 20 can start creating the video.

[0031] Sheet 30 is bound to pages 2 onwards of coloring book 300. Sheet 30 is rectangular. Markers 303 are placed at the four corners of sheet 30. Markers 303 are square. Outlines 304-306 are drawn inside the four markers 303 to define the area to be colored.

[0032] Outlines 304-306 correspond to coloring patterns. For example, outline 304 corresponds to coloring pattern "Vegetable-1", outline 305 corresponds to coloring pattern "Vegetable-2", and outline 306 corresponds to coloring pattern "Vegetable-3". Marker 303 contains pattern identification information that can identify the coloring patterns corresponding to outlines 304-306.

[0033] The final page of coloring book 300 contains message 307, which prompts the user to take a picture of themselves and display the video.

[0034] <System Functional Configuration> [Server device 10] Figure 4(a) shows the functional configuration of the server device 10. In the server device 10, information processing by software stored in the storage unit 102 is specifically realized by the control unit 101, which is an example of hardware, and can be executed as each functional unit.

[0035] The server device 10 includes an imaging data storage unit 105, a correction unit 106, an identification information identification unit 107, a coloring pattern identification unit 108, a database 109, a pattern information identification unit 110, a pattern image generation unit 111, a character image generation unit 112, and a video data generation unit 115. The server device 10 also preferably includes a user image generation unit 113 and an image storage unit 114.

[0036] The imaging data storage unit 105 stores imaging data obtained by imaging the sheet 30 including the marker 303. Specifically, imaging data obtained by imaging the sheet 30 including the marker 303 using the imaging unit 206 of the user terminal 20 is acquired via the communication unit 203, the communication network 2, and the communication unit 103, and the acquired imaging data is stored.

[0037] The correction unit 106 corrects the image data obtained by imaging the sheet 30 based on the marker 303. Specifically, it corrects the deviation in the planar coordinate position as a plane correction amount, using the position of the marker 303 as a reference.

[0038] The identification information identification unit 107 identifies the pattern identification information contained in the marker 303 based on the imaging data obtained by imaging the sheet 30. Specifically, the pattern identification information is information that can identify the coloring pattern. The pattern identification information is, for example, a two-dimensional code. The identification information identification unit 107 identifies the pattern identification information by identifying the two-dimensional code. It is preferable to use an AR marker as the pattern identification information, but it is also possible to use identification information other than an AR marker, such as a two-dimensional code like a QR code (registered trademark).

[0039] The coloring pattern identification unit 108 identifies the coloring pattern based on the pattern identification information identified by the identification information identification unit 107.

[0040] Specifically, as shown in Figure 5(a), the coloring pattern identification table 109a is stored in the database 109. In the coloring pattern identification table 109a, a coloring pattern is associated with each pattern identification information.

[0041] The coloring pattern identification unit 108 accesses the database 109, searches the coloring pattern identification table 109a for pattern identification information identified by the identification information identification unit 107, and identifies the coloring pattern corresponding to the retrieved pattern identification information.

[0042] The pattern information identification unit 110 identifies the contour and the position of the contour that correspond to the coloring pattern identified by the coloring pattern identification unit 108.

[0043] Specifically, as shown in Figure 5(b), the pattern information identification table 109b is stored in the database 109. In the pattern information identification table 109b, the outlines and their positions are associated for each coloring pattern.

[0044] The pattern information identification unit 110 accesses the database 109, searches the pattern information identification table 109b for the coloring pattern identified by the coloring pattern identification unit 108, and identifies the contour and the position of the contour corresponding to the retrieved coloring pattern.

[0045] The pattern image generation unit 111 generates a coloring pattern image by trimming the contour to include the inside of the contour, based on the contour and the position of the contour identified by the pattern information identification unit 110. Specifically, it extracts the edge information and position of the contour of the coloring pattern, trims the contour and the part including the inside of the contour, and generates a coloring pattern image.

[0046] The character image generation unit 112 generates a character image that includes the coloring pattern image generated by the pattern image generation unit 111. Specifically, it generates a character image by adding a torso, limbs, and face to the coloring pattern image. The data to be added (not shown) is stored in the database 109 in advance. Note that the character image including the coloring pattern image may be generated by a method different from that of this embodiment, such as by deforming the coloring pattern image.

[0047] The imaging data storage unit 105 stores imaging data obtained by imaging the user. Specifically, the imaging data storage unit 105 acquires imaging data obtained by imaging the user using the imaging unit 206 of the user terminal 20 via the communication unit 203, the communication network 2, and the communication unit 103, and stores the acquired imaging data.

[0048] The user image generation unit 113 generates a user image that fits within a predetermined frame based on the image data obtained by capturing a user. Specifically, it detects the faces of people included in the image data and generates a user image by fitting the detected faces within a predetermined frame size.

[0049] The character image generation unit 112 generates a character image that includes the user image generated by the user image generation unit 113. Specifically, it generates the character image by adding a torso and limbs to the user image. The data to be added (not shown) is stored in the database 109 in advance. Note that the character image including the user image may be generated by a method different from that of this embodiment, such as by deforming the user image.

[0050] The image storage unit 114 stores character images including coloring pattern images and character images including user images. Specifically, when the image storage unit 114 receives a save operation using the input unit 204 of the user terminal 20, it stores image display data for displaying the character image including the coloring pattern image or the character image including the user image.

[0051] The video data generation unit 115 generates video data for playing a video that uses character images, including coloring pattern images, generated by the character image generation unit 112. The video data includes image data for displaying the video using the character images and audio data for playing background music. The image data includes data for moving the limbs and data for changing facial expressions that have been added to the character images, including the coloring pattern images. Specifically, the video data generation unit 115 generates streaming data for playing a video that uses character images, including the coloring pattern images.

[0052] Furthermore, it is preferable that the video data generation unit 115 generates video data for playing a video that uses character images, including user images, generated by the character image generation unit 112. The video data includes image data for displaying the video that uses the character images and audio data for playing background music. The image data includes data for moving the limbs attached to the character images, including the user images. Specifically, the video data generation unit 115 generates streaming data for playing a video that uses character images, including user images.

[0053] Furthermore, it is preferable that the video data generation unit 115 generates video data for displaying a video in which a character image including a coloring pattern image is added to a character image including a user image. The video data includes image data for displaying a video using the character image and audio data for playing background music. The image data includes data for moving the limbs and changing facial expressions attached to the character image including the coloring pattern image, and data for moving the limbs attached to the character image including the user image. Specifically, the video data generation unit 115 generates streaming data for displaying a video in which a character image including a coloring pattern image is added to a character image including a user image.

[0054] [User terminal 20] Figure 4(b) shows the functional configuration of the user terminal 20. In the user terminal 20, information processing by software stored in the memory unit 202 is specifically realized by the control unit 201, which is an example of hardware, and can be executed as each functional unit.

[0055] The user terminal 20 includes a video display unit 207. Preferably, the user terminal 20 includes a character image display unit 208, a save operation unit 209, a saved image list display unit 210, and a playback operation unit 211.

[0056] The video display unit 207 displays a video using character images, including a coloring pattern image, based on the video data generated by the video data generation unit 115. Preferably, the video display unit 207 displays a video that adds a user image to the character image, including the coloring pattern image, based on the video data generated by the video data generation unit 115. Preferably, the video display unit 207 displays a video that adds a user image to the character image, including the coloring pattern image, based on the video data generated by the video data generation unit 115. Specifically, the video display unit 207 acquires the streaming data generated by the video data generation unit 115 via the communication unit 103, the communication network 2, and the communication unit 203, and outputs a video signal based on the acquired streaming data to the output unit 205.

[0057] The character image display unit 208 displays the character image generated by the character image generation unit 112. Specifically, the character image display unit 208 outputs a video signal to the output unit 205 based on the image display data of the character image acquired via the communication unit 103, the communication network 2, and the communication unit 203.

[0058] The save operation unit 209 accepts save operations to save the character image displayed on the character image display unit 208. Specifically, it accepts predetermined operations on the input unit 204 as save operations.

[0059] The saved image list display unit 210 displays a list of character images saved by the image storage unit 114. Specifically, when a save operation is received by the input unit 204, the image storage unit 114 outputs a video signal to the output unit 205 for displaying a list of character images based on the image display data of the saved character image list acquired via the communication network 2 and the communication unit 203.

[0060] The playback operation unit 211 accepts playback operations for videos that use character images displayed in a list by the saved image list display unit 210. Specifically, it accepts predetermined operations on the input unit 204 as playback operations.

[0061] It should be noted that the arrangement of each functional unit described in this embodiment does not have to be the arrangement exemplified here. In other words, the coloring video display system may be configured by providing at least a portion of the functional units provided on the server device 10 on the user terminal 20. Alternatively, each functional unit may be provided on multiple external devices instead of the server device 10. In other words, various information and programs may be distributed and stored on multiple external devices using blockchain technology or the like.

[0062] <Processing Flow> Next, the video creation method will be explained with reference to Figures 6 and 7. The user reads the two-dimensional code 302 contained in the coloring book 300 using the user terminal 20 and accesses the server device 10. When the server device 10 receives an access request from the user terminal 20, it displays a menu screen on the user terminal 20. On the menu screen, the user can choose to capture an image of the coloring book or capture an image of the user. If the user selects to capture an image of the coloring book on the menu screen, the process shown in Figure 6 is performed, and if the user selects to capture an image of the user on the menu screen, the process shown in Figure 7 is performed.

[0063] [Image of a coloring page] Figure 6 is a processing flowchart for displaying a video using character images, including coloring pattern images, with the coloring video display system 1.

[0064] As shown in Figure 6, when the user captures an image of the sheet 30 including the marker 303 using the user terminal 20 (S1), the user terminal 20 transmits the image data to the server device 10 via the communication network 2 (S2).

[0065] The server device 10 stores the imaging data obtained when the user images the sheet 30 including the marker 303 in the imaging data storage unit 105 (S3).

[0066] Next, the server device 10 corrects the image data obtained by imaging the sheet 30 in the correction unit 106 based on the marker 303 (S4). Alternatively, the image data may be stored after the plane correction.

[0067] Next, the server device 10 identifies the pattern identification information contained in the marker 303 based on the imaging data obtained by imaging the sheet 30 (S5).

[0068] Next, the server device 10 identifies a coloring pattern in the coloring pattern identification unit 108 based on the pattern identification information identified by the identification information identification unit 107 (S6).

[0069] Next, the server device 10, in the pattern information identification unit 110, identifies the contour and the position of the contour that correspond to the coloring pattern identified by the coloring pattern identification unit 108 (S7).

[0070] Next, the server device 10 generates a coloring pattern image in the pattern image generation unit 111 by trimming the contour to include the inside of the contour, based on the contour and the position of the contour identified by the pattern information identification unit 110 (S8).

[0071] Next, the server device 10 generates a character image in the character image generation unit 112, which includes the coloring pattern image generated by the pattern image generation unit 111 (S9).

[0072] Next, the server device 10 transmits the image display data of the character image to the user terminal 20 via the communication network 2 (S10).

[0073] Next, the user terminal 20 displays the character image generated by the character image generation unit 112 on the character image display unit 208 (S11).

[0074] Next, when the user terminal 20 receives an operation from the save operation unit 209 (S12), it sends a save request to the server device 10 via the communication network 2 (S13).

[0075] Next, the server device 10 saves the character image in the image storage unit 114 (S14).

[0076] Next, the server device 10 transmits image display data of the saved list of character images stored in the image storage unit 114 to the user terminal 20 via the communication network 2 (S15).

[0077] Next, the user terminal 20 displays a list of character images saved by the image storage unit 114 on the saved image list display unit 210 (S16).

[0078] Next, when the user terminal 20 receives a playback operation request for a video using character images displayed in the saved image list display unit 210 via the playback operation unit 211 (S17), it sends a video playback request to the server device 10 via the communication network 2 (S18).

[0079] Next, the server device 10 generates video data in the video data generation unit 115 for playing a video that uses character images, including the coloring pattern images generated by the character image generation unit 112 (S19).

[0080] Next, the server device 10 transmits the video data generated by the video data generation unit 115 to the user terminal 20 via the communication network 2 (S20). Then, the user terminal 20 displays the video on the video display unit 207 based on the video data generated by the video data generation unit 115 (S21).

[0081] [User imaging] Figure 7 is a processing flowchart for displaying a video using character images, including a user image, using the coloring video display system 1.

[0082] As shown in Figure 7, when the user captures an image of themselves using the user terminal 20 (S30), the user terminal 20 transmits the image data to the server device 10 via the communication network 2 (S31).

[0083] The server device 10 stores the imaging data obtained by imaging the user in the imaging data storage unit 105 (S32).

[0084] Next, the server device 10 generates a user image within a predetermined frame based on the imaging data obtained by imaging the user in the user image generation unit 113 (S33).

[0085] Next, the server device 10 generates a character image in the character image generation unit 112, which includes the user image generated by the user image generation unit 113 (S34).

[0086] Next, the server device 10 transmits the image display data of the character image to the user terminal 20 via the communication network 2 (S35).

[0087] Next, the user terminal 20 displays the character image generated by the character image generation unit 112 on the character image display unit 208 (S36).

[0088] Next, when the user terminal 20 receives an operation from the save operation unit 209 (S37), it sends a save request to the server device 10 via the communication network 2 (S38).

[0089] Next, the server device 10 saves the character image in the image storage unit 114 (S39).

[0090] Next, the server device 10 transmits a list of character images stored in the image storage unit 114 to the user terminal 20 via the communication network 2 (S40).

[0091] Next, the user terminal 20 displays a list of character images saved by the image storage unit 114 on the saved image list display unit 210 (S41).

[0092] Next, when the user terminal 20 receives a playback operation request for a video using character images displayed in the saved image list display unit 210 via the playback operation unit 211 (S42), it sends a video playback request to the server device 10 via the communication network 2 (S43).

[0093] Next, the server device 10 generates video data in the video data generation unit 115 for playing a video that uses character images, including user images, generated by the character image generation unit 112 (S44).

[0094] Next, the server device 10 transmits the video data generated by the video data generation unit 115 to the user terminal 20 via the communication network 2 (S45).

[0095] Next, the user terminal 20 displays a video in the video display unit 207 based on the video data generated by the video data generation unit 115 (S46).

[0096] As shown in Figure 6, the video data generation unit 115 generates video data for playing a video that uses character images including coloring pattern images, and the video display unit 207 can then display a video that uses only character images including coloring pattern images based on the video data generated by the video data generation unit 115.

[0097] Furthermore, as shown in Figure 7, the video data generation unit 115 generates video data for playing a video that uses character images including user images, and the video display unit 207 can display a video that uses only character images including user images based on the video data generated by the video data generation unit 115.

[0098] On the other hand, as shown in Figures 6 and 7, the video data generation unit 115 generates video data for playing a video that uses character images including coloring pattern images, and video data for playing a video that uses character images including user images. Based on the video data generated by the video data generation unit 115, the video display unit 207 can display a video that combines character images including coloring pattern images with character images including user images.

[0099] In other words, when the save operation unit 209 of the user terminal 20 receives a save operation, the character image including the coloring pattern image and the character image including the user image are saved in the image storage unit 114 of the server device 10.

[0100] Then, the saved image list display unit 210 of the user terminal 20 displays a list of character images, including the coloring pattern images saved by the image storage unit 114, and character images, including the user's image.

[0101] At this time, when the playback operation unit 211 of the user terminal 20 receives a playback operation, the video data generation unit 115 of the server device 10 generates video data for playing a video that uses character images including a coloring pattern image generated by the character image generation unit 112 and character images including a user image.

[0102] Then, the user terminal 20 displays a video in the video display unit 207 based on the video data generated by the video data generation unit 115. As a result, a video is displayed in which character images, including a coloring pattern image, are combined with character images, including the user's image.

[0103] <Example of using the coloring video display system> Next, we will explain an example of how to use the coloring video display system 1 using Figures 8 and 9.

[0104] As shown in Figure 8(a), the user purchases a coloring book 300. The user scans the two-dimensional code 302 on the coloring book 300 with the user terminal 20 to access the server device 10. When the user terminal 20 accesses the server device 10, the menu screen W1 is displayed on the user terminal 20.

[0105] The menu screen W1 includes a coloring image capture selection unit W11 for selecting the capture of a coloring image, a user image capture selection unit W12 for selecting the capture of a user image, and a saved image list display operation unit W13 for displaying saved images in a list, and accepts any of these input operations.

[0106] On menu screen W1, operating the coloring image capture selection unit W11 transitions to the coloring image capture screen W2 (Figure 8(b)) for capturing a coloring image. On menu screen W1, operating the user image capture selection unit W12 transitions to the user image capture screen W4 (Figure 9(d)) for capturing a user image. On menu screen W1, operating the saved image list display operation unit W13 transitions to the saved image list display screen W3 (Figure 8(c)) on the saved image list display unit 210, which displays a list of saved character images.

[0107] As shown in Figure 8(b), the coloring image capture screen W2 includes a coloring image capture selection unit W11, a user image capture selection unit W12, a saved image list display operation unit W13, an image capture screen display unit W21 that displays the capture screen, an image capture operation unit W22 that captures the image on the image capture screen display unit W21, a character image display unit 208, a character image display unit W23 that displays a character image including a coloring pattern image, a save operation unit 209, and a save operation unit W24 for performing a save operation to save the character image.

[0108] On the coloring page image capture screen W2, when the user operates the capture operation unit W22 to capture the sheet 30 including the marker 303, a character image including the coloring page pattern image is displayed on the character image display unit W23. When the character image is displayed on the character image display unit W23, the save operation unit W24 saves the character image.

[0109] When the user image capture selection unit W12 is operated on the coloring image capture screen W2, the screen transitions to the user image capture screen W4 (Figure 9(d)) for capturing a user image. When the user saves an image using the saved image list display operation unit W13 on the coloring image capture screen W2, the screen transitions to the saved image list display screen W3 (Figure 8(c)).

[0110] As shown in Figure 8(c), the saved image list display screen W3 includes a coloring image capture selection unit W11, a user image capture selection unit W12, a saved image list display operation unit W13, a saved list display unit W31 that displays the saved list, and a playback operation unit 211, which includes a playback operation unit W32 for playing a video that uses the saved character images. In this example, the saved list display unit W31 displays character images including coloring pattern images.

[0111] On the saved image list display screen W3, operating the coloring image capture selection unit W11 transitions to the coloring image capture screen W2 (Figure 8(b)). Operating the user image capture selection unit W12 transitions to the user image capture screen W4 (Figure 9(d)) for capturing user images. On the saved image list display screen W3, performing a playback operation using the playback operation unit W32 transitions to the video playback screen W5 (Figure 9(f)).

[0112] As shown in Figure 9(d), the user image capture screen W4 includes a coloring image capture selection unit W11, a user image capture selection unit W12, a saved image list display operation unit W13, an image capture screen display unit W41 that displays the image capture screen, an image capture operation unit W42 that captures the image on the image capture screen display unit W41, a character image display unit 208 which is a character image display unit W43 that displays a character image including the user image, a save operation unit 209 which is a save operation unit W44 for performing a save operation to save the character image, and a camera switching unit W45 that switches between the front camera and the rear camera.

[0113] On the user image acquisition screen W4, when the user operates the acquisition operation unit W42 to acquire an image of the user, a character image including the user image is displayed on the character image display unit W43. When a character image is displayed on the character image display unit W43, the character image is saved when the save operation unit W44 performs a save operation.

[0114] When the user image capture screen W4 is accessed via the coloring image capture selection unit W11, the screen transitions to the coloring image capture screen W2 (Figure 8(b)). When the user image capture screen W4 is accessed via the saved image list display operation unit W13, the screen transitions to the saved image list display screen W3 (Figure 9(e)).

[0115] As shown in Figure 9(e), in this example, the saved image list display section W31 of the saved image list display screen W3 displays character images including coloring pattern images and character images including user images. When a playback operation is performed using the playback operation section W32 on the saved image list display screen W3, the screen transitions to the video playback screen W5 shown in Figure 9(f).

[0116] As shown in Figure 9(f), the video playback screen W5 is a video display unit 207 and includes a video display unit W51 for displaying videos.

[0117] The video display unit W51 displays a video based on video data for playing a video that uses character images. In this example, a video is displayed that combines a character image containing a coloring pattern image with a character image containing a user image. Specifically, a dance video is displayed in which the character image containing the coloring pattern image and the character image containing the user image dance. In this dance video, the character image containing the coloring pattern image moves its limbs and changes its facial expression. The character image containing the user image also moves its limbs. In addition, background music is played in conjunction with the display of the dance video.

[0118] In this embodiment, an example is given in which a dance video using coloring pages is displayed in the coloring video display system 1, but the type of video to be displayed is not limited to dance videos. For example, various types of videos can be applied to this embodiment, such as a video in which character images participate in a sports day, or a video in which character images ride a vehicle and go on an adventure.

[0119] The video playback screen W5 includes a save list display operation unit W52. Performing an operation on the save list display operation unit W52 ends video playback and returns to the save image list display screen W3.

[0120] As described above, the coloring video display system 1 of this embodiment can display videos that use character images, including coloring pattern images, so that users can enjoy coloring.

[0121] Furthermore, the coloring video display system 1 of this embodiment can display videos that use character images including user images and character images including coloring pattern images, so that users can enjoy coloring.

[0122] Furthermore, in the coloring video display system 1 of this embodiment, saved character images are displayed in a list, and the system accepts playback requests for videos that use the character images displayed in the list, so that the user's preferences can be reflected in the videos.

[0123] Although the present invention has been described above with reference to preferred embodiments, it goes without saying that the present invention is not limited to the embodiments described above, and various modifications can be made within the scope of the present invention. [Explanation of symbols]

[0124] 1: Coloring video display system 10: Server equipment 20: User terminal 30: Sheet 303: Marker

Claims

1. A coloring video display system that displays videos using coloring pages, A sheet on which an outline defining the area to be colored is drawn, and a marker containing pattern identification information that can identify the coloring pattern corresponding to the outline is attached around the outline, An imaging data storage unit that stores imaging data obtained by imaging the sheet so as to include the marker, A correction unit that corrects the plane of the image data obtained by imaging the aforementioned sheet based on the marker, An identification information identification unit identifies the pattern identification information contained in the marker based on imaging data obtained by imaging the sheet, A coloring pattern identification unit identifies the coloring pattern based on the pattern identification information identified by the identification information identification unit, A pattern information identification unit identifies the contour and the position of the contour that correspond to the coloring pattern identified by the coloring pattern identification unit, A pattern image generation unit generates a coloring pattern image by trimming the contour to include the inside of the contour, based on the contour and the position of the contour identified by the pattern information identification unit. A character image generation unit generates a character image that includes the coloring pattern image generated by the pattern image generation unit, A video data generation unit generates video data for playing a video that uses character images, including the aforementioned coloring pattern image, The system includes a video display unit that displays a video using character images, including the coloring pattern images, based on the video data generated by the video data generation unit. A coloring book video display system.

2. The aforementioned imaging data storage unit stores imaging data obtained by imaging the user, The system includes a user image generation unit that generates a user image contained within a predetermined frame based on imaging data obtained by imaging the user, The character image generation unit generates a character image that includes the user image generated by the user image generation unit. The aforementioned video data generation unit generates video data for playing a video in which a character image including the coloring pattern image is added to a character image including the user image. The video display unit displays a video based on the video data generated by the video data generation unit, which adds a character image including the user image to a character image including the coloring pattern image. The coloring book video display system according to claim 1.

3. A character image display unit that displays the character image generated by the character image generation unit, A save operation unit that receives a save operation to save the character image displayed by the character image display unit, An image storage unit that saves a character image when the aforementioned save operation is received, The image storage unit displays a list of character images saved by the image storage unit, The system includes a playback operation unit that receives a playback operation for a video using character images displayed in the saved image list display unit. The coloring book video display system according to claim 1 or 2.

Images