Computer program, information processing device, and information processing method

Abstract

To provide a computer program, an information processing device and an information processing method which facilitate creation of a storyboard.SOLUTION: A computer program displays, on a touch-panel integrated display of a computer, multiple objects corresponding to respective scenes forming a storyboard chronologically together with thumb nails of images rendered for respective scenes, receives on the display an object that corresponds to a scene to be edited from among the objects, displays an editing screen for the scene corresponding to the selected object on the same screen as the multiple objects, receives rendering editing to the scene on the displayed editing screen, displays in the same screen a reproduction button for reproducing the multiple scenes as a moving image according to a set time axis, and when the reproduction button is selected on the display, reproduces the multiple scenes as a moving image.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to a computer program, an information processing device, and an information processing method for supporting the creation of a storyboard. [Background technology]

[0002] In video and animation production, storyboards are primarily created in order to share the image. These days, instead of the traditional hand-drawn rough sketches, it is possible to create storyboards by arranging pictures drawn using image editing software. In order to visualize the video to be produced, a function that plays the arranged pictures continuously and outputs them as a video has been proposed as a production support technology (Patent Document 1, etc.).

[0003] Due to technological advances in image processing hardware, the functions of image editing programs have become more and more diverse, and even if a program has a storyboard creation function, it is not possible to remove other functions. For users who only want the storyboard creation function, using such an image editing program is a redundant configuration, as it requires high costs due to the other functions and hardware with higher processing power. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2005-285076 A Summary of the Invention [Problem to be solved by the invention]

[0005] An object of the present invention is to provide a computer program, an information processing device, and an information processing method with limited functions so that storyboards can be easily created. [Means for solving the problem]

[0006] A computer program according to an embodiment of the present disclosure causes a computer having a display with a built-in touch panel to display on the display a plurality of objects corresponding to a plurality of frames constituting a storyboard in chronological order along with thumbnails of images drawn for each frame, accept on the display a selection of an object corresponding to a frame to be edited from among the plurality of objects, display an editing screen for the frame corresponding to the selected object on the same screen as the plurality of objects, accept drawing edits for the frame on the displayed editing screen, display on the same screen a play button for playing the plurality of frames as a video in accordance with a set time axis, and, when the play button is selected on the display, execute a process of playing the plurality of frames as a video.

[0007] An information processing device according to one embodiment of the present disclosure includes a display with a built-in touch panel and a processing unit, wherein the processing unit displays on the display a plurality of objects corresponding to a plurality of frames constituting a storyboard in chronological order together with thumbnails of images drawn for each frame, accepts selection of an object corresponding to a frame to be edited from the plurality of objects, displays an editing screen for the frame corresponding to the selected object on the same screen as the plurality of objects, accepts drawing edits for the frame on the displayed editing screen, displays on the same screen a play button for playing a video of the plurality of frames according to a set time axis, and includes a processing unit for playing a video of the plurality of frames when the play button is selected on the display.

[0008] In an information processing method of one embodiment of the present disclosure, a computer equipped with a display with a built-in touch panel displays on the display a plurality of objects corresponding to a plurality of frames constituting a storyboard in chronological order along with thumbnails of images drawn for each frame, accepts on the display a selection of an object corresponding to a frame to be edited from the plurality of objects, displays an editing screen for the frame corresponding to the selected object on the same screen as the plurality of objects, accepts drawing edits for the frame on the displayed editing screen, displays on the same screen a play button for playing back the plurality of frames as a video in accordance with a set timeline, and when the play button is selected on the display, plays back the plurality of frames as a video.

[0009] In the field of image processing, displaying thumbnails of each frame and accepting image editing operations may differ in the way memory is referenced and the way it is expanded in memory when calling the OS (Operating System) or the software that is the basis for program-based control. In this case, accepting edits and displaying videos while holding both in a temporary storage area as data to be displayed on the same screen is not easy even in a simple configuration, and since it is not always used in an environment with sufficient hardware resources such as memory, there have been few examples of this being realized in conventional image editing programs. Furthermore, the above-mentioned way of reference and the way of expansion in memory may change due to updates to the OS or the underlying software.

[0010] In order to realize a computer program with limited functions to easily create storyboards, it is necessary not only to simplify the functions but also to devise a software configuration to simplify the functions. As described above, particularly in information terminals such as tablet information terminals or smartphones that have limited memory and hardware resources for drawing, displaying thumbnails and accepting edits, which have different ways of referencing memory, have not been performed on the same screen.

[0011] The computer program, information processing device, and information processing method disclosed herein enable display of thumbnails of each frame of a storyboard and editing of images for a selected frame on the same screen on a tablet terminal or a smartphone equipped with a display with a built-in touch panel.

[0012] A computer program according to one embodiment of the present disclosure may cause the computer to display an interface for switching the display method of the plurality of objects corresponding to the plurality of frames while displaying the editing screen, and to execute a process for switching the display method of the plurality of objects each time the interface is selected.

[0013] A computer program according to an embodiment of the present disclosure may cause the computer to execute a process of displaying a waveform of a sound to be output in synchronization with the video when the play button is selected, approximately parallel to the plurality of objects which are the previous plurality of frames arranged in chronological order.

[0014] A computer program according to one embodiment of the present disclosure may cause the computer to execute a process of superimposing the plurality of frames on the plurality of objects arranged in chronological order, displaying a cursor movable in the arranged direction, and switching between and displaying the plurality of frames in accordance with the movement of the cursor.

[0015] A computer program according to an embodiment of the present disclosure may cause the computer to execute a process of displaying a playback time in the video corresponding to each of the frames. Effect of the Invention

[0016] According to the computer program of the present disclosure, while displaying thumbnails of each frame, it is possible to select any one of the thumbnails and edit the frame on the same screen while checking the thumbnails of the previous and next frames. Furthermore, video playback that displays the edited frames in chronological order can be performed without switching screens. In this way, a computer program limited to the functions necessary for storyboard production is realized. [Brief description of the drawings]

[0017] [Figure 1] 1 is a block diagram showing a configuration of an information processing device according to the present disclosure. [Diagram 2] 11 is a flowchart illustrating an example of a processing procedure executed by an information processing device. [Diagram 3] 11 is a flowchart illustrating an example of a processing procedure executed by an information processing device. [Figure 4] 11 is a flowchart illustrating an example of a processing procedure executed by an information processing device. [Diagram 5] FIG. 13 is a diagram showing an example of a storyboard creation screen. [Figure 6] FIG. 13 is a diagram showing another example of the storyboard creation screen. [Figure 7] FIG. 13 is a diagram showing another example of the storyboard creation screen. [Figure 8] 13 shows an example of a storyboard creation screen during video playback. [Figure 9] 13 shows an example of the storyboard creation screen with the display switching button selected. [Figure 10] FIG. 13 is a diagram showing another example of the storyboard creation screen. [Figure 11] FIG. 13 is a diagram showing another example of the storyboard creation screen. [Figure 12] FIG. 13 is a diagram showing an example of a storyboard creation screen in a modified example. [Figure 13] An explanatory diagram of a comment addition button. [Figure 14] 13 is a flowchart showing an example of a processing procedure for accepting an operation on the second editing screen. [Figure 15] 13 shows an example of a display of a second editing screen. [Figure 16] 13 shows an example of a display of a second editing screen. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present disclosure will be specifically described with reference to the drawings showing embodiments thereof. In the following embodiments, a method for creating a storyboard using a computer program according to the present disclosure will be described.

[0019] 1 is a block diagram showing a configuration of an information processing device 1 according to the present disclosure. The information processing device 1 is a tablet terminal equipped with a display with a built-in touch panel as a display unit 13. The information processing device 1 may be a smartphone. The information processing device 1 may be a desktop or laptop personal computer.

[0020] The information processing device 1 includes a processing unit 10, a storage unit 11, a communication unit 12, a display unit 13, an operation unit 14, and a voice input / output unit 15.

[0021] The processing unit 10 includes a processor 100 using a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and / or an MPU (Micro-Processing Unit). The processing unit 10 includes a memory 101 such as a DRAM (Dynamic Random Access Memory), an SRAM (Static Random Access Memory), or an SDRAM (Synchronous Dynamic Random Access Memory). The processing unit 10 stores data generated during processing in the memory 101 or the storage unit 11, and executes calculations based on a storyboard program P1 stored in the storage unit 11.

[0022] The storage unit 11 uses a non-volatile memory such as a flash memory or an SSD (Solid State Drive). The storage unit 11 stores data used by the processing unit 10. The storage unit 11 stores a storyboard program P1. The storyboard program P1 may be a storyboard program P9 stored in a storage medium 9 that is read by the processing unit 10 and copied into the storage unit 11. The storyboard program P1 may be a program downloaded by the processing unit 10 from another program server device via the communication unit 12 and stored in the storage unit 11 in an executable manner.

[0023] The communication unit 12 realizes communication with the server device 2 via a network including the Internet or a carrier network. Specifically, the communication unit 12 is a network card. The communication unit 12 may be a wireless communication module that connects to a carrier network, or may be a wireless communication module for Wi-Fi (registered trademark). The processing unit 10 can transmit and receive data to and from an external device such as a server device (not shown) via the communication unit 12.

[0024] The display unit 13 is a display such as a liquid crystal display or an organic EL (Electro Luminescence) display. The display unit 13 displays a GUI (Graphical User Interface) of the storyboard program P1. The display unit 13 is a touch panel built-in display. The display unit 13 is not limited to a touch panel built-in type.

[0025] The operation unit 14 is a user interface capable of inputting and outputting data to and from the processing unit 10. In one example, the operation unit 14 is a touch panel built into the display unit 13. The operation unit 14 may include a pen for the touch panel. The operation unit 14 may be a mouse, a keyboard, or the like. The operation unit 14 may include a physical button.

[0026] The voice input / output unit 15 includes a speaker and a microphone. The processing unit 10 can output voice from the speaker and accept voice input from the microphone. The processing unit 10 may use the voice input / output unit 15 as part of the operation by the operation unit 14. In other words, the processing unit 10 may recognize the operation content from the input voice by the microphone and accept the operation in an interactive format with the voice output by the speaker.

[0027] The general-purpose information processing device 1 configured in this manner realizes easy creation of storyboards based on the storyboard program P1. Figures 2 to 4 are flowcharts showing an example of processing procedures executed by the information processing device 1. When the storyboard program P1 is selected and started by the user and a new storyboard creation menu displayed by the storyboard program P1 is selected, the processing unit 10 executes the following processing procedures.

[0028] The processing unit 10 displays a storyboard creation screen including an object that is a reduced version of the first frame that constitutes a new storyboard and an editing screen for the first frame on the display unit 13 (step S101). In step S101, the processing unit 10 displays the editing screen by expanding drawing data for the entire frame in the memory 101, while displaying the object by referring to drawing data for the corresponding frame from the memory 101. The drawing data is described in vector format or raster format for each layer in association with the identification data of the frame.

[0029] The processing unit 10 judges whether or not an editing operation on the editing screen is accepted by the operation unit 14 (step S102). If it is judged that the editing operation is accepted (S102: YES), the processing unit 10 draws on the editing screen in accordance with the editing operation (step S103) and advances the process to step S104. The result of drawing in step S103 is reflected in the object corresponding to the first frame by referencing it from the memory 101.

[0030] If it is determined that the editing operation on the editing screen is not accepted (S102: NO), the processing unit 10 advances the process to step S104.

[0031] The processing unit 10 judges whether an operation to add a new frame or to duplicate a frame has been accepted (step S104). The storyboard creation screen includes an interface that accepts an operation to add a new blank frame and an interface that accepts an operation to duplicate a selected frame (see FIG. 5), and the processing unit 10 accepts an operation from the operation unit 14 of either interface. If it is judged that an operation to add a new frame or to duplicate a frame has not been accepted (S104: NO), the processing unit 10 returns the process to step S102.

[0032] If it is determined that a new addition or duplication of a frame has been accepted (S104: YES), the processing unit 10 stores (moves) the drawing data of the drawing result on the editing screen of the first frame in association with the identification data of the frame to a temporary storage area of ​​the storage unit 11 or another area of ​​the memory 101 (step S105), and deletes the editing screen of the first frame from the storyboard creation screen (step S106). The processing unit 10 switches the reference destination of the drawing data of the object of the first frame from the memory 101 to the storage destination (step S107), and displays the object so as to be distinguishable from the object of the newly added or duplicated frame (step S108).

[0033] The processing unit 10 assigns identification data (second) to the newly added or duplicated frame as a second frame (step S109), and displays an editing screen for the new frame on the storyboard creation screen (step S110). The objects of the new frame are displayed alongside the first frame at a location corresponding to a later time than the first frame (step S111). In step S110, the processing unit 10 expands the drawing data of the entire frame in the memory 101 and displays the editing screen, similar to the first frame in step S101. In step S111, the processing unit 10 displays the objects by referencing the drawing data for the corresponding frame from the memory 101.

[0034] The processing unit 10 judges whether an editing operation for a new frame is to be accepted by the operation unit 14 on the storyboard creation screen (step S112). If it is judged that the editing operation is to be accepted (S112: YES), the processing unit 10 draws on the editing screen in accordance with the editing operation (step S113) and advances the process to step S114. The result of drawing in step S113 is reflected in the object corresponding to the new frame by reference from the memory 101.

[0035] If it is determined that the editing operation on the editing screen is not accepted (S112: NO), the processing unit 10 advances the process to step S114.

[0036] The processing unit 10 judges whether an operation for newly adding or duplicating a third or subsequent frame has been received (step S114). If it is judged that an operation for newly adding or duplicating a frame has been received (S114: YES), the processing unit 10 stores (moves) the drawing data of the drawing result on the editing screen that was being displayed in association with the identification data of the target frame in a temporary storage area of ​​the storage unit 11 or another area of ​​the memory 101 (step S115), and deletes the editing screen that was displayed before the new frame was added from the storyboard creation screen (step S116). The processing unit 10 switches the reference destination of the drawing data of the object of the frame corresponding to the editing screen that was displayed before the addition from the memory 101 to the storage destination (step S117), and displays the object in a manner distinguishable from the object of the new frame (step S118).

[0037] The processing unit 10 assigns identification data to the newly added or duplicated frame as the frame next to the frame whose editing screen was displayed before the addition (step S119). The processing unit 10 displays the editing screen for the new frame within the storyboard creation screen (step S120), displays the objects of the new frame alongside the objects of the frame whose editing screen was displayed before the addition (step S121), and proceeds to the process of step S128.

[0038] If it is determined in step S114 that an operation to add a new frame or to duplicate a frame has not been accepted (S114: NO), the processing unit 10 determines whether an operation to select an object of another frame on the operation unit 14 has been accepted (step S122).

[0039] When it is determined that an operation to select an object in another frame has been accepted (S122: YES), the processing unit 10 stores (moves) the drawing data of the drawing result on the editing screen that was being displayed in association with the identification data of the target frame in a temporary storage area of ​​the storage unit 11 or another area of ​​the memory 101 (step S123). The processing unit 10 deletes the editing screen that was displayed before the object in the other frame was selected from the storyboard creation screen (step S124). The processing unit 10 switches the reference destination of the drawing data of the object in the frame corresponding to the editing screen that was displayed before the object in the other frame was selected from the memory 101 to the storage destination (step S125), and displays the object in a manner distinguishable from the object in the selected frame (step S126). The processing unit 10 displays the editing screen for the selected frame in the storyboard creation screen (step S127), and proceeds to the process of step S128.

[0040] The processing unit 10 judges whether or not a video playback based on a plurality of frames including the frame being edited has been selected (step S128). If it is judged that a video playback has not been selected (S128: NO), the processing unit 10 returns the process to step S112 and accepts editing operations on the displayed editing screen, selection of other frames, addition of new frames, and the like.

[0041] If it is determined that video playback has been selected (S128: YES), the processing unit 10 plays back a plurality of frames including the added frame in chronological order as a video (step S129). In step S129, the processing unit 10 displays frames including drawing data drawn in accordance with a set time axis in chronological order. The processing unit 10 switches and displays one frame at a set time interval, such as every two seconds, for example. As will be described later, a display time may be set for each frame. The processing unit 10 may display the frames by superimposing them on the editing screen being displayed, or may display them in another available area within the storyboard creation screen.

[0042] The processing unit 10 determines whether or not the end of the storyboard program P1 has been selected (step S130). If it is determined that the end of the storyboard program P1 has not been selected (S130: NO), the processing unit 10 returns the process to step S112.

[0043] If it is determined that the end of the storyboard program P1 has been selected (S130: YES), the processing unit 10 ends the storyboard program P1.

[0044] As shown in Figures 2 to 4, in order to display an object including thumbnails of each of the multiple frames on the storyboard creation screen, while displaying an editing screen for one of the multiple frames, and to accept the addition or selection of a frame, processing is required to expand drawing data on memory 101 and switch the reference destination. In order to optimize the drawing processing using an OS or existing development engine, such as a function to accept drawings, the way data on memory 101 is handled and referenced must be made compatible with those existing programs, making it difficult to realize a simple interface as described below. By implementing the above-mentioned processing procedure, it becomes possible to create storyboards with a simple interface.

[0045] The processing procedure shown in FIG. 2 to FIG. 4 will be described with reference to a specific example of the interface of the storyboard program P1. FIG. 5 is a diagram showing an example of a storyboard creation screen 300. In FIG. 5, the storyboard creation screen 300 includes an editing screen 301 of the first frame and an object 302 corresponding to a thumbnail of the first frame. In FIG. 5, since new storyboard data has just been created, there is only one frame. When a user draws a picture on the editing screen 301 displayed on the display of the display unit 13 using a touch pen for a tablet or the like, the trajectory is accepted by the operation unit 14 as is, and a preview is displayed as shown in FIG. 5 by the processing unit 10 based on the drawing data. Enlarging and reducing within the editing screen 301 can be performed by pinching out, pinching in, or the like on the editing screen 301. The editing operation on the editing screen 301 is not limited to a drawing operation such as drawing a line, but editing functions such as Redo and Undo assigned to gestures using a touch pen or a user's finger may be realized to realize an intuitive operation. The preview is also reflected in a thumbnail of an object 302 corresponding to the frame displayed on the editing screen 301.

[0046] The storyboard creation screen 300 includes a delete button 303 that accepts the deletion of a frame currently being displayed on the editing screen, a new add button 304 that accepts an operation to newly add a blank frame, and a duplicate button 305 that accepts an operation to duplicate a selected frame.

[0047] Although the processing performed when the Delete button 303 is selected is not described in the processing procedures of Figures 2 to 4, the processing unit 10 deletes the display of the selected frame and deletes the data of the corresponding frame from the memory 101 and the storage unit 11.

[0048] When the Add New button 304 is selected, a new blank frame is inserted as the frame following the frame that was displayed by the processing of steps S104 and S114 in the processing procedure of Figures 2 to 4. The insertion position is not limited to this, and it may be inserted as the previous frame.

[0049] When the Duplicate button 305 is selected, a duplicate of the displayed frame is inserted as the next frame through the processing of steps S104 and S114 in the processing procedure of Figures 2 to 4. The drawing data associated with the frame is duplicated and stored in association with the identification data of another frame. The duplicated frame can be further edited, such as by deleting or adding additional drawings, on the edit screen 301, independently of the original frame.

[0050] 5, an editing work area 3010 for the editing screen 301 may include a layer switching button 3011. On the editing screen 301, a button 3012 for inserting image data into the editing screen 301, a pointer button 3013 for selecting an area or point on the editing screen 301, an eraser button 3014 for erasing the selected location, and a drawing button 3015 for drawing on the selected location with the selected pen type are displayed. The user can draw on the editing screen 301 by successively selecting the layer switching button 3011, the insert button 3012, the pointer button 3013, the eraser button 3014, and the drawing button 3015.

[0051] A zoom slide 3016 may be displayed for the editing screen 301. The zoom slide 3016 is displayed when the eraser button 3014 or the drawing button 3015 is selected. When an operation is performed on the zoom slide 3016, the size of the eraser is enlarged or reduced, or the size of the drawing pen (brush) is enlarged or reduced.

[0052] The storyboard creation screen 300 also includes a video playback button 306. The video playback button 306 is always displayed, but may be configured to be enabled when the number of frames reaches two or more.

[0053] The storyboard creation screen 300 includes a menu button 307 for terminating the storyboard program P1. When the menu button 307 is selected, for example, the processing procedure of step S130 shown in the flowcharts of FIGS. 2-4 is executed, and the storyboard program P1 is terminated.

[0054] The storyboard creation screen 300 includes a scale 308 that indicates the elapsed time setting for each of the multiple frames. The scale 308 includes graduations at a predetermined unit interval. The scale 308 allows the user to check the playback time of the video corresponding to each frame. In the storyboard creation screen 300 of FIG. 5, since there is only one frame, one second is assigned to each frame according to the initial setting, and it is shown that the frame takes up one second from its start to its end. The scale 308 allows the user to set the length of time each frame occupies, as shown in FIG. 6, depending on the length of the object 302 in the direction along the scale 308. As shown in FIG. 11, a marker 311 can be placed on the scale 308, and the user can use the marker to consider the allocation of frames that allows scene switching that is particularly timed with audio data or the like.

[0055] The storyboard creation screen 300 includes a sound receiving section 309 for receiving sound data such as voice and music to be output in synchronization with the playback of a plurality of frames, in parallel with the arrangement direction of the objects 302 on the screen. The storyboard creation screen 300 in Fig. 5 is blank and does not receive sound.

[0056] Fig. 6 is a diagram showing another example of the storyboard creation screen 300. The storyboard creation screen 300 in Fig. 6 shows a state after two frames have been added. Components common to Fig. 5 are given the same reference numerals and detailed description is omitted. In the storyboard creation screen 300 in Fig. 6, a plurality of objects 302 corresponding to a plurality of frames constituting the storyboard being created are arranged in chronological order together with thumbnails of images drawn for each frame. The direction in which the objects 302 are arranged is the horizontal direction (horizontal direction, width direction) of the image in Fig. 6, but is not limited thereto and may be the vertical direction (vertical direction, height direction) of the image.

[0057] Of the multiple objects 302, the object 302 corresponding to the frame to be edited is displayed highlighted, and the objects 302 of the other frames are displayed masked in gray, making it possible to distinguish between the selected frame and the others.

[0058] 6, the time taken by the first frame is doubled. The time may be extended by an operation of stretching the object 302 in accordance with the scale 308. The scale 308 may be made grabbable.

[0059] Fig. 7 is a diagram showing another example of the storyboard creation screen 300. In Fig. 7, a knob 3021 for changing the time occupied by a selected frame of the object 302 is displayed. In the storyboard creation screen 300 in Fig. 7, the start time of the selected frame is advanced by 0.5 seconds, and the end time of the selected frame is adjusted to be delayed by 0.5 seconds, compared to the storyboard creation screen 300 in Fig. 6. In this way, the storyboard program P1 of the present disclosure makes it possible to intuitively adjust the flow of frames and the timing of each frame relative to the whole, while displaying the editing screen 301 for the selected frame.

[0060] Returning to FIG. 6, the description will be continued. In the storyboard creation screen 300 in FIG. 6, a cursor 312 movable in the time axis direction is displayed superimposed on a plurality of objects 302 in which a plurality of frames are arranged in chronological order. The cursor 312 can be moved by a user's operation. For example, when an input operation for specifying a time is performed on a time display section 313 that displays an elapsed time, the cursor 312 moves to the location of the specified time. The cursor 312 itself may be movable. When the movement of the cursor 312 is detected, the processing section 10 may display an image of a frame corresponding to the position of the cursor 312 in the same range as the editing screen 301. This makes it possible to display a part of the flow by switching frames like an animation, not limited to playing from the beginning of the moving image, as shown in FIG. 8 described later.

[0061] The user can use the operation unit 14 to select an object 302 and display and edit the frames on the editing screen 301, move the cursor 312 to switch between frames, or play the object as a video using the video playback button 306.

[0062] By using the storyboard program P1 of the present disclosure, an editing screen 301 and a plurality of objects 302 corresponding to thumbnails of each frame arranged in chronological order can be displayed side by side on the same storyboard creation screen 300, and any frame to be edited can be selected from the plurality of objects 302. The editing target on the editing screen 301 is switched to the one corresponding to the selected frame in response to an operation. While performing a drawing operation on the frame, the user can immediately check a preview by selecting the video playback button 306 on the same screen.

[0063] Fig. 8 shows an example of the storyboard creation screen 300 when playing back a moving image. Reference numeral 4 in Fig. 8 indicates the range in which the moving image is played back on the storyboard creation screen 300. As shown in Fig. 8, when the moving image play button 306 is selected, the moving image is played back in the same range as the layout at the time of editing up to that point, within the editing area of ​​the editing screen 301. This allows the user to check successively how the moving image will look while editing each frame.

[0064] 8, a cursor 312 that moves in the direction of the time axis of the object 302 of each frame is displayed. The cursor 312 can be moved along the time axis by being superimposed on a plurality of objects 302 that arrange a plurality of frames in chronological order. The cursor 312 may be controlled so as to move in accordance with the display of the moving image within the range 4. This allows the user to confirm which frame is being played back from the position of the cursor 312 on the plurality of objects 302.

[0065] Returning to Fig. 6, the explanation will be continued. A display switching button 310 is shown on the storyboard creation screen 300 in Fig. 6. The display switching button 310 is a button for reducing the range including the objects 302, the scale 308, and the sound receiving portion 309 arranged in chronological order, and for increasing the range that the editing screen 301 occupies within the display portion 13.

[0066] Fig. 9 shows an example of the storyboard creation screen 300 in which the display switch button 310 has been selected. The storyboard creation screen 300 shown in Fig. 9 shows a state in which the display switch button 310 has been selected by the user on the storyboard creation screen 300 shown in Fig. 6, and the area displaying the objects 302 and the like has been reduced. As a result, the storyboard creation screen 300 shown in Fig. 9 has an expanded range of the editing screen 301 and the editing work area 3010 compared to the storyboard creation screen 300 shown in Fig. 6, improving the operability of the user's editing operations on the frames.

[0067] Conversely, when the display switch button 310 is selected on the storyboard creation screen 300 shown in Fig. 9, the range including the objects 302, scale 308, and sound receiving unit 309 arranged in chronological order returns to the range shown in Fig. 6, and the editing screen 301 returns to its initial range. In this way, a display method in which the arrangement of the objects 302 for checking the connections with other frames and the editing screen 301 can be grasped at once on the same screen, and an editing screen 301 that is easy for the user, the storyboard creator, to edit can be easily switched by operating a single interface, improving operability.

[0068] The variations of the storyboard creation screen 300 when the display switching button 310 is selected are not limited to the above. Fig. 10 is a diagram showing another example of the storyboard creation screen 300. Fig. 10 shows a state in which the object 302 is folded and hidden as a method of displaying the object 302. In this way, it is also possible to temporarily hide the object 302 within the same screen.

[0069] FIG. 11 is a diagram showing another example of the storyboard creation screen 300. In the storyboard creation screen 300 of FIG. 11, sounds such as voice, sound effects, or background music are set in the sound receiving unit 309. In the sound receiving unit 309, when the range of the sound receiving unit 309 is selected and then the new addition button 304 is selected, an addition operation can be performed from a list of sounds such as voice, sound effects, or background music. In addition, the sound setting may be performed by selecting a button (a musical note symbol shown in FIG. 5, etc.) in the range of the sound receiving unit 309 or by pressing and holding the range in the sound receiving unit 309 and then adding the sound from a list of sounds that is displayed. In FIG. 11, the sound receiving unit 309 displays the waveform of the sound to be output in parallel with the objects 302 arranged in time series and in synchronization with the video. The waveform displayed in the sound receiving unit 309 can be shifted in the time series direction with respect to the arrangement of the objects 302.

[0070] In the storyboard creation screen 300 shown in FIG. 11, a marker 311 is placed on a scale 308. The user can select and place the marker 311 at any position on the scale 308 using the operation unit 14. The processing unit 10 stores the playback time from the beginning of the video when a plurality of frames corresponding to the position of the marker 311 on the scale 308 are played as a video. The user can place the marker 311 while referring to the waveform of the sound to be output synchronously, and the operation of selecting a frame to be displayed at that time and setting the time of each frame is simplified. The position of the marker 311 may be received by displaying a reception interface for the marker 311 during playback as a video. The marker 311 can be placed on the time axis corresponding to the position of the cursor 312 when the reception interface is tapped during playback of the video. The processing unit 10 stores the time on the video corresponding to the position of the marker 311 separately from each other. When each marker 311 is selected, it is also possible to display the frame at the time corresponding to the marker 311.

[0071] Based on the storyboard program P1, the processing unit 10 can also realize other functions. For example, the menu of the storyboard program P1 may include a comment addition menu and a write menu. When the comment addition menu is selected, the processing unit 10 accepts input of text data explaining the action for each frame, and stores the input text data in the storage unit 11 in association with the frame identification data. When the write menu is selected, the processing unit 10 outputs the frame number, the image drawn for the frame, the written text data, and the time stamp in a table format. The processing unit 10 may store document data, such as PDF data, in the storage unit 11 as table format data.

[0072] By using the storyboard program P1, storyboards can be created with simple operations. Users who have difficulty communicating through hearing can also use the storyboard program P1 to easily draw pictures and write text by hand and play them back on the same screen for communication, so the use of the program is not limited to creating storyboards.

[0073] (Modification) In the modified example, a function based on the above-mentioned storyboard program P1 is further added. Fig. 12 is a diagram showing an example of a storyboard creation screen 300 in the modified example. The storyboard creation screen 300 in the modified example has an add comment button 3017 added to the editing work area 3010, in comparison with the storyboard creation screen 300 shown in Fig. 5. The add comment button 3017 corresponds to the add comment menu described above.

[0074] FIG. 13 is an explanatory diagram of the comment add button 3017. FIG. 13 shows an example of a display after the comment add button 3017 is selected on the storyboard creation screen 300 and drawing is performed. When the comment add button 3017 is selected, a free trajectory can be drawn using a touch panel pen, similar to the drawing button 3015. Until another button is selected, the trajectory drawn on the editing screen 301 is displayed on the layer for comments. The drawing data on the layer for comments is stored as comment data for explaining the operation for each frame in association with the identification data of the selected frame. In FIG. 13, an example of comment data is shown as a trajectory 50. The type (color, thickness, etc.) of the pen (brush) used to draw the comment can be selected so as to be distinguished from the pen (brush) used when the drawing button 3015 is selected. In FIG. 13, the trajectory 50 is drawn, for example, as handwritten characters indicating a wind-blown performance and a line indicating a fluttering movement.

[0075] It is possible to set whether or not the comment data is to be output superimposed on each frame when the storyboard data is played back as a video. If it is set to be output superimposed, when video playback is selected (FIG. 4, S128: YES), the processing unit 10 plays back the comment data in chronological order with the comment trajectory 50 for each frame superimposed on each frame. The processing unit 10 can output the video to be played back as video data, and at that time, it is also possible to set whether to output the comment data trajectory 50 as a video with or without superimposition. This allows the user to add information that supplements the movement expressed in the frame to be added to each frame in the storyboard state of the animation or live-action video that the user imagines. By being able to play back the video with the comments superimposed, the user can imagine the same thing as the direction of the animation or live-action video that is completed based on the storyboard with other users who share the storyboard.

[0076] Returning to FIG. 12, the description will be continued. The storyboard creation screen 300 of the modified example includes a screen switching button 32 in an editing work area 3010. In the storyboard creation screen 300 shown in FIG. 12, editing on an editing screen 301 is selected. The screen switching button 32 realizes switching between a second editing screen 320 (see FIG. 15) which is not a screen for drawing on each frame like the editing screen 301, but allows writing of a picture of each frame and text explaining the action in that frame in a table format known as a storyboard. The second editing screen 320 is a screen which allows previewing and editing the result when printed out. In other words, it is possible to output, for example, as document data such as PDF data, while keeping the display mode on the second editing screen 320. The second editing screen 320 will be described in detail later.

[0077] The process of accepting the switch to the second editing screen 320 is performed, for example, when it is determined to be NO in step S104 in the processing procedure shown in Figures 2 to 4. The processing unit 10 proceeds with the process in response to whether an operation to switch to the second editing screen 320 has been performed on the storyboard creation screen 300 in addition to the editing operation to the editing screen in step S102 and the operation to add a new frame or duplicate in step S104.

[0078] 14 is a flowchart showing an example of a processing procedure for accepting an operation on the second editing screen 320. The processing unit 10 executes the following processing when an operation for switching to the second editing screen 320 is performed. When the screen switching button 32 is selected again during the processing, the processing returns to the processing procedure shown in FIG. 2 to FIG. 4.

[0079] The processing unit 10 displays the second editing screen 320 (step S301). The processing unit 10 judges whether or not text input is selected on the second editing screen 320 (step S302). When it is judged that text input is selected (S302: YES), the processing unit 10 accepts the selection of a field for inputting text on the screen (step S303) and accepts text editing for the selected field (step S304). In step S304, the processing unit 10 displays the input text in an editable state, and may display a soft keyboard or may accept text from a keyboard connected to the information processing device 1. The processing unit 10 stores the text in association with the identification data of the input field frame (step S305), and returns the process to step S302.

[0080] If it is determined that text input is not selected (S302: NO), the processing unit 10 determines whether or not comment input is selected (step S306). If it is determined that comment input is selected (S306: YES), the processing unit 10 accepts free trajectory drawing on the layer that covers the whole (step S307), and stores the drawing data in association with a position on the layer that covers the tabular form (step S308). In step S308, the processing unit 10 may separate the layers by page of the print output and store the drawing data in each layer. The processing unit 10 returns the process to step S302, and accepts drawings and updates the drawing data while comment input is selected.

[0081] If it is determined that comment input has not been selected (S306: NO), the processing unit 10 determines whether or not a specific operation has been performed on the display area of ​​the frame (step S309). The specific operation is, for example, a double tap. If it is determined that a specific operation has not been performed (S309: NO), the processing unit 10 returns the process to step S302.

[0082] If it is determined that a specific operation has been performed (S309: YES), the processing unit 10 identifies the identification data of the frame that is the object of the operation (step S310), and displays the editing screen 301 of the frame of the identified identification data, i.e., the screen of Fig. 5, etc. (step S311). The processing unit 10 ends the display of the second editing screen 320, and moves the process to the processing procedure of Figs. 2 to 4.

[0083] 15 and 16 show display examples of the second editing screen 320. As shown in Fig. 15 and 16, in the second editing screen 320, frames are arranged vertically, and each frame is shown in a table format with the frame number, a note explaining the action in the frame, and the frame timestamp. The frames may be divided into a predetermined number (for example, five) of frames for each page when printed out. The boundary between the first and second pages is displayed in the second editing screen 320 in Fig. 16.

[0084] The second editing screen 320 includes an add comment button 321 and a text input button 322. When the user selects the add comment button 321, as described with reference to FIG. 13, the user can draw a free trajectory using a touch panel pen. The trajectory drawn on the second editing screen 320 is displayed on a layer for comments. In FIG. 15, an example of comment data is shown as a trajectory 51. The layer for comments covers the entire second editing screen 320 or is one page at a time, so that it is possible to draw lines expressing movement across multiple frames, handwritten characters, and the like, as shown in FIG. 15. It is preferable that the user can select whether or not to output the trajectory 51 together with the printed output.

[0085] The second editing screen 320 shown in Fig. 16 shows an example of the display when the text input button 322 is selected. In Fig. 16, the user has touched the comment area of ​​the second frame with a touch panel pen, causing the comment area to become an editing area 323 and displaying a cursor. In this way, since text can be input using a keyboard or a software keyboard, operability is improved.

[0086] In addition, the storyboard program P1 may be configured to call, as an additional function, a counter function within the program that measures the tempo (BPM: Beats Per Minute) of sounds such as voices, sound effects, or background music to be synchronized when playing back a plurality of frames. This counter function allows the user to measure and understand the tempo at the timing of tapping while playing back the sound. Based on the measured tempo, the processing unit 10 may automatically determine the unit of time for each frame. For example, in the case of 150 BPM, the processing unit 10 may set 0.4 seconds (=60 seconds x 1 / 150) per frame as the minimum unit, and assign a frame that was set to 1 second to a length that is a multiple of the minimum unit, such as 0.8 seconds or 1.2 seconds.

[0087] The storyboard program P1 may further include a setting function of a background image reflected on all frames. The background image may be set for each frame. As the background image, a line drawing such as a perspective ruler that assists drawing is stored in the storage unit 11. When the user selects the background image setting menu, the processing unit 10 may accept a selection from background image candidates stored in the storage unit 11 or an image provided by a Web server to which the information processing device 1 can be connected for communication. The background image may be set from an image (including a photo for tracing) stored in a storage (camera roll) that can be shared with other applications of the information processing device 1 by the user, and may be uploaded to the above-mentioned Web server so that other users can use it. It is preferable that the background image can be selected whether to include it when playing the video or not when playing the video, similarly to the comment data.

[0088] The embodiments disclosed above are illustrative in all respects and are not restrictive. The scope of the present invention is defined by the claims, and includes all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0089] 1. Information processing device 10 Processing section 13 Display unit (display) 14 Operation unit (touch panel) P1 Storyboard Program

Claims

1. A computer with a touch panel display, displaying on the display a plurality of objects corresponding to a plurality of frames constituting the storyboard, together with thumbnails of images drawn for each frame, in chronological order; accepting, on the display, a selection of an object corresponding to a frame to be edited from among the plurality of objects; displaying an edit screen for a frame corresponding to the selected object on the same screen as the plurality of objects; On the displayed editing screen, drawing and editing of the frame is accepted, displaying a playback button on the same screen for playing back the plurality of frames as a video in accordance with a set time axis; When the play button is selected on the display, the plurality of frames are played as a video; The playback time of the video corresponding to each of the frames is displayed in a manner that allows the playback time to be changed by an operable knob output to an end of the object depending on the length in the time series direction of each of the corresponding objects. A computer program that executes a process.

2. The computer includes: displaying an interface for switching a display method of the plurality of objects corresponding to the plurality of frames while displaying the editing screen; Each time the interface is selected, a display method for the plurality of objects is switched.

2. A computer program product as claimed in claim 1, which is adapted to carry out a process.

3. The computer includes: When the play button is selected, a waveform of a sound to be output in synchronization with the video is displayed substantially parallel to the plurality of objects in which the plurality of previous frames are arranged in chronological order.

2. A computer program product as claimed in claim 1, which is adapted to carry out a process.

4. The computer includes: superimposing the plurality of frames on the plurality of objects arranged in chronological order, and displaying a cursor movable in the arranged direction; The plurality of frames are displayed in a switching manner in response to the movement of the cursor. A computer program product according to any one of claims 1 to 3, which is adapted to carry out a process.

5. The computer includes: Accept the drawing of a comment for the frame to be edited on a layer separate from the said frame, storing the drawing data of the comment in association with the identification data of the frame; Accepts a setting as to whether or not to superimpose the comment when playing the video. A computer program product according to any one of claims 1 to 3, which is adapted to carry out a process.

6. The computer includes: displaying a second editing screen in which the plurality of frames are arranged vertically and identification data of each frame, a comment explaining the action in each frame, and a time stamp of each frame are displayed in a table format; Accepting text editing for the comment on the second editing screen and drawing with a free trajectory on a layer covering the second editing screen A computer program product according to any one of claims 1 to 3, which is adapted to carry out a process.

7. A computer having a display with a built-in touch panel, displaying on the display a plurality of objects corresponding to a plurality of frames constituting the storyboard, together with thumbnails of images drawn for each frame, in chronological order; accepting, on the display, a selection of an object corresponding to a frame to be edited from among the plurality of objects; displaying an edit screen for a frame corresponding to the selected object on the same screen as the plurality of objects; On the displayed editing screen, drawing and editing of the frame is accepted, displaying a playback button on the same screen for playing back the plurality of frames as a video in accordance with a set time axis; When the play button is selected on the display, the plurality of frames are played as a video; displaying an interface for switching a display method of the plurality of objects corresponding to the plurality of frames while displaying the editing screen; Each time the interface is selected, a display method for the plurality of objects is switched. A computer program that executes a process.

8. A computer having a display with a built-in touch panel, displaying on the display a plurality of objects corresponding to a plurality of frames constituting the storyboard, together with thumbnails of images drawn for each frame, in chronological order; accepting, on the display, a selection of an object corresponding to a frame to be edited from among the plurality of objects; displaying an edit screen for a frame corresponding to the selected object on the same screen as the plurality of objects; On the displayed editing screen, drawing and editing of the frame is accepted, displaying a playback button on the same screen for playing back the plurality of frames as a video in accordance with a set time axis; When the play button is selected on the display, the plurality of frames are played as a video; displaying a second editing screen in which the plurality of frames are arranged vertically and identification data of each frame, a comment explaining the action in each frame, and a time stamp of each frame are displayed in a table format; Accepting text editing for the comment on the second editing screen and drawing with a free trajectory on a layer covering the second editing screen A computer program that executes a process.

9. A built-in touch panel display and a processing unit; The processing unit includes: displaying on the display a plurality of objects corresponding to a plurality of frames constituting the storyboard, together with thumbnails of images drawn for each frame, in chronological order; accepting a selection of an object corresponding to a frame to be edited from among the plurality of objects; displaying an edit screen for a frame corresponding to the selected object on the same screen as the plurality of objects; On the displayed editing screen, drawing and editing of the frame is accepted, displaying a playback button on the same screen for playing back the plurality of frames as a video in accordance with a set time axis; When the play button is selected on the display, the plurality of frames are played as a video; The playback time of the video corresponding to each of the frames is displayed in a manner that allows the playback time to be changed by an operable knob output to an end of the object depending on the length in the time series direction of each of the corresponding objects. An information processing device comprising a processing unit.

10. A computer equipped with a display having a built-in touch panel, displaying on the display a plurality of objects corresponding to a plurality of frames constituting the storyboard, together with thumbnails of images drawn for each frame, in chronological order; accepting, on the display, a selection of an object corresponding to a frame to be edited from among the plurality of objects; displaying an edit screen for a frame corresponding to the selected object on the same screen as the plurality of objects; On the displayed editing screen, drawing and editing of the frame is accepted, displaying a playback button on the same screen for playing back the plurality of frames as a video in accordance with a set time axis; When the play button is selected on the display, the plurality of frames are played as a video; The playback time of the video corresponding to each of the frames is displayed in a manner that allows the playback time to be changed by an operable knob output to an end of the object depending on the length in the time series direction of each of the corresponding objects. Information processing methods.

Images