Computer program, information processing device, and information processing method

Abstract

To provide a computer program, an information processing device, and an information processing method capable of easily creating a storyboard.SOLUTION: A computer program causes a computer with a built-in touch panel display to perform processing for: arranging and displaying a plurality of objects corresponding to each of a plurality of frames constituting a storyboard in chronological order along with thumbnails of images drawn for each frame, on the display; accepting a selection of an object corresponding to a frame to be edited from among the plurality of objects displayed, on the display; displaying an editing screen of a frame corresponding to the selected object in the same screen together with the plurality of objects; accepting drawing edits to the frame on the editing screen; displaying a play button, which plays the plurality of frames as a video in accordance with the set timeline, in the same screen; and playing the plurality of frames as a video when the play button is selected on the display.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present invention relates to a computer program, an information processing apparatus, and an information processing method for assisting in creating storyboards.

Background Art

[0002] In the fields of video production and animation production, primarily for image sharing, storyboards are created by arranging simple hand-drawn punch drawings in chronological order. These days, it has become possible to create storyboards by arranging pictures drawn using image editing software instead of the conventional hand-drawn punch-drawing storyboards. To visualize the video to be produced, functions such as continuously playing the arranged images and outputting them as a moving image have also been proposed as production support technologies (see Patent Document 1, etc.).

[0003] Due to the technological progress of hardware related to image processing, the functions of image editing programs have expanded enormously in various ways. Even if a program has a function for creating storyboards, other functions cannot be excluded. For users who only want the function of creating storyboards, using such an image editing program results in a redundant configuration, such as increased costs due to the presence of other functions or the need for higher-performance hardware.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] An object of the present invention is to provide a computer program, an information processing apparatus, and an information processing method with functions focused on easily creating storyboards.

Means for Solving the Problems

[0006] A computer program according to an embodiment of the present disclosure causes a computer including a display with a built-in touch panel to display, on the display, a plurality of objects respectively corresponding to a plurality of frames constituting a picture storyboard in time series together with thumbnails of images drawn for each frame, accept a selection of an object corresponding to a frame to be edited from among the plurality of objects on the display, display an editing screen for the frame corresponding to the selected object within the same screen as the plurality of objects, accept drawing editing to the frame on the displayed editing screen, display a playback button for playing back the plurality of frames as a video in accordance with a set time axis within the same screen, and execute a process of playing back the plurality of frames as a video when the playback button is selected on the display.

[0007] An information processing apparatus according to an embodiment of the present disclosure includes a display with a built-in touch panel and a processing unit. The processing unit causes the display to display, in time series, a plurality of objects respectively corresponding to a plurality of frames constituting a picture storyboard together with thumbnails of images drawn for each frame, accepts a selection of an object corresponding to a frame to be edited from among the plurality of objects, displays an editing screen for the frame corresponding to the selected object within the same screen as the plurality of objects, accepts drawing editing to the frame on the displayed editing screen, displays a playback button for playing back the plurality of frames as a video in accordance with a set time axis within the same screen, and includes a processing unit that plays back the plurality of frames as a video when the playback button is selected on the display.

[0008] An information processing method according to an embodiment of the present disclosure includes a computer having a display with a built-in touch panel that arranges a plurality of objects corresponding to a plurality of frames constituting a picture book in time series together with thumbnails of images drawn for each frame and displays them on the display, receives a selection of an object corresponding to a frame to be edited from among the plurality of objects on the display, displays an editing screen for the frame corresponding to the selected object within the same screen as the plurality of objects, receives drawing editing for the frame on the displayed editing screen, displays a playback button for playing back the plurality of frames as a video in accordance with a set time axis within the same screen, and when the playback button is selected on the display, plays back the plurality of frames as a video.

[0009] In the field of image processing, with regard to the display of thumbnails for each frame and the reception of image editing operations, there may be differences in the way of referring to memory and the way of expanding to memory when calling software that is the basis of control based on an OS (Operating System) or a program. In this case, even if it is a simple configuration, it is not easy to receive editing or display a video while holding all data to be displayed within the same screen in a temporary storage area, and since hardware resources such as memory are not always used in a sufficient environment, there have been few examples realized by conventional image editing programs. Furthermore, the above-described ways of referring and expanding to memory may be changed by an update of the OS or an update of the underlying software.

[0010] In realizing a computer program with functions limited so that a picture book can be easily created, not only should the functions be simply narrowed down, but the software configuration also requires ingenuity to be simple. As described above, especially in an information terminal such as a tablet-type information terminal or a smartphone where there are limitations on memory and hardware resources for drawing, the display of thumbnails with different ways of referring to memory and the reception of editing have not been executed within the same screen.

[0011] In the computer program, information processing apparatus, and information processing method of the present disclosure, on a tablet terminal equipped with a display with a built-in touch panel, or a smartphone or the like, the display of thumbnails of each frame of a picture book and the editing of an image to a selected frame are realized within the same screen.

[0012] The computer program according to an embodiment of the present disclosure may cause the computer to display an interface for switching a display method of the plurality of objects corresponding to the plurality of frames while the editing screen is being displayed, and execute a process of switching the display method of the plurality of objects each time the interface is selected.

[0013] The computer program according to an embodiment of the present disclosure may cause the computer to execute a process of displaying a waveform of a sound output in synchronization with the video when the play button is selected, substantially in parallel with the plurality of objects arranged in time series of the plurality of frames.

[0014] The computer program according to an embodiment of the present disclosure may cause the computer to display a cursor that can be moved in the arranged direction by superimposing it on the plurality of objects arranged in time series of the plurality of frames, and execute a process of switching and displaying the plurality of frames in response to the movement of the cursor.

[0015] The computer program according to an embodiment of the present disclosure may cause the computer to execute a process of displaying a playback time in the video corresponding to each frame.

Advantages of the Invention

[0016] According to the computer program of the present disclosure, while displaying thumbnails of each frame, any one of the thumbnails can be selected, and frames can be edited within the same screen while checking the thumbnails of the previous and subsequent frames. Furthermore, video playback that displays the edited frames in chronological order can also be executed without switching the screen. In this way, a computer program focused on the functions necessary for creating storyboards is realized.

Brief Description of the Drawings

[0017]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Embodiments for Carrying Out the Invention

[0018] The present disclosure will be specifically described with reference to the drawings showing its embodiments. In the following embodiments, a method for creating a picture story using the computer program of the present disclosure will be described.

[0019] FIG. 1 is a block diagram showing the configuration of the information processing apparatus 1 of the present disclosure. The information processing apparatus 1 is a tablet terminal including a display with a built-in touch panel as the display unit 13. The information processing apparatus 1 may be a smartphone. The information processing apparatus 1 may be a desktop or laptop personal computer.

[0020] The information processing apparatus 1 includes a processing unit 10, a storage unit 11, a communication unit 12, a display unit 13, an operation unit 14, and an audio input / output unit 15.

[0021] The processing unit 10 includes a processor 100 using a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and / or an MPU (Micro-Processing Unit). The processing unit 10 includes a memory 101 such as a DRAM (Dynamic Random Access Memory), an SRAM (Static Random Access Memory), or an SDRAM (Synchronous Dynamic Random Access Memory). The processing unit 10 stores the data generated during processing in the memory 101 or the storage unit 11, and executes calculations based on the picture story program P1 stored in the storage unit 11.

[0022] The storage unit 11 uses a non-volatile memory such as a flash memory or an SSD (Solid State Drive). The storage unit 11 stores the data used by the processing unit 10. The storage unit 11 stores the picture content program P1. The picture content program P1 may be a copy that the processing unit 10 reads out the picture content program P9 stored in the storage medium 9 and copies it to the storage unit 11. The picture content program P1 may also be downloaded by the processing unit 10 from another program server device via the communication unit 12 and stored in the storage unit 11 in an executable manner.

[0023] The communication unit 12 realizes communication with the server device 2 via a network including the Internet or a carrier network. Specifically, the communication unit 12 is a network card. The communication unit 12 may be a wireless communication module connected to a carrier network, or may be a wireless communication module for Wi-Fi (registered trademark). The processing unit 10 can transmit and receive data to and from an external device such as a server device (not shown) by the communication unit 12.

[0024] The display unit 13 is a display such as a liquid crystal display or an organic EL (Electro Luminescence) display. The display unit 13 displays the GUI (Graphical User Interface) of the picture content program P1. The display unit 13 is a display with a built-in touch panel. The display unit 13 is not limited to being a display with a built-in touch panel.

[0025] The operation unit 14 is a user interface capable of input and output with the processing unit 10. In one example, the operation unit 14 is a touch panel built into the display unit 13. The operation unit 14 may include a pen for the touch panel. The operation unit 14 may be a mouse, a keyboard, etc. The operation unit 14 may include physical buttons.

[0026] The audio input / output unit 15 includes a speaker and a microphone. The processing unit 10 can output sound from the speaker and receive audio input from the microphone. The processing unit 10 may use the audio input / output unit 15 as part of the operation by the operation unit 14. That is, the processing unit 10 may recognize the operation content from the input audio by the microphone and receive the operation in the form of dialogue with the sound output from the speaker.

[0027] The general-purpose information processing apparatus 1 configured as described above realizes easy creation of a picture content based on the picture content program P1. FIGS. 2-4 are flowcharts showing an example of the processing procedure executed by the information processing apparatus 1. When the picture content program P1 is selected and activated by the user and the creation menu of a new picture content displayed by the picture content program P1 is selected, the processing unit 10 executes the following processing procedure.

[0028] The processing unit 10 displays on the display unit 13 a picture content creation screen including an object obtained by shrinking the first frame constituting the new picture content and an editing screen for the first frame (step S101). In step S101, the processing unit 10 displays the editing screen by expanding and displaying the drawing data of the entire frame in the memory 101, and for the object, refers to the drawing data for the corresponding frame from the memory 101 for display. The drawing data is described in vector format or raster format for each layer in association with the identification data of the frame.

[0029] The processing unit 10 determines whether to receive an editing operation on the editing screen by the operation unit 14 (step S102). If it is determined that an editing operation is received (S102: YES), the processing unit 10 draws on the editing screen according to the editing operation (step S103) and advances the process to step S104. The result drawn in step S103 is reflected in the object corresponding to the first frame by referring to it from the memory 101.

[0030] When it is determined that the editing operation on the editing screen is not accepted (S102: NO), the processing unit 10 proceeds with the processing to step S104 as it is.

[0031] The processing unit 10 determines whether an operation of newly adding or duplicating a frame is accepted (step S104). The drawing frame creation screen includes an interface for accepting an operation of newly adding a blank frame and an interface for accepting an operation of duplicating the selected frame (see FIG. 5), and the processing unit 10 accepts an operation by the operation unit 14 of any one of the interfaces. When it is determined that an operation of newly adding or duplicating a frame is not accepted (S104: NO), the processing unit 10 returns the processing to step S102.

[0032] When it is determined that a new addition or duplication of a frame is accepted (S104: YES), the processing unit 10 stores (moves) the drawing data of the drawing result on the editing screen of the first frame in the temporary storage area of the storage unit 11 or another area of the memory 101 in association with the identification data of the frame (step S105), and deletes the editing screen of the first frame from the drawing frame creation screen (step S106). The processing unit 10 switches the reference destination of the drawing data of the object of the first frame from the memory 101 to the storage destination (step S107), and displays the object so as to be distinguishable from the object of the newly added or duplicated frame (step S108).

[0033] The processing unit 10 assigns identification data (second) as the second frame to the newly added or duplicated new frame (step S109), and displays an editing screen for the new frame within the drawing frame creation screen (step S110). The object of the new frame is displayed side by side with the first frame at a position corresponding to a time later than that of the first frame (step S111). In step S110, the processing unit 10 expands and displays the drawing data of the entire frame in the memory 101 for the editing screen in the same manner as the first frame in step S101. In step S111, the processing unit 10 displays the object by referring to the drawing data for the corresponding frame from the memory 101.

[0034] The processing unit 10 determines whether to accept an editing operation on a new frame at the drawing content creation screen by the operation unit 14 (step S112). When it is determined that the editing operation is accepted (S112: YES), the processing unit 10 draws on the editing screen according to the editing operation (step S113), and proceeds to step S114. The result drawn in step S113 is reflected in the object corresponding to the new frame by reference from the memory 101.

[0035] When it is determined that the editing operation on the editing screen is not accepted (S112: NO), the processing unit 10 proceeds to step S114 as it is.

[0036] The processing unit 10 determines whether an operation of newly adding or duplicating the third and subsequent frames is accepted (step S114). When it is determined that the operation of newly adding or duplicating a frame is accepted (S114: YES), the processing unit 10 stores (moves) the drawing data of the drawing result on the editing screen being displayed in the temporary storage area of the storage unit 11 or another area of the memory 101 in association with the identification data of the target frame (step S115), and deletes the editing screen previously displayed from the drawing content creation screen before adding a new frame (step S116). The processing unit 10 switches the reference destination of the drawing data of the object of the frame corresponding to the editing screen previously displayed to the storage destination from the memory 101 (step S117), and displays the object so as to be distinguishable from the object of the new frame (step S118).

[0037] The processing unit 10 assigns identification data as the next frame of the frame for which the editing screen was displayed before addition to the newly added or duplicated new frame (step S119). The processing unit 10 displays the editing screen for the new frame within the drawing content creation screen (step S120), displays the object of the new frame side by side with the object of the frame for which the editing screen was displayed before addition (step S121), and proceeds to step S128.

[0038] When the processing unit 10 determines in step S114 that an operation for newly adding or duplicating a frame has not been received (S114: NO), the processing unit 10 determines whether an operation for selecting an object of another frame has been received by the operation unit 14 (step S122).

[0039] When it is determined that an operation for selecting an object of another frame has been received (S122: YES), the processing unit 10 stores (moves) the drawing data of the drawing result on the editing screen being displayed in the temporary storage area of the storage unit 11 or another area of the memory 101 in association with the identification data of the target frame (step S123). The processing unit 10 deletes the editing screen displayed before selecting the object of another frame from the drawing container creation screen (step S124). The processing unit 10 switches the reference destination of the drawing data of the object of the frame corresponding to the editing screen displayed before selecting the object of another frame from the memory 101 to the storage destination (step S125), and causes the object to be displayed distinguishable from the object of the selected frame (step S126). The processing unit 10 displays the editing screen for the selected frame within the drawing container creation screen (step S127), and advances the process to step S128.

[0040] The processing unit 10 determines whether video playback based on a plurality of frames including the frame being edited has been selected (step S128). When it is determined that video playback has not been selected (S128: NO), the processing unit 10 returns the process to step S112 and accepts an editing operation on the editing screen being displayed, selection of another frame, addition of a new frame, etc.

[0041] When it is determined that video playback is selected (S128: YES), the processing unit 10 plays a plurality of frames including the added frames as a video in time series (step S129). In step S129, the processing unit 10 displays the frames including the drawing data drawn according to the set time axis in time series. For example, the processing unit 10 switches and displays one frame every set time such as 2 seconds. As will be described later, a display time may be set for each frame. The processing unit 10 may display it by superimposing it on the editing screen being displayed, or may display it in another available area within the storyboard creation screen.

[0042] The processing unit 10 determines whether the end of the storyboard program P1 is selected (step S130). When it is determined that the end of the storyboard program P1 is not selected (S130: NO), the processing unit 10 returns the process to step S112.

[0043] When it is determined that the end of the storyboard program P1 is selected (S130: YES), the processing unit 10 ends the storyboard program P1.

[0044] As shown in FIGS. 2-4, in order to display an object including thumbnails of a plurality of frames in the storyboard creation screen, display an editing screen for any one of the plurality of frames, and accept frame addition and selection, it is necessary to perform processes such as expanding the drawing data on the memory 101 and switching the reference destination. In order to optimize the drawing process using an OS or an existing development engine, such as a function for accepting drawing, the handling and reference method of the data on the memory 101 need to be compatible with those existing programs. Therefore, it has been difficult to realize a simple interface as described later. By implementing the above-described processing procedure, it becomes possible to create a storyboard with a simple interface.

[0045] Regarding the processing procedures shown in FIGS. 2-4, the functions will be described with reference to a specific example of the interface of the picture content program P1. FIG. 5 is a diagram showing an example of the picture content creation screen 300. In FIG. 5, the picture content creation screen 300 includes an editing screen 301 for the first frame and an object 302 corresponding to the thumbnail of the first frame. In FIG. 5, since the picture content data has just been newly created, there is only one frame. When the user draws a picture on the editing screen 301 being displayed on the display of the display unit 13 using a touch pen for a tablet or the like, the trajectory is directly received by the operation unit 14, and a preview is displayed as shown in FIG. 5 based on the drawing data by the processing unit 10. Enlargement and reduction within the editing screen 301 can be implemented by pinch-out, pinch-in, etc. on the editing screen 301. In the editing operation on the editing screen 301, not limited to the drawing operation of drawing a line, etc., editing functions such as Redo and Undo assigned to respective gestures using a touch pen or the user's finger may be realized to realize an intuitive operation. The preview is also reflected in the thumbnail of the object 302 corresponding to the frame displayed on the editing screen 301.

[0046] The picture content creation screen 300 includes a delete button 303 for accepting the deletion of the frame being displayed on the editing screen, a new addition button 304 for accepting the operation of newly adding a blank frame, and a duplicate button 305 for accepting the operation of duplicating the selected frame.

[0047] The processing when the delete button 303 is selected is not described in the processing procedures of FIGS. 2-4, but the processing unit 10 deletes the display of the selected frame and deletes the corresponding frame data on the memory 101 and the storage unit 11.

[0048] When the new addition button 304 is selected, a new blank frame is inserted as the next frame of the displayed frame by the processing of steps S104 and S114 in the processing procedures of FIGS. 2-4. The insertion position is not limited to this, and it may be inserted as the previous frame.

[0049] When the copy button 305 is selected, among the processing procedures of FIGS. 2-4, the copied frames of the frames being displayed are inserted as the next frame by the processing of steps S104 and S114. The drawing data associated with the frames is copied and stored in association with the identification data of another frame. For the copied frames, on the editing screen 301, further drawing and editing such as deletion can be performed independently of the original frames.

[0050] As shown in FIG. 5, the editing work area 3010 for the editing screen 301 may include a layer switching button 3011. For the editing screen 301, an image data insertion button 3012 into the editing screen 301, a pointer button 3013 for selecting a range or point on the editing screen 301, an eraser button 3014 for erasing the selected location, and a drawing button 3015 for drawing at the selected location with the currently selected pen type are displayed. The user can sequentially select the layer switching button 3011, the insertion button 3012, the pointer button 3013, the eraser button 3014, and the drawing button 3015 to draw on the editing screen 301.

[0051] A zoom slide 3016 for the editing screen 301 may be displayed. The zoom slide 3016 is displayed when the eraser button 3014 or the drawing button 3015 is selected. When operating on the zoom slide 3016, the size of the eraser is enlarged or reduced, or the size of the drawing pen (brush) is enlarged or reduced.

[0052] The picture content creation screen 300 further includes a video playback button 306. The video playback button 306 is always displayed, but may be enabled when the number of frames becomes two or more.

[0053] The picture content creation screen 300 includes a menu button 307 for ending the picture content program P1. When the menu button 307 is selected, for example, the processing procedure of step S130 shown in the flowcharts of FIGS. 2-4 is executed, and the picture content program P1 ends.

[0054] The drawing composition creation screen 300 includes a scale 308 indicating the elapsed time setting for each of a plurality of frames. The scale 308 includes graduations at predetermined unit intervals. With the scale 308, it is possible to check the playback time in the video corresponding to each frame. In the drawing composition creation screen 300 of FIG. 5, since there is only one frame, 1 second is assigned to each frame according to the initial setting, and it is shown that the time from the start to the end of the frame is 1 second. With the scale 308, as shown in FIG. 6, it is possible to set the length of time occupied by each frame according to the length of the object 302 in the direction along the scale 308. As shown in FIG. 11, a marker 311 can be placed on the scale 308, and the user can also consider the assignment of frames such that scene transitions with particular timing, such as with audio data, can be achieved by the marker.

[0055] The drawing composition creation screen 300 includes a sound reception unit 309 for receiving sound data such as audio or music that is output in synchronization with the playback of a plurality of frames, parallel to the arrangement direction of the objects 302 on the screen. In the drawing composition creation screen 300 of FIG. 5, no sound is received and it is blank.

[0056] FIG. 6 is a diagram showing another example of the drawing composition creation screen 300. The drawing composition creation screen 300 of FIG. 6 shows the state after two additional frames are added. The same reference numerals are given to the components common to FIG. 5, and detailed description thereof is omitted. In the drawing composition creation screen 300 of FIG. 6, a plurality of objects 302 corresponding to the plurality of frames constituting the drawing composition being created are arranged in time series together with the thumbnails of the images drawn for each frame. The direction in which the objects 302 are arranged is the horizontal direction (lateral direction, width direction) in the image in FIG. 6, but is not limited thereto, and may be the vertical direction (longitudinal direction, height direction) in the image.

[0057] Among the plurality of objects 302, the object 302 corresponding to the frame to be edited is highlighted, and the objects 302 of the other frames are masked in gray and displayed, so that the selected frame and the others can be distinguished.

[0058] In the drawing content creation screen 300 of FIG. 6, the time occupied by the first frame is extended by a factor of two. In accordance with the scale 308, the time may be extended by an operation of stretching the object 302. The graduations of the scale 308 may be made grabable.

[0059] FIG. 7 is a diagram showing another example of the drawing content creation screen 300. In FIG. 7, a knob 3021 for changing the time occupied by a frame is displayed for the selected frame among the objects 302. In the drawing content creation screen 300 of FIG. 7, it is shown that the start time of the frame being selected is advanced by 0.5 seconds and, further, the end time of the frame being selected is adjusted to be 0.5 seconds later as compared with the drawing content creation screen 300 of FIG. 6. Thus, with the drawing content program P1 of the present disclosure, while displaying the editing screen 301 for the frame being selected, the flow of frames and the adjustment of the timing for the whole of each frame can be adjusted intuitively.

[0060] Returning to FIG. 6 to continue the description. In the drawing content creation screen 300 of FIG. 6, a cursor 312 that can move in the time axis direction is displayed superimposed on a plurality of objects 302 in which a plurality of frames are arranged in time series. The cursor 312 can be moved by a user operation. For example, when an input operation for designating time is performed on a time display unit 313 that displays the elapsed time, the cursor 312 moves to the location of the designated time. A moving operation may be possible for the cursor 312 itself. When the movement of the cursor 312 is detected, the processing unit 10 may display an image of a frame corresponding to the position of the cursor 312 in the same range as the editing screen 301. Thereby, not only can it be played back from the beginning of the video as shown in FIG. 8 described later, but it is also possible to display a part of the flow by switching frames like an animation.

[0061] The user can select the object 302 by operating the operation unit 14 to display and edit frames on the editing screen 301, switch and display frames while moving the cursor 312, or play it as a video by operating the video playback button 306.

[0062] By using the picture composition program P1 of the present disclosure, a plurality of objects 302 corresponding to the editing screen 301 and the thumbnails of each frame arranged in time series are arranged and displayed within the same picture composition creation screen 300, and any one of the frames to be edited can be arbitrarily selected from the plurality of objects 302. The editing target of the editing screen 301 can be switched according to the operation to the one corresponding to the selected frame. The user can immediately check the preview by selecting the video playback button 306 within the same screen while performing a drawing operation on the frame.

[0063] FIG. 8 shows an example of the picture composition creation screen 300 during video playback. Reference numeral 4 shown in FIG. 8 indicates the range within which the video is played within the picture composition creation screen 300. As shown in FIG. 8, when the video playback button 306 is selected, the video is played within the editing area of the editing screen 301 in the same range as the layout during the previous editing. Thereby, the user can sequentially check how it looks when made into a video while editing each frame.

[0064] On the picture composition creation screen 300 of FIG. 8, a cursor 312 that moves in the direction of the time axis of the object 302 of each frame is displayed. The cursor 312 can be superimposed on a plurality of objects 302 arranged in time series and moved along the time axis. The cursor 312 may be controlled to move in accordance with the display of the video within the range 4. Thereby, the user can check which frame the frame being played is at the position of the cursor 312 on the plurality of objects 302.

[0065] Returning to FIG. 6, the description will continue. On the drawing content creation screen 300 of FIG. 6, a display switching button 310 is shown. The display switching button 310 is a button for reducing the range including the objects 302 arranged in time series, the scale 308, and the sound reception unit 309, and increasing the range occupied in the display unit 13 of the editing screen 301.

[0066] FIG. 9 shows an example of the drawing content creation screen 300 when the display switching button 310 is selected. The drawing content creation screen 300 shown in FIG. 9 shows a state where the display switching button 310 is selected by the user on the drawing content creation screen 300 shown in FIG. 6, and the area for displaying the objects 302 and the like is reduced. As a result, the range of the editing screen 301 and the editing work area 3010 is expanded in the drawing content creation screen 300 shown in FIG. 9 compared to the drawing content creation screen 300 shown in FIG. 6, and the operability of the user's editing operation on the frames is improved.

[0067] When the display switching button 310 is selected on the drawing content creation screen 300 shown in FIG. 9, conversely, the range including the objects 302 arranged in time series, the scale 308, and the sound reception unit 309 returns to the range shown in FIG. 6, and the editing screen 301 returns to the initial range. In this way, the arrangement of the objects 302 for confirming the connection with other frames and the editing screen 301 can be easily switched by an operation on one interface so that they can be grasped at once in the same screen, and the operability is improved.

[0068] The variations of the drawing content creation screen 300 when the display switching button 310 is selected are not limited to this. FIG. 10 is a diagram showing another example of the drawing content creation screen 300. In FIG. 10, as a display method of the object 302, a state where the object 302 is folded and hidden is shown. In this way, it is also possible to temporarily hide the object 302 on the same screen.

[0069] FIG. 11 is a diagram showing another example of the drawing content creation screen 300. In the drawing content creation screen 300 of FIG. 11, sounds such as voices, sound effects, or BGM are set in the sound reception unit 309. In the sound reception unit 309, after selecting within the range of the sound reception unit 309 and then selecting the new addition button 304, an addition operation is possible from the list of sounds such as voices, sound effects, or BGM that are displayed. Additionally, for sound settings, an addition operation may also be possible by selecting a button (musical note symbol shown in FIG. 5 etc.) within the range of the sound reception unit 309 or from the list of sounds that is displayed when the range within the sound reception unit 309 is long-pressed. In FIG. 11, in the sound reception unit 309, waveforms of sounds that are output in synchronization with the video are displayed parallel to the objects 302 arranged in time series. The waveforms displayed in the sound reception unit 309 can be shifted in the time series direction with respect to the arrangement of the objects 302.

[0070] In the drawing content creation screen 300 shown in FIG. 11, a marker 311 is attached to the scale 308. The marker 311 can be arranged by the user selecting an arbitrary location on the scale 308 with the operation unit 14. The processing unit 10 stores the playback time from the start of the video when a plurality of frames corresponding to the position of the marker 311 on the scale 308 are played back as a video. The user can move the marker 311 while referring to the waveform of the sound to be output synchronously, and operations such as selecting the frame to be displayed at that time or setting the time of each frame are facilitated. The position of the marker 311 may be received by displaying the reception interface of the marker 311 during playback as a video. The marker 311 can be moved to the position on the time axis corresponding to the position of the cursor 312 when the reception interface is tapped during video playback. The processing unit 10 stores the times on the video corresponding to the position of the marker 311 separately. When each marker 311 is selected, it is also possible to display the frame corresponding to the time of that marker 311.

[0071] Based on the drawing content program P1, the processing unit 10 can also realize other functions. For example, the menu of the drawing content program P1 may include a comment addition menu and a write-out menu. When the comment addition menu is selected, the processing unit 10 receives the input of text data explaining the operation for each frame, and stores the input text data in the storage unit 11 in association with the frame identification data. When the write-out menu is selected, the processing unit 10 outputs the frame number, the image drawn for the frame, the written text data, and the timestamp in a tabular format. The processing unit 10 may store document data such as PDF data in the storage unit 11 as tabular data.

[0072] By using the drawing content program P1, a drawing content can be created with simple operations. The drawing content program P1 can also be used for users who have difficulty in auditory communication to easily draw handwritten pictures and characters and play them back easily within the same screen for communication, and the application is not limited to only creating drawing contents.

[0073] (Modification example) In the modification example, functions based on the above-described drawing content program P1 are further added. FIG. 12 is a diagram showing an example of the drawing content creation screen 300 in the modification example. The drawing content creation screen 300 in the modification example has a comment addition button 3017 added to the editing work area 3010 with respect to the drawing content creation screen 300 shown in FIG. 5. The comment addition button 3017 corresponds to the above-described comment addition menu.

[0074] FIG. 13 is an explanatory diagram of the comment addition button 3017. FIG. 13 shows an example of the display after the comment addition button 3017 is selected and drawn on the drawing content creation screen 300. When the comment addition button 3017 is selected, similar to the drawing button 3015, it can be drawn in a free trajectory using a pen for the touch panel. Until another button is selected, the trajectory drawn on the editing screen 301 is displayed on the layer for comments. The drawing data on the layer for comments is stored as comment data for explaining the operations for each frame in association with the identification data of the currently selected frame. In FIG. 13, an example of the comment data is shown by the trajectory 50. The type (color, thickness, etc.) of the pen (brush) when drawing comments can be selected so as to be distinguishable from the pen (brush) when the drawing button 3015 is selected. In FIG. 13, the trajectory 50 is drawn, for example, as handwritten characters indicating an effect of wind blowing and a line indicating a dancing motion.

[0075] It is possible to set whether to output the drawing content data overlaid on each frame when the drawing content data is played back as a video with respect to the comment data. When it is set to output overlaid, when video playback is selected (FIG. 4, S128: YES), the processing unit 10 plays back the comment trajectory 50 for each frame in time series in a state where it is overlaid on each frame. The processing unit 10 can output the played-back video as video data, but at that time, it is also possible to set whether to output the video in a state where the trajectory 50 of the comment data is overlaid or in a state where it is not overlaid. Thereby, in the state of the drawing content of the animation or live-action video imagined by the user, information can be added to supplement the motion expressed by the frames to be added to each frame. Since the video can be played back with the comment overlaid, the same thing can be recalled as an effect on the animation or live-action video completed based on the drawing content by other users who share the drawing content.

[0076] Returning to FIG. 12, the description will continue. The picture content creation screen 300 of the modification example includes a screen switching button 32 within the editing work area 3010. In the picture content creation screen 300 shown in FIG. 12, the editing on the editing screen 301 is selected. The screen switching button 32 is not a screen for drawing on each frame like the editing screen 301, but enables switching between the editing screen 301 and a second editing screen 320 (see FIG. 15) where text can be written to describe the picture pattern of each frame and the actions in that frame in a tabular format called a conventional so-called picture content. The second editing screen 320 is a screen where the result when printed can be previewed and edited. That is, output in document data such as PDF data is possible in the state displayed on the second editing screen 320. The second editing screen 320 will be described in detail later.

[0077] The process of accepting the switching to the second editing screen 320 is performed, for example, when it is determined as NO in step S104 in the processing procedures shown in FIGS. 2 - 4. The processing unit 10 proceeds with the processing according to whether an operation to switch to the second editing screen 320 is performed in addition to either the editing operation on the editing screen in step S102 or the operation of newly adding or duplicating a frame in step S104 on the picture content creation screen 300.

[0078] FIG. 14 is a flowchart showing an example of the processing procedure for accepting operations on the second editing screen 320. When an operation to switch to the second editing screen 320 is performed, the processing unit 10 executes the following processing. If the screen switching button 32 is selected again during the processing, the processing procedures shown in FIGS. 2 - 4 are returned to.

[0079] The processing unit 10 displays the second editing screen 320 (step S301). The processing unit 10 determines whether text input is selected on the second editing screen 320 (step S302). If it is determined that text input is selected (S302: YES), the processing unit 10 accepts, on the screen, the selection of the field for inputting text (step S303), and accepts text editing for the selected field (step S304). In step S304, the processing unit 10 may display the input text in an editable state, display the soft keyboard, or accept text from the keyboard connected to the information processing apparatus 1. The processing unit 10 stores the text in association with the identification data of the input field command (step S305), and returns the process to step S302.

[0080] If it is determined that text input is not selected (S302: NO), the processing unit 10 determines whether comment input is selected (step S306). If the processing unit 10 determines that comment input is selected (S306: YES), it accepts the drawing of a free trajectory on the layer covering the whole (step S307), and stores the drawing data in association with the position on the layer covering the tabular mode (step S308). In step S308, the processing unit 10 may divide the layers in units of pages of print output and store the drawing data for each. The processing unit 10 returns the process to step S302, and while comment input is selected, accepts drawing and updates the drawing data.

[0081] If it is determined that comment input is not selected (S306: NO), the processing unit 10 determines whether a specific operation has been performed on the display area of the command (step S309). The specific operation is, for example, a double tap. If it is determined that the specific operation has not been performed (S309: NO), the processing unit 10 returns the process to step S302.

[0082] When it is determined that a specific operation has been performed (S309: YES), the processing unit 10 identifies the identification data of the frame to be operated on (step S310), and displays the editing screen 301 of the identified frame, that is, the screen shown in FIG. 5 or the like (step S311). The processing unit 10 ends the display of the second editing screen 320 and moves the process to the processing procedures shown in FIGS. 2-4.

[0083] FIGS. 15 and 16 show an example of the display of the second editing screen 320. In the second editing screen 320, as shown in FIGS. 15 and 16, the frames are arranged vertically, and for each frame, the frame number, the writing explaining the action in the frame, and the time stamp of the frame are represented in a tabular format. The frames may be divided for each page when printing and output, for example, every predetermined number (for example, 5). In the second editing screen 320 of FIG. 16, the boundary between the first page and the second page is displayed.

[0084] The second editing screen 320 includes a comment addition button 321 and a text input button 322. When the user selects the comment addition button 321, as described with reference to FIG. 13, it can be drawn with a free trajectory using a pen for the touch panel. The trajectory drawn on the second editing screen 320 is displayed on the layer for comments. In FIG. 15, an example of comment data is shown by the trajectory 51. Since the layer for comments covers the entire second editing screen 320 or one page at a time, as shown in FIG. 15, it is possible to draw lines expressing movement across multiple frames, handwritten characters, etc. Whether to output the trajectory 51 together at the time of printing output may be selectable.

[0085] The second editing screen 320 shown in FIG. 16 shows an example of the display when the text input button 322 is selected. In FIG. 16, when the user touches the comment area of the second frame using a pen for the touch panel, it is shown that the comment area becomes the editing area 323 and a cursor is displayed. In this way, since text can be input using a keyboard or a software keyboard, etc., the operability is improved.

[0086] In addition, the picture composition program P1 may be configured to be able to call, within the program, a counter function that measures the tempo (BPM: Beat Per Minute) of sounds such as voices, sound effects, or BGM that are synchronized during the playback of a plurality of frames as an additional function. With this counter function, the user can measure and grasp the tempo at the timing of tapping while playing the sound. Based on the measured tempo, the processing unit 10 may automatically determine the unit of time for each frame. For example, if the tempo is 150 BPM, the processing unit 10 may use 0.4 seconds (= 60 seconds × 1 / 150) per frame as the minimum unit, and assign frames that were originally 1 second in length to lengths that are multiples of the minimum unit, such as 0.8 seconds or 1.2 seconds.

[0087] The picture composition program P1 may further include a function for setting a background image that is reflected in all frames. The background image may be set for each frame. As the background image, store line drawings such as a scale ruler for assisting drawing in the storage unit 11. When the user selects the background image setting menu, the processing unit 10 may be configured to accept a selection from the background image candidates stored in the storage unit 11 or images provided by a web server to which the information processing apparatus 1 can be communicatively connected. The background image can also be set from images (including photos for tracing) stored in a storage (camera roll) that can be shared with other applications of the information processing apparatus 1 by the user, and may be uploadable to the above-described web server so that it can be used by other users. It is preferable that the background image can be selected in the same way as the comment data whether to include it during video playback or not to include it during video playback.

[0088] The embodiments disclosed as above are illustrative in all respects and not restrictive. The scope of the present invention is indicated by the scope of the claims, and includes all modifications within the meaning and scope equivalent to the scope of the claims.

Explanation of Reference Numerals

[0089] 1 Information processing apparatus 10 Processing unit 13 Display unit (display) 14 Operation unit (touch panel) P1 Picture control program

Claims

1. A computer with a touch panel display, displaying on the display a plurality of objects corresponding to a plurality of frames constituting the storyboard, together with thumbnails of images drawn for each frame, in chronological order; accepting, on the display, a selection of an object corresponding to a frame to be edited from among the plurality of objects; displaying an edit screen for a frame corresponding to the selected object on the same screen as the plurality of objects; On the displayed editing screen, drawing and editing of the frame is accepted, displaying a playback button on the same screen for playing back the plurality of frames as a video in accordance with a set time axis; When the play button is selected on the display, the plurality of frames are played as a video. A computer program that executes a process.

2. The computer includes: displaying an interface for switching a display method of the plurality of objects corresponding to the plurality of frames while displaying the editing screen; Each time the interface is selected, a display method for the plurality of objects is switched.

2. A computer program product as claimed in claim 1, which is adapted to carry out a process.

3. The computer includes: When the playback button is selected, a waveform of a sound to be output in synchronization with the video is displayed approximately parallel to the plurality of objects in which the previous plurality of frames are arranged in chronological order.

2. A computer program product as claimed in claim 1, which is adapted to carry out a process.

4. The computer includes: superimposing the plurality of frames on the plurality of objects arranged in chronological order, and displaying a cursor movable in the arranged direction; The plurality of frames are displayed in a switching manner in response to the movement of the cursor. A computer program product according to any one of claims 1 to 3, which is adapted to carry out a process.

5. The computer includes: Displaying the playback time of the video corresponding to each of the frames A computer program product according to any one of claims 1 to 3, which is adapted to carry out a process.

6. The computer includes: Accept the drawing of a comment for the frame to be edited on a layer separate from the said frame, storing the drawing data of the comment in association with the identification data of the frame; Accepts a setting as to whether or not to superimpose the comment when playing the video. A computer program product according to any one of claims 1 to 3, which is adapted to carry out a process.

7. The computer includes: displaying a second editing screen in which the plurality of frames are arranged vertically and identification data of each frame, a comment explaining the action in each frame, and a time stamp of each frame are displayed in a table format; Accepting text editing for the comment on the second editing screen and drawing with a free trajectory on a layer covering the second editing screen A computer program product according to any one of claims 1 to 3, which is adapted to carry out a process.

8. A built-in touch panel display and a processing unit; The processing unit includes: displaying on the display a plurality of objects corresponding to a plurality of frames constituting the storyboard, together with thumbnails of images drawn for each frame, in chronological order; accepting a selection of an object corresponding to a frame to be edited from among the plurality of objects; displaying an edit screen for a frame corresponding to the selected object on the same screen as the plurality of objects; On the displayed editing screen, drawing and editing of the frame is accepted, displaying a playback button on the same screen for playing back the plurality of frames in a moving image in accordance with a set time axis; When the play button is selected on the display, the plurality of frames are played back as a moving image. An information processing device comprising a processing unit.

9. A computer equipped with a display having a built-in touch panel, displaying on the display a plurality of objects corresponding to a plurality of frames constituting the storyboard, together with thumbnails of images drawn for each frame, in chronological order; accepting, on the display, a selection of an object corresponding to a frame to be edited from among the plurality of objects; displaying an edit screen for a frame corresponding to the selected object on the same screen as the plurality of objects; On the displayed editing screen, drawing and editing of the frame is accepted, displaying a playback button on the same screen for playing back the plurality of frames in a moving image in accordance with a set time axis; When the play button is selected on the display, the plurality of frames are played back as a moving image. Information processing methods.

Images