Reproduction Device, Reproduction Method, and Program

Abstract

To reproduce a video synchronized with an audio reproduction position during audio data reproduction without processing the audio data in accordance with a video reproduction rate when the audio and the video are separately recorded by a variable frame rate recording function.SOLUTION: A reproduction device that reproduces audio data having the same audio recording rate and audio reproduction rate and video data recorded separately at the same timing as the audio data and having different video recording rates and video reproduction rates includes audio reproduction position acquisition means that acquires an audio reproduction position at a predetermined timing in the audio data, video reproduction position calculation means that calculates a video reproduction position corresponding to the audio reproduction position on the basis of the audio reproduction position, the video recording rate, and the video reproduction rate, and frame reading means that reads a video frame corresponding to the video reproduction position from the video data, and the video frame read by the frame reading means is reproduced.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a playback device, a playback method, and a program.

Background Art

[0002] In recent years, in video and audio recording and playback devices that record and play back audio and video, a "variable frame rate recording technology" that records video at a frame rate different from the frame rate during video playback has been used. For example, in this variable frame rate recording technology, video is recorded at 120 fps (frames per second), and the recorded video data is played back at 60 fps, so that the video is played back at twice the speed, that is, a slow motion video (slow motion movie) is obtained. Also, for example, in this variable frame rate recording technology, video is recorded at 60 fps, and the recorded video data is played back at 120 fps, so that the video is played back at twice the speed, that is, a fast motion video (fast motion movie) is obtained. These days, many products equipped with a variable frame rate recording technology (variable frame rate recording function) that can obtain such slow motion videos and fast motion videos are already on the market.

[0003] In many of the existing products described above, while audio is recorded and played back in real time, video is recorded and played back at a variable frame rate, so there is a problem that audio and video cannot be synchronized. To solve this problem, many of the existing products record video and audio separately when recording them with the variable frame rate recording function. Furthermore, the user creates video content with synchronized audio and video by adding (composing) separately recorded audio data to the video data through processes such as post-production.

[0004] In order to solve the above problems, the technology of Patent Document 1 has been proposed. In the technology of Patent Document 1, when a user records video, the processing of the audio data recorded at that time is performed in accordance with the frame rate of video playback to generate audio data during video playback, thereby creating video content with synchronized video and audio.

[0005] Here, with reference to FIG. 7, a conventional video and audio playback device that plays back video data and audio data separately recorded at the same timing with a variable frame rate recording function will be described. During recording with the variable frame rate recording function, the frame rate of video recording is the same as the frame rate of audio recording, but the frame rate of video playback is different from the frame rate of audio playback. FIG. 7 is a block diagram showing a configuration example of such a conventional video and audio playback device. As shown in FIG. 7, the video and audio playback device 70 includes a system controller 700, a first recording medium 701, a second recording medium 702, a drive unit 703, an audio decoding processing unit 704, an audio processing unit 705, an audio output unit 706, and an output unit 707. Further, the video and audio playback device 70 includes a video decoding processing unit 710, a frame reading unit 711, a video synthesizing unit 712, a video output unit 713, a display unit 714, and a memory 720. The system controller 700 controls various controls and operations in the video and audio playback device 70. The audio data (encoded audio data) recorded with the variable frame rate recording function is stored in the first recording medium 701. Also, the video data (encoded still image data or moving image data) recorded with the variable frame rate recording function is stored in the second recording medium 702.

[0006] To play back video and audio, the user gives a playback instruction to the system controller 700 of the video and audio playback device 70 to play back the audio data and video data to be played back. In the video and audio playback device 70, when the system controller 700 receives the playback instruction, the video and audio playback process is performed as follows.

[0007] First, the drive unit 703 acquires audio data from the first recording medium 701 and video data from the second recording medium 702. At this time, since the frame rate of video playback and the frame rate of audio playback are different, the audio data playback and the video data playback cannot be performed simultaneously. Therefore, in the video and audio playback device 70, the audio data playback and the video data playback are performed separately.

[0008] The audio decoding processing unit 704 decodes the audio data acquired by the drive unit 703 and stores the decoded audio data in the memory 720. The audio processing unit 705 reads out the audio data (or audio frame) to be played back in the decoded audio data stored in the memory 720 and transmits the read audio data to the audio output unit 706. The output unit 707 such as a speaker outputs the audio data from the audio output unit 706 as audio. Through such processing, the audio data to be played back is played back.

[0009] On the other hand, the video decoding processing unit 710 decodes the video data acquired by the drive unit 703 and stores the decoded video data in the memory 720. The frame reading unit 711 reads out the video frame to be played back in the decoded video data stored in the memory 720. The video synthesizing unit 712 synthesizes status information such as an icon indicating the playback state onto the video frame read by the frame reading unit 711 and transmits the synthesized video frame to the video output unit 713. The display unit 714 displays the synthesized video frame from the video output unit 713. The display unit 714 is, for example, a liquid crystal display, an organic EL display, or the like. Through such processing, the video data to be played back is played back.

Prior Art Documents

Patent Documents

[0010]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0011] As described above, in a conventional video and audio playback device, video data and audio data are played back separately. However, since the playback of audio data and the playback of video data cannot be performed simultaneously, when trying to check the image of a shooting scene using only audio data, it is not possible to check the video corresponding to (synchronized with) the part where the audio data is being played back (audio playback part). Therefore, for example, at a shooting site of a movie or the like, there is a problem that it is difficult for a user to grasp the image of the video taken (the image of the shooting scene). That is, it is difficult to grasp what scene the video started in when this sound was played.

[0012] Also, even if the technology of Patent Document 1 described above is used to process audio data and the processed audio data can be played back simultaneously with video data, since the audio data processed at the video playback rate is played back, the actual audio of the shooting scene cannot be correctly checked. Therefore, it did not solve the problem of difficulty in grasping the image of the shooting scene.

[0013] An object of the present invention is to enable the playback of a video synchronized with the audio playback position during the playback of audio data without processing the audio data according to the video playback rate when the audio and video are separately recorded with a variable frame rate recording function.

Means for Solving the Problems

[0014] In order to achieve the above object, a playback device of the present invention that plays back audio data with the same audio recording rate and audio playback rate, and video data with different video recording rates and video playback rates that are separately recorded at the same timing as the audio data, at a predetermined timing in the audio data At the time on the time axisVoice playback position acquisition means for acquiring a voice playback position, video playback position calculation means for calculating a video playback position corresponding to the voice playback position based on the voice playback position, the video recording rate, and the video playback rate, and frame reading means for reading a video frame corresponding to the video playback position from the video data, wherein the video frame read by the frame reading means is played back.

Effect of the Invention

[0015] According to the present invention, when voice and video are separately recorded with a variable frame rate recording function, it is possible to play back video synchronized with the voice playback position during voice data playback without processing the voice data according to the video playback rate. Therefore, the user can easily check the video synchronized with the voice playback position during voice data playback.

Brief Description of the Drawings

[0016]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Modes for Carrying Out the Invention

[0017] Hereinafter, each embodiment of the present invention will be described in detail with reference to the drawings. However, the configurations described in the following embodiments are merely examples, and the scope of the present invention is not limited by the configurations described in each embodiment.

[0018] For a video and audio playback device according to each embodiment of the present invention, when there are audio data and video data separately recorded at the same timing with a variable frame rate recording function, during the playback of the audio data, a video synchronized with the audio playback position (the playback position of the audio data) can be played (displayed). Note that the video and audio playback device according to each embodiment of the present invention is also simply referred to as a "playback device".

[0019] Hereinafter, the frame rate of video recording is also simply referred to as the "video recording rate", and the frame rate of video playback is also simply referred to as the "video playback rate". The frame rate (frame frequency) of audio recording is also simply referred to as the "audio recording rate", and the frame rate of audio playback is also simply referred to as the "audio playback rate". For the audio data and video data separately recorded at the same timing with a variable frame rate recording function, the audio recording rate is the same as the audio playback rate, and the video recording rate is different from the video playback rate. Also, the video recording rate is the same as the audio recording rate, but the video playback rate is different from the audio playback rate.

[0020] Hereinafter, the audio data and video data separately recorded at the same timing with a variable frame rate recording function are also simply referred to as "audio data" and "video data", respectively.

[0021] <First Embodiment> Hereinafter, the first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a configuration example of a video and audio playback device according to the first embodiment of the present invention.

[0022] As shown in FIG. 1, the video and audio playback device 10 includes a system controller 100, a first recording medium 101, a second recording medium 102, a drive unit 103, an audio decoding processing unit 104, an audio processing unit 105, an audio output unit 106, and an output unit 107. The video and audio playback device 10 also includes an audio playback position acquisition unit 108, a video playback position calculation unit 109, a video decoding processing unit 110, a frame reading unit 111, a video composition unit 112, a video output unit 113, a display unit 114, and a memory 120. The system controller 100 functions as a control unit (control means) that controls various controls and operations in the video and audio playback device 10. The system controller 100 may be configured by a CPU. The first recording medium 101 is a recording medium for storing audio data, and audio data recorded with a variable frame rate recording function is stored in the first recording medium 101. The second recording medium 102 is a recording medium for storing video data, and video data recorded with a variable frame rate recording function is stored in the second recording medium 102. Note that the video data stored in the second recording medium 102 may be moving image data or still image data.

[0023] In the first embodiment, the video and audio playback device 10 is configured to include the first recording medium 101 and the second recording medium 102. However, in the embodiments of the present invention, the first recording medium 101 and the second recording medium 102 may be configured by one recording medium or may be configured by a plurality of recording mediums.

[0024] The video and audio playback device 10 has the system controller 100, the audio playback position acquisition unit 108, and the video playback position calculation unit 109 as main parts of the present invention. The audio playback position acquisition unit 108 acquires the audio playback position and transmits the acquired audio playback position to the video playback position calculation unit 109. The video playback position calculation unit 109 calculates the video playback position based on the video recording rate, the video playback rate, and the audio playback position, and transmits the calculated video playback position to the frame reading unit 111. The relationship between the video playback position and the audio playback position will be described later.

[0025] Except for the system controller 100, the audio reproduction position acquisition unit 108, and the video reproduction position calculation unit 109, the other components of the video and audio reproduction device 10 are the same as those of the conventional video and audio reproduction device 70. That is, each process performed by the first recording medium 101 to the output unit 107 of the video and audio reproduction device 10 of the present invention is basically the same as each process performed by the first recording medium 701 to the output unit 707 of the conventional video and audio reproduction device 70. Also, each process performed by the video decoding processing unit 110 and the video synthesis unit 112 of the video and audio reproduction device 10 of the present invention is basically the same as each process performed by the video decoding processing unit 710 and the video synthesis unit 712 of the conventional video and audio reproduction device 70. Further, each process performed by the video output unit 113, the display unit 114, and the memory 120 of the video and audio reproduction device 10 of the present invention is basically the same as each process performed by the video output unit 713, the display unit 714, and the memory 720 of the conventional video and audio reproduction device 70.

[0026] Also, the frame reading unit 111 of the video and audio reproduction device 10 of the present invention reads out a video frame corresponding to the video reproduction position from the decoded video data stored in the memory 120. The frame reading unit 111 functions as frame reading means.

[0027] Here, with reference to FIG. 2, the relationship between the video playback position, the audio playback position, and the video frame synchronized with the audio playback position in the present invention will be described. FIG. 2 is a conceptual diagram showing the relationship between the video playback position, the audio playback position, and the video frame synchronized with the audio playback position in the present invention. In FIG. 2, the video data recorded at the video recording rate of 120P is being played back at the video playback rate of 60P. At the same timing, the audio data recorded at the audio recording rate of 120P is being played back at the audio playback rate of 120P. The audio data 210 indicates the audio data being played back. The position 211 within the audio data 210 is the audio playback position T1, and the audio playback position T1 is represented in time units. For example, the audio playback position T1 is t1 seconds. The specific meaning of the audio playback position will be described later. The video data 200 is the video data corresponding to the audio data 210. The video playback position T2 (that is, the time T2 at which the video data is played back) is calculated based on the audio playback position T1, the video recording rate, and the video playback rate. The video playback position T2 is the video playback position from the start of video playback (the timing when video playback starts). The video playback position T2 is also represented in time units. For example, the video playback position T2 is t2 seconds. The reference numeral 201 represents the video frame corresponding to the video playback position T2 within the video data 200. The "video frame corresponding to the video playback position T2" represented by the reference numeral 201 becomes the "video frame synchronized with the audio playback position T1".

[0028] In the video and audio playback device 10 according to the first embodiment, when the system controller 100 receives an audio playback instruction from the user to play the audio data, the audio playback process is performed, and the audio data is played back by the output unit 107 such as a speaker. The audio data played back by the output unit 107 is the decoded audio data stored in the memory 120.

[0029] In the video and audio playback device 10 according to the first embodiment, during the playback of audio data, the system controller 100 is configured to receive a video playback request notification from the user, which is "to request the display of video frames synchronized with the audio playback position", at an arbitrary timing. This arbitrary timing is determined by the user spontaneously, that is, it is a predetermined timing desired by the user. For example, the predetermined timing is the timing when a user operation (such as a key operation on the operation panel by the user or an operation of an external button by the user) for notifying the system controller 100 of the video playback request notification from the user is performed. The system controller 100 has an operation panel to be described later and also functions as an input unit (input means) for receiving video playback request notifications and various operations from the user. The system controller 100 receives the video playback request notification by a key operation on the operation panel by the user during the playback of the audio data. When the system controller 100 receives a video playback request notification from the user, it transmits the fact that "there has been a video playback request notification" and the timing (hereinafter referred to as "notification timing" or "notification location") at which the video playback request notification was received to the audio playback position acquisition unit 108.

[0030] Note that the video playback request notification may be received during the playback of the audio data by a key operation on the operation panel of each part other than the system controller 100 of the video and audio playback device 10 (for example, the display unit 114) by the user. In this case, the display unit 114 having the operation panel also functions as an input unit (input means) for receiving video playback request notifications and various operations from the user. Also, the video playback request notification may be received when a key operation by the user (for example, a key operation to temporarily stop the playback of the audio data) is performed during the playback of the audio data. Further, the system controller 100 may be configured to separately receive a video playback request notification from the user by, for example, an external button or the like.

[0031] The audio playback position acquisition unit 108 acquires the time from the timing when audio playback starts (hereinafter also referred to as the "beginning of audio playback"), which is the audio playback position, to the notification timing, and transmits the acquired audio playback position to the video playback position calculation unit 109. The audio playback position T1 is the time from the beginning of audio playback to the notification timing, and the audio playback position T1 in FIG. 2 is an example of the audio playback position acquired by the audio playback position acquisition unit 108. The video playback position calculation unit 109 calculates the time that becomes the video playback position based on the video recording rate, the video playback rate, and the audio playback position using the following formula (1), and transmits the calculated video playback position T2 to the frame reading unit 111. [Formula 1] T2 = T1 × (FRr ÷ FRp) However, in the above formula (1), the time T2 is the video playback position, the time T1 is the audio playback position, FRr is the video recording rate, and FRp is the video playback rate.

[0032] The frame reading unit 111 reads out the video frame corresponding to the video playback position T2 (the video frame synchronized with the audio playback position T1) from the decoded video data stored in the memory 120 using the video frame reading method. Here, regarding the "video frame reading method" used by the frame reading unit 111, since there are various "video frame reading methods" that utilize existing video playback technologies, any method may be used.

[0033] In order to improve the visibility of the video, the frame reading unit 111 may be configured to read out a plurality of video frames including the video frame corresponding to the video playback position T2 according to the user's request. For example, the frame reading unit 111 reads out the video frame corresponding to the video playback position T2 and also reads out the video frames a predetermined number of frames before the video frame corresponding to the video playback position T2. Further, the frame reading unit 111 reads out the video frame corresponding to the video playback position T2 and also reads out the video frames a predetermined number of frames after the video frame corresponding to the video playback position T2. The playback of the video by the plurality of video frames read out by the frame reading unit 111 may be performed on the display unit 114 according to the video playback rate. The display unit 114 functions as a display means and is composed of, for example, a liquid crystal display, an organic EL display, etc., and displays the video frame (or a plurality of video frames) read out by the frame reading unit 111. Further, the system controller 100 causes the display unit 114 to display the video frame (or a plurality of video frames) read out by the frame reading unit 111 as display control means.

[0034] Next, the operation of the video and audio playback device according to the first embodiment will be described. FIG. 3 is a flowchart showing the operation of the video and audio playback device 10 according to the first embodiment.

[0035] As described above, the system controller 100 receives a video playback request notification and various operations from the user, and by executing a predetermined computer program, comprehensively controls the operations of each part of the video and audio playback device 10 and executes the processing of FIG. 3. Further, the system controller 100 and the audio playback position acquisition unit 108 function as audio playback position acquisition means. The system controller 100 and the video playback position calculation unit 109 function as video playback position calculation means. In the following description, S means step.

[0036] First, in S301, the system controller 100 plays the audio data by a key operation on the operation panel by the user (key operation to play the audio data). Next, in S302, the system controller 100 determines whether the playback of the audio data has ended by a key operation on the operation panel by the user. When the system controller 100 determines that the playback of the audio data has ended, the process ends. On the other hand, in S302, when the system controller 100 determines that the playback of the audio data has not ended, that is, when it determines that the audio data is being played, the process proceeds to S303.

[0037] In S303, the system controller 100 determines whether there is a video playback request notification from the user by a key operation on the operation panel by the user. When the system controller 100 determines that there is a video playback request notification from the user, the process proceeds to S304. On the other hand, in S303, when the system controller 100 determines that there is no video playback request notification from the user, the process returns to S301 to continue playing the audio data.

[0038] In S304, after the system controller 100 transmits "there is a video playback request notification" and the notification location to the audio playback position acquisition unit 108, it causes the audio playback position acquisition unit 108 to acquire the audio playback position of the notification location. Next, in S305, the system controller 100 causes the video playback position calculation unit 109 to calculate the video playback position from the video recording rate, the video playback rate, and the audio playback position. Specifically, in S305, as described above, the video playback position calculation unit 109 uses the above formula 1 to calculate the video playback position (time T2).

[0039] Next, in S306, the system controller 100 causes the frame reading unit 111 to read out a video frame corresponding to the video playback position (time T2) from the decoded video data stored in the memory 120. Thus, by the process performed in S306, a video frame synchronized with the audio playback position (time T1) in the decoded video data stored in the memory 120 is acquired.

[0040] Next, in S307, the system controller 100 causes the display unit 114 such as a display to display the video frame acquired in S306, thereby playing back a video frame synchronized with the audio playback position during the playback of the audio data. Specifically, the system controller 100 transmits the video frame acquired in S306 to the video output unit 113, and causes the display unit 114 to display the video frame from the video output unit 113. In S307, if necessary, the system controller 100 may cause the video composition unit 112 to compose predetermined information onto the video frame acquired in S306, and transmit the composed video frame to the video output unit 113.

[0041] Next, in S308, the system controller 100 checks whether there is a pause request from the user to "temporarily pause the playback of the audio data" by a key operation on the operation panel by the user. When the system controller 100 confirms that there is no pause request from the user, it returns the process to S301 and continues the playback of the audio data. On the other hand, in S308, when the system controller 100 confirms that there is a pause request from the user, it pauses the playback of the audio data and then repeats the process performed in S308.

[0042] Next, the user interface screen of the video and audio playback device of the present invention will be described. FIG. 4 is a diagram showing an example of the user interface screen of the video and audio playback device according to the first embodiment. The example of the user interface screen shown in FIG. 4 is displayed on the display unit 114 having an operation panel. The display unit 114 having an operation panel may be composed of a liquid crystal display corresponding to a touch panel.

[0043] In FIG. 4, the portion indicated by reference numeral 400 is an example of the actual user interface screen displayed on the display unit 114 having an operation panel, and the portion indicated by reference numeral 401 is an example of the screen configuration of the operation panel. Also, in FIG. 4, the portion indicated by reference numeral 402 is an example of the video frame displayed on the display unit 114, and an enlarged view of the operation panel indicated by reference numeral 401 (hereinafter referred to as the "enlarged operation panel view") is indicated by reference numeral 410. In this user interface screen 400, not only the video frame 402 but also a volume meter indicating the volume of the audio data being played, the file name of the audio data, the playback time of the audio data, etc. are displayed. In this embodiment, the recording rate and the playback rate of the audio data are the same, but since the recording rate and the playback rate of the video data are different, if played simultaneously, the actual playback time will be different and the timing will be off. Therefore, the system controller 100 as the display control means controls so as not to play the video data during the playback of the audio data. That is, the system controller 100 does not cause the display unit 114 as the display means to display the video frame of the video data corresponding to the playback position of the audio data being played during the playback of the audio data. Instead, the system controller 100 causes the display unit 114 to display the video frame corresponding to the video playback position (that is, the video frame corresponding to the video playback position) as the video frame synchronized with the audio playback position acquired in S305.

[0044] Here, each key of the operation panel composed of a touch panel will be described using the enlarged view of the operation panel indicated by reference numeral 410. Also, key 411 is a key for pausing the playback of audio data, key 412 is a key for fast-forwarding audio data, and key 413 is a key for rewinding audio data. Furthermore, key 414 is a key for playing the previous scene, and key 415 is a key for playing the next scene.

[0045] Also, during the pause of audio by a key operation (screen touch operation of key 411) for pausing the playback of audio data, the operation panel will be displayed as the screen of the operation panel indicated by reference numeral 420. On the screen of the operation panel indicated by reference numeral 420, key 421 is a key for playing audio data, key 422 is a key for fast-forwarding by a unit time, and key 423 is a key for rewinding by a unit time.

[0046] Note that when the system controller 100 pauses the playback of audio data due to a screen touch operation of key 411 on the operation panel by the user during the playback of audio data, the system controller 100 may accept a video playback request notification in conjunction with the user's pause request. In this case, when the system controller 100 accepts a video playback request notification in conjunction with the user's pause request, it transmits "There is a video playback request notification" and the notification location (the timing when the playback of audio data was paused) to the audio playback position acquisition unit 108.

[0047] Furthermore, when the user operates key 412 in FIG. 4 to fast forward the audio data, or when the user operates key 413 in FIG. 4 to rewind the audio data, the system controller 100 may receive a video playback request notification at the time of the destination time. Similarly, when the user operates key 422 in FIG. 4 to fast forward the paused audio data by a unit time, the system controller 100 may receive a video playback request notification at the time of the destination time. Also, when the user operates key 423 in FIG. 4 to rewind the paused audio data by a unit time, the system controller 100 may receive a video playback request notification at the time of the destination time. That is, when the audio data is fast forwarded or rewound, if the system controller 100 receives a video playback request notification, it transmits the fact that "there has been a video playback request notification" and the notification location to the audio playback position acquisition unit 108. The notification location (the timing at which the video playback request notification was received) in these cases is not the timing at which the audio data was fast forwarded or rewound, but the time of the destination of the audio data.

[0048] As described above, in the video and audio playback device according to the first embodiment, during the playback of audio data, the system controller 100 is configured to receive a video playback request notification at an arbitrary timing desired by the user. Therefore, according to the video and audio playback device according to the first embodiment, the user can view a video synchronized with the audio playback position at a predetermined timing desired by the user (i.e., at a predetermined timing determined according to the user operation) during the playback of the audio data.

[0049] <Second Embodiment> Hereinafter, a second embodiment of the present invention will be described.

[0050] The configuration of the video and audio playback device according to the second embodiment of the present invention is the same as the configuration of the video and audio playback device 10 according to the first embodiment shown in FIG. 1, and thus the description thereof will be omitted.

[0051] In the video and audio playback device according to the second embodiment, during the playback of audio data, the system controller is configured to automatically receive a video playback request notification at a timing corresponding to the volume change of the audio being played. The timing corresponding to the volume change of the audio being played is, for example, the timing when the volume of the audio being played (the audio data being played) exceeds a predetermined threshold value.

[0052] Here, the operation of the video and audio playback device according to the second embodiment will be described. FIG. 5 is a flowchart showing the operation of the video and audio playback device according to the second embodiment.

[0053] First, in S501, the system controller plays the audio data by a key operation on the operation panel by the user (a key operation to play the audio data). Next, in S502, the system controller determines whether the playback of the audio data has ended by a key operation on the operation panel by the user. When the system controller determines that the playback of the audio data has ended, the process ends. On the other hand, in S502, when the system controller determines that the playback of the audio data has not ended, that is, when it determines that the audio data is being played, the process proceeds to S503.

[0054] In S503, the system controller determines whether the volume of the audio being played exceeds a predetermined threshold value. When it determines that the volume of the audio being played exceeds the predetermined threshold value, the process proceeds to S504. On the other hand, in S503, when the system controller determines that the volume of the audio being played does not exceed the predetermined threshold value, the process returns to S501 and the playback of the audio data continues.

[0055] In S504, after the system controller transmits to the audio playback position acquisition unit the timing at which the volume of the audio being played back exceeds a predetermined threshold (hereinafter, also simply referred to as "the corresponding location"), the system controller causes the audio playback position acquisition unit to acquire the audio playback position of the corresponding location. In the flow shown in FIG. 5, it is assumed that the audio playback position at the timing when the volume of the audio exceeds a predetermined threshold is acquired in S504, but the audio playback position at the timing when the amount of change in the volume of the audio exceeds a predetermined threshold may be acquired instead.

[0056] The processing after S505 (the processing performed in S505 to S508) is the same as the processing after S305 (the processing performed in S305 to S308) of the first embodiment, and thus the description of these processes is omitted. In this way, by the processing performed in S501 to S508, a video synchronized with the audio playback position according to the volume change of the audio being played back is played back.

[0057] As described above, in the video-audio playback device according to the second embodiment, during the playback of audio data, the system controller is configured to automatically receive a video playback request notification at a predetermined timing (the timing according to the volume change of the audio being played back). Therefore, according to the video-audio playback device according to the second embodiment, for example, in a movie shooting site where audio and video are separately recorded with a variable frame rate recording function, an audio check operation can be easily performed.

[0058] That is, when a person in charge of an audio check operation at a movie shooting site performs an audio check such as a rattling sound, even if only the audio data is played back using a conventional video-audio playback device, there is a problem that it is not immediately clear which shooting scene the audio being played back belongs to. By using the video-audio playback device according to the second embodiment, this problem can be solved.

[0059] That is, in S504, if the system controller sets the location where the pachinko sound is emitted as the timing when the volume of the voice being played exceeds a predetermined threshold or the timing when there is a volume change exceeding the predetermined threshold, the video frame at the location where the pachinko sound is emitted can be displayed on the display unit. Therefore, the person in charge of the voice check work at the movie shooting site can perform the voice check work after confirming the video of the actual shooting scene.

[0060] <Third Embodiment> Hereinafter, a third embodiment of the present invention will be described.

[0061] Since the configuration of the video and audio playback device according to the third embodiment of the present invention is the same as the configuration of the video and audio playback device 10 according to the first embodiment shown in FIG. 1, the description thereof will be omitted.

[0062] In the video and audio playback device according to the third embodiment, during the playback of audio data, the system controller is configured to automatically receive a video playback request notification at regular timing. This regular timing is, for example, the timing when a predetermined time interval set by the user has elapsed.

[0063] Here, the operation of the video and audio playback device according to the third embodiment will be described. FIG. 6 is a flowchart showing the operation of the video and audio playback device according to the third embodiment.

[0064] First, in S600, the system controller initializes the previous measurement start time Ts. Next, in S601, the system controller plays the audio data by a key operation on the operation panel by the user (key operation for playing the audio data), and measures the elapsed time from the start of the audio data playback as the current time. During the playback of the audio data, the system controller is configured to continuously measure the current time.

[0065] Next, in S602, the system controller determines whether the playback of the audio data has ended based on the key operation of the operation panel by the user. If the system controller determines that the playback of the audio data has ended, the process ends. On the other hand, in S602, if the system controller determines that the playback of the audio data has not ended, that is, if it determines that the audio data is being played back, the process proceeds to S603.

[0066] In S603, the system controller determines whether a predetermined time interval has elapsed based on the following formula (2). If it determines that the predetermined time interval has elapsed, that is, if the following formula (2) holds, the process proceeds to S604. On the other hand, in S603, if the system controller determines that the predetermined time interval has not elapsed, that is, if the following formula (2) does not hold, the process returns to S601 and the playback of the audio data continues. [Formula 2] Current time - Ts > Specified time However, in the above formula (2), the specified time is a predetermined time interval, Ts is the previous measurement start time, and the current time is the current time measured by the system controller in S603.

[0067] In S604, the system controller substitutes the current time into the previous measurement start time Ts, after notifying that "there is a video playback request" and transmitting the corresponding location to the audio playback position acquisition unit, causes the audio playback position acquisition unit to acquire the audio playback position of the corresponding location. Incidentally, the "corresponding location" mentioned in S604 means the timing when a predetermined time interval has elapsed.

[0068] The processes after S605 (the processes performed in S605 to S608) are the same as the processes after S305 (the processes performed in S305 to S308) of the first embodiment, so the description of these processes is omitted. In this way, the processes performed in S600 to S608 cause the video to be played back periodically (at a predetermined time interval).

[0069] As described above, in the video and audio playback device according to the third embodiment, during the playback of audio data, the system controller is configured to automatically receive a video playback request notification at regular intervals. Therefore, according to the video and audio playback device according to the third embodiment, video frames are automatically played back at predetermined time intervals.

[0070] By the way, at a movie shooting site, when a person in charge of audio check performs an audio check, there is a request to confirm, to some extent in real time, what kind of video corresponds to the audio while checking the audio. However, the process of sequentially displaying video data in accordance with the playback rate of audio data increases the processing load on the system controller, resulting in the problem that the processing load on the system controller is heavy. By using the video and audio playback device according to the third embodiment, this problem can be solved.

[0071] That is, in order to reduce the processing load on the system controller, in the video and audio playback device according to the third embodiment, it is sufficient to play back video frames at a predetermined time interval that is at least wider than the video playback rate. If the user pre-sets a "predetermined time interval" that is at least wider than the video playback rate, the problem of heavy processing load can be solved.

[0072] As described above, the preferred embodiments of the present invention have been described. However, the present invention is not limited to the above-described embodiments, and various modifications and changes are possible within the scope of the gist thereof. For example, all the functions realized by the system controller, the audio playback position acquisition unit, and the video playback position calculation unit in the second and third embodiments described above may be realized by a single video and audio playback device. Further, all the functions realized by the system controller, the audio playback position acquisition unit, and the video playback position calculation unit in the first, second, and third embodiments described above may be realized by a single video and audio playback device. In this case, the system controller and the audio playback position acquisition unit function as the first audio playback position acquisition means and also function as the second audio playback position acquisition means.

[0073] Furthermore, the video and audio playback device (playback device) according to the embodiment of the present invention can also be applied to mobile terminals such as smartphones and tablet terminals, and imaging devices such as digital video cameras.

[0074] In addition, the present invention can also be realized by a process in which a program that realizes one or more functions of each of the above-described embodiments is supplied to a system or device via a storage medium such as a network or a memory, and one or more processors of a computer of the system or device read and execute the program. Further, the present invention can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

Explanation of Reference Numerals

[0075] 10 Video and audio playback device 100 System controller 101 First recording medium 102 Second recording medium 103 Drive unit 104 Audio decoding processing unit 105 Audio processing unit 106 Audio output unit 107 Output unit 108 Audio playback position acquisition unit 109 Video playback position calculation unit 110 Video decoding processing unit 111 Frame reading unit 112 Video synthesis unit 113 Video output unit 114 Display unit 120 Memory

Claims

1. A playback device that plays back audio data with the same audio recording rate and audio playback rate, and video data with different video recording rates and video playback rates that are separately recorded at the same timing as the audio data, audio playback position acquisition means for acquiring the audio playback position at the time on the time axis at a predetermined timing in the audio data; video playback position calculation means for calculating a video playback position corresponding to the audio playback position based on the audio playback position, the video recording rate, and the video playback rate; frame reading means for reading out a video frame corresponding to the video playback position from the video data; comprising: A playback device characterized by playing back the video frame read by the frame reading means.

2. The playback device according to claim 1, wherein the audio playback position acquisition means acquires the audio playback position based on the audio data.

3. The playback device according to claim 2, wherein the audio playback position acquisition means acquires the audio playback position at a timing when the volume exceeds a predetermined threshold value or when the amount of change in volume exceeds a predetermined threshold value in the audio data.

4. The playback device according to claim 1, wherein the audio playback position acquisition means acquires the audio playback position at the predetermined timing determined according to a user operation.

5. The playback device according to claim 4, wherein the predetermined timing is the timing at which an operation by the user for notifying the audio playback position acquisition means of a video playback request from the user is performed.

6. The playback device according to claim 4, wherein the predetermined timing is the timing at which an operation by the user for temporarily stopping the playback of the audio data is performed.

7. The playback device according to claim 4, wherein when an operation by the user for fast-forwarding or rewinding the audio data is performed, the predetermined timing is the time of the destination of movement of the audio data.

8. The playback device according to any one of claims 1 to 7, wherein the video playback position calculation means calculates the video playback position using the calculation formula of the audio playback position × (the video recording rate ÷ the video playback rate).

9. The playback device according to claim 4, wherein the frame reading means reads out a plurality of video frames including the video frame corresponding to the video playback position in response to a request from the user.

10. The playback device according to claim 4, wherein the frame reading means reads out, in response to a request from the user, the video frame corresponding to the video playback position and also reads out video frames a predetermined number of frames before the video frame corresponding to the video playback position.

11. The playback device according to claim 4, wherein the frame reading means reads out, in response to a request from the user, the video frame corresponding to the video playback position and also reads out video frames a predetermined number of frames after the video frame corresponding to the video playback position.

12. The playback device according to claim 1, wherein the predetermined timing is regular timing and is the timing at which a predetermined time interval set by the user has elapsed.

13. A playback device that plays back audio data with the same audio recording rate and audio playback rate and video data with different video recording rates and video playback rates, separately recorded at the same timing as the audio data, a first audio playback position acquisition means for acquiring an audio playback position at a time on the time axis at a predetermined timing determined by an operation by the user during playback of the audio data; a second audio playback position acquisition means for automatically acquiring an audio playback position at a predetermined timing during playback of the audio data; video playback position calculation means for calculating a video playback position using the calculation formula of the audio playback position × (the video recording rate ÷ the video playback rate) based on the audio playback position acquired by the first audio playback position acquisition means or the second audio playback position acquisition means, the video recording rate, and the video playback rate; frame reading means for reading out a video frame corresponding to the video playback position from the video data; comprising: A playback device, characterized in that the video frame read out by the frame reading means is played back.

14. The playback device according to claim 13, wherein the frame reading means reads out a plurality of video frames including the video frame corresponding to the video playback position in response to a request from the user.

15. The playback device according to claim 13, wherein the frame reading means reads, in response to a request from the user, not only the video frame corresponding to the video playback position but also video frames a predetermined number of frames before the video frame corresponding to the video playback position.

16. The playback device according to claim 13, wherein the frame reading means reads, in response to a request from the user, not only the video frame corresponding to the video playback position but also video frames a predetermined number of frames after the video frame corresponding to the video playback position.

17. A playback method for playing back audio data having the same audio recording rate and audio playback rate and video data having different video recording rates and video playback rates recorded separately at the same timing as the audio data, an audio playback position acquisition step of acquiring an audio playback position at a time on the time axis at a predetermined timing determined by an operation by the user during playback of the audio data; a video playback position calculation step of calculating a video playback position using the calculation formula of the audio playback position × (the video recording rate ÷ the video playback rate) based on the audio playback position, the video recording rate, and the video playback rate; a frame reading step of reading a video frame corresponding to the video playback position from the video data; comprising: A playback method characterized by playing back the video frame read in the frame reading step.

18. A playback method for playing back audio data having the same audio recording rate and audio playback rate and video data having different video recording rates and video playback rates recorded separately at the same timing as the audio data, an audio playback position acquisition step of automatically acquiring an audio playback position at a time on the time axis at a predetermined timing during playback of the audio data; a video playback position calculation step of calculating a video playback position using the calculation formula of the audio playback position × (the video recording rate ÷ the video playback rate) based on the audio playback position, the video recording rate, and the video playback rate; a frame reading step of reading a video frame corresponding to the video playback position from the video data; comprising: A playback method characterized by playing back the video frame read in the frame reading step.

19. A playback method for playing back audio data with the same audio recording rate and audio playback rate, and video data with different video recording rates and video playback rates recorded separately at the same timing as the audio data, comprising: a first audio playback position acquisition step of acquiring an audio playback position at a time on the time axis at a predetermined timing determined by an operation by a user during playback of the audio data; a second audio playback position acquisition step of automatically acquiring an audio playback position at a predetermined timing during playback of the audio data; a video playback position calculation step of calculating a video playback position using the calculation formula of the audio playback position × (the video recording rate ÷ the video playback rate) based on the audio playback position acquired by the first audio playback position acquisition step or the second audio playback position acquisition step, the video recording rate, and the video playback rate; a frame reading step of reading out a video frame corresponding to the video playback position from the video data; characterized by comprising: playing back the video frame read out in the frame reading step. **Claim 20**: A program for causing a computer to execute each means of the playback device according to any one of Claims 1 to 16.

Images