Information processing device, information processing method, and program

Abstract

To provide an information processing device, an information processing method, and a program displaying display images with uniformity in the number of display frames for each attribute of the display image.SOLUTION: In a server 14, a CPU 60 acquires image data to be edited including moving image data, acquires first display image data of a plurality of frames, generates second display image data of a plurality of frames determined according to the number of frames of the first display image data from the image data to be edited, and displays a display image of a plurality of frames indicated by the second display image data of a plurality of frames on a display. The image data to be edited and the first display image data of the plurality of frames are image data having common attributes. The second display image data of the plurality of frames includes second display image data for a plurality of frames of moving image corresponding to still image data of a plurality of frames constituting at least a part of moving image data.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] The technology disclosed herein relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] Japanese Patent Application Publication No. 2014-146939 discloses a photobook creation terminal that extracts image data from moving image data and lays out pages in a photobook. The photobook creation terminal described in Japanese Patent Application Publication No. 2014-146939 includes an input means, a page allocation means, an image number calculation means, an image extraction means, and a layout means. The input means inputs moving image data. The page allocation means allocates scenes included in the moving image data to one or more pages of the photobook. The image number calculation means calculates the number of frames required for a scene for each page allocated by the page allocation means. The image extraction means extracts image data for the required number of frames calculated by the image number calculation means for each scene of the moving image data. The layout means lays out the image data extracted by the image extraction means on the pages allocated by the page allocation means. Summary of the Invention

[0003] However, in the technology described in JP 2014-146939 A, the required number of frames of image data is calculated without considering attributes such as events that are similar between image data. Therefore, when image data is classified by attributes such as events, the number of frames of image data may vary greatly between attributes.

[0004] One embodiment of the technology disclosed herein provides an information processing device, an information processing method, and a program that can display a display image with uniformity in the number of display frames for each attribute of the display image. [Means for solving the problem]

[0005] A first aspect of the technology disclosed herein is an information processing device comprising a processor and a memory built into or connected to the processor, wherein the processor acquires image data to be edited including moving image data, acquires first display image data of a plurality of frames, generates second display image data of a plurality of frames from the image data to be edited, the second display image data being determined according to at least one of the number of frames of the first display image data and the time interval between frames of the first display image data, and displays on a display a plurality of frames of display images indicated by the second display image data of the plurality of frames, wherein the image data to be edited and the first display image data of the plurality of frames are image data having common attributes, and the second display image data of the plurality of frames includes second display image data for a moving image of a plurality of frames corresponding to still image data of a plurality of frames constituting at least a part of the moving image data.

[0006] A second aspect of the technology of the present disclosure is the information processing device according to the first aspect, in which the number of frames of the first display image data is equal to or greater than the number of frames of the second display image data.

[0007] A third aspect of the technology disclosed herein is an information processing device according to the first or second aspect, in which the image data to be edited and the first display image data are classified by attribute, and the processor acquires the image data to be edited and multiple frames of the first display image data as image data having a common attribute based on the attribute.

[0008] A fourth aspect of the technology of the present disclosure is an information processing device according to any one of the first to third aspects, in which the attributes include at least one of date, event, subject, imaging location, photographer, and imaging device model.

[0009] A fifth aspect of the technology of the present disclosure is an information processing device according to any one of the first to third aspects, in which attributes of image data to be edited are identified by performing image analysis on the image data to be edited.

[0010] A sixth aspect of the technology disclosed herein is an information processing device according to any one of the first to fifth aspects, in which the first display image data is assigned an imaging time at which imaging for the first display image data was performed for each frame, and the processor derives the number of frames of the second display image data for a moving image according to an interval between imaging times within a first display image data block in which the imaging time interval between adjacent frames is equal to or less than a predetermined interval and the first display image data is consecutive.

[0011] A seventh aspect of the technology disclosed herein is an information processing device relating to any one of the first to sixth aspects, in which the video data includes first video data having a first attribute as an attribute and second video data having a second attribute as an attribute different from the first attribute, and in which the processor executes a frame number suppression generation process to generate second display image data for video from the video data with a frame number determined according to the smaller of the first frame number and the second frame number when a first frame number of second display image data for video corresponding to the first video data differs from a second frame number of second display image data for video corresponding to the second video data.

[0012] An eighth aspect of the technology disclosed herein is an information processing device according to the seventh aspect, in which a processor executes a frame number suppression generation process when the degree of change over time in the ratio of the data amount of moving image data to the data amount of image data to be edited exceeds a predetermined degree.

[0013] A ninth aspect of the technology disclosed herein is an information processing device according to the eighth aspect, in which the frame number suppression generation process is a process of generating second display image data for moving images from moving image data with a number of frames determined according to the smaller of the first frame number and the second frame number and the ratio before the degree exceeds a default degree.

[0014] A tenth aspect of the technology of the present disclosure is an information processing device relating to any one of the first to ninth aspects, in which a processor generates second display image data for moving images with a number of frames determined according to the time required for imaging to obtain the moving image data.

[0015] An eleventh aspect of the technology disclosed herein is an information processing device relating to any one of the first to tenth aspects, in which a processor limits the maximum number of frames of the second display image data for moving images in accordance with the time required for imaging to obtain the moving image data.

[0016] A twelfth aspect of the technology of the present disclosure is an information processing device relating to any one of the first to eleventh aspects, in which still image data of multiple frames is a group of still image data whose similarity derived according to the result obtained by performing image recognition processing on moving image data and at least one of the time intervals between frames in the moving image data is within a predetermined range.

[0017] A thirteenth aspect of the technology of the present disclosure is an information processing device relating to any one of the first to twelfth aspects, in which a group of still image data of multiple frames is a result obtained by performing image recognition processing on moving image data, and the similarity derived according to at least one of the time intervals between frames in the moving image data is outside a predetermined range.

[0018] A 14th aspect of the technology of the present disclosure is an information processing device relating to any one of the 1st to 13th aspects, in which the display displays, for multiple frames of display images, a display image based on second display image data for moving images and a display image based on second display image data other than the second display image data for moving images in a distinguishable manner.

[0019] A fifteenth aspect of the technology disclosed herein is an information processing device relating to any one of the first to fourteenth aspects, in which a display displays multiple frames of display images based on second display image data for moving images generated from the same moving image data, on a strip-shaped background extending along the time axis, in a manner distinguishable from display images based on second display image data other than the second display image data for the moving image.

[0020] A 16th aspect of the technology disclosed herein is an information processing device relating to any one of the first to fifteenth aspects, in which the second display image data of multiple frames includes image data for time zone overlapping display of multiple frames generated from edited image data obtained by capturing images during overlapping time periods, and the display displays the second display images of multiple frames in chronological order, and displays the images for time zone overlapping display of multiple frames indicated by the image data for time zone overlapping display of multiple frames in a manner arranged in correspondence with positions indicating the time periods on the time axis.

[0021] A 17th aspect of the technology of the present disclosure is an information processing device relating to the 16th aspect, in which a display displays a plurality of frames of time zone overlap display images, in which a time zone overlap display image for moving images corresponding to moving images and a time zone overlap display image for still images corresponding to still images are arranged separately in correspondence with positions indicating the time zones on a time axis.

[0022] An 18th aspect of the technology of the present disclosure is an information processing device relating to any one of the 1st to 17th aspects, in which the second display image data for at least one frame of a moving image among the second display image data for a moving image of multiple frames corresponds to still image data of multiple frames constituting at least a part of the moving image data.

[0023] A 19th aspect of the technology of the present disclosure is an information processing method including obtaining image data to be edited including moving image data, obtaining first display image data of a plurality of frames, generating second display image data of a plurality of frames from the image data to be edited, the second display image data being determined according to at least one of the number of frames of the first display image data and the time interval between frames of the first display image data, and displaying, on a display, display images of a plurality of frames indicated by the second display image data of the plurality of frames, wherein the image data to be edited and the first display image data of the plurality of frames are image data having common attributes, and the second display image data of the plurality of frames includes second display image data for a moving image of a plurality of frames corresponding to still image data of a plurality of frames that constitute at least a part of the moving image data.

[0024] A twentieth aspect of the technology disclosed herein is a program for causing a computer to execute processing, the processing including: acquiring image data to be edited including moving image data; acquiring first display image data of a plurality of frames; generating second display image data of a plurality of frames from the image data to be edited, the second display image data being determined according to at least one of the number of frames of the first display image data and the time interval between frames of the first display image data; and displaying, on a display, display images of a plurality of frames indicated by the second display image data of the plurality of frames, wherein the image data to be edited and the first display image data of the plurality of frames are image data having common attributes, and the second display image data of the plurality of frames includes second display image data for a moving image of a plurality of frames corresponding to still image data of a plurality of frames constituting at least a portion of the moving image data. Effect of the Invention

[0025] According to one embodiment of the technique of the present disclosure, an effect is obtained in that a display image can be displayed with uniformity in the number of display frames for each attribute of the display image. [Brief description of the drawings]

[0026] [Figure 1] 1 is a conceptual diagram showing a schematic configuration of an information processing system according to an embodiment. [Diagram 2] 2 is a block diagram showing an example of a hardware configuration of an electrical system of a user device included in the information processing system according to the embodiment. FIG. [Diagram 3] 2 is a block diagram showing an example of a hardware configuration of an electrical system of a server included in the information processing system according to the embodiment. FIG. [Figure 4] FIG. 1 is a conceptual diagram illustrating a use of an information processing system according to an embodiment. [Diagram 5] 2 is a block diagram showing an example of the contents stored in a storage of a user device included in the information processing system according to the embodiment. FIG. [Figure 6] FIG. 2 is a conceptual diagram showing an example of a configuration of captured image data according to the embodiment. [Figure 7] 1 is a conceptual diagram illustrating a conventional method for generating list image data. [Figure 8] 11 is a block diagram showing an example of main functions of a CPU when a list image data generation process is executed by the CPU of a server included in an information processing system according to an embodiment. FIG. [Figure 9] 11 is a block diagram showing an example of processing contents when a CPU of a server included in an information processing system according to an embodiment operates as a DB construction unit. FIG. [Figure 10] 11 is a block diagram showing an example of processing contents when a CPU of a server included in an information processing system according to an embodiment operates as a DB construction unit and an image analysis unit. FIG. [Figure 11] 11 is a block diagram showing an example of processing contents when a CPU of a server included in an information processing system according to an embodiment operates as a DB construction unit and an image analysis unit. FIG. [Figure 12] 11 is a block diagram showing an example of processing contents when a CPU of a server included in an information processing system according to an embodiment operates as a DB construction unit and an image analysis unit. FIG. [Figure 13]10 is a conceptual diagram showing an example of the configuration of an annual event image file DB constructed by a CPU of a server included in the information processing system according to the embodiment. FIG. [Figure 14] FIG. 13 is a conceptual diagram showing an example of the configuration of a 2018 first event image file DB. [Figure 15] FIG. 13 is a conceptual diagram showing an example of the configuration of a 2019 first event image file DB. [Figure 16] 11 is a block diagram showing an example of processing contents when a CPU of a server included in an information processing system according to an embodiment operates as an acquisition unit. FIG. [Figure 17] 11 is a block diagram showing an example of processing contents when a CPU of a server included in the information processing system according to the embodiment operates as an image data selection unit. FIG. [Figure 18] 11 is a block diagram showing an example of processing contents when a CPU of a server included in the information processing system according to the embodiment operates as an image data selection unit, a determination unit, and an acquisition unit. FIG. [Figure 19] 11 is a block diagram showing an example of processing contents when a CPU of a server included in the information processing system according to the embodiment operates as an acquisition unit, a recording time determination unit, and a frame number derivation unit. FIG. [Figure 20] 11 is a block diagram showing an example of processing contents when a CPU of a server included in an information processing system according to an embodiment operates as an acquisition unit, a reduced display image data generation unit, a determination unit, and a frame number derivation unit. FIG. [Figure 21] 11 is a block diagram showing an example of processing contents when a CPU of a server included in the information processing system according to the embodiment operates as an image data selection unit, an acquisition unit, and a determination unit. FIG. [Figure 22] 11 is a block diagram showing an example of processing contents when a CPU of a server included in an information processing system according to an embodiment operates as a determination unit, a file number counting unit, and a frame number counting unit. FIG. [Figure 23]A block diagram showing an example of the processing contents when the CPU of a server included in the information processing system according to the embodiment operates as a file number counting unit, a determination unit, a frame number counting unit, a missing frame number calculation unit, an acquisition unit, and a reduced display image data generation unit. [Figure 24] 11 is a block diagram showing an example of processing contents when a CPU of a server included in the information processing system according to the embodiment operates as an acquisition unit, a missing frame number calculation unit, and a reduced display image data generation unit. FIG. [Diagram 25] 11 is a block diagram showing an example of processing contents when a CPU of a server included in the information processing system according to the embodiment operates as a determination unit, a list image data generation unit, and a transmission unit. FIG. [Figure 26] 11 is an example of list image data obtained by executing the list image data generating process according to the embodiment. [Figure 27] 11 is a screen diagram showing an example of a mode in which a list image shown by list image data obtained by executing the list image data generation process according to the embodiment is displayed on a display of a user device. FIG. [Figure 28A] 10 is a flowchart showing an example of the flow of a list image data generating process according to the embodiment. [Figure 28B] This is a continuation of the flowchart shown in FIG. 28A. [Figure 28C] This is a continuation of the flowchart shown in Figure 28B. [Figure 28D] This is a continuation of the flowchart shown in Figure 28C. [Figure 29] 13 is a screen diagram showing a first modified example of a display form of a list image displayed on a display of a user device included in the information processing system according to the embodiment. FIG. [Diagram 30] 13 is a screen diagram showing a second modified example of a display form of a list image displayed on a display of a user device included in the information processing system according to the embodiment. FIG. [Diagram 31] 13 is a modified example of list image data obtained by executing the list image data generating process according to the embodiment. [Diagram 32] 32 is a screen diagram showing an example of a state in which list image data indicated by the list image data shown in FIG. 31 is displayed on a display of a user device. FIG. [Diagram 33] 11 is a conceptual diagram showing an example of a form of a block in which the interval between the imaging times of adjacent frames is equal to or shorter than a predetermined interval. FIG. [Diagram 34] 13 is a flowchart showing a modified example of the flow of the list image data generating process according to the embodiment. [Diagram 35] 11 is a block diagram showing an example of a configuration in which a frame number suppression generation process is executed by a CPU of a server included in an information processing system according to an embodiment. FIG. [Figure 36A] 11 is a flowchart showing an example of the flow of a frame number suppression generation process according to the embodiment. [Figure 36B] This is a continuation of the flowchart shown in FIG. 36A. [Figure 37] FIG. 13 is a conceptual diagram showing a first modified example of a frame number derivation table used by a CPU of a server included in the information processing system according to the embodiment. [Figure 38] FIG. 13 is a conceptual diagram showing a second modified example of the frame number derivation table used by the CPU of the server included in the information processing system according to the embodiment. [Figure 39] 1 is a block diagram showing an example of a manner in which a server-side program is installed on a computer in a server from a storage medium on which the server-side program is stored. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027] An example of an embodiment of an information processing device, an information processing method, and a program according to the technology of the present disclosure will be described with reference to the attached drawings.

[0028] First, the terms used in the following description will be explained.

[0029] CPU is an abbreviation for "Central Processing Unit". RAM is an abbreviation for "Random Access Memory". SSD is an abbreviation for "Solid State Drive". HDD is an abbreviation for "Hard Disk Drive". EEPROM is an abbreviation for "Electrically Erasable and Programmable Read Only Memory". ASIC is an abbreviation for "Application Specific Integrated Circuit". PLD is an abbreviation for "Programmable Logic Device". FPGA is an abbreviation for "Field-Programmable Gate Array". SoC is an abbreviation for "System-on-a-chip". CMOS is an abbreviation for "Complementary Metal Oxide Semiconductor". CCD is an abbreviation for "Charge Coupled Device". EL is an abbreviation for "Electro-Luminescence". UI is an abbreviation for "User Interface". USB is an abbreviation for "Universal Serial Bus". GPU is an abbreviation for "Graphics Processing Unit". GPS is an abbreviation for "Global Positioning System". RTC is an abbreviation for "Real Time Clock". ID is an abbreviation for "Identification". Exif is an abbreviation for "exchangeable image file format". WAN is an abbreviation for "Wide Area Network". LAN is an abbreviation for "Local Area Network". DB is an abbreviation for "Database". JPEG is an abbreviation for "Joint Photographic Experts Group". MPEG is an abbreviation for "Moving Picture Experts Group".

[0030] 1, an information processing system 10 includes a user device 12 and a server 14. The user device 12 is a terminal device, such as a smartphone, that transmits and receives input information and / or image information by a user 16 to and from the server 14. However, this is merely an example, and the user device 12 may be a tablet terminal, a personal computer, a wearable terminal, and / or a digital camera, or other terminal device.

[0031] The information processing system 10 is used by a number of users 16. Each user 16 is assigned a user device 12. For example, the user 16 is the owner of the user device 12.

[0032] The user device 12 is communicatively connected to the server 14 via a network 19. An example of the network 19 is a WAN such as the Internet. The network 19 is not limited to a WAN, but may be a LAN, or a network in which a LAN and a WAN are connected. The user device 12 and the network 19 may be connected by a wireless communication method or a wired communication method. The server 14 and the network 19 may be connected by a wireless communication method or a wired communication method. Although not shown in the example shown in FIG. 1, the network 19 includes, for example, a base station. The network 19 establishes communication between the user device 12 and the server 14, and transmits and receives various information between the user device 12 and the server 14.

[0033] The server 14 receives a request from the user device 12 via the network 19 and provides a service in response to the request via the network 19 to the user device 12 that made the request.

[0034] For example, the user device 12 requests the server 14 to process image data (e.g., the "image file group" shown in FIG. 4). The server 14 processes the image data provided by the user device 12 and provides the processed image data (e.g., the "list image data" shown in FIG. 4) to the user device 12. The server 14 is an example of an "information processing device" according to the technology of the present disclosure.

[0035] As an example, as shown in FIG. 2, the user device 12 includes a computer 22, an imaging device 24, a clock 26, a communication I / F 28, a GPS receiver 30, an accepting device 32, a display 34, a microphone 36, a speaker 38, and an external I / F 40. The computer 22 includes a CPU 42, a storage 44, and a memory 46. The CPU 42, the storage 44, and the memory 46 are connected to a bus 48. The imaging device 24, the clock 26, the communication I / F 28, the GPS receiver 30, the accepting device 32, the display 34, the microphone 36, the speaker 38, and the external I / F 40 are also connected to the bus 48. In the example shown in FIG. 2, for convenience of illustration, one bus is shown as the bus 48, but the bus 48 includes a data bus, an address bus, a control bus, and the like.

[0036] The CPU 42 controls the entire user device 12. The storage 44 stores various parameters and various programs. The storage 44 is a non-volatile storage device. Here, an EEPROM is used as an example of the storage 44, but the storage 44 is not limited to this and may be an SSD and / or an HDD. The memory 46 is a volatile storage device. The memory 46 is used as a work memory by the CPU 42 and temporarily stores various information. Here, a DRAM is used as an example of the memory 46, but the storage 44 is not limited to this and may be another type of volatile storage device such as an SRAM.

[0037] The imaging device 24 is a device that generates image data. The imaging device 24 has, for example, a CMOS image sensor, and is equipped with a zoom mechanism and a focus adjustment mechanism. Here, a CMOS image sensor is exemplified as the image sensor of the imaging device 24, but the image sensor is not limited to this and may be another type of image sensor such as a CCD image sensor. The imaging device 24 captures an image of a subject according to an instruction from the CPU 42. The imaging device 24 captures an image of the subject to generate image data indicating the subject. The CPU 42 acquires the image data generated by the imaging device 24, and stores the acquired image data in the storage 44.

[0038] The clock 26 acquires the current time. The clock 26 is, for example, an RTC, and receives a supply of driving power from a power supply system that is separated from the power supply system for the computer 22, and continues to keep track of the current time (year, month, day, hour, minute, and second) even when the computer 22 is shut down. The clock 26 outputs the current time to the CPU 42 every time the current time is updated.

[0039] The communication I / F 28 is connected to the network 19 via a wireless communication system, and controls the transmission and reception of various information between the CPU 42 and the server 14 via the network 19. Note that, although a wireless communication system is illustrated here as an example, the technology of the present disclosure is not limited to this, and a wired communication system may also be used.

[0040] The GPS receiver 30 receives radio waves from multiple GPS satellites (not shown) in response to an instruction from the CPU 42, and outputs reception result information indicating the reception result to the CPU 42. The CPU 42 calculates GPS information as position specifying information capable of specifying the current position of the user device 12, based on the reception result information input from the GPS receiver 30. The GPS information is, for example, latitude, longitude, and altitude capable of specifying the current position of the user device 12.

[0041] The receiving device 32 receives instructions from the user 16, etc. Examples of the receiving device 32 include a touch panel 32A and a hard key. The instructions received by the receiving device 32 are acquired by the CPU 42. The receiving device 32 may receive instructions from the user 16, etc. by voice input via the microphone 36.

[0042] The display 34 displays various information under the control of the CPU 42. An example of the display 34 is a liquid crystal display. Note that the display 34 is not limited to a liquid crystal display, and other types of displays such as an EL display may also be used.

[0043] In this embodiment, an out-cell type touch panel display is used in which the touch panel 32A is overlaid on the surface of the display area of ​​the display 34. However, the out-cell type touch panel display is merely one example, and for example, an on-cell type or in-cell type touch panel display can also be applied.

[0044] The microphone 36 converts the collected sound into an electrical signal, and outputs the electrical signal obtained by converting the sound to the CPU 42. The speaker 38 converts an electrical signal input from a specific device (e.g., the CPU 42) into sound, and outputs the sound obtained by converting the electrical signal to the outside of the user device 12.

[0045] The external I / F 40 is responsible for transmitting and receiving various types of information between the user device 12 and devices external to the user device 12. An example of the external I / F 40 is a USB interface. A user device, a personal computer, a server, a USB memory, a memory card, and / or a printer, etc., are connected to the USB interface.

[0046] As an example, as shown in Fig. 3, the server 14 includes a computer 50, a communication I / F 52, a receiving device 54, a display 56, and an external I / F 58. The computer 50 includes a CPU 60, a storage 62, and a memory 64. The CPU 60, the storage 62, and the memory 64 are connected to a bus 66. The communication I / F 52, the receiving device 54, the display 56, and the external I / F 58 are also connected to the bus 66. Note that in the example shown in Fig. 3, for convenience of illustration, one bus is shown as the bus 66, but the bus 66 includes a data bus, an address bus, a control bus, and the like.

[0047] The CPU 60 controls the entire server 14. The storage 62 stores various parameters and various programs. The storage 62 is a non-volatile storage device. Here, an SSD is used as an example of the storage 62, but the storage 62 may be an EEPROM and / or an HDD. The memory 64 is a volatile storage device. The memory 64 is used as a work memory by the CPU 60 and temporarily stores various information. Here, a DRAM is used as an example of the memory 64, but the storage 62 may be another type of volatile storage device such as an SRAM. The CPU 60 is an example of a "processor" according to the technology of the present disclosure, and the storage 62 and the memory 64 are an example of a "memory" according to the technology of the present disclosure.

[0048] The communication I / F 52 is communicably connected to the network 19 and controls the exchange of various information between the CPU 60 and the user device 12 via the network 19 .

[0049] The receiving device 54 receives instructions from an administrator of the server 14, etc. Examples of the receiving device 54 include a remote controller, a touch panel, and / or a hard key. The instructions received by the receiving device 54 may include instructions input by voice via a microphone, etc. The instructions received by the receiving device 54 are acquired by the CPU 60.

[0050] The display 56 displays various information under the control of the CPU 60. An example of the display 56 is a liquid crystal display. Note that the display 56 is not limited to a liquid crystal display, and other types of displays such as an EL display may also be used.

[0051] The external I / F 58 controls the transmission and reception of various information between the server 14 and devices existing outside the server 14. An example of the external I / F 58 is a USB interface. A user device, a personal computer, a server, a USB memory, a memory card, and / or a printer, etc. are connected to the USB interface.

[0052] As an example, as shown in FIG. 4, in an information processing system 10, a user device 12 uploads a group of image files to a server 14, and the server 14 manages the uploaded group of image files. The group of image files includes a plurality of image files. An image file is, for example, a file including captured image data obtained by capturing an image of a subject by the user device 12. The image files are broadly classified into still image files and video files. A still image file includes captured image data for one frame. A video file includes captured image data for multiple frames obtained by capturing an image of a subject at a default frame rate (for example, 30 frames / second).

[0053] The server 14 generates list image data by performing image processing on the group of image files and provides the list image data to the user device 12. The list image data refers to, for example, image data in which a plurality of reduced display image data are organized in a list format. The reduced display image data is image data in which the captured image data is reduced. In other words, the reduced display image data is image data in which the size and image quality are reduced compared to the captured image data. A specific example of reduced display image data is thumbnail image data that shows a thumbnail image.

[0054] The user 16 can view the list image shown by the list image data via the display 34 of the user device 12. The list image includes a plurality of reduced display images shown by a plurality of reduced display image data. In the example shown in Fig. 4, a plurality of reduced display images corresponding to a plurality of still image files (hereinafter also referred to as "still image file reduced display images") and a plurality of reduced display images corresponding to a plurality of video files (hereinafter also referred to as "video file reduced display images") are shown. A video identification mark is displayed in the center of the video file reduced display image as a mark that can identify the video file reduced display image.

[0055] The user 16 selects an image for reduced display from the list image via the touch panel 32A. When an image for reduced display is selected by the user 16, an image file corresponding to the selected image for reduced display is downloaded from the server 14 to the user device 12. In the example shown in FIG. 4, an image for reduced display of a video file has been selected by the user 16, so the video file corresponding to the image for reduced display of the video file selected by the user 16 is downloaded from the server 14 to the user device 12.

[0056] As an example, as shown in FIG. 5, the image file group is stored in the storage 44 of the user device 12. The image file group in the storage 44 is uploaded from the user device 12 to the server 14 in response to an instruction received by the reception device 32. In the example shown in FIG. 5, a plurality of image file groups including a first image file group and a second image file group are stored in the storage 44. Each of the image file groups is stored, for example, in a year-specific folder. For example, the year-specific folder in which the first image file group is stored is a folder indicating 2018. The folder indicating 2018 stores the image file group obtained by capturing images by the imaging device 24 in 2018. In addition, for example, the year-specific folder in which the second image file group is stored is a folder indicating 2019. The folder indicating 2019 stores the image file group obtained by capturing images by the imaging device 24 in 2019.

[0057] As an example, as shown in Fig. 6, captured image data included in a still image file and a video file has original image data and related information. The original image data and related information are in one-to-one correspondence for each captured image data. The original image data is still image data. Thus, a still image file contains original image data for one frame, and a video file contains original image data for multiple frames.

[0058] The related information includes the file name of the corresponding image file and imaging condition information indicating the conditions of imaging performed to obtain the corresponding original image data (hereinafter, also referred to as "imaging conditions"). An example of the imaging condition information is Exif information. In the example shown in FIG. 6, the imaging condition information includes date information, imaging position information, photographer information, model information, and captured image data identifier. The date information is information indicating the date on which one frame of imaging was performed. An example of the date information is the year, month, day, hour, minute, and second acquired from the clock 26 at the timing when one frame of imaging was performed. The imaging position information is information indicating the position where imaging was performed by the imaging device 24. An example of the imaging position information is GPS information calculated at the timing when one frame of imaging was performed. The photographer information is information indicating the photographer, and the model information is information indicating the model of the imaging device 24. The photographer information and the model information are registered in advance in, for example, the user device 12. The captured image data identifier is an identifier that can identify the corresponding captured image data. Therefore, one captured image data identifier is assigned to a still image file, and multiple captured image data identifiers are assigned to a moving image file.

[0059] Generally, one frame of reduced display image data is generated for one image file. Therefore, when the number of image files differs among the image files uploaded from the user device 12 to the server 14, the number of frames of the reduced display image data included in the list image data corresponding to the image files also differs.

[0060] For example, as shown in FIG. 7, the list image data for the image file group obtained by capturing images at the 2018 athletic meet (hereinafter also referred to as the "2018 athletic meet image file group") includes nine frames of reduced display image data. In contrast, the image file group obtained by capturing images at the 2019 athletic meet (hereinafter also referred to as the "2019 athletic meet image file group") includes two still image files and two video files. Therefore, when one frame of reduced display image data is generated for one image file, the list image data for the 2019 athletic meet image file group includes four reduced display image data. The four reduced display image data refer to two still image file reduced display image data corresponding to two still image files and two video file reduced display image data corresponding to two video files.

[0061] In this way, the number of frames of the reduced display image data corresponding to the 2019 Athletic Meet image file group is fewer than the number of frames of the reduced display image data corresponding to the 2018 Athletic Meet image file group. In other words, even though the image files in 2018 and 2019 were captured and obtained at the same event, the "Athletic Meet," the number of frames of the reduced display image data is not the same between 2018 and 2019.

[0062] In view of such circumstances, as shown in Fig. 8 as an example, in the server 14, the CPU 60 executes a list image data generating process. A list image data generating program 72 is stored in the storage 62 of the server 14. The CPU 60 reads out the list image data generating program 72 from the storage 62. The CPU 60 then executes the list image data generating program 72 read out from the storage 62 on the memory 64, thereby operating as a DB constructing unit 60A, an image analyzing unit 60B, an acquiring unit 60C, a determining unit 60D, a reduced display image data generating unit 60E, an image data selecting unit 60F, a recording time specifying unit 60G, a frame number deriving unit 60H, a frame number counting unit 60I, a file number counting unit 60J, a missing frame number calculating unit 60K, a list image data generating unit 60L, and a transmitting unit 60M. In other words, the list image data generation process is realized by the CPU 60 operating as a DB construction unit 60A, an image analysis unit 60B, an acquisition unit 60C, a judgment unit 60D, an image data generation unit for reduced display 60E, an image data selection unit 60F, a recording time determination unit 60G, a frame number derivation unit 60H, a frame number counting unit 60I, a file number counting unit 60J, a missing frame number calculation unit 60K, a list image data generation unit 60L, and a transmission unit 60M.

[0063] The CPU 60 executes the list image data generating process to obtain the edit target image data including the moving image file and obtain the first display image data of a plurality of frames. The CPU 60 also executes the list image data generating process to generate the second display image data of a plurality of frames determined according to the number of frames of the first display image data from the edit target image data. The CPU 60 also executes the list image data generating process to display the display image of a plurality of frames indicated by the second display image data of a plurality of frames on the display 34. Here, the edit target image data and the first display image data of a plurality of frames are image data having a common attribute. The second display image data of a plurality of frames includes the second display image data of a plurality of frames for the moving image corresponding to the still image data of a plurality of frames constituting at least a part of the moving image data. An example of the list image data generating process will be described in more detail below.

[0064] As an example, as shown in FIG. 9, when the image data group transmitted from the user device 12 is received by the communication I / F 52, the DB construction unit 60A acquires the image data group received by the communication I / F 52. The DB construction unit 60A uses the acquired image data group to construct an image file DB 74 in the storage 62. In the image file DB 74, file configuration image data is associated with each file name indicating an image file. The file configuration image data refers to captured image data that constitutes an image file, that is, captured image data included in an image file. In the example shown in FIG. 9, a file name with an extension of "jpg" indicating the file format of JPEG is a still image file. Also, in the example shown in FIG. 9, a file name with an extension of "mpg" indicating the file format of MPEG is a moving image file. Note that the file format of a still image file is not limited to JPEG and may be another file format. Also, the file format of a moving image file is not limited to MPEG and may be another file format.

[0065] As an example, as shown in FIG. 10, the image analysis unit 60B acquires an image file from the image file DB 74, and extracts captured image data from the acquired image file. Then, the image analysis unit 60B performs image analysis on the captured image data extracted from the image file to identify the subject appearing in the captured image data. Here, image analysis using a cascade classifier is performed on the captured image data. However, this is merely an example, and other types of image analysis such as pattern matching may be used, and any type of image analysis may be used as long as it is capable of identifying the subject appearing in the captured image data.

[0066] For example, subjects that are identified by image analysis by the image analysis unit 60B as subjects captured in captured image data obtained by capturing images at an athletic meet include, for example, a red and white cap, a rope, gloves, and gym clothes.

[0067] The image analysis unit 60B performs image analysis on the captured image data to generate subject identification information capable of identifying a subject, and outputs the subject identification information to the DB creation unit 60A.

[0068] As an example, as shown in FIG. 11, the storage 62 stores a small event specification table 76. The small event specification table 76 is a table used to specify a small event. Here, a small event refers to an event larger in scale than a large event. For example, if an athletic meet is a large event, the events occurring during the athletic meet and the elements for establishing the athletic meet are small events. Examples of small events related to the athletic meet include sprinting, ball throwing, and tug-of-war. For example, if skiing is a large event, small events related to skiing are snowy mountains, slopes, skiers, and snowboarders. For example, if a concert is a large event, small events related to the concert are a stage, an audience, and a conductor. For example, if swimming in the sea is a large event, small events related to swimming in the sea are the sea, a sandy beach, and a parasol.

[0069] The DB creator 60A identifies a small event and derives a major event based on the identified small event. The DB creator 60A refers to a small event identification table 76 to identify the small event. In the small event identification table 76, the small event is associated with a reference subject. The reference subject is an element that is predetermined as a necessary element that constitutes a small event.

[0070] In the example shown in Fig. 11, sprinting, ball tossing, and tug-of-war are shown as examples of small events related to an athletic meet. In the example shown in Fig. 11, a finish tape and a track are shown as examples of elements constituting the small event of sprinting, i.e., reference subjects. In addition, a red ball, a white ball, and a basket are shown as examples of elements constituting the small event of ball tossing, i.e., reference subjects. Furthermore, a rope and gloves are shown as examples of elements constituting the small event of tug-of-war, i.e., reference subjects.

[0071] The DB creator 60A refers to the small event identification table 76 to identify a small event corresponding to a subject identified by the subject identification information input from the image analyzer 60B. That is, a small event associated with a reference subject that is the same as or similar to the subject identified by the subject identification information input from the image analyzer 60B is derived from the small event identification table 76. The processes shown in Figs. 10 and 11 are performed on captured image data included in all image files in the image file DB 74.

[0072] 12, the DB creator 60A identifies a major event based on the minor event identified by referring to the minor event identification table 76 (see FIG. 11). The storage 62 stores a major event identification table 78. In the major event identification table 78, major events are associated with minor events related to the major events (hereinafter also referred to as "related minor events").

[0073] In the example shown in FIG. 12, athletic meet, skiing, a concert, swimming in the sea, etc. are shown as specific examples of major events. In the example shown in FIG. 12, sprinting, tug-of-war, ball toss, etc. are shown as examples of related minor events associated with the major event of athletic meet. In the example shown in FIG. 12, a snowy mountain and a lift are shown as examples of related minor events associated with the major event of skiing. In the example shown in FIG. 12, a stage and audience seats are shown as examples of related minor events associated with the major event of concert. In the example shown in FIG. 12, the sea, a sandy beach, and parasols are shown as examples of related minor events associated with the major event of swimming in the sea.

[0074] The DB construction unit 60A identifies a major event corresponding to a minor event identified by the image analysis unit 60B performing image analysis on an image file, by referring to the major event identification table 78. That is, a major event associated with an associated minor event that is the same as or similar to the minor event derived from the minor event identification table 76 is derived from the major event identification table 78.

[0075] In addition, the DB creator 60A acquires captured image data that is determined to be a major event from the image analyzer 60B, and extracts date information from the acquired captured image data. The date information is extracted from the imaging condition information (see FIG. 6) of the related information included in the captured image data.

[0076] The DB creator 60A identifies major events and extracts date information for each captured image data for all image files included in the image file DB 74. The DB creator 60A then classifies the image files by year and by major event, thereby creating an annual event image file DB 80.

[0077] As an example, as shown in Fig. 13, the annual event image file DB 80 classifies image files by year and by major event. The annual event image file DB 80 shown in Fig. 13 associates each image file with a file name, original image data, major event, minor event, date information, image capture location information, photographer information, model information, and image capture time information.

[0078] The file name, original image data, date information, imaging position information, photographer information, and model information are obtained from imaging condition information (see FIG. 6) associated with the captured image data. Although not shown, the original image data includes an image data identifier (see FIG. 6). The captured image data identifier is also obtained from imaging condition information associated with the captured image data. The imaging time information is information indicating the time required for imaging to obtain a moving image file (hereinafter also referred to as "imaging time"). The imaging time information is calculated, for example, from the number of frames of the captured image data included in the moving image file and the default frame rate used in imaging to obtain the moving image file. Note that this is merely an example, and if imaging time information is added to the moving image file, the imaging time information may be acquired from the moving image file. Addition of imaging time information to a moving image file is realized, for example, by measuring the time from when imaging for a moving image starts to when it ends by the CPU 42, and adding the measured time to the moving image file.

[0079] 14 and 15 show specific examples of the annual event image file DB 80. In Fig. 14, a DB in which image files captured in 2018 are classified by year and by event (hereinafter also referred to as the "2018 event image file DB") is shown as an example of the annual event image file DB 80.

[0080] In the example shown in Fig. 14, the 2018 event image file DB is divided into 2018 1st to Nth event image file DBs for each major event, 1st to Nth events. In the example shown in Fig. 14, an athletic meet is shown as an example of the first event. In other words, the 2018 1st event image file DB contains image files captured at the major event, the 2018 athletic meet.

[0081] In the example shown in Fig. 15, the 2019 event image file DB is divided into 2019 1st to Nth event image file DBs for each major event, 1st to Nth events. In the example shown in Fig. 15, an athletic meet is also shown as an example of the first event. In other words, the 2019 1st event image file DB contains image files captured at the major event, the 2019 athletic meet.

[0082] As an example, as shown in FIG. 16, the acquisition unit 60C acquires a still image file from the yearly event image file DB 80. The reduced display image data generation unit 60E extracts original image data from the still image file acquired by the acquisition unit 60C, and generates reduced display image data by reducing the extracted original image data. The reduced display image data generation unit 60E assigns a still image file identifier that can identify the corresponding still image file to the reduced display image data. The reduced display image data generation unit 60E also associates the generated reduced display image data with the corresponding still image file and stores it in the yearly event image file DB 80. As a result, the reduced display image data is classified by year and by major event, similar to the image files.

[0083] The acquisition of still image files by the acquisition unit 60C, the generation of image data for reduced display by the reduced display image data generation unit 60E, the assignment of a still image file identifier by the reduced display image data generation unit 60E, the association of the image data for reduced display to the still image file by the reduced display image data generation unit 60E, and the storage of the image data for reduced display in the annual event image file DB80 by the reduced display image data generation unit 60E are performed for all still image files included in the annual event image file DB80.

[0084] The still image file identifier is used as a key for calling up a still image file corresponding to the reduced-size display image data. For example, when reduced-size display image data is provided from the server 14 to the user device 12 and then selected by the user 16, the server 14 retrieves the still image file identified from the still image file identifier from the annual event image file DB 80 and provides it to the user device 12.

[0085] 17, the image data selection unit 60F selects, as the processing target image file group, an image file group to which date information indicating the date of the most recent year has been added from the annual event image file DB 80. In the example shown in FIG 17, the image file group for 2018 has been selected as the processing target image file group.

[0086] As an example, as shown in FIG. 18, a year storage area is set in the memory 64. The year storage area is a storage area in which the year included in date information is stored. When the image data selection unit 60F selects a group of image files to be processed from the year event image file DB 80, it acquires date information from the group of image files to be processed and stores the year included in the date indicated by the acquired date information in the year storage area. In the example shown in FIG. 18, "2018" is stored in the year storage area.

[0087] The determination unit 60D determines whether or not a video file is included in the group of image files to be processed selected by the image data selection unit 60F. If the determination unit 60D determines that a video file is not included in the group of image files to be processed selected by the image data selection unit 60F, the determination unit 60D adds one year to the year stored in the year storage area. If the determination unit 60D determines that a video file is included in the group of image files to be processed selected by the image data selection unit 60F, the determination unit 60D instructs the acquisition unit 60C to acquire the video file.

[0088] 19, when acquisition of video files is instructed by determination unit 60D, acquisition unit 60C acquires all video files from the group of target image files selected by image data selection unit 60F. Acquisition unit 60C outputs all video files acquired from the group of target image files to reduced display image data generation unit 60E, recording time identification unit 60G, and frame number derivation unit 60H.

[0089] The recording time determination unit 60G determines the recording time of the video file input from the acquisition unit 60C. For example, the image capture time indicated by the image capture time information is used as the recording time.

[0090] The frame number derivation unit 60H derives the number of frames for reduced video display based on the recording time identified by the recording time identification unit 60G. The frame number for reduced video display refers to the number of frames of image data for reduced video file display showing the image for reduced video file display described above. The storage 62 stores a frame number derivation table 82. In the frame number derivation table 82, the recording time and the frame number for reduced video display are associated with each other.

[0091] In the example shown in Fig. 19, the number of frames for reduced video display increases as the recording time becomes longer. Specifically, when the recording time is less than 1 second, the number of frames for reduced video display is "1", when the recording time is 1 second or more and less than 5 seconds, the number of frames for reduced video display is "2", and when the recording time is 5 seconds or more and less than 10 seconds, the number of frames for reduced video display is "3". Note that this is merely an example, and the contents of the frame number derivation table 82 may be variable values ​​that are changed according to instructions accepted by the accepting device 32 or 54. Furthermore, the contents of the frame number derivation table 82 may be variable values ​​that are changed according to various conditions such as the number of image files (e.g., video files) included in the image file group to be processed and / or the number of frames of captured image data included in the video files.

[0092] 20, the reduced-size display image data generating unit 60E extracts original image data from each of the multiple captured image data included in the moving image file input from the acquiring unit 60C. Then, the reduced-size display image data generating unit 60E generates reduced-size display image data by reducing the extracted original image data.

[0093] The reduced display image data generating unit 60E assigns a captured image data identifier to the reduced display image data, which can identify the corresponding captured image data. The reduced display image data generating unit 60E also associates the generated reduced display image data with the corresponding captured image data, and stores the associated data in the annual event image file DB 80.

[0094] The captured image data identifier is used as a key for calling up a video file corresponding to the reduced display image data. For example, when reduced display image data is provided from the server 14 to the user device 12 and then selected by the user 16, the server 14 retrieves the video file identified by the captured image file identifier from the annual event image file DB 80 and provides it to the user device 12.

[0095] The determination unit 60D determines whether or not the processing by the reduced display image data generation unit 60E for all video files included in the group of image files to be processed (hereinafter also referred to as "video file reduction processing") has been completed. Here, the video file reduction processing refers to the generation of reduced display image data, the assignment of video file identifiers, the association of the reduced display image data with the video files, and the storage of the reduced display image data in the annual event image file DB 80.

[0096] When the determination unit 60D determines that the video file reduction process has been completed for all video files included in the group of image files to be processed, it adds one year to the year in the year storage area. When the determination unit 60D determines that the processing by the reduced display image data generation unit 60E has not been completed for all video files included in the group of image files to be processed, it instructs the reduced display image data generation unit 60E to continue the video file reduction process. When the determination unit 60D instructs the reduced display image data generation unit 60E to continue the video file reduction process, the reduced display image data generation unit 60E continues the video file reduction process.

[0097] When the determination unit 60D determines that the video file reduction process has been completed for all video files included in the group of image files to be processed, the image data selection unit 60F acquires the year from the year storage area, as shown in Fig. 21. Then, the image data selection unit 60F reselects the group of image files for the year acquired from the year storage area as the group of image files to be processed.

[0098] The acquisition unit 60C acquires one piece of event information from the group of image files to be processed selected by the image data selection unit 60F. The event information is information indicating a major event. The determination unit 60D determines whether or not a video file is included in the group of image files related to the major event (hereinafter also referred to as a "notable event") indicated by the event information acquired by the acquisition unit 60C.

[0099] If the determination unit 60D determines that the image files related to the event of interest do not include a video file, it then determines whether or not all event information has been acquired by the acquisition unit 60C from the image files to be processed. If the determination unit 60D determines that all event information has not been acquired by the acquisition unit 60C from the image files to be processed, it instructs the acquisition unit 60C to continue acquiring event information. In response, the acquisition unit 60C continues acquiring event information from the image files to be processed. If the determination unit 60D determines that all event information has been acquired by the acquisition unit 60C from the image files to be processed, it adds one year to the year in the year storage area.

[0100] On the other hand, when the determination unit 60D determines that the image file group related to the event of interest includes a video file, it then determines whether or not the reduced display image data for the previous year includes common event information. Here, the reduced display image data for the previous year refers to reduced display image data associated with an image data group to which a date one year earlier than the processing target image data group currently selected by the image data selection unit 60F has been assigned. In addition, the common event information refers to information indicating a major event that is the same as or similar to the event of interest. The determination of whether or not the reduced display image data for the previous year includes common event information is performed by determining whether or not the image data group for the previous year includes a major event that is the same as or similar to the event of interest.

[0101] As an example, as shown in Fig. 22, when the determination unit 60D determines that the reduced display image data for the previous year does not include common event information, it then determines whether or not all event information has been acquired by the acquisition unit 60C from the group of image files to be processed. When the determination unit 60D determines that all event information has not been acquired by the acquisition unit 60C from the group of image files to be processed, it instructs the acquisition unit 60C to continue acquiring event information. In response, the acquisition unit 60C continues acquiring event information from the group of image files to be processed. When the determination unit 60D determines that all event information has been acquired by the acquisition unit 60C from the group of image files to be processed, the list image data generation process proceeds to (A) shown in Fig. 25.

[0102] On the other hand, if the judgment unit 60D determines that the reduced display image data for the previous year contains common event information, it instructs the frame number counting unit 60I to start counting the number of frames, and instructs the file number counting unit 60J to start counting the number of files.

[0103] When instructed to start counting by determination unit 60D, frame number counting unit 60I counts the number of frames of reduced display image data for major events (hereinafter also referred to as "common events") indicated by the common event information from the image files for the previous year in annual event image file DB 80. When instructed to start counting by determination unit 60D, file number counting unit 60J counts the number of image files in the image file group to be processed for the common event.

[0104] 23, the determination unit 60D determines whether the number of frames counted by the frame number counting unit 60I is greater than the number of image files counted by the file number counting unit 60J. If the determination unit 60D determines that the number of frames counted by the frame number counting unit 60I is equal to or less than the number of image files counted by the file number counting unit 60J, it instructs the reduced display image data generating unit 60E to generate reduced display image data. If the determination unit 60D determines that the number of frames counted by the frame number counting unit 60I is greater than the number of image files counted by the file number counting unit 60J, it instructs the missing frame number calculating unit 60K to calculate the missing frame number.

[0105] When the reduced-size display image data generating unit 60E is instructed by the determining unit 60D to generate reduced-size display image data, the reduced-size display image data generating unit 60E obtains one frame of captured image data from the moving image file obtained by the obtaining unit 60C, i.e., the moving image file currently being processed. Note that the group of image files to be processed including the moving image file obtained by the obtaining unit 60C is an example of "image data to be edited" according to the technology of the present disclosure.

[0106] The reduced display image data generating unit 60E extracts original image data from the acquired captured image data, and reduces the extracted original image data to generate reduced display image data for one frame. The reduced display image data generating unit 60E also assigns a captured image data identifier capable of identifying the corresponding captured image data to the reduced display image data. The reduced display image data generating unit 60E then associates the generated reduced display image data with the corresponding captured image data, and stores them in the annual event image file DB 80. After that, the list image data generating process proceeds to (A) shown in FIG.

[0107] The missing frame number calculation section 60K calculates the number of missing frames, which is a value obtained by subtracting the number of image files counted by the file number counting section 60J from the number of frames counted by the frame number counting section 60I.

[0108] As an example, as shown in FIG. 24, the reduced display image data generating unit 60E acquires captured image data for the number of frames obtained by adding 1 to the number of missing frames calculated by the missing frame number calculating unit 60K from the video file acquired by the acquiring unit 60C, i.e., the video file currently being processed (hereinafter, also referred to as the "adjusted frame number"). Here, when the number of adjustment frames is multiple, the reduced display image data generating unit 60E derives the captured image data for the number of adjustment frames, i.e., the similarity between the captured image data of multiple frames. The similarity is derived according to the result obtained by performing image recognition processing (e.g., image recognition processing by a cascade classifier and / or pattern matching, etc.) on multiple original image data included in the captured image data of multiple frames. The reduced display image data generating unit 60E acquires captured image data for which the derived similarity is within a predetermined range, for the number of frames obtained by adding 1 to the missing frame number, from the video file.

[0109] The reduced display image data generating unit 60E extracts original image data from each of the captured image data for the adjusted number of frames, and reduces the extracted original image data to generate reduced display image data for the adjusted number of frames. Here, the reduced display image data for the adjusted number of frames is an example of "second display image data for multiple frames of moving images corresponding to multiple frames of still image data constituting at least a part of the moving image data" according to the technology of the present disclosure.

[0110] The reduced display image data generating unit 60E also assigns a captured image data identifier to the reduced display image data, which can identify the corresponding captured image data. The reduced display image data generating unit 60E then associates the generated reduced display image data with the corresponding captured image data and stores them in the annual event image file DB 80. After that, the list image data generating process proceeds to (A) shown in FIG.

[0111] After the reduced display image data for the number of adjustment frames is generated and stored in the year event image file DB80, as shown in FIG. 25 as an example, the determination unit 60D determines whether or not all event information has been acquired by the acquisition unit 60C from the image file group to be processed. If the determination unit 60D determines that all event information has not been acquired by the acquisition unit 60C from the image file group to be processed, the list image data generation process proceeds to (B) shown in FIG. 21, and the determination unit 60D instructs the acquisition unit 60C to continue acquiring event information. On the other hand, if the determination unit 60D determines that all event information has been acquired by the acquisition unit 60C from the image file group to be processed, it adds one year to the year in the year storage area. Then, the determination unit 60D determines whether or not the processes shown in FIG. 21 to FIG. 24 have been executed on the image files in the year event image file DB80.

[0112] 9 to 24 have not been executed for all image file groups in the year event image file DB 80, the list image data generation process proceeds to (C) shown in Fig. 21, and the determination unit 60D instructs the image data selection unit 60F to resume selection of image file groups to be processed. Here, resuming selection of image file groups to be processed refers to the process of reselecting, from the year event image file DB 80, image file groups corresponding to the year in the year storage area as image file groups to be processed.

[0113] On the other hand, if the determining unit 60D determines that the processes shown in FIGS. 9 to 24 have been executed for all image files in the annual event image file DB 80, it instructs the list image data generating unit 60L to generate list image data.

[0114] When the determination unit 60D instructs the list image data generation unit 60L to generate list image data, the list image data generation unit 60L acquires all reduced display image data from the annual event image file DB80, classifying them by year and by event. Then, the list image data generation unit 60L generates list image data (see FIG. 26) by year and by major event from all acquired reduced display image data. Here, the list image data generated by the list image data generation unit 60L is an example of "first display image data of multiple frames" and "second display image data of multiple frames" according to the technology of the present disclosure. In this embodiment, the list image data is generated by the list image data generation unit 60L, thereby acquiring the "first display image data" according to the technology of the present disclosure.

[0115] In this embodiment, the term "yearly unit" means one year unit. However, this is merely an example, and M-year units (M: natural number equal to or greater than 2) may be used. One year unit and major events are examples of "common attributes" according to the technology of the present disclosure. Furthermore, major events are an example of "events" according to the technology of the present disclosure. Among the reduced display image data of multiple frames related to the same major event, the reduced display image data of year N is an example of "first display image data" according to the technology of the present disclosure, and the reduced display image data of year N+1 is an example of "second display image data" according to the technology of the present disclosure.

[0116] The transmission unit 60M transmits the list image data generated by the list image data generation unit 60L to the user device 12 that is the source of the image data group via the communication I / F 52. The user device 12 that is the source of the image data group refers to the user device that provided the image data group to the server 14 (for example, the user device 12 shown in FIG. 9).

[0117] As an example, as shown in FIG. 26, the CPU 60 executes list image data generation processing to acquire a group of 2019 athletic meet image files, and generates image data for reduced display of the above-mentioned adjusted number of frames (hereinafter also referred to as "image data group for displaying adjusted frames") from one video file included in the acquired group of 2019 athletic meet image files. In addition, the CPU 60 executes list image data generation processing to generate a 20 The reduced display image data included in the list image data corresponding to the 18th Athletic Meet image file group is obtained.

[0118] Therefore, the number of frames of the reduced display image data included in the list image data corresponding to the 2018 Athletic Meet image file group and the number of frames of the reduced display image data included in the list image data corresponding to the 2019 Athletic Meet image file group are unified. Note that this is merely an example, and the technology of the present disclosure is not limited to this. If an adjustment frame display image data group is generated, the number of frames of the reduced display image data included in the list image data corresponding to the 2018 Athletic Meet image file group may be greater than the number of frames of the reduced display image data included in the list image data corresponding to the 2019 Athletic Meet image file group.

[0119] In addition, here, the image data group for displaying the adjustment frames is generated from one video file, but the technology of the present disclosure is not limited to this, and the image data group for displaying the adjustment frames may be generated from multiple video files. In this case, the number of adjustment frames may be distributed and allocated to the multiple video files, and reduced display image data for the allocated number of frames may be generated from each image file.

[0120] In addition, although an athletic meet is exemplified as a major event here, the technology of the present disclosure is not limited to this, and the number of frames of reduced display image data corresponding to a group of image files in one year unit is unified between common major events held in one year unit. Also, one year unit is merely an example, and the number of frames of reduced display image data is unified even between common major events held in multiple years (for example, the opening ceremony of the Olympic Games, the closing ceremony of the Olympic Games, and / or a specific Olympic sport).

[0121] In the example shown in FIG. 26, the list image data relating to the 2018 Athletic Meet image file group is an example of "multiple frames of first display image data" according to the technology of the present disclosure. In the example shown in FIG. 26, the 2019 Athletic Meet image file group is an example of "edited image data" according to the technology of the present disclosure. In the example shown in FIG. 26, the video file on which the adjustment frame display image data group is based is an example of "video image data" according to the technology of the present disclosure. In the example shown in FIG. 26, the captured image data for multiple frames included in the video file on which the adjustment frame display image data group is based is an example of "multiple frames of still image data" according to the technology of the present disclosure.

[0122] When the list image data is received by the user device 12, as shown in Fig. 27 as an example, a list image indicated by the list image data is displayed on the display 34. On the display 34, for the images for reduced display of a plurality of frames, an image group indicated by the image data group for displaying an adjustment frame (hereinafter also referred to as "image group for displaying an adjustment frame") is displayed in a manner that allows distinction from reduced display images other than the image group for displaying an adjustment frame.

[0123] Specifically, the reduced display images of multiple frames included in the list image are displayed on a strip-shaped background extending along the time axis in such a manner that the group of images for displaying adjustment frames generated from the same video file can be distinguished from the reduced display images of multiple frames other than the group of images for displaying adjustment frames. In the example shown in Fig. 27, the group of images for displaying adjustment frames is visually distinguishable from the reduced display images other than the group of images for displaying adjustment frames by applying a different color to the background (hatched area) of the group of images for displaying adjustment frames from the background of the reduced display images other than the group of images for displaying adjustment frames.

[0124] In the following, for ease of explanation, one frame of image included in the group of images for displaying adjustment frames will be referred to as an "image for displaying adjustment frames," and data showing an image for displaying adjustment frames will be referred to as "image data for displaying adjustment frames."

[0125] Here, the adjustment frame display image data is an example of "second display image data for moving images" according to the technology of the present disclosure. The adjustment frame display image is an example of "display image based on second display image data for moving images" according to the technology of the present disclosure. The adjustment frame display image group is an example of "display image of multiple frames based on second display image data for moving images" according to the technology of the present disclosure. The reduced display images other than the adjustment frame display image group among the multiple reduced display images included in the list image related to the 2019 Athletic Meet image file group are an example of "display image based on the second display image data other than the second display image data for moving images" according to the technology of the present disclosure.

[0126] In this case, the image data for displaying the adjustment frame is generated from some of the original image data, not all of the original image data included in one video file in the 2019 Athletic Meet image file group, but the technology of the present disclosure is not limited to this, and the image data for displaying the adjustment frame may be generated from each of all the original image data included in the video file. For example, when the recording time of the video file is less than one second, the image data for displaying the adjustment frame may be generated from each of all the original image data included in the video file.

[0127] Next, the operation of the information processing system 10 will be described.

[0128] An example of the flow of the list image data generation process is shown in Figures 28A to 28D. Note that the flow of the list image data generation process shown in Figures 28A to 28D is an example of an "information processing method" according to the technique of the present disclosure.

[0129] In the list image data generation process shown in Fig. 28A, first, in step ST10, the determination unit 60D determines whether or not the image files transmitted from the user device 12 have been received by the communication I / F 52. In step ST10, if the image files transmitted from the user device 12 have not been received by the communication I / F 52, the determination is negative, and the list image data generation process proceeds to step ST80 shown in Fig. 28D. In step ST10, if the image files transmitted from the user device 12 have been received by the communication I / F 52, the determination is positive, and the list image data generation process proceeds to step ST12.

[0130] In step ST12, the DB creator 60A uses the image files received by the communication I / F 52 to create an image file DB 74 in the storage 62 (see FIG. 9), and then the list image data generating process proceeds to step ST14.

[0131] In step ST14, the image analysis unit 60B performs image analysis on the image files included in the image file DB 74 constructed in step ST12 (see FIGS. 10 and 11), and then the list image data generation process proceeds to step ST16.

[0132] In step ST16, the DB construction unit 60A constructs an annual event image file DB 80 by classifying the image files included in the image file DB 74 by year and by major event based on the results of the image analysis performed in step ST14 (see Figures 12 to 15), and then the list image data generation process proceeds to step ST18.

[0133] In step ST18, the acquisition unit 60C acquires unprocessed still image files from the annual event image file DB 80 constructed in step ST16 (see FIG. 16), and then the list image data generation process proceeds to step ST20. The still image file yet to be processed refers to a still image file for which the processes in steps ST20 and ST21 have not yet been performed.

[0134] In step ST20, the reduced display image data generating section 60E generates reduced display image data from the still image file acquired in step ST18 (see FIG. 16), and then the list image data generating process proceeds to step ST21.

[0135] In step ST21, the reduced display image data generation unit 60E associates the reduced display image data generated in step ST20 with the corresponding still image file and stores it in the annual event image file DB80 (see FIG. 16), and then the list image data generation process proceeds to step ST22.

[0136] In step ST22, the determination unit 60D determines whether or not the processes of steps ST20 and ST21 have been performed for all still image files. If the processes of steps ST20 and ST21 have not been performed for all still image files in step ST22, the determination is negative, and the list image data generation process proceeds to step ST18. If the processes of steps ST20 and ST21 have been performed for all still image files in step ST22, the determination is positive, and the list image data generation process proceeds to step ST24 shown in FIG. 28B.

[0137] In step ST24 shown in FIG. 28B, the image data selection unit 60F selects from the annual event image file DB80 a group of image files to which the date of the most recent year has been assigned as a group of image files to be processed (see FIG. 17), and then the list image data generation process proceeds to step ST26.

[0138] In step ST26, the image data selection unit 60F stores the year of the date assigned to the group of image files to be processed selected in step ST24 in the year storage area (see FIG. 18), and then the list image data generation process proceeds to step ST28.

[0139] In step ST28, the determination unit 60D determines whether or not a video file is included in the group of image files to be processed selected in step ST24 (see FIG. 18). If a video file is not included in the group of image files to be processed selected in step ST24 in step ST28, the determination is negative, and the list image data generation process proceeds to step ST42. If a video file is included in the group of image files to be processed selected in step ST24 in step ST28, the determination is positive, and the list image data generation process proceeds to step ST30.

[0140] In step ST30, the acquiring unit 60C acquires unprocessed video files from the group of image files to be processed selected in step ST24 (see FIG. 19), and then the list image data generating process proceeds to step ST32. In this step ST30, the unprocessed video files refer to video files that have not yet been subjected to the processes in steps ST32 to ST38.

[0141] In step ST32, the recording time identifying unit 60G identifies the recording time of the video file acquired in step ST30 (see FIG. 19), and thereafter, the list image data generating process proceeds to step ST34.

[0142] In step ST34, the frame number derivation unit 60H derives the number of frames for reduced video display according to the recording time identified in step ST32, using the frame number derivation table 82 (see FIG. 19), and then the list image data generation process proceeds to step ST36.

[0143] In step ST36, the reduced display image data generating unit 60E generates reduced display image data for the number of frames for reduced display of the moving image derived in step ST34 (see FIG. 20), and thereafter, the list image data generating process proceeds to step ST38.

[0144] In step ST38, the reduced display image data generation unit 60E associates the reduced display image data generated in step ST36 with the corresponding captured image data and stores it in the annual event image file DB80 (see Figure 20), and then the list image data generation process proceeds to step ST40.

[0145] In step ST40, the determination unit 60D determines whether or not the processes of steps ST32 to ST38 have been performed for all video files included in the group of image files to be processed selected in step ST24 (see FIG. 20). If the processes of steps ST32 to ST38 have not been performed for all video files included in the group of image files to be processed selected in step ST24 in step ST40, the determination is negative and the list image data generation process proceeds to step ST30. If the processes of steps ST32 to ST38 have been performed for all video files included in the group of image files to be processed selected in step ST24 in step ST40, the determination is positive and the list image data generation process proceeds to step ST42.

[0146] In step ST42, the image data selection unit 60F adds one year to the year in the year storage area (see FIG. 20), and thereafter, the list image data generation process proceeds to step ST44 shown in FIG. 28C.

[0147] In step ST44 shown in Figure 28C, the image data selection unit 60F selects a group of image data from the year event image file DB80 that has been assigned the date of the year in the year memory area as a group of image files to be processed (see Figure 21), and then the list image data generation process proceeds to step ST46.

[0148] In step ST46, the acquisition unit 60C acquires unprocessed event information from the group of image files to be processed selected in step ST44 (see FIG. 21), and then the list image data generation process proceeds to step ST48. In step ST46, the unprocessed event information refers to event information that has not yet been used in the processes after step ST48.

[0149] In step ST48, the determination unit 60D determines whether or not a video file is included in the image files of the target event indicated by the event information acquired in step ST46 among the target image files selected in step ST44 (see FIG. 21). If a video file is not included in the image files of the target event indicated by the event information acquired in step ST46 among the target image files selected in step ST44, the determination is negative and the list image data generation process proceeds to step ST68 shown in FIG. 28D. If a video file is included in the image files of the target event indicated by the event information acquired in step ST46 among the target image files selected in step ST44, the determination is positive and the list image data generation process proceeds to step ST50.

[0150] In step ST50, the determination unit 60D determines whether or not common event information is included in the reduced display image data corresponding to the image data group for the previous year in the annual event image file DB 80 (see FIG. 21). In step ST50, if the reduced display image data corresponding to the image data group for the previous year in the annual event image file DB 80 does not include common event information, the determination is negative, and the list image data generation process proceeds to step ST68 shown in FIG. 28D. In step ST50, if the reduced display image data corresponding to the image data group for the previous year in the annual event image file DB 80 includes common event information, the determination is positive, and the list image data generation process proceeds to step ST52.

[0151] In step ST52, the acquiring unit 60C acquires unprocessed common event video files from the group of image data to be processed selected in step ST44 (see the acquiring unit 60C in Figs. 23 and 24), and then the list image data generation process proceeds to step ST54. In this step ST52, the unprocessed common event image files refer to video files that are related to common event information included in the reduced display image data for the previous year from the group of image data to be processed selected in step ST44, and that have not yet been used in the processing of step ST62 or step ST64.

[0152] In step ST54, the frame number counting section 60I counts the number of frames of reduced display image data for the previous year for common events (major events indicated by the common event information) (see FIG. 22), and then the list image data generating process proceeds to step ST56. In this step ST54, the previous year refers to the year before the year in the year storage area.

[0153] In step ST56, the file number counting unit 60J counts the number of image files in the group of image data to be processed selected in step ST44 for the common event (the major event indicated by the common event information) (see FIG. 22), and then the list image data generation process proceeds to step ST58.

[0154] In step ST58, it is determined whether the number of frames counted in step ST54 is greater than the number of image files counted in step ST56 (see FIG. 23). In step ST58, if the number of frames counted in step ST54 is equal to or less than the number of image files counted in step ST56, the determination is negative, and the list image data generation process proceeds to step ST64 shown in FIG. 28D. In step ST58, if the number of frames counted in step ST54 is greater than the number of image files counted in step ST56, the determination is positive, and the list image data generation process proceeds to step ST60.

[0155] In step ST60, the missing frame number calculation unit 60K calculates the number of missing frames using the number of frames counted in step ST54 and the number of image files counted in step ST56 (see FIG. 23), and then the list image data generation process proceeds to step ST62.

[0156] In step ST62, the reduced display image data generation unit 60E generates reduced display image data for the adjusted number of frames obtained by adding 1 to the number of missing frames calculated in step ST60 from the common event moving image file acquired in step ST52 (see Figure 24), and then the list image data generation process proceeds to step ST68 shown in Figure 28D.

[0157] In step ST64 shown in FIG. 28D, the reduced display image data generation unit 60E generates one frame of reduced display image data from the common event moving image file acquired in step ST52 (see FIG. 23), and then the list image data generation process proceeds to step ST66.

[0158] In step ST66, the reduced display image data generation unit 60E associates the reduced display image data generated in step ST64 with the corresponding captured image data and stores it in the annual event image file DB80 (see Figure 23), and then the list image data generation process proceeds to step ST68.

[0159] In step ST68, the determination unit 60D determines whether or not the process of step ST46 has been performed for all the event information included in the processing target image data group selected in step ST44 (see FIG. 25). In step ST68, if the process of step ST46 has not been performed for all the event information included in the processing target image data group selected in step ST44, the determination is negative, and the list image data generation process proceeds to step ST46 shown in FIG. 28C. In step ST68, if the process of step ST46 has been performed for all the event information included in the processing target image data group selected in step ST44, the determination is positive, and the list image data generation process proceeds to step ST70.

[0160] In step ST70, the determination section 60D adds one year to the year in the year storage area (see FIG. 25), and thereafter, the list image data generating process proceeds to step ST72.

[0161] In step ST72, the determination unit 60D determines whether or not the process of step ST44 has been performed for all image files in the yearly event image file DB 80 (see FIG. 25). If the process of step ST44 has not been performed for all image files in the yearly event image file DB 80 in step ST72, the determination is negative, and the list image data generation process proceeds to step ST44 shown in FIG. 28C. If the process of step ST44 has been performed for all image files in the yearly event image file DB 80 in step ST72, the determination is positive, and the list image data generation process proceeds to step ST74.

[0162] In step ST74, the list image data generation unit 60L generates list image data by classifying all reduced display image data in the annual event image file DB80 by year and by major event (see Figures 25 and 26), and then the list image data generation process proceeds to step ST76.

[0163] In step ST76, the determination unit 60D determines whether or not the timing for transmitting the list image data (hereinafter also referred to as "transmission timing") has arrived. Examples of the transmission timing include the timing when an instruction to start transmitting the list image data is received by the receiving device 32 or 54, or the timing when a predetermined time (e.g., 10 seconds) has elapsed since the processing of step ST74 ended.

[0164] In step ST76, if the transmission timing has not arrived, the determination is negative, and the determination in step ST76 is performed again. In step ST76, if the transmission timing has arrived, the determination is positive, and the list image data generation process proceeds to step ST78.

[0165] In step ST78, the transmission section 60M transmits the list image data generated in step ST74 to the user device 12 via the communication I / F 52 (see FIG. 25), and then the list image data generation process proceeds to step ST80.

[0166] In step ST80, the determination unit 60D determines whether or not a condition for terminating the list image data generation process (hereinafter also referred to as a "list image data generation process termination condition") has been satisfied. One example of the list image data generation process termination condition is that an instruction to terminate the list image data generation process has been accepted by the acceptance device 32 or 54. In step ST80, if the list image data generation process termination condition is not satisfied, the determination is negative, and the list image data generation process proceeds to step ST10 shown in FIG. 28A. In step ST80, if the list image data generation process termination condition is satisfied, the determination is positive, and the list image data generation process ends.

[0167] As described above, in this embodiment, in the server 14, the number of adjustment frames determined according to the number of frames of the reduced display image data included in the list image data corresponding to the 2018 Athletic Meet image file group is generated from the 2019 Athletic Meet image file group (see FIG. 26). The reduced display image data included in the list image data corresponding to the 2018 Athletic Meet image file group and the 2019 Athletic Meet image file group have a common attribute of one year and an attribute of an athletic meet. Then, the display 34 displays the number of adjustment frames for the adjustment frame display image (see FIG. 27). Therefore, according to this configuration, the list image can be displayed on the display 34 with uniformity in the number of display frames for each attribute of the reduced display image data.

[0168] Furthermore, in this embodiment, the number of frames of the adjustment frame display image is limited to be equal to or less than the number of frames of the reduced display image data included in the list image data corresponding to the 2018 Athletic Meet image file group (see FIG. 26). Therefore, according to this configuration, compared to a case where the number of frames of the adjustment frame display image is not limited at all, it is possible to prevent the number of frames of the reduced display image data from exceeding the number of frames of the reduced display image data included in the list image data corresponding to the 2018 Athletic Meet image file group.

[0169] Also, in this embodiment, in the server 14, the CPU 60 acquires a group of 2019 Athletic Meet image files and a plurality of frames of reduced display image data corresponding to the group of 2018 Athletic Meet image files as image data having a common attribute (see FIG. 26). Then, list image data corresponding to the group of 2019 Athletic Meet image files is generated from the group of 2019 Athletic Meet image files. The number of frames of the list image data corresponding to the group of 2019 Athletic Meet image files is the same as the number of frames of reduced display image data included in the list image data corresponding to the group of 2018 Athletic Meet image files. Therefore, according to this configuration, the number of frames of reduced display image data having a common attribute can be unified.

[0170] In this embodiment, the common attribute of the reduced display image data of multiple frames corresponding to the 2019 Athletic Meet image file group and the 2018 Athletic Meet image file group is a major event called an athletic meet. Therefore, according to this configuration, the number of frames of reduced display image data having the common attribute of an athletic meet can be unified.

[0171] The common attribute is not limited to the major event, and instead of or together with the major event, the date, the subject, the image capturing location, the photographer, and / or the model of the image capturing device may be adopted. Also, instead of the major event, the common attribute may be a minor event. Also, although the year unit is exemplified as the common attribute in the above, the season unit (e.g., spring, summer, fall, winter), month unit, week unit, or day unit may be used.

[0172] In this embodiment, the attributes of the image files (e.g., major events) are identified by performing image analysis on the image files. Therefore, this configuration reduces the effort required to identify the attributes of the image files compared to identifying the attributes of the image files while visually checking the contents of the image files.

[0173] In this embodiment, the captured image data for the number of adjustment frames acquired from the video file by the reduced display image data generating unit 60E is captured image data whose similarity is within a predetermined range. The reduced display image data generating unit 60E generates reduced display image data for the number of adjustment frames from the captured image data for the number of adjustment frames. Therefore, this configuration can meet the needs of users who want to check subtle differences between reduced display image data.

[0174] In this embodiment, the similarity is derived based on the results of image analysis, but the technology disclosed herein is not limited to this, and the similarity may be derived by taking into account the time interval between frames of captured image data obtained from a video file, instead of or together with the results of image analysis.

[0175] In addition, the captured image data for the number of adjustment frames acquired from the video file by the reduced display image data generating unit 60E may be captured image data whose similarity is outside the predetermined range. In this case, it is possible to meet the needs of users who do not need similar reduced display image data.

[0176] In this embodiment, the display 34 displays the reduced display images of multiple frames in a manner that allows a distinction between the adjustment frame display image group and the reduced display images other than the adjustment frame display image group. The display 34 also displays the reduced display images of multiple frames included in the list image in a manner that allows a distinction between the adjustment frame display image group generated from the same video file and the reduced display images of multiple frames other than the adjustment frame display image group on a band-shaped background extending along the time axis. Therefore, according to this configuration, the user can visually distinguish between the adjustment frame display image group and the reduced display images other than the adjustment frame display image group.

[0177] In the above embodiment, the reduced display images of multiple frames included in the list image are displayed in a row along the time axis, but the technology of the present disclosure is not limited to this. For example, the display 34 may display the reduced display images of multiple frames included in the list image related to the 2019 Athletic Meet image file group in separate rows. In this case, when the time periods of imaging performed to obtain each of the reduced display images of multiple frames overlap, it is preferable to display the reduced display images of multiple frames in separate rows.

[0178] Here, as a specific example, a case will be described in which the list image data relating to the 2019 athletic meet image file group includes image data for reduced display of multiple frames obtained by capturing images in overlapping time periods (hereinafter also referred to as "image data for time period overlapping display of multiple frames"). In this case, as shown in FIG. 29 as an example, the display 34 displays the images for time period overlapping display of multiple frames in chronological order, which are indicated by the image data for time period overlapping display of multiple frames. The display 34 also displays the images for time period overlapping display of multiple frames in a manner in which the images for time period overlapping display of multiple frames are arranged in correspondence with each other at positions indicating the time periods in which the images for time period overlapping display of multiple frames are overlapping with each other on the time axis. In the example shown in FIG. 29, an upper section and a lower section are provided on the time axis, and the images for time period overlapping display are displayed in a corresponding arrangement in the upper section and the lower section at positions indicating the time periods in which the images for time period overlapping display of multiple frames. Therefore, according to this configuration, it is possible to visually grasp the temporal relationship between the images for time period overlapping display of multiple frames indicated by the image data for reduced display of multiple frames obtained by capturing images in overlapping time periods.

[0179] In the example shown in Fig. 29, the time zone overlap display image is displayed in two stages, but this is merely one example, and the image may be displayed in three or more stages. This display method of displaying images in multiple stages is effective when the time periods during which images were captured by multiple photographers overlap.

[0180] Also, as shown in FIG. 30, a reduced display screen of a plurality of frames arranged along the time axis is Another reduced-size image may be displayed in a pop-up manner between the images, in such a way that the other reduced-size image is displayed corresponding to a position on the time axis indicating the time at which the image was captured.

[0181] In the above embodiment, only the adjustment frame display image based on the adjustment frame display image data for one frame generated from the original image data for one frame is displayed in the display area of ​​the adjustment frame display image for one frame, but the technology of the present disclosure is not limited to this. For example, the adjustment frame display image for a plurality of frames represented by the adjustment frame display image data for a plurality of frames generated from the original image data for a plurality of frames (for example, the "digest version reduced display image data" shown in FIG. 31) may be displayed in sequence in the display area of ​​the adjustment frame display image for one frame. The digest version reduced display image data shown in FIG. 31 corresponds to a plurality of original image data constituting at least a part of the video file from which the digest version reduced display image data is generated. In the example shown in FIG. 32, the adjustment frame display image for three frames represented by the adjustment frame display image data for three frames constituting the digest version reduced display image data are displayed in sequence in the display area of ​​the adjustment frame display image for one frame. According to this configuration, since it is not necessary to display the adjustment frame display images for a plurality of frames side by side, it is possible to grasp the outline of the video file even if the display space is limited.

[0182] In the above embodiment, the number of frames of the adjustment frame display image data is derived according to the number of frames counted by the frame number counting unit 60I, but the technology of the present disclosure is not limited to this. For example, the number of frames of the adjustment frame display image data may be derived according to the interval of the imaging time in one block (hereinafter also referred to as the "display image data block") of the reduced display image data group for the previous year for the common event (in the example shown in FIG. 26, all reduced display image data included in the list image data related to the 2018 athletic meet image file group). For example, as shown in FIG. 33, the number of frames of the adjustment frame display image data may be derived according to the average value of the imaging time interval in a block in which the imaging time interval (hereinafter also referred to as the "imaging time interval") between adjacent frames is equal to or less than a default interval (for example, 1 second) among the reduced display image data group for the previous year for the common event. Note that instead of the average value, the median or the mode may be adopted. Note that, when there are multiple display image data blocks, the average value, median, or mode of the imaging time interval between the reduced display image data included in all the display image data blocks may be calculated and used.

[0183] In this way, in order to derive the number of frames of the adjustment frame display image data according to the interval between the imaging times in the display image data block, the CPU 60 executes list image data generation processing, as shown in Fig. 34 as an example. The flowchart shown in Fig. 34 differs from the flowchart in Fig. 28C in that it includes steps ST59A to ST59C between step ST58 and step ST60.

[0184] In the list image data generating process shown in FIG. 34, the CPU 60 calculates the image capture time intervals for the reduced display image data for the previous year for common events in step ST59A, and then the list image data generating process proceeds to step ST59B.

[0185] In step ST59B, the CPU 60 determines whether or not there is a block in the group of reduced display image data for the common event for the previous year, the image capture time interval calculated in step ST59A being equal to or shorter than the default interval. The default interval may be a fixed value or a variable value that is changed according to given conditions.

[0186] In step ST59B, if there is no block in the group of reduced display image data for common events for the previous year whose image capture time interval calculated in step ST59A is equal to or shorter than the preset interval, the determination is negative, and the list image data generation process proceeds to step ST60. In step ST59B, if there is a block in the group of reduced display image data for common events for the previous year whose image capture time interval calculated in step ST59A is equal to or shorter than the preset interval, the determination is positive, and the list image data generation process proceeds to step ST59C.

[0187] In step ST59C, the CPU 60 selects captured image data from the common event moving image file acquired in step ST52 according to the imaging time interval in the block, and then the list image data generation process proceeds to step ST59D. In this step ST59C, captured image data is selected from the common event moving image file at a time interval corresponding to the average imaging time interval in the block.

[0188] In step ST59D, the CPU 60 extracts original image data from the captured image data selected in step ST59C, and generates reduced display image data by reducing the extracted original image data. The list image data generation process then proceeds to step ST68 shown in FIG. 28D.

[0189] According to the configuration shown in Figures 33 and 34, even if the adjustment frame number is not calculated based on the frame number by the frame number counting unit 60I, the number of frames of the image data for adjustment frame display can be made closer to the number of frames of the image data for reduced display for the previous year.

[0190] In the above embodiment, the 2019 athletic meet image file group is exemplified, but the image data group related to 2019 may also be an image data group related to a major event other than an athletic meet. Here, as an example, consider a case where the image data group related to 2019 includes a first moving image file having a first attribute and a second moving image file having a second attribute. Note that here, the first moving image file is an example of the "first moving image data" related to the technology of the present disclosure, and the second moving image file is an example of the "second moving image data" related to the technology of the present disclosure. In addition, the first attribute is, for example, a major event such as an athletic meet, and the second attribute is, for example, a major event such as a concert.

[0191] 35, the CPU 60 executes a frame number suppression generation process. A frame number suppression generation process program 84 is stored in the storage 62. The frame number suppression generation process is realized by the CPU 60 reading the frame number suppression generation process program 84 from the storage 62 and executing the frame number suppression generation process program 84 on the memory 64.

[0192] By executing a frame number suppression generation process, when a first frame number of image data for displaying an adjustment frame corresponding to a first moving image file differs from a second frame number of image data for displaying an adjustment frame corresponding to a second moving image file, the CPU 60 generates image data for displaying an adjustment frame from the moving image file with a frame number determined according to the smaller of the first and second frame numbers.

[0193] An example of the flow of the frame number suppression generation process is shown in Figures 36A and 36B. The frame number suppression generation process shown in Figures 36A and 36B is executed by the CPU 60 on the condition that the process of step ST74 shown in Figure 28D has ended. Note that, for convenience of explanation, the following description will be given on the assumption that the image file group for 2018 and the image file group for 2019 are to be processed.

[0194] In the frame number suppression and generation process shown in FIG. 36A, first, in step ST100, the CPU 60 calculates the data amount of the image file group for 2018, and then the frame number suppression and generation process proceeds to step ST102.

[0195] In step ST102, the CPU 60 calculates the data amount of the video image files in the image file group for 2018, and then the frame number suppression generation process proceeds to step ST104.

[0196] In step ST104, the CPU 60 calculates the 2018 video data ratio, and then the frame number suppression generation process proceeds to step ST106. The 2018 video data ratio is the ratio of the data amount calculated in step ST102 to the data amount calculated in step ST100.

[0197] In step ST106, the CPU 60 calculates the data amount of the image files for 2019, and then the frame number suppression generation process proceeds to step ST108.

[0198] In step ST108, the CPU 60 calculates the data amount of the video image files in the image file group for 2019, and then the frame number suppression generation process proceeds to step ST110.

[0199] In step ST110, the CPU 60 calculates the 2019 video data ratio, and then the frame number suppression generation process proceeds to step ST112. The 2019 video data ratio is the ratio of the data amount calculated in step ST108 to the data amount calculated in step ST106.

[0200] In step ST112, the CPU 60 calculates the increase rate of the moving image data ratio, and then the frame number suppression generation process proceeds to step ST114. Note that the increase rate is an example of the "degree of change in ratio over time" according to the technique of the present disclosure.

[0201] In step ST114, it is determined whether the increase rate calculated in step ST112 is greater than a default increase rate (e.g., 30%). The default increase rate is an example of a "default degree" according to the technology of the present disclosure. The default increase rate may be a fixed value or a variable value that is changed according to given conditions. In step ST114, if the increase rate calculated in step ST112 is equal to or less than the default increase rate, the determination is negative, and the frame number suppression and generation process proceeds to step ST100. In step ST114, if the increase rate calculated in step ST112 is greater than the default increase rate, the determination is positive, and the frame number suppression and generation process proceeds to step ST116 shown in FIG. 36B.

[0202] In step ST116 shown in Fig. 36B, the CPU 60 calculates the number of frames of reduced display image data corresponding to the moving image file for each attribute. Specifically, the number of frames of reduced display image data corresponding to the first moving image file and the number of frames of reduced display image data corresponding to the second moving image file are calculated. Note that in this step ST116, the number of frames of reduced display image data is calculated on the premise that one frame of reduced display image data is generated for one frame of captured image data.

[0203] In the next step ST118, the CPU 60 judges whether the number of frames is different between the attributes. That is, the CPU 60 judges whether the number of frames of the reduced display image data corresponding to the first moving image file is different from the number of frames of the reduced display image data corresponding to the second moving image file. In step ST118, if the number of frames is the same between the attributes, the judgment is negative, and the frame number suppression generation process ends. In step ST118, if the number of frames is different between the attributes, the judgment is positive, and the frame number suppression generation process proceeds to step ST120.

[0204] In step ST120, the CPU 60 sets the number of frames of reduced display image data corresponding to the moving image file, i.e., the number of frames of adjustment frame display image data included in the list image data generated in step ST74 shown in Fig. 28D, to the minimum number of frames. Here, the minimum number of frames refers to the number of frames of reduced display image data corresponding to the first moving image file and the number of frames of reduced display image data corresponding to the second moving image file, whichever is smaller. After the processing of this step ST120 is executed, the frame number suppression generation processing ends.

[0205] When the frame number suppression generation process is executed in this manner, image data for displaying the adjustment frame is generated from the moving image file with a frame number determined according to the smaller of the first frame number and the second frame number, so that the number of frames of the image data for displaying the adjustment frame is not excessively increased.

[0206] In addition, when the frame number suppression generation process is executed, the number of frames of the image data for displaying the adjustment frame is set to the minimum number of frames on the condition that the growth rate exceeds the default growth rate. Therefore, compared to when the number of frames of the image data for displaying the adjustment frame is set to the minimum number of frames regardless of the growth rate, it is possible to suppress excessive restriction of the number of frames of the image data for displaying the adjustment frame. Note that, although the growth rate across years from 2018 to 2019 is exemplified here, the technology of the present disclosure is not limited to this, and the growth rate across seasons (for example, spring, summer, fall, and winter), months, weeks, or days may be calculated.

[0207] The number of frames of the image data for displaying an adjustment frame may be determined according to the smaller of the first number of frames and the second number of frames (for example, the above-mentioned minimum number of frames) and the ratio before the increase rate exceeds the default increase rate (for example, the above-mentioned 2018 video data ratio). In this case, for example, the CPU 60 derives the number of frames of the image data for displaying an adjustment frame using a frame number derivation table 86 shown in FIG. 37. In the frame number derivation table 86 shown in FIG. 37, the number of frames of the image data for displaying an adjustment frame is associated with the minimum number of frames and the 2018 video data ratio. In this way, by adopting the number of frames determined according to the minimum number of frames and the 2018 video data ratio as the number of frames of the image data for displaying an adjustment frame, it is possible to prevent the number of frames of the image data for displaying an adjustment frame from being excessive or insufficient, compared to a case in which the number of frames of the image data for displaying an adjustment frame is determined based only on the minimum number of frames or the 2018 video data ratio.

[0208] In addition, instead of using the frame number derivation table 86, the CPU 60 may calculate the number of frames of the image data for displaying the adjustment frame using an arithmetic formula in which the minimum number of frames and the 2018 video data ratio are independent variables and the number of frames of the image data for displaying the adjustment frame is the dependent variable.

[0209] In the above embodiment, an example of calculating the number of adjustment frames obtained by adding 1 to the number of missing frames has been described, but the technology of the present disclosure is not limited to this. For example, the number of frames of the image data for displaying an adjustment frame may be derived using a frame number derivation table 88 shown in FIG. 38. In the frame number derivation table 88 shown in FIG. 38, the number of frames of the image data for displaying an adjustment frame are associated with the number of missing frames and the imaging time interval. Therefore, the CPU 60 may calculate the number of missing frames and the imaging time interval, and derive the number of frames of the image data for displaying an adjustment frame from the frame number derivation table 88 according to the calculated number of missing frames and the imaging time interval. In addition, the CPU 60 may calculate the number of frames of the image data for displaying an adjustment frame using an arithmetic expression in which the number of missing frames and the imaging time interval are independent variables and the number of frames of the image data for displaying an adjustment frame is a dependent variable.

[0210] Also, the number of frames determined according to the recording time of the moving image file may be adopted as the number of frames of the image data for displaying the adjustment frame. In this case, for example, the CPU 60 may derive the number of frames of the image data for displaying the adjustment frame by using a table that applies the frame number derivation table 82 shown in FIG. 19. The table that applies the frame number derivation table 82 refers to a table that applies the number of frames of the image data for displaying the adjustment frame instead of the frame number for displaying the reduced moving image of the frame number derivation table 82. In this way, by adopting the number of frames determined according to the recording time of the moving image file as the number of frames of the image data for displaying the adjustment frame, it is possible to prevent the number of frames of the image data for displaying the adjustment frame from being excessive or insufficient, compared to the case where the number of frames of the image data for displaying the adjustment frame is determined without any consideration of the recording time of the moving image file.

[0211] In addition, the maximum number of frames of the image data for displaying the adjustment frame may be limited according to the recording time of the moving image file. This makes it possible to prevent the number of frames of the image data for displaying the adjustment frame from increasing excessively, compared to when the number of frames of the image data for displaying the adjustment frame is determined without taking into account the recording time of the moving image file.

[0212] In the above embodiment, the list image data generating process is executed by the server 14, but the technology of the present disclosure is not limited to this. For example, the list image data generating process may be executed in a distributed manner by a plurality of devices. For example, a storage server that stores at least one of the image file DB 74 shown in FIG. 9 and the yearly event image file DB 80 shown in FIG. 17, an image analysis server that executes image analysis on the image file DB 74 and the yearly event image file DB 80, and a process execution server that executes processes other than image analysis may be provided. In addition, the list image data generating process may be executed by a personal computer and / or a user device instead of the server 14.

[0213] In the above embodiment, an example was described in which the list image data generation program 72 and the frame number suppression generation processing program 84 (hereinafter, when there is no need to distinguish between these programs, they will be referred to as the "server-side program" without reference numbers) are stored in the storage 62, but the technology of the present disclosure is not limited to this. For example, as shown in FIG. 39, the server-side program may be stored in a storage medium 100. The storage medium 100 is a non-transitory storage medium. An example of the storage medium 100 is any portable storage medium such as an SSD or a USB memory. The server-side program is an example of a "program" according to the technology of the present disclosure.

[0214] The server-side program stored in the storage medium 100 is installed in the computer 50. The CPU 60 executes the list image data generation process in accordance with the list image data generation program 72, and executes the frame number suppression generation process in accordance with the frame number suppression generation process program 84. Note that, for convenience of explanation, in the following, when there is no need to distinguish between the list image data generation process and the frame number suppression generation process, they will be referred to as the "server-side process."

[0215] In addition, the server-side program may be stored in a memory unit of another computer or server device connected to computer 50 via a communication network (not shown), and the server-side program may be downloaded and installed on computer 50 in response to a request from server 14.

[0216] It is not necessary to store all of the server-side programs in the storage unit of another computer or server connected to computer 50, or in storage 62; only a portion of the server-side programs may be stored.

[0217] 39, the CPU 60 is a single CPU, but it may be a plurality of CPUs. Also, a GPU may be applied in place of the CPU 60 or together with the CPU 60.

[0218] 39 illustrates a computer 50, but the technology of the present disclosure is not limited thereto, and a device including an ASIC, an FPGA, and / or a PLD may be applied instead of the computer 50. Also, instead of the computer 50, a combination of a hardware configuration and a software configuration may be used.

[0219] The hardware resources for executing the server-side processing described in the above embodiment can be various processors as shown below. An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing server-side processing by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, which is a processor having a circuit configuration designed specifically for executing a specific process, such as an FPGA, PLD, or ASIC. Each processor has a built-in or connected memory, and each processor executes the server-side processing by using the memory.

[0220] The hardware resource that executes the server-side processing may be composed of one of these various processors, or may be composed of a combination of two or more processors of the same type or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the server-side processing may be a single processor.

[0221] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes server-side processing. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute server-side processing on a single IC chip, as typified by SoC. In this way, server-side processing is realized using one or more of the above various processors as hardware resources.

[0222] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor elements. Also, the above server-side processing is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be changed, without departing from the spirit of the invention.

[0223] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above description and illustrations, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the above description and illustrations omit explanations of technical common sense that do not require explanation in order to enable the implementation of the technology of the present disclosure.

[0224] In this specification, "A and / or B" is the same as "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0225] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or standard was specifically and individually indicated to be incorporated by reference.

Claims

1. A processor; A memory included in or connected to the processor; The processor, Acquire image data to be edited, including video data; Obtaining a first display image data of a plurality of frames; generating, from the image data to be edited, a plurality of frames of second display image data determined in accordance with at least one of the number of frames of the first display image data and a time interval between frames of the first display image data; displaying a plurality of frames of display images represented by the plurality of frames of second display image data on a display; the edit target image data and the first display image data of the plurality of frames are image data having a common attribute, The second display image data of the plurality of frames includes second display image data of the plurality of frames for a moving image corresponding to still image data of the plurality of frames constituting at least a part of the moving image data. Information processing device.

2. The information processing device according to claim 1 , wherein the number of frames of the first display image data is equal to or greater than the number of frames of the second display image data.

3. the edit target image data and the first display image data are classified according to the attributes, 3 . The information processing apparatus according to claim 1 , wherein the processor acquires the edit target image data and the first display image data of the plurality of frames as image data having the common attribute based on the attribute.

4. The information processing apparatus according to claim 1 , wherein the attributes include at least one of a date, an event, a subject, an image capturing position, a photographer, and a model of an image capturing device.

5. The information processing apparatus according to claim 1 , wherein the attribute of the image data to be edited is identified by performing an image analysis on the image data to be edited.

6. The first display image data is provided with an image capturing time at which the first display image data was captured for each frame, 6. The information processing device according to claim 1, wherein the processor derives the number of frames of the second display image data for the moving image according to an interval of the imaging times within a first display image data block in which the interval of the imaging times between adjacent frames is equal to or less than a predetermined interval and the first display image data is continuous.

7. the video data includes first video data having a first attribute as the attribute, and second video data having a second attribute as the attribute, the second attribute being different from the first attribute; 7. An information processing device as described in any one of claims 1 to 6, wherein the processor executes a frame number suppression generation process to generate the second display image data for moving images from the moving image data with a number of frames determined according to the smaller of the first frame number and the second frame number when a first frame number of the second display image data for moving images corresponding to the first moving image data differs from a second frame number of the second display image data for moving images corresponding to the second moving image data.

8. The information processing apparatus according to claim 7 , wherein the processor executes the frame number suppression generation process when a degree of change over time in a ratio of a data amount of the moving image data to a data amount of the image data to be edited exceeds a predetermined degree.

9. 9. The information processing device of claim 8, wherein the frame number suppression generation process is a process of generating the second display image data for the moving image from the moving image data with a number of frames determined according to the smaller of the first frame number and the second frame number and the ratio before the degree exceeds the default degree.

10. The information processing device according to claim 1 , wherein the processor generates the second display image data for the moving image with a number of frames determined according to a time required for capturing an image to obtain the moving image data.

11. The information processing device according to claim 1 , wherein the processor limits a maximum number of frames of the second display image data for moving images in accordance with a time required for capturing images to obtain the moving image data.

12. 12. An information processing device as claimed in any one of claims 1 to 11, wherein the still image data of the multiple frames is a group of still image data whose similarity derived according to at least one of the results obtained by performing image recognition processing on the video data and the time intervals between frames in the video data is within a predetermined range.

13. 13. An information processing device as claimed in any one of claims 1 to 12, wherein the still image data of the multiple frames is a group of still image data whose similarity derived according to at least one of the results obtained by performing image recognition processing on the video data and the time intervals between frames in the video data is outside a predetermined range.

14. An information processing device as described in any one of claims 1 to 13, wherein the display displays the display images of the multiple frames in a manner that distinguishes between display images based on the second display image data for the moving image and display images based on the second display image data other than the second display image data for the moving image.

15. An information processing device as described in any one of claims 1 to 14, wherein the display displays multiple frames of display images based on the second display image data for the moving image generated from the same moving image data, on a strip-shaped background extending along a time axis, in a manner distinguishable from display images based on the second display image data other than the second display image data for the moving image.

16. The second display image data of the plurality of frames includes time period overlapping display image data of the plurality of frames generated from the edit target image data obtained by capturing images in mutually overlapping time periods, The information processing device according to any one of claims 1 to 15, wherein the display displays the second display images of the multiple frames in chronological order, and displays the images for time zone overlapping display of the multiple frames indicated by the image data for time zone overlapping display of the multiple frames in a manner arranged in correspondence with the position indicating the time zone on the time axis.

17. The information processing device described in claim 16, wherein the display displays the images for time zone overlap display of the multiple frames in a manner in which a time zone overlap display image for moving images corresponding to moving images and a time zone overlap display image for still images corresponding to still images are arranged separately in correspondence with positions indicating the time zones on the time axis.

18. 18. An information processing device as claimed in any one of claims 1 to 17, wherein the second display image data for at least one frame of the second display image data for the moving image corresponds to still image data of multiple frames constituting at least a part of the moving image data.

19. Obtaining image data to be edited, including moving image data; Obtaining a first display image data of a plurality of frames; generating, from the image data to be edited, a plurality of frames of second display image data determined in accordance with at least one of the number of frames of the first display image data and a time interval between frames of the first display image data; and displaying a plurality of frames of display images represented by the plurality of frames of second display image data on a display; the edit target image data and the first display image data of the plurality of frames are image data having a common attribute, The second display image data of the plurality of frames includes second display image data of the plurality of frames for a moving image corresponding to still image data of the plurality of frames constituting at least a part of the moving image data. Information processing methods.

20. A program for causing a computer to execute a process, The process comprises: Obtaining image data to be edited, including moving image data; Obtaining a first display image data of a plurality of frames; generating, from the image data to be edited, a plurality of frames of second display image data determined in accordance with at least one of the number of frames of the first display image data and a time interval between frames of the first display image data; and displaying a plurality of frames of display images represented by the plurality of frames of second display image data on a display; the edit target image data and the first display image data of the plurality of frames are image data having a common attribute, The second display image data of the plurality of frames includes second display image data of the plurality of frames for a moving image corresponding to still image data of the plurality of frames constituting at least a part of the moving image data. program.

Images