Information processing device, information processing method, and program

Abstract

To provide an information processing device, an information processing method, and a program displaying display images with uniformity in the number of display frames for each attribute of the display image.SOLUTION: In a server 14, a CPU 60 acquires image data to be edited including moving image data, acquires first display image data of a plurality of frames, generates second display image data of a plurality of frames determined according to the number of frames of the first display image data from the image data to be edited, and displays a display image of a plurality of frames indicated by the second display image data of a plurality of frames on a display. The image data to be edited and the first display image data of the plurality of frames are image data having common attributes. The second display image data of the plurality of frames includes second display image data for a plurality of frames of moving image corresponding to still image data of a plurality of frames constituting at least a part of moving image data.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The technology disclosed herein relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] Japanese Patent Application Laid-Open Publication No. 2014-146939 discloses a photobook creation terminal that extracts image data from moving image data and lays out pages in a photobook. The photobook creation terminal described in Japanese Patent Application Laid-Open Publication No. 2014-146939 includes an input means, a page allocation means, an image number calculation means, an image extraction means, and a layout means. The input means inputs moving image data. The page allocation means allocates scenes included in the moving image data to one or more pages of the photobook. The image number calculation means calculates the number of frames required for a scene for each page allocated by the page allocation means. The image extraction means extracts image data for each scene of the moving image data with the required number of frames calculated by the image number calculation means. The layout means lays out the image data extracted by the image extraction means on the pages allocated by the page allocation means. Summary of the Invention

[0003] However, in the technology described in JP 2014-146939 A, the required number of frames of image data is calculated without considering attributes such as events that are similar between image data. Therefore, when image data is classified by attributes such as events, the number of frames of image data may vary greatly between attributes.

[0004] One embodiment of the technology disclosed herein provides an information processing device, an information processing method, and a program that can display a display image with a consistent number of display frames for each attribute of the display image. [Means for solving the problem]

[0005] A first aspect of the technology of the present disclosure is an information processing device comprising a processor and a memory built into or connected to the processor, wherein the processor acquires image data to be edited including moving image data, acquires first display image data of a plurality of frames, generates second display image data of a plurality of frames from the image data to be edited, the second display image data being determined according to at least one of the number of frames of the first display image data and the time interval between frames of the first display image data, and displays display images of a plurality of frames indicated by the second display image data of a plurality of frames on a display, wherein the image data to be edited and the first display image data of a plurality of frames are image data having common attributes, and the second display image data of a plurality of frames includes second display image data for a moving image of a plurality of frames corresponding to still image data of a plurality of frames that constitute at least a part of the moving image data.

[0006] A second aspect of the technique of the present disclosure is the information processing device according to the first aspect, in which the number of frames of the first display image data is equal to or greater than the number of frames of the second display image data.

[0007] A third aspect of the technology disclosed herein is an information processing device according to the first or second aspect, in which the image data to be edited and the first image data for display are classified by attribute, and the processor acquires the image data to be edited and multiple frames of the first image data for display as image data having a common attribute based on the attribute.

[0008] A fourth aspect of the technology of the present disclosure is an information processing device according to any one of the first to third aspects, in which the attributes include at least one of date, event, subject, imaging location, photographer, and imaging device model.

[0009] A fifth aspect of the technology of the present disclosure is an information processing device according to any one of the first to third aspects, in which the attributes of the image data to be edited are identified by performing image analysis on the image data to be edited.

[0010] A sixth aspect of the technology of the present disclosure is an information processing device according to any one of the first to fifth aspects, in which the first display image data is assigned with the imaging time at which the image for the first display image data was captured for each frame, and the processor derives the number of frames of the second display image data for moving images according to the interval between the imaging times within a first display image data block in which the interval between the imaging times of adjacent frames is less than a predetermined interval and the first display image data is continuous.

[0011] A seventh aspect of the technology of the present disclosure is an information processing device according to any one of the first to sixth aspects, in which the video data includes first video data having a first attribute as an attribute and second video data having a second attribute as an attribute different from the first attribute, and in which, when the first number of frames of the second display image data for video corresponding to the first video data differs from the second number of frames of the second display image data for video corresponding to the second video data, the processor executes a frame number suppression generation process to generate the second display image data for video from the video data with a frame number determined according to the smaller of the first number of frames and the second number of frames.

[0012] An eighth aspect of the technology of the present disclosure is an information processing device according to the seventh aspect, in which the processor executes a frame number suppression generation process when the degree of change over time in the ratio of the data volume of moving image data to the data volume of image data to be edited exceeds a predetermined degree.

[0013] A ninth aspect of the technology of the present disclosure is an information processing device according to the eighth aspect, in which the frame number suppression generation process is a process of generating second display image data for moving images from moving image data with a number of frames determined according to the smaller of the first frame number and the second frame number and the proportion before the degree exceeds a default degree.

[0014] A tenth aspect of the technology of the present disclosure is an information processing device according to any one of the first to ninth aspects, in which a processor generates second display image data for moving images with a number of frames determined according to the time required for capturing images to obtain the moving image data.

[0015] An eleventh aspect of the technology of the present disclosure is an information processing device according to any one of the first to tenth aspects, in which the processor limits the maximum number of frames of the second display image data for moving images according to the time required for capturing images to obtain the moving image data.

[0016] A twelfth aspect of the technology of the present disclosure is an information processing device according to any one of the first to eleventh aspects, in which multiple frames of still image data are a group of still image data whose similarity, derived according to the results obtained by performing image recognition processing on moving image data and at least one of the time intervals between frames in the moving image data, is within a predetermined range.

[0017] A thirteenth aspect of the technology of the present disclosure is an information processing device according to any one of the first to twelfth aspects, in which multiple frames of still image data are a group of still image data whose similarity, derived according to the results obtained by performing image recognition processing on moving image data and at least one of the time intervals between frames in the moving image data, is outside a predetermined range.

[0018] A fourteenth aspect of the technology of the present disclosure is an information processing device according to any one of the first to thirteenth aspects, in which the display displays, for multiple frames of display images, display images based on second display image data for moving images and display images based on second display image data other than the second display image data for moving images in a manner that allows them to be distinguished.

[0019] A fifteenth aspect of the technology of the present disclosure is an information processing device according to any one of the first to fourteenth aspects, in which the display displays multiple frames of display images based on second display image data for moving images generated from the same moving image data on a strip-shaped background extending along the time axis in a manner that is distinguishable from display images based on second display image data other than the second display image data for moving images.

[0020] A 16th aspect of the technology disclosed herein is an information processing device according to any one of the first to fifteenth aspects, in which the second display image data of multiple frames includes image data for overlapping time periods of multiple frames generated from image data to be edited obtained by capturing images during overlapping time periods, and the display displays the second display images of multiple frames in chronological order, and displays the time period overlapping display images of multiple frames indicated by the time period overlapping display image data of multiple frames in a manner arranged in correspondence with positions indicating the time periods on the time axis.

[0021] A 17th aspect of the technology of the present disclosure is an information processing device according to the 16th aspect, in which the display displays multiple frames of time zone overlap display images, with time zone overlap display images for moving images corresponding to moving images and time zone overlap display images for still images corresponding to still images, arranged separately in accordance with positions indicating the time zones on the time axis.

[0022] An 18th aspect of the technology of the present disclosure is an information processing device according to any one of the 1st to 17th aspects, in which the second display image data for at least one frame of moving images among the second display image data for multiple frames of moving images corresponds to still image data of multiple frames that constitute at least a part of the moving image data.

[0023] A 19th aspect of the technology of the present disclosure is an information processing method including: acquiring image data to be edited including moving image data; acquiring first display image data of a plurality of frames; generating second display image data of a plurality of frames from the image data to be edited, the second display image data being determined according to at least one of the number of frames of the first display image data and the time interval between frames of the first display image data; and displaying, on a display, display images of a plurality of frames indicated by the second display image data of a plurality of frames, wherein the image data to be edited and the first display image data of a plurality of frames are image data having common attributes, and the second display image data of a plurality of frames includes second display image data of a plurality of frames for a moving image corresponding to still image data of a plurality of frames that constitute at least a part of the moving image data.

[0024] A twentieth aspect of the technology of the present disclosure is a program for causing a computer to execute processing, the processing including: acquiring image data to be edited including moving image data; acquiring first display image data of a plurality of frames; generating second display image data of a plurality of frames from the image data to be edited, the second display image data being determined according to at least one of the number of frames of the first display image data and the time interval between frames of the first display image data; and displaying, on a display, display images of a plurality of frames indicated by the second display image data of a plurality of frames, wherein the image data to be edited and the first display image data of a plurality of frames are image data having common attributes, and the second display image data of a plurality of frames includes second display image data of a plurality of frames for a moving image corresponding to still image data of a plurality of frames that constitute at least a part of the moving image data. [Effects of the Invention]

[0025] According to one embodiment of the technique of the present disclosure, an effect is obtained in which display images can be displayed with a consistent number of display frames for each attribute of the display image. [Brief explanation of the drawings]

[0026] [Figure 1] 1 is a conceptual diagram illustrating a schematic configuration of an information processing system according to an embodiment. [Figure 2] FIG. 2 is a block diagram showing an example of a hardware configuration of an electrical system of a user device included in the information processing system according to the embodiment. [Figure 3] FIG. 2 is a block diagram illustrating an example of a hardware configuration of an electrical system of a server included in the information processing system according to the embodiment. [Figure 4] FIG. 1 is a conceptual diagram illustrating a use of an information processing system according to an embodiment. [Figure 5] FIG. 2 is a block diagram showing an example of the contents stored in a storage of a user device included in the information processing system according to the embodiment. [Figure 6] FIG. 2 is a conceptual diagram illustrating an example of the configuration of captured image data according to the embodiment. [Figure 7] FIG. 10 is a conceptual diagram illustrating a conventional method for generating list image data. [Figure 8] 10 is a block diagram showing an example of main functions of a CPU when a list image data generation process is executed by the CPU of a server included in the information processing system according to the embodiment. FIG. [Figure 9] 10 is a block diagram showing an example of processing contents when a CPU of a server included in an information processing system according to an embodiment operates as a DB construction unit. FIG. [Figure 10] 10 is a block diagram showing an example of processing contents when a CPU of a server included in the information processing system according to the embodiment operates as a DB construction unit and an image analysis unit. FIG. [Figure 11] 10 is a block diagram showing an example of processing contents when a CPU of a server included in the information processing system according to the embodiment operates as a DB construction unit and an image analysis unit. FIG. [Figure 12] 10 is a block diagram showing an example of processing contents when a CPU of a server included in the information processing system according to the embodiment operates as a DB construction unit and an image analysis unit. FIG. [Figure 13]10 is a conceptual diagram showing an example of the configuration of an annual event image file DB constructed by a CPU of a server included in the information processing system according to the embodiment. FIG. [Figure 14] FIG. 10 is a conceptual diagram showing an example of the configuration of a 2018 first event image file DB. [Figure 15] FIG. 1 is a conceptual diagram showing an example of the configuration of a 2019 first event image file DB. [Figure 16] 10 is a block diagram showing an example of processing contents when a CPU of a server included in an information processing system according to an embodiment operates as an acquisition unit. FIG. [Figure 17] 10 is a block diagram showing an example of processing contents when a CPU of a server included in the information processing system according to the embodiment operates as an image data selection unit. FIG. [Figure 18] 10 is a block diagram showing an example of processing contents when a CPU of a server included in the information processing system according to the embodiment operates as an image data selection unit, a determination unit, and an acquisition unit. FIG. [Figure 19] 10] FIG. 10 is a block diagram showing an example of processing contents when a CPU of a server included in the information processing system according to the embodiment operates as an acquisition unit, a recording time determination unit, and a frame number derivation unit. [Figure 20] 10 is a block diagram showing an example of processing contents when a CPU of a server included in an information processing system according to an embodiment operates as an acquisition unit, a reduced display image data generation unit, a determination unit, and a frame number derivation unit. FIG. [Figure 21] 10 is a block diagram showing an example of processing contents when a CPU of a server included in the information processing system according to the embodiment operates as an image data selection unit, an acquisition unit, and a determination unit. FIG. [Figure 22] FIG. 10 is a block diagram showing an example of processing contents when a CPU of a server included in the information processing system according to the embodiment operates as a determination unit, a file number counting unit, and a frame number counting unit. [Figure 23]A block diagram showing an example of processing contents when the CPU of a server included in the information processing system according to the embodiment operates as a file number counting unit, a determination unit, a frame number counting unit, a missing frame number calculation unit, an acquisition unit, and a reduced display image data generation unit. [Figure 24] 10 is a block diagram showing an example of processing contents when a CPU of a server included in an information processing system according to an embodiment operates as an acquisition unit, a missing frame number calculation unit, and a reduced display image data generation unit. FIG. [Figure 25] 10 is a block diagram showing an example of processing contents when a CPU of a server included in the information processing system according to the embodiment operates as a determination unit, a list image data generation unit, and a transmission unit. FIG. [Figure 26] 10 is an example of list image data obtained by executing the list image data generation process according to the embodiment. [Figure 27] 10A and 10B are diagrams illustrating an example of a screen in which a list image shown by list image data obtained by executing the list image data generation process according to the embodiment is displayed on a display of a user device. [Figure 28A] 10 is a flowchart illustrating an example of the flow of a list image data generation process according to the embodiment. [Figure 28B] This is a continuation of the flowchart shown in FIG. 28A. [Figure 28C] This is a continuation of the flowchart shown in FIG. 28B. [Figure 28D] This is a continuation of the flowchart shown in FIG. 28C. [Figure 29] FIG. 10 is a screen diagram showing a first modified example of a display mode of a list image displayed on a display of a user device included in the information processing system according to the embodiment. [Figure 30] FIG. 10 is a screen diagram showing a second modified example of the display mode of a list image displayed on the display of a user device included in the information processing system according to the embodiment. [Figure 31] 10 is a modified example of list image data obtained by executing the list image data generation process according to the embodiment. [Figure 32] 32 is a screen diagram showing an example of a state in which list image data indicated by the list image data shown in FIG. 31 is displayed on the display of a user device. FIG. [Figure 33] FIG. 10 is a conceptual diagram showing an example of a configuration of blocks in which the interval between the imaging times of adjacent frames is equal to or shorter than a predetermined interval. [Figure 34] 10 is a flowchart showing a modified example of the flow of the list image data generation process according to the embodiment. [Figure 35] FIG. 10 is a block diagram showing an example of a configuration in which a frame number suppression generation process is executed by a CPU of a server included in an information processing system according to an embodiment. [Figure 36A] 10 is a flowchart illustrating an example of the flow of a frame number suppression generation process according to the embodiment. [Figure 36B] This is a continuation of the flowchart shown in FIG. 36A. [Figure 37] FIG. 10 is a conceptual diagram showing a first modified example of a frame number derivation table used by a CPU of a server included in an information processing system according to an embodiment. [Figure 38] FIG. 10 is a conceptual diagram showing a second modified example of the frame number derivation table used by the CPU of the server included in the information processing system according to the embodiment. [Figure 39] FIG. 10 is a block diagram showing an example of how a server-side program is installed on a computer in a server from a storage medium on which the server-side program is stored. DETAILED DESCRIPTION OF THE INVENTION

[0027] An example of an embodiment of an information processing device, an information processing method, and a program according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0028] First, the terms used in the following description will be explained.

[0029] CPU is an abbreviation for "Central Processing Unit". RAM is an abbreviation for "Random Access Memory". SSD is an abbreviation for "Solid State Drive". HDD is an abbreviation for "Hard Disk Drive". EEPROM is an abbreviation for "Electrically Erasable and Programmable Read Only Memory". ASIC is an abbreviation for "Application Specific Integrated Circuit". PLD is an abbreviation for "Programmable Logic Device". FPGA is an abbreviation for "Field-Programmable Gate Array". SoC is an abbreviation for "System-on-a-chip". CMOS is an abbreviation for "Complementary Metal Oxide Semiconductor". CCD is an abbreviation for "Charge Coupled Device". EL is an abbreviation for "Electro-Luminescence". UI is an abbreviation for "User Interface". USB is an abbreviation for "Universal Serial Bus". GPU is an abbreviation for "Graphics Processing Unit". GPS is an abbreviation for "Global Positioning System". RTC is an abbreviation for "Real Time Clock". ID is an abbreviation for "Identification". Exif is an abbreviation for "exchangeable image file format". WAN is an abbreviation for "Wide Area Network". LAN is an abbreviation for "Local Area Network". DB is an abbreviation for "Database". JPEG is an abbreviation for "Joint Photographic Experts Group". MPEG is an abbreviation for "Moving Picture Experts Group".

[0030] 1, an information processing system 10 includes a user device 12 and a server 14. The user device 12 is a terminal device, such as a smartphone, that transmits and receives input information and / or image information from a user 16 to and from the server 14. However, this is merely an example, and the user device 12 may also be a tablet terminal, a personal computer, a wearable terminal, and / or a digital camera, or other terminal device.

[0031] The information processing system 10 is used by multiple users 16. Each user 16 is assigned a user device 12. For example, the user 16 is the owner of the user device 12.

[0032] The user device 12 is communicatively connected to the server 14 via a network 19. An example of the network 19 is a WAN such as the Internet. The network 19 is not limited to a WAN, but may be a LAN, or a network in which a LAN and a WAN are connected. The user device 12 and the network 19 may be connected via wireless communication or wired communication. The server 14 and the network 19 may be connected via wireless communication or wired communication. Although not shown in the example shown in FIG. 1 , the network 19 may include, for example, a base station. The network 19 establishes communication between the user device 12 and the server 14 and transmits and receives various information between the user device 12 and the server 14.

[0033] The server 14 receives a request from the user device 12 via the network 19 and provides a service according to the request to the user device 12 via the network 19 .

[0034] For example, the user device 12 requests the server 14 to process image data (e.g., the "image file group" shown in FIG. 4). The server 14 processes the image data provided by the user device 12 and provides the processed image data (e.g., the "list image data" shown in FIG. 4) to the user device 12. The server 14 is an example of an "information processing device" according to the technology of the present disclosure.

[0035] 2, the user device 12 includes a computer 22, an image capture device 24, a clock 26, a communication I / F 28, a GPS receiver 30, an accepting device 32, a display 34, a microphone 36, a speaker 38, and an external I / F 40. The computer 22 includes a CPU 42, a storage 44, and a memory 46. The CPU 42, the storage 44, and the memory 46 are connected to a bus 48. The image capture device 24, the clock 26, the communication I / F 28, the GPS receiver 30, the accepting device 32, the display 34, the microphone 36, the speaker 38, and the external I / F 40 are also connected to the bus 48. Note that in the example shown in FIG. 2, for convenience of illustration, a single bus is shown as the bus 48, but the bus 48 also includes a data bus, an address bus, a control bus, and the like.

[0036] The CPU 42 controls the entire user device 12. The storage 44 stores various parameters and programs. The storage 44 is a non-volatile storage device. Here, an EEPROM is used as an example of the storage 44, but the storage 44 is not limited to this and may be an SSD and / or an HDD. The memory 46 is a volatile storage device. The memory 46 is used as a work memory by the CPU 42 and temporarily stores various information. Here, a DRAM is used as an example of the memory 46, but the storage 44 is not limited to this and may be another type of volatile storage device such as an SRAM.

[0037] The imaging device 24 is a device that generates image data. The imaging device 24 has, for example, a CMOS image sensor and is equipped with a zoom mechanism, a focus adjustment mechanism, and the like. Note that, although a CMOS image sensor is exemplified here as the image sensor of the imaging device 24, this is not limiting and other types of image sensors such as a CCD image sensor may also be used. The imaging device 24 captures an image of a subject in accordance with instructions from the CPU 42. Then, the imaging device 24 captures an image of the subject and generates image data representing the subject. The CPU 42 acquires the image data generated by the imaging device 24 and stores the acquired image data in the storage 44.

[0038] The clock 26 acquires the current time. The clock 26 is, for example, an RTC, and receives a supply of driving power from a power supply system that is separated from the power supply system for the computer 22. Even when the computer 22 is shut down, the clock 26 continues to keep track of the current time (year, month, day, hour, minute, and second). The clock 26 outputs the current time to the CPU 42 every time the current time is updated.

[0039] The communication I / F 28 is connected to the network 19 via wireless communication and controls the exchange of various information between the CPU 42 and the server 14 via the network 19. Note that although a wireless communication method is exemplified here, the technology of the present disclosure is not limited to this and may also be a wired communication method.

[0040] The GPS receiver 30 receives radio waves from multiple GPS satellites (not shown) in response to instructions from the CPU 42, and outputs reception result information indicating the reception result to the CPU 42. Based on the reception result information input from the GPS receiver 30, the CPU 42 calculates GPS information as location identification information that can identify the current location of the user device 12. The GPS information is, for example, latitude, longitude, and altitude that can identify the current location of the user device 12.

[0041] The receiving device 32 receives instructions from the user 16 or the like. Examples of the receiving device 32 include a touch panel 32A and hard keys. The instructions received by the receiving device 32 are acquired by the CPU 42. The receiving device 32 may also receive instructions from the user 16 or the like by voice input via the microphone 36.

[0042] The display 34 displays various types of information under the control of the CPU 42. An example of the display 34 is a liquid crystal display. However, the display 34 is not limited to a liquid crystal display, and other types of displays such as an EL display may also be used.

[0043] In this embodiment, an out-cell type touch panel display is used in which the touch panel 32A is overlaid on the surface of the display area of ​​the display 34. However, the out-cell type touch panel display is merely an example, and it is also possible to apply, for example, an on-cell type or an in-cell type touch panel display.

[0044] The microphone 36 converts the collected sound into an electrical signal and outputs the electrical signal obtained by converting the sound to the CPU 42. The speaker 38 converts an electrical signal input from a specific device (e.g., the CPU 42) into sound and outputs the sound obtained by converting the electrical signal to the outside of the user device 12.

[0045] The external I / F 40 controls the exchange of various information with devices external to the user device 12. An example of the external I / F 40 is a USB interface. The USB interface is connected to a user device, a personal computer, a server, a USB memory, a memory card, and / or a printer.

[0046] 3, the server 14 includes a computer 50, a communication I / F 52, a receiving device 54, a display 56, and an external I / F 58. The computer 50 includes a CPU 60, a storage 62, and a memory 64. The CPU 60, the storage 62, and the memory 64 are connected to a bus 66. The communication I / F 52, the receiving device 54, the display 56, and the external I / F 58 are also connected to the bus 66. In the example shown in FIG. 3, for convenience of illustration, a single bus is shown as the bus 66, but the bus 66 also includes a data bus, an address bus, a control bus, and the like.

[0047] The CPU 60 controls the entire server 14. The storage 62 stores various parameters and various programs. The storage 62 is a non-volatile storage device. Here, an SSD is used as an example of the storage 62, but the storage 62 is not limited to this and may be an EEPROM and / or an HDD, etc. The memory 64 is a volatile storage device. The memory 64 is used as a work memory by the CPU 60 and temporarily stores various information. Here, a DRAM is used as an example of the memory 64, but the memory 64 is not limited to this and may be another type of volatile storage device such as an SRAM. The CPU 60 is an example of a "processor" according to the technology of the present disclosure, and the storage 62 and the memory 64 are examples of "memory" according to the technology of the present disclosure.

[0048] The communication I / F 52 is communicably connected to the network 19 and controls the exchange of various information between the CPU 60 and the user device 12 via the network 19 .

[0049] The receiving device 54 receives instructions from an administrator of the server 14 or the like. Examples of the receiving device 54 include a remote controller, a touch panel, and / or a hard key. The instructions received by the receiving device 54 may also include instructions input by voice via a microphone or the like. The instructions received by the receiving device 54 are acquired by the CPU 60.

[0050] The display 56 displays various types of information under the control of the CPU 60. An example of the display 56 is a liquid crystal display. However, the display 56 is not limited to a liquid crystal display, and other types of displays such as an EL display may also be used.

[0051] The external I / F 58 controls the exchange of various information with devices external to the server 14. An example of the external I / F 58 is a USB interface. A user device, a personal computer, a server, a USB memory, a memory card, and / or a printer, etc., are connected to the USB interface.

[0052] As an example, as shown in FIG. 4, in an information processing system 10, a user device 12 uploads a group of image files to a server 14, and the server 14 manages the uploaded group of image files. The group of image files includes a plurality of image files. An image file is, for example, a file containing captured image data obtained by capturing an image of a subject using the user device 12. Image files are broadly classified into still image files and video files. A still image file contains captured image data for one frame. A video file contains captured image data for multiple frames obtained by capturing an image of a subject at a predetermined frame rate (for example, 30 frames per second).

[0053] The server 14 performs image processing on the group of image files to generate list image data and provides it to the user device 12. List image data refers to, for example, image data in which multiple pieces of reduced-size image data for display are compiled in a list format. The reduced-size image data for display is image data obtained by reducing the captured image data. In other words, the reduced-size image data for display is image data that is reduced in size and image quality compared to the captured image data. A specific example of the reduced-size image data for display is thumbnail image data that shows a thumbnail image.

[0054] The user 16 can view the list image displayed by the list image data via the display 34 of the user device 12. The list image includes a plurality of thumbnail images displayed by a plurality of thumbnail image data. In the example shown in FIG. 4, a plurality of thumbnail images corresponding to a plurality of still image files (hereinafter also referred to as "still image file thumbnail images") and a plurality of thumbnail images corresponding to a plurality of video files (hereinafter also referred to as "video file thumbnail images") are displayed. A video identification mark is displayed in the center of the video file thumbnail image as a mark that can identify the video file thumbnail image.

[0055] The user 16 selects a thumbnail image from the list image via the touch panel 32A. When the user 16 selects a thumbnail image, an image file corresponding to the selected thumbnail image is downloaded from the server 14 to the user device 12. In the example shown in FIG. 4 , a thumbnail image for a moving image file has been selected by the user 16, and so the moving image file corresponding to the thumbnail image for the moving image file selected by the user 16 is downloaded from the server 14 to the user device 12.

[0056] As an example, as shown in FIG. 5 , a group of image files is stored in the storage 44 of the user device 12. The group of image files in the storage 44 is uploaded from the user device 12 to the server 14 in response to an instruction received by the receiving device 32. In the example shown in FIG. 5 , a plurality of groups of image files including a first group of image files and a second group of image files are stored in the storage 44. Each of the groups of image files is stored in, for example, a yearly folder. For example, the yearly folder in which the first group of image files is stored is a folder representing 2018. The folder representing 2018 stores a group of image files obtained by capturing images using the imaging device 24 in 2018. Furthermore, for example, the yearly folder in which the second group of image files is stored is a folder representing 2019. The folder representing 2019 stores a group of image files obtained by capturing images using the imaging device 24 in 2019.

[0057] As an example, as shown in Figure 6, captured image data included in a still image file and a moving image file includes original image data and related information. The original image data and related information are associated one-to-one for each captured image data. The original image data is still image data. Therefore, a still image file contains original image data for one frame, and a moving image file contains original image data for multiple frames.

[0058] The related information includes the file name of the corresponding image file and imaging condition information indicating the imaging conditions (hereinafter also referred to as "imaging conditions") used to obtain the corresponding original image data. An example of the imaging condition information is Exif information. In the example shown in FIG. 6, the imaging condition information includes date information, imaging location information, photographer information, model information, and a captured image data identifier. The date information is information indicating the date on which one frame of image data was captured. An example of the date information is the year, month, day, hour, minute, and second obtained from the clock 26 when one frame of image data was captured. The imaging location information is information indicating the location where image data was captured by the imaging device 24. An example of the imaging location information is GPS information calculated when one frame of image data was captured. The photographer information is information indicating the photographer, and the model information is information indicating the model of the imaging device 24. The photographer information and the model information are registered in advance in, for example, the user device 12. The captured image data identifier is an identifier that can identify the corresponding captured image data. Therefore, one captured image data identifier is assigned to a still image file, and multiple captured image data identifiers are assigned to a moving image file.

[0059] Generally, one frame of reduced display image data is generated for one image file. Therefore, if the number of image files differs among the image files uploaded from the user device 12 to the server 14, the number of frames of the reduced display image data included in the list image data corresponding to the image files also differs.

[0060] For example, as shown in FIG. 7, the list image data for the image file group obtained by capturing images at the 2018 athletic meet (hereinafter also referred to as the "2018 athletic meet image file group") includes nine frames of reduced-size image data. In contrast, the image file group obtained by capturing images at the 2019 athletic meet (hereinafter also referred to as the "2019 athletic meet image file group") includes two still image files and two video files. Therefore, if one frame of reduced-size image data is generated for one image file, the list image data for the 2019 athletic meet image file will include four reduced-size image data. The four reduced-size image data refer to two still image file reduced-size image data corresponding to two still image files and two video file reduced-size image data corresponding to two video files.

[0061] Thus, the number of frames of the reduced-size image data for display corresponding to the 2019 Athletic Meet image file group is fewer than the number of frames of the reduced-size image data for display corresponding to the 2018 Athletic Meet image file group. In other words, even though the image files in 2018 and 2019 were captured at the same event, "Athletics Meet," the number of frames of the reduced-size image data for display is not the same between 2018 and 2019.

[0062] In view of these circumstances, as an example, as shown in Fig. 8, in the server 14, a list image data generation process is executed by the CPU 60. A list image data generation program 72 is stored in the storage 62 of the server 14. The CPU 60 reads the list image data generation program 72 from the storage 62. The CPU 60 then executes the list image data generation program 72 read from the storage 62 on the memory 64, thereby operating as a DB construction unit 60A, an image analysis unit 60B, an acquisition unit 60C, a determination unit 60D, a reduced display image data generation unit 60E, an image data selection unit 60F, a recording time determination unit 60G, a frame number derivation unit 60H, a frame number counting unit 60I, a file number counting unit 60J, a missing frame number calculation unit 60K, a list image data generation unit 60L, and a transmission unit 60M. That is, the list image data generation process is realized by the CPU 60 operating as a DB construction unit 60A, an image analysis unit 60B, an acquisition unit 60C, a judgment unit 60D, a reduced display image data generation unit 60E, an image data selection unit 60F, a recording time determination unit 60G, a frame number derivation unit 60H, a frame number counting unit 60I, a file number counting unit 60J, a missing frame number calculation unit 60K, a list image data generation unit 60L, and a transmission unit 60M.

[0063] The CPU 60 executes a list image data generation process to acquire image data to be edited, including a moving image file, and acquires multiple frames of first display image data. The CPU 60 also executes the list image data generation process to generate multiple frames of second display image data, determined according to the number of frames of the first display image data, from the image data to be edited. The CPU 60 also executes the list image data generation process to display multiple frames of display images indicated by the multiple frames of second display image data on the display 34. Here, the image data to be edited and the multiple frames of first display image data are image data having common attributes. The multiple frames of second display image data include multiple frames of second display image data for moving images corresponding to the multiple frames of still image data that constitute at least a portion of the moving image data. An example of the list image data generation process will be described in more detail below.

[0064] As an example, as shown in FIG. 9 , when a group of image data transmitted from the user device 12 is received by the communication I / F 52, the DB creator 60A acquires the group of image data received by the communication I / F 52. The DB creator 60A uses the acquired group of image data to create an image file DB 74 in the storage 62. In the image file DB 74, file constituent image data is associated with each file name indicating an image file. The file constituent image data refers to captured image data that constitutes an image file, i.e., captured image data included in an image file. In the example shown in FIG. 9 , file names with the extension "jpg," which indicates the JPEG file format, are still image files. In addition, in the example shown in FIG. 9 , file names with the extension "mpg," which indicates the MPEG file format, are moving image files. Note that the file format of still image files is not limited to JPEG and may be other file formats. In addition, the file format of moving image files is not limited to MPEG and may be other file formats.

[0065] As an example, as shown in FIG. 10, the image analysis unit 60B acquires an image file from the image file DB 74 and extracts captured image data from the acquired image file. The image analysis unit 60B then performs image analysis on the captured image data extracted from the image file to identify subjects appearing in the captured image data. Here, image analysis using a cascade classifier is performed on the captured image data. However, this is merely one example, and other image analysis methods such as pattern matching may also be used. Any method of image analysis may be used as long as it is capable of identifying subjects appearing in the captured image data.

[0066] For example, subjects that are captured in captured image data obtained by capturing images at an athletic meet and that are identified by image analysis by the image analysis unit 60B include, for example, a red and white cap, a rope, gloves, and gym clothes.

[0067] The image analysis unit 60B performs image analysis on the captured image data to output subject identification information that can identify the subject to the DB creation unit 60A.

[0068] As an example, as shown in FIG. 11 , a small event identification table 76 is stored in the storage 62. The small event identification table 76 is a table used to identify small events. Here, a small event refers to an event that is larger in scale than a large event. For example, if an athletic meet is considered a large event, the small events are the events that occur during the athletic meet and the elements that make the athletic meet possible. Examples of small events related to the athletic meet include sprints, ball throwing, and tug-of-war. For example, if skiing is considered a large event, the small events related to skiing include snowy mountains, slopes, skiers, and snowboarders. For example, if a concert is considered a large event, the small events related to the concert include the stage, audience seats, and conductor. For example, if swimming in the sea is considered a large event, the small events related to swimming in the sea include the sea, the beach, and parasols.

[0069] The DB creator 60A identifies minor events and derives major events based on the identified minor events. To identify the minor events, the DB creator 60A refers to a minor event identification table 76. In the minor event identification table 76, minor events are associated with reference subjects. A reference subject is an element that is predetermined as a necessary element that constitutes a minor event.

[0070] In the example shown in Fig. 11, a sprint, a ball toss, and a tug-of-war are shown as examples of small events related to an athletic meet. In the example shown in Fig. 11, a finish line tape and a track are shown as examples of elements that make up the small event of a sprint, i.e., examples of reference subjects. In addition, a red ball, a white ball, and a basket are shown as examples of elements that make up the small event of a ball toss, i.e., examples of reference subjects. Furthermore, a rope and gloves are shown as examples of elements that make up the small event of a tug-of-war, i.e., examples of reference subjects.

[0071] The DB creator 60A refers to the small event identification table 76 to identify a small event corresponding to the subject identified by the subject identification information input from the image analyzer 60B. That is, a small event associated with a reference subject that is the same as or similar to the subject identified by the subject identification information input from the image analyzer 60B is derived from the small event identification table 76. The processes shown in Figures 10 and 11 are performed on the captured image data included in all image files in the image file DB 74.

[0072] As an example, as shown in Fig. 12, the DB construction unit 60A identifies a major event based on the minor event identified by referring to the minor event identification table 76 (see Fig. 11). The storage 62 stores a major event identification table 78. In the major event identification table 78, major events are associated with minor events related to the major events (hereinafter also referred to as "related minor events").

[0073] In the example shown in FIG. 12, specific examples of major events include sports day, skiing, a concert, and swimming in the sea. In the example shown in FIG. 12, sprints, tug-of-war, and ball toss are shown as examples of related minor events associated with the major event of sports day. In addition, in the example shown in FIG. 12, snowy mountains and lifts are shown as examples of related minor events associated with the major event of skiing. In addition, in the example shown in FIG. 12, a stage and audience seats are shown as examples of related minor events associated with the major event of a concert. In addition, in the example shown in FIG. 12, the sea, a sandy beach, and parasols are shown as examples of related minor events associated with the major event of swimming in the sea.

[0074] The DB construction unit 60A identifies a major event corresponding to a minor event identified by the image analysis unit 60B performing image analysis on the image file, by referring to the major event identification table 78. That is, the major event associated with an associated minor event that is the same as or similar to the minor event derived from the minor event identification table 76 is derived from the major event identification table 78.

[0075] The DB creator 60A also acquires captured image data that has been identified as a major event from the image analyzer 60B and extracts date information from the acquired captured image data. The date information is extracted from the imaging condition information (see FIG. 6) of the related information included in the captured image data.

[0076] The DB creator 60A identifies major events and extracts date information for each captured image data item for all image files included in the image file DB 74. The DB creator 60A then classifies the image files by year and by major event, thereby creating an annual event image file DB 80.

[0077] 13, the annual event image file DB 80 classifies image files by year and by major event. The annual event image file DB 80 shown in Fig. 13 associates each image file with a file name, original image data, major event, minor event, date information, image capture location information, photographer information, model information, and image capture time information.

[0078] The file name, original image data, date information, imaging location information, photographer information, and model information are obtained from imaging condition information (see FIG. 6) associated with the captured image data. Although not shown, the original image data includes a captured image data identifier (see FIG. 6). The captured image data identifier is also obtained from imaging condition information associated with the captured image data. The imaging time information is information indicating the time required for capturing images to obtain a video file (hereinafter also referred to as "capture time"). The imaging time information is calculated, for example, from the number of frames of captured image data included in the video file and the default frame rate used for capturing images to obtain the video file. Note that this is merely an example, and if the video file has imaging time information attached to it, the imaging time information may be acquired from the video file. Attaching the imaging time information to a video file is realized, for example, by having the CPU 42 measure the time from the start to the end of capturing the video and attaching the measured time to the video file.

[0079] 14 and 15 show specific examples of the annual event image file DB 80. Fig. 14 shows, as an example of the annual event image file DB 80, a DB in which image files captured in 2018 are classified by year and by event (hereinafter also referred to as the "2018 event image file DB").

[0080] In the example shown in Fig. 14, the 2018 event image file DB is divided into 2018 1st to Nth event image file DBs, each corresponding to a major event, namely, the 1st to Nth events. In the example shown in Fig. 14, an athletic meet is shown as an example of the 1st event. In other words, the 2018 1st event image file DB contains image files captured at the 2018 athletic meet, a major event.

[0081] In the example shown in Fig. 15, the 2019 event image file DB is divided into 2019 1st to Nth event image file DBs, each corresponding to a major event, namely, the 1st to Nth events. In the example shown in Fig. 15, an athletic meet is also shown as an example of the 1st event. In other words, the 2019 1st event image file DB contains image files captured at the 2019 athletic meet, a major event.

[0082] As an example, as shown in FIG. 16, the acquisition unit 60C acquires still image files from the annual event image file DB 80. The reduced-size display image data generation unit 60E extracts original image data from the still image files acquired by the acquisition unit 60C and generates reduced-size display image data by reducing the extracted original image data. The reduced-size display image data generation unit 60E assigns a still image file identifier to the reduced-size display image data, which can identify the corresponding still image file. The reduced-size display image data generation unit 60E also associates the generated reduced-size display image data with the corresponding still image file and stores it in the annual event image file DB 80. As a result, the reduced-size display image data is classified by year and by major event, just like the image files.

[0083] The acquisition of still image files by the acquisition unit 60C, the generation of image data for reduced display by the image data for reduced display generation unit 60E, the assignment of still image file identifiers by the image data for reduced display generation unit 60E, the association of the image data for reduced display with the still image file by the image data for reduced display generation unit 60E, and the storage of the image data for reduced display in the annual event image file DB80 by the image data for reduced display generation unit 60E are performed for all still image files included in the annual event image file DB80.

[0084] The still image file identifier is used as a key for calling up a still image file corresponding to the reduced-size display image data. For example, when reduced-size display image data is provided from the server 14 to the user device 12 and then selected by the user 16, the server 14 retrieves the still image file identified by the still image file identifier from the annual event image file DB 80 and provides it to the user device 12.

[0085] 17, the image data selection unit 60F selects, as the image file group to be processed, a group of image files to which date information indicating the date of the earliest year has been assigned from the annual event image file DB 80. In the example shown in Fig. 17, the image file group for 2018 has been selected as the image file group to be processed.

[0086] As an example, as shown in FIG. 18, a year storage area is set in the memory 64. The year storage area is a storage area in which the year included in date information is stored. When the image data selection unit 60F selects a group of image files to be processed from the year event image file DB 80, it acquires date information from the group of image files to be processed and stores the year included in the date indicated by the acquired date information in the year storage area. In the example shown in FIG. 18, "2018" is stored in the year storage area.

[0087] The determination unit 60D determines whether or not a moving image file is included in the group of image files to be processed selected by the image data selection unit 60F. If the determination unit 60D determines that a moving image file is not included in the group of image files to be processed selected by the image data selection unit 60F, it adds one year to the year stored in the year storage area. If the determination unit 60D determines that a moving image file is included in the group of image files to be processed selected by the image data selection unit 60F, it instructs the acquisition unit 60C to acquire the moving image file.

[0088] 19, when acquisition of video files is instructed by determination unit 60D, acquisition unit 60C acquires all video files from the group of image files to be processed selected by image data selection unit 60F. Acquisition unit 60C outputs all video files acquired from the group of image files to be processed to reduced display image data generation unit 60E, recording time determination unit 60G, and frame number derivation unit 60H.

[0089] The recording time determination unit 60G determines the recording time of the video file input from the acquisition unit 60C. For example, the image capture time indicated by the image capture time information is used as the recording time.

[0090] The frame number derivation unit 60H derives the number of frames for reduced video display based on the recording time determined by the recording time determination unit 60G. The number of frames for reduced video display refers to the number of frames of the image data for reduced video file display that shows the image for reduced video file display described above. The storage 62 stores a frame number derivation table 82. In the frame number derivation table 82, the recording time is associated with the number of frames for reduced video display.

[0091] 19, the longer the recording time, the more frames there are for displaying a reduced video. Specifically, if the recording time is less than 1 second, the number of frames for displaying a reduced video is "1," if the recording time is between 1 second and 5 seconds, the number of frames for displaying a reduced video is "2," and if the recording time is between 5 seconds and 10 seconds, the number of frames for displaying a reduced video is "3." Note that this is merely an example, and the contents of the frame number derivation table 82 may be variable values ​​that are changed in accordance with instructions received by the receiving device 32 or 54. Furthermore, the contents of the frame number derivation table 82 may be variable values ​​that are changed in accordance with various conditions, such as the number of image files (e.g., video files) included in the image file group to be processed and / or the number of frames of captured image data included in the video files.

[0092] 20, the reduced-size display image data generating unit 60E extracts original image data from each of the multiple captured image data included in the moving image file input from the acquiring unit 60C. Then, the reduced-size display image data generating unit 60E generates reduced-size display image data by reducing the extracted original image data.

[0093] The reduced display image data generating unit 60E assigns a captured image data identifier to the reduced display image data, which can identify the corresponding captured image data. The reduced display image data generating unit 60E also associates the generated reduced display image data with the corresponding captured image data and stores the associated data in the annual event image file DB 80.

[0094] The captured image data identifier is used as a key to call up a video file corresponding to the reduced-size display image data. For example, when the reduced-size display image data is provided from the server 14 to the user device 12 and then selected by the user 16, the server 14 retrieves the video file identified by the captured image file identifier from the annual event image file DB 80 and provides it to the user device 12.

[0095] The determination unit 60D determines whether or not the processing by the reduced-size display image data generation unit 60E (hereinafter also referred to as the "moving image file reduction process") has been completed for all moving image files included in the group of image files to be processed. Here, the moving image file reduction process refers to the generation of reduced-size display image data, the assignment of moving image file identifiers, the association of the reduced-size display image data with the moving image files, and the storage of the reduced-size display image data in the annual event image file DB 80.

[0096] If the determination unit 60D determines that the video file reduction process has been completed for all video files included in the group of image files to be processed, it adds one year to the year in the year storage area. If the determination unit 60D determines that the processing by the reduced display image data generation unit 60E has not been completed for all video files included in the group of image files to be processed, it instructs the reduced display image data generation unit 60E to continue the video file reduction process. When the determination unit 60D instructs the reduced display image data generation unit 60E to continue the video file reduction process, the reduced display image data generation unit 60E continues the video file reduction process.

[0097] When the determination unit 60D determines that the video file reduction process has been completed for all video files included in the group of image files to be processed, the image data selection unit 60F acquires the year from the year storage area, as shown in Figure 21. Then, the image data selection unit 60F reselects the group of image files for the year acquired from the year storage area as the group of image files to be processed.

[0098] The acquisition unit 60C acquires one piece of event information from the group of image files to be processed selected by the image data selection unit 60F. The event information is information indicating a major event. The determination unit 60D determines whether or not a video file is included in the group of image files related to the major event (hereinafter also referred to as a "notable event") indicated by the event information acquired by the acquisition unit 60C.

[0099] If the determination unit 60D determines that the image file group related to the event of interest does not include a video file, it then determines whether all event information has been acquired by the acquisition unit 60C from the image file group to be processed. If the determination unit 60D determines that all event information has not been acquired by the acquisition unit 60C from the image file group to be processed, it instructs the acquisition unit 60C to continue acquiring event information. In response, the acquisition unit 60C continues acquiring event information from the image file group to be processed. If the determination unit 60D determines that all event information has been acquired by the acquisition unit 60C from the image file group to be processed, it adds one year to the year in the year storage area.

[0100] On the other hand, if the determination unit 60D determines that the image file group related to the featured event includes a video file, it then determines whether common event information is included in the reduced-size display image data for the previous year. Here, the reduced-size display image data for the previous year refers to reduced-size display image data associated with an image data group that has a date assigned one year earlier than the target image data group currently selected by the image data selection unit 60F. Furthermore, common event information refers to information indicating a major event that is the same as or similar to the featured event. The determination of whether common event information is included in the reduced-size display image data for the previous year is performed by determining whether the image data group for the previous year includes a major event that is the same as or similar to the featured event.

[0101] As an example, as shown in FIG. 22, if the determination unit 60D determines that the reduced display image data for the previous year does not contain common event information, it then determines whether all event information has been acquired by the acquisition unit 60C from the group of image files to be processed. If the determination unit 60D determines that all event information has not been acquired by the acquisition unit 60C from the group of image files to be processed, it instructs the acquisition unit 60C to continue acquiring event information. In response, the acquisition unit 60C continues acquiring event information from the group of image files to be processed. If the determination unit 60D determines that all event information has been acquired by the acquisition unit 60C from the group of image files to be processed, the list image data generation process proceeds to (A) shown in FIG. 25.

[0102] On the other hand, if the judgment unit 60D determines that the image data for reduced display for the previous year contains common event information, it instructs the frame number counting unit 60I to start counting the number of frames and instructs the file number counting unit 60J to start counting the number of files.

[0103] When instructed to start counting by determination unit 60D, frame number counting unit 60I counts the number of frames of image data for reduced display from the image file group for the previous year in the annual event image file DB 80, for the major event (hereinafter also referred to as "common event") indicated by the common event information. When instructed to start counting by determination unit 60D, file number counting unit 60J counts the number of image files in the image file group to be processed, for the common event.

[0104] 23, determination unit 60D determines whether the number of frames counted by frame number counting unit 60I is greater than the number of image files counted by file number counting unit 60J. If determination unit 60D determines that the number of frames counted by frame number counting unit 60I is equal to or less than the number of image files counted by file number counting unit 60J, it instructs reduced display image data generation unit 60E to generate reduced display image data. If determination unit 60D determines that the number of frames counted by frame number counting unit 60I is greater than the number of image files counted by file number counting unit 60J, it instructs missing frame number calculation unit 60K to calculate the number of missing frames.

[0105] When the determination unit 60D instructs the generation of reduced display image data, the reduced display image data generation unit 60E acquires one frame of captured image data from the moving image file acquired by the acquisition unit 60C, i.e., the moving image file currently being processed. Note that the group of image files to be processed, including the moving image file acquired by the acquisition unit 60C, is an example of "image data to be edited" according to the technology of the present disclosure.

[0106] The reduced display image data generating unit 60E extracts original image data from the acquired captured image data and reduces the extracted original image data to generate reduced display image data for one frame. The reduced display image data generating unit 60E also assigns a captured image data identifier to the reduced display image data, which can identify the corresponding captured image data. The reduced display image data generating unit 60E then associates the generated reduced display image data with the corresponding captured image data and stores it in the annual event image file DB 80. Thereafter, the list image data generating process proceeds to (A) shown in FIG.

[0107] The missing frame number calculation unit 60K calculates the number of missing frames, which is a value obtained by subtracting the number of image files counted by the file number counting unit 60J from the number of frames counted by the frame number counting unit 60I.

[0108] As an example, as shown in FIG. 24, the reduced-size display image data generation unit 60E acquires captured image data for a number of frames (hereinafter also referred to as the “adjusted number of frames”) obtained by adding one to the number of missing frames calculated by the missing frame number calculation unit 60K from the video file acquired by the acquisition unit 60C, i.e., the video file currently being processed. Here, if the number of adjustment frames is multiple, the reduced-size display image data generation unit 60E derives the captured image data for the number of adjustment frames, i.e., the similarity between the captured image data for multiple frames. The similarity is derived, for example, according to the results obtained by performing image recognition processing (e.g., image recognition processing using a cascade classifier and / or pattern matching) on ​​multiple pieces of original image data included in the captured image data for multiple frames. The reduced-size display image data generation unit 60E acquires captured image data for a number of frames (e.g., the number of missing frames plus one) whose derived similarity falls within a predetermined range from the video file.

[0109] The reduced display image data generating unit 60E extracts original image data from each of the captured image data for the adjusted number of frames and reduces the extracted original image data to generate reduced display image data for the adjusted number of frames. Here, the reduced display image data for the adjusted number of frames is an example of "second display image data for a moving image of multiple frames corresponding to still image data of multiple frames that constitute at least a part of the moving image data" according to the technology of the present disclosure.

[0110] The reduced display image data generating unit 60E also assigns a captured image data identifier to the reduced display image data, which can identify the corresponding captured image data.The reduced display image data generating unit 60E then associates the generated reduced display image data with the corresponding captured image data and stores them in the annual event image file DB 80.Then, the list image data generating process proceeds to (A) shown in FIG.

[0111] After the reduced-size display image data for the number of adjustment frames is generated and stored in the yearly event image file DB 80, as shown in FIG. 25 as an example, the determination unit 60D determines whether or not all event information has been acquired by the acquisition unit 60C from the image file group to be processed. If the determination unit 60D determines that the acquisition unit 60C has not acquired all event information from the image file group to be processed, the list image data generation process proceeds to (B) shown in FIG. 21, and the determination unit 60D instructs the acquisition unit 60C to continue acquiring event information. On the other hand, if the determination unit 60D determines that the acquisition unit 60C has acquired all event information from the image file group to be processed, it adds one year to the year in the year storage area. Then, the determination unit 60D determines whether or not the processes shown in FIGS. 21 to 24 have been performed on the image files in the yearly event image file DB 80.

[0112] 9 to 24 have not been executed for all image file groups in the yearly event image file DB 80, the list image data generation process proceeds to (C) shown in Fig. 21, and the determination unit 60D instructs the image data selection unit 60F to resume selection of image file groups to be processed. Here, resuming selection of image file groups to be processed refers to the process of reselecting, from the yearly event image file DB 80, image files that correspond to the year in the year storage area as image file groups to be processed.

[0113] On the other hand, if the determining unit 60D determines that the processes shown in FIGS. 9 to 24 have been executed for all image files in the annual event image file DB 80, it instructs the list image data generating unit 60L to generate list image data.

[0114] When the determination unit 60D instructs the list image data generation unit 60L to generate list image data, the list image data generation unit 60L acquires all reduced display image data from the annual event image file DB 80, classifying the data by year and by event. Then, the list image data generation unit 60L generates list image data (see FIG. 26) from all acquired reduced display image data by year and by major event. Here, the list image data generated by the list image data generation unit 60L is an example of "first display image data of multiple frames" and "second display image data of multiple frames" according to the technology of the present disclosure. Furthermore, in this embodiment, the "first display image data" according to the technology of the present disclosure is acquired by generating list image data by the list image data generation unit 60L.

[0115] In this embodiment, "yearly units" refers to units of one year. However, this is merely an example, and units of M years (M: a natural number equal to or greater than 2) may also be used. Yearly units and major events are examples of "common attributes" according to the technology of the present disclosure. Furthermore, major events are an example of "events" according to the technology of the present disclosure. Among multiple frames of reduced display image data related to the same major event, reduced display image data for year N is an example of "first display image data" according to the technology of the present disclosure, and reduced display image data for year N+1 is an example of "second display image data" according to the technology of the present disclosure.

[0116] The transmitting unit 60M transmits the list image data generated by the list image data generating unit 60L to the user device 12 that is the provider of the image data group via the communication I / F 52. The user device 12 that is the provider of the image data group refers to the user device that provided the image data group to the server 14 (for example, the user device 12 shown in FIG. 9).

[0117] As an example, as shown in FIG. 26, the CPU 60 executes the list image data generation process to acquire a group of 2019 athletic meet image files, and generates image data for reduced display with the above-mentioned adjusted number of frames (hereinafter also referred to as "image data group for displaying adjusted frames") from one moving image file included in the acquired group of 2019 athletic meet image files. In addition, the CPU 60 executes the list image data generation process to generate a group of 20 The reduced display image data included in the list image data corresponding to the 2018 athletic meet image file group is acquired.

[0118] Therefore, the number of frames of the reduced display image data included in the list image data corresponding to the 2018 Athletic Meet image file group is the same as the number of frames of the reduced display image data included in the list image data corresponding to the 2019 Athletic Meet image file group. Note that this is merely an example, and the technology of the present disclosure is not limited to this. As long as an adjustment frame display image data group is generated, the number of frames of the reduced display image data included in the list image data corresponding to the 2018 Athletic Meet image file group may be greater than the number of frames of the reduced display image data included in the list image data corresponding to the 2019 Athletic Meet image file group.

[0119] Although the image data group for displaying adjustment frames is generated from one video file here, the technology of the present disclosure is not limited to this, and the image data group for displaying adjustment frames may be generated from multiple video files. In this case, the number of adjustment frames is distributed and allocated to the multiple video files, and reduced display image data for the allocated number of frames is generated from each image file.

[0120] Furthermore, although an athletic meet is used as an example of a major event here, the technology of the present disclosure is not limited to this, and the number of frames of reduced display image data corresponding to a group of image files in one-year units is standardized between common major events held in one-year units. Furthermore, one-year units are merely an example, and the number of frames of reduced display image data is also standardized between common major events held in multiple-year units (for example, the opening ceremony of the Olympic Games, the closing ceremony of the Olympic Games, and / or specific Olympic sports).

[0121] In the example shown in FIG. 26, the list image data for the 2018 Athletic Meet image file group is an example of "multiple frames of first display image data" according to the technology of the present disclosure. In the example shown in FIG. 26, the 2019 Athletic Meet image file group is an example of "image data to be edited" according to the technology of the present disclosure. In the example shown in FIG. 26, the moving image file that is the basis of the adjustment frame display image data group is an example of "moving image data" according to the technology of the present disclosure. In the example shown in FIG. 26, the captured image data for multiple frames included in the moving image file that is the basis of the adjustment frame display image data group is an example of "multiple frames of still image data" according to the technology of the present disclosure.

[0122] 27, the display 34 displays a list of images indicated by the list of image data. The display 34 displays the images for reduced display of multiple frames in a manner that allows a distinction between the image group indicated by the image data group for displaying adjustment frames (hereinafter also referred to as "image group for displaying adjustment frames") and the images for reduced display other than the image group for displaying adjustment frames.

[0123] Specifically, the reduced display images of multiple frames included in the list image are displayed on a strip-shaped background extending along the time axis in a manner that allows a distinction to be made between the group of images for displaying adjustment frames that were generated from the same video file and the reduced display images of multiple frames other than the group of images for displaying adjustment frames. In the example shown in Fig. 27, the background (hatched area) of the group of images for displaying adjustment frames is given a color that is different from the background of the reduced display images other than the group of images for displaying adjustment frames, thereby making it possible to visually distinguish the group of images for displaying adjustment frames from the reduced display images other than the group of images for displaying adjustment frames.

[0124] For ease of explanation, hereinafter, one frame of image included in the group of images for displaying adjustment frames will be referred to as an "image for displaying adjustment frames," and data representing the image for displaying adjustment frames will be referred to as "image data for displaying adjustment frames."

[0125] Here, the adjustment frame display image data is an example of "second display image data for moving images" according to the technology of the present disclosure. The adjustment frame display image is an example of "display image based on second display image data for moving images" according to the technology of the present disclosure. The adjustment frame display image group is an example of "display image of multiple frames based on second display image data for moving images" according to the technology of the present disclosure. Among the multiple reduced display images included in the list image related to the 2019 Athletic Meet image file group, reduced display images other than the adjustment frame display image group are an example of "display image based on the second display image data other than second display image data for moving images" according to the technology of the present disclosure.

[0126] Note that, here, the image data for displaying the adjustment frame is generated from some of the original image data, not all of the original image data contained in one video file in the 2019 Athletic Meet image file group, but the technology of the present disclosure is not limited to this, and the image data for displaying the adjustment frame may be generated from each of the original image data contained in the video file. For example, if the recording time of the video file is less than one second, the image data for displaying the adjustment frame may be generated from each of the original image data contained in the video file.

[0127] Next, the operation of the information processing system 10 will be described.

[0128] 28A to 28D show an example of the flow of the list image data generation process. Note that the flow of the list image data generation process shown in Fig. 28A to 28D is an example of an "information processing method" according to the technique of the present disclosure.

[0129] 28A, first, in step ST10, the determination unit 60D determines whether or not the image file group transmitted from the user device 12 has been received by the communication I / F 52. If the image file group transmitted from the user device 12 has not been received by the communication I / F 52 in step ST10, the determination is negative, and the list image data generation process proceeds to step ST80 shown in Fig. 28D. If the image file group transmitted from the user device 12 has been received by the communication I / F 52 in step ST10, the determination is positive, and the list image data generation process proceeds to step ST12.

[0130] In step ST12, the DB creator 60A uses the image files received by the communication I / F 52 to create an image file DB 74 in the storage 62 (see FIG. 9), and then the list image data generation process proceeds to step ST14.

[0131] In step ST14, the image analysis unit 60B performs image analysis on the image files included in the image file DB 74 constructed in step ST12 (see FIGS. 10 and 11), and then the list image data generation process proceeds to step ST16.

[0132] In step ST16, the DB construction unit 60A constructs an annual event image file DB80 by classifying the image files included in the image file DB74 by year and by major event based on the results of the image analysis performed in step ST14 (see Figures 12 to 15), and then the list image data generation process proceeds to step ST18.

[0133] In step ST18, the acquisition unit 60C acquires unprocessed still image files from the annual event image file DB 80 constructed in step ST16 (see FIG. 16), and then the list image data generation process proceeds to step ST20. The processed still image file refers to a still image file for which the processing in steps ST20 and ST21 has not yet been performed.

[0134] In step ST20, the reduced display image data generating section 60E generates reduced display image data from the still image file acquired in step ST18 (see FIG. 16), and then the list image data generating process proceeds to step ST21.

[0135] In step ST21, the reduced display image data generation unit 60E associates the reduced display image data generated in step ST20 with the corresponding still image file and stores it in the annual event image file DB80 (see Figure 16), and then the list image data generation process proceeds to step ST22.

[0136] In step ST22, the determination unit 60D determines whether or not the processes of steps ST20 and ST21 have been performed for all still image files. If the processes of steps ST20 and ST21 have not been performed for all still image files in step ST22, the determination is negative, and the list image data generation process proceeds to step ST18. If the processes of steps ST20 and ST21 have been performed for all still image files in step ST22, the determination is positive, and the list image data generation process proceeds to step ST24 shown in FIG. 28B.

[0137] In step ST24 shown in Figure 28B, the image data selection unit 60F selects from the annual event image file DB80 a group of image files assigned dates from the earliest year as a group of image files to be processed (see Figure 17), and then the list image data generation process proceeds to step ST26.

[0138] In step ST26, the image data selection unit 60F stores the year of the date assigned to the group of image files to be processed selected in step ST24 in the year storage area (see Figure 18), and then the list image data generation process proceeds to step ST28.

[0139] In step ST28, the determination unit 60D determines whether or not a video file is included in the group of image files to be processed selected in step ST24 (see FIG. 18). In step ST28, if a video file is not included in the group of image files to be processed selected in step ST24, the determination is negative, and the list image data generation process proceeds to step ST42. In step ST28, if a video file is included in the group of image files to be processed selected in step ST24, the determination is positive, and the list image data generation process proceeds to step ST30.

[0140] In step ST30, the acquisition unit 60C acquires unprocessed video files from the group of image files to be processed selected in step ST24 (see FIG. 19), and then the list image data generation process proceeds to step ST32. In this step ST30, unprocessed video files refer to video files that have not yet been processed in steps ST32 to ST38.

[0141] In step ST32, the recording time specifying unit 60G specifies the recording time of the video file acquired in step ST30 (see FIG. 19), and then the list image data generating process proceeds to step ST34.

[0142] In step ST34, the frame number derivation unit 60H uses the frame number derivation table 82 to derive the number of frames for displaying a reduced video according to the recording time identified in step ST32 (see FIG. 19), and then the list image data generation process proceeds to step ST36.

[0143] In step ST36, the reduced display image data generating unit 60E generates reduced display image data for the number of frames for reduced display of the moving image derived in step ST34 (see FIG. 20), and then the list image data generating process proceeds to step ST38.

[0144] In step ST38, the reduced display image data generation unit 60E associates the reduced display image data generated in step ST36 with the corresponding captured image data and stores it in the annual event image file DB80 (see Figure 20), and then the list image data generation process proceeds to step ST40.

[0145] In step ST40, the determination unit 60D determines whether or not the processes of steps ST32 to ST38 have been performed for all video image files included in the group of image files to be processed selected in step ST24 (see FIG. 20). In step ST40, if the processes of steps ST32 to ST38 have not been performed for all video image files included in the group of image files to be processed selected in step ST24, the determination is negative, and the list image data generation process proceeds to step ST30. In step ST40, if the processes of steps ST32 to ST38 have been performed for all video image files included in the group of image files to be processed selected in step ST24, the determination is positive, and the list image data generation process proceeds to step ST42.

[0146] In step ST42, the image data selection unit 60F adds one year to the year in the year storage area (see FIG. 20), and then the list image data generation process proceeds to step ST44 shown in FIG. 28C.

[0147] In step ST44 shown in Figure 28C, the image data selection unit 60F selects a group of image data in the year memory area from the year event image file DB80 that has been assigned a year date as a group of image files to be processed (see Figure 21), and then the list image data generation process proceeds to step ST46.

[0148] In step ST46, the acquisition unit 60C acquires unprocessed event information from the group of image files to be processed selected in step ST44 (see FIG. 21), and then the list image data generation process proceeds to step ST48. In step ST46, unprocessed event information refers to event information that has not yet been used in the processes from step ST48 onwards.

[0149] In step ST48, the determination unit 60D determines whether or not a video file is included in the image files of the event of interest indicated by the event information acquired in step ST46, among the image files of the image files of the image files of the image file of interest selected in step ST44 (see FIG. 21). If a video file is not included in the image files of the event of interest indicated by the event information acquired in step ST46, among the image files of the image files of the image files of the image file of interest selected in step ST44, the determination is negative, and the list image data generation process proceeds to step ST68 shown in FIG. 28D. If a video file is included in the image files of the event of interest indicated by the event information acquired in step ST46, among the image files of the image files of the image files of the image file of interest selected in step ST44, the determination is positive, and the list image data generation process proceeds to step ST50.

[0150] In step ST50, the determination unit 60D determines whether or not common event information is included in the reduced display image data corresponding to the image data group for the previous year in the annual event image file DB 80 (see FIG. 21). If common event information is not included in the reduced display image data corresponding to the image data group for the previous year in the annual event image file DB 80 in step ST50, the determination is negative, and the list image data generation process proceeds to step ST68 shown in FIG. 28D. If common event information is included in the reduced display image data corresponding to the image data group for the previous year in the annual event image file DB 80 in step ST50, the determination is positive, and the list image data generation process proceeds to step ST52.

[0151] In step ST52, the acquisition unit 60C acquires unprocessed common event video files from the group of image data to be processed selected in step ST44 (see the acquisition unit 60C in FIGS. 23 and 24), and then the list image data generation process proceeds to step ST54. In this step ST52, the unprocessed common event video files refer to video files that are related to common event information included in the reduced display image data for the previous year from the group of image data to be processed selected in step ST44, and that have not yet been used in the processing of step ST62 or step ST64.

[0152] In step ST54, the frame number counting unit 60I counts the number of frames of the reduced display image data for the previous year for the common event (the major event indicated by the common event information) (see FIG. 22), and then the list image data generation process proceeds to step ST56. In this step ST54, the previous year refers to the year before the year in the year storage area.

[0153] In step ST56, the file number counting unit 60J counts the number of image files in the group of image data to be processed selected in step ST44 for the common event (the major event indicated by the common event information) (see Figure 22), and then the list image data generation process proceeds to step ST58.

[0154] In step ST58, it is determined whether the number of frames counted in step ST54 is greater than the number of image files counted in step ST56 (see FIG. 23). In step ST58, if the number of frames counted in step ST54 is equal to or less than the number of image files counted in step ST56, the determination is negative, and the list image data generation process proceeds to step ST64 shown in FIG. 28D. In step ST58, if the number of frames counted in step ST54 is greater than the number of image files counted in step ST56, the determination is positive, and the list image data generation process proceeds to step ST60.

[0155] In step ST60, the missing frame number calculation unit 60K calculates the number of missing frames using the number of frames counted in step ST54 and the number of image files counted in step ST56 (see Figure 23), and then the list image data generation process proceeds to step ST62.

[0156] In step ST62, the reduced display image data generation unit 60E generates reduced display image data for the adjusted number of frames obtained by adding 1 to the number of missing frames calculated in step ST60 from the common event moving image file acquired in step ST52 (see Figure 24), and then the list image data generation process proceeds to step ST68 shown in Figure 28D.

[0157] In step ST64 shown in FIG. 28D, the reduced display image data generation unit 60E generates one frame of reduced display image data from the common event moving image file acquired in step ST52 (see FIG. 23), and then the list image data generation process proceeds to step ST66.

[0158] In step ST66, the reduced display image data generation unit 60E associates the reduced display image data generated in step ST64 with the corresponding captured image data and stores it in the annual event image file DB80 (see Figure 23), and then the list image data generation process proceeds to step ST68.

[0159] In step ST68, the determination unit 60D determines whether or not the processing of step ST46 has been performed for all of the event information included in the group of image data to be processed selected in step ST44 (see FIG. 25). If the processing of step ST46 has not been performed for all of the event information included in the group of image data to be processed selected in step ST44 in step ST68, the determination is negative, and the list image data generation processing proceeds to step ST46 shown in FIG. 28C. If the processing of step ST46 has been performed for all of the event information included in the group of image data to be processed selected in step ST44 in step ST68, the determination is positive, and the list image data generation processing proceeds to step ST70.

[0160] In step ST70, the determination section 60D adds one year to the year in the year storage area (see FIG. 25), and then the list image data generation process proceeds to step ST72.

[0161] In step ST72, the determination unit 60D determines whether or not the processing of step ST44 has been performed for all image files in the yearly event image file DB 80 (see FIG. 25). If the processing of step ST44 has not been performed for all image files in the yearly event image file DB 80 in step ST72, the determination is negative, and the list image data generation processing proceeds to step ST44 shown in FIG. 28C. If the processing of step ST44 has been performed for all image files in the yearly event image file DB 80 in step ST72, the determination is positive, and the list image data generation processing proceeds to step ST74.

[0162] In step ST74, the list image data generation unit 60L generates list image data by classifying all of the image data for reduced display in the annual event image file DB80 by year and by major event (see Figures 25 and 26), and then the list image data generation process proceeds to step ST76.

[0163] In step ST76, the determination unit 60D determines whether or not the timing for transmitting the list image data (hereinafter also referred to as "transmission timing") has arrived. Examples of the transmission timing include the timing when an instruction to start transmitting the list image data is accepted by the accepting device 32 or 54, or the timing when a predetermined time (e.g., 10 seconds) has elapsed since the processing of step ST74 ended.

[0164] In step ST76, if the transmission timing has not arrived, the determination is negative and the determination of step ST76 is made again. In step ST76, if the transmission timing has arrived, the determination is positive and the list image data generation process proceeds to step ST78.

[0165] In step ST78, the transmission unit 60M transmits the list image data generated in step ST74 to the user device 12 via the communication I / F 52 (see FIG. 25), and then the list image data generation process proceeds to step ST80.

[0166] In step ST80, the determination unit 60D determines whether or not a condition for terminating the list image data generation process (hereinafter also referred to as a "list image data generation process termination condition") has been satisfied. One example of the list image data generation process termination condition is that an instruction to terminate the list image data generation process has been accepted by the acceptance device 32 or 54. In step ST80, if the list image data generation process termination condition is not satisfied, the determination is negative, and the list image data generation process proceeds to step ST10 shown in FIG. 28A. In step ST80, if the list image data generation process termination condition is satisfied, the determination is positive, and the list image data generation process ends.

[0167] As described above, in this embodiment, the server 14 generates, from the 2019 Athletic Meet image file group, image data for displaying adjusted frames, the number of adjustment frames determined according to the number of frames of the reduced display image data included in the list image data corresponding to the 2018 Athletic Meet image file group (see FIG. 26). The reduced display image data included in the list image data corresponding to the 2018 Athletic Meet image file group and the 2019 Athletic Meet image file group share the attributes of one year and athletic meet. The display 34 then displays images for displaying adjusted frames, the number of adjustment frames (see FIG. 27). Therefore, with this configuration, the list image can be displayed on the display 34 with a consistent number of display frames for each attribute of the reduced display image data.

[0168] Furthermore, in this embodiment, the number of frames of the adjustment frame display image is limited to be equal to or less than the number of frames of the reduced display image data included in the list image data corresponding to the 2018 Athletic Meet image file group (see FIG. 26). Therefore, with this configuration, compared to when there is no limit on the number of frames of the adjustment frame display image, it is possible to prevent the number of frames of the reduced display image data from exceeding the number of frames of the reduced display image data included in the list image data corresponding to the 2018 Athletic Meet image file group.

[0169] Furthermore, in this embodiment, the CPU 60 of the server 14 acquires a group of 2019 Athletic Meet image files and a plurality of frames of reduced display image data corresponding to a group of 2018 Athletic Meet image files as image data having common attributes (see FIG. 26). Then, list image data corresponding to the 2019 Athletic Meet image file group is generated from the group of 2019 Athletic Meet image files. The number of frames of the list image data corresponding to the 2019 Athletic Meet image file group is the same as the number of frames of the reduced display image data included in the list image data corresponding to the 2018 Athletic Meet image file group. Therefore, with this configuration, the number of frames of reduced display image data having common attributes can be unified.

[0170] In this embodiment, the common attribute of the multiple frames of reduced-size image data corresponding to the 2019 Athletic Meet image file group and the 2018 Athletic Meet image file group is the major event of the Athletic Meet. Therefore, with this configuration, the number of frames of reduced-size image data having the common attribute of the Athletic Meet can be unified.

[0171] The common attribute is not limited to the major event, and the date, subject, imaging location, photographer, and / or imaging device model may be used instead of or in addition to the major event. Also, the common attribute may be a minor event instead of a major event. Furthermore, although the above example uses a year as the common attribute, the common attribute may be a seasonal attribute (e.g., spring, summer, fall, winter), a monthly attribute, a weekly attribute, or a daily attribute.

[0172] In this embodiment, the attributes of the image file group (e.g., major events) are identified by performing image analysis on the image file group. Therefore, this configuration reduces the effort required to identify the attributes of the image file group compared to identifying the attributes of the image file group while visually checking the contents of the image file group.

[0173] In this embodiment, the captured image data for the number of adjustment frames acquired from the video file by the reduced-size display image data generation unit 60E is captured image data whose similarity falls within a predetermined range. The reduced-size display image data generation unit 60E generates reduced-size display image data for the number of adjustment frames from the captured image data for the number of adjustment frames. Therefore, this configuration can meet the needs of users who want to check subtle differences between the reduced-size display image data.

[0174] In this embodiment, the similarity is derived based on the results of image analysis, but the technology of the present disclosure is not limited to this, and the similarity may be derived by taking into account the time interval between frames of captured image data obtained from a video file, instead of or in addition to the results of image analysis.

[0175] Furthermore, the captured image data for the number of adjusted frames acquired from the video file by the reduced-size image data generating unit 60E may be captured image data whose similarity is outside the predetermined range. In this case, it is possible to meet the needs of users who do not need similar reduced-size image data.

[0176] Furthermore, in this embodiment, the display 34 displays the reduced-size display images of multiple frames in a manner that allows a distinction between the group of adjustment frame display images and the reduced-size display images other than the group of adjustment frame display images. Furthermore, the display 34 displays the reduced-size display images of multiple frames included in the list image in a manner that allows a distinction between the group of adjustment frame display images generated from the same video file and the reduced-size display images of multiple frames other than the group of adjustment frame display images, on a strip-shaped background extending along the time axis. Therefore, this configuration allows the user to visually distinguish between the group of adjustment frame display images and the reduced-size display images other than the group of adjustment frame display images.

[0177] In the above embodiment, an example was described in which multiple frames of thumbnail images included in a list image are displayed in a row along a time axis, but the technology of the present disclosure is not limited to this. For example, the display 34 may display multiple frames of thumbnail images included in a list image related to the 2019 Athletic Meet image file group in separate rows. In this case, if the time periods during which the images were captured to obtain each of the multiple frames of thumbnail images overlap, it is preferable to display the multiple frames of thumbnail images in separate rows.

[0178] Here, as a specific example, a case will be described in which the list image data for the 2019 athletic meet image file group includes multiple frames of reduced-size display image data (hereinafter also referred to as "multiple-frame time-zone overlapping display image data") obtained by capturing images during overlapping time periods. In this case, as shown in FIG. 29 as an example, the display 34 displays multiple frames of time-zone overlapping display images represented by the multiple frames of time-zone overlapping display image data in chronological order. The display 34 also displays the multiple frames of time-zone overlapping display images in a manner in which they are aligned in correspondence with each other at positions on the time axis indicating the time periods in which the multiple frames of time-zone overlapping display images overlap. In the example shown in FIG. 29, an upper section and a lower section are provided on the time axis, and the time-zone overlapping display images are aligned in correspondence with each other at positions indicating the time periods in which the multiple frames overlap. Therefore, with this configuration, it is possible to visually grasp the temporal relationship between the multiple frames of time-zone overlapping display images represented by the multiple frames of reduced-size display image data obtained by capturing images during overlapping time periods.

[0179] 29 shows an example in which the time period overlap display image is displayed in two stages, but this is merely one example, and the image may be displayed in three or more stages. This display method of displaying images in multiple stages is effective when the time periods in which multiple reduced display images obtained by each of multiple photographers were taken overlap.

[0180] Also, as shown in FIG. 30, a reduced display screen of a plurality of frames arranged along the time axis is Other reduced images may be displayed in a pop-up format between the images, in which case the other reduced images are displayed at positions on the time axis that correspond to the times at which the images were captured.

[0181] Furthermore, in the above embodiment, only an adjustment frame display image based on one frame of adjustment frame display image data generated from one frame of original image data is displayed in the display area of ​​one frame of adjustment frame display image, but the technology of the present disclosure is not limited to this. For example, multiple frames of adjustment frame display images represented by multiple frames of adjustment frame display image data generated from multiple frames of original image data (e.g., the "digest version reduced display image data" shown in FIG. 31) may be displayed in sequence in the display area of ​​one frame of adjustment frame display image. The digest version reduced display image data shown in FIG. 31 corresponds to multiple original image data constituting at least a portion of the video file from which the digest version reduced display image data is generated. In the example shown in FIG. 32, three frames of adjustment frame display images represented by three frames of adjustment frame display image data constituting the digest version reduced display image data are displayed in sequence in the display area of ​​one frame of adjustment frame display image. This configuration eliminates the need to display multiple frames of adjustment frame display images side by side, making it possible to grasp the overview of the video file even when display space is limited.

[0182] Furthermore, in the above embodiment, the number of frames of the adjustment frame display image data is derived based on the number of frames counted by the frame counting unit 60I. However, the technology of the present disclosure is not limited to this. For example, the number of frames of the adjustment frame display image data may be derived based on the interval between capture times within one block (hereinafter also referred to as a "display image data block") of the group of reduced display image data for the previous year for a common event (in the example shown in FIG. 26, all reduced display image data included in the list image data for the 2018 athletic meet image file group). For example, as shown in FIG. 33, the number of frames of the adjustment frame display image data may be derived based on the average value of the capture time intervals within a block of the group of reduced display image data for the previous year for a common event, in which the capture time interval between adjacent frames (hereinafter also referred to as the "capture time interval") is equal to or less than a predetermined interval (e.g., 1 second). Note that instead of the average value, a median or mode may be used. Note that if there are multiple display image data blocks, the average value, median, or mode of the capture time intervals between the reduced display image data included in all the display image data blocks may be calculated and used.

[0183] In this way, in order to derive the number of frames of the adjustment frame display image data in accordance with the interval between imaging times within the display image data block, the CPU 60 executes a list image data generation process, as shown in Fig. 34 as an example. The flowchart shown in Fig. 34 differs from the flowchart in Fig. 28C in that steps ST59A to ST59C are provided between step ST58 and step ST60.

[0184] In the list image data generating process shown in FIG. 34, the CPU 60 calculates the image capturing time interval for the reduced display image data for the previous year for the common event in step ST59A, and then the list image data generating process proceeds to step ST59B.

[0185] In step ST59B, the CPU 60 determines whether or not there is a block in the group of reduced-size display image data for the common event for the previous year, for which the image capture time interval calculated in step ST59A is equal to or shorter than a predetermined interval. Note that the predetermined interval may be a fixed value or a variable value that changes according to given conditions.

[0186] In step ST59B, if there is no block in the group of reduced display image data for the previous year for common events for which the imaging time interval calculated in step ST59A is equal to or shorter than the predetermined interval, the determination is negative, and the list image data generation process proceeds to step ST60.In step ST59B, if there is a block in the group of reduced display image data for the previous year for common events for which the imaging time interval calculated in step ST59A is equal to or shorter than the predetermined interval, the determination is positive, and the list image data generation process proceeds to step ST59C.

[0187] In step ST59C, the CPU 60 selects captured image data from the common event moving image file acquired in step ST52 according to the image capture time interval within the block, and then the list image data generation process proceeds to step ST59D. In this step ST59C, captured image data is selected from the common event moving image file at a time interval corresponding to the average image capture time interval within the block.

[0188] In step ST59D, the CPU 60 extracts original image data from the captured image data selected in step ST59C and generates image data for reduced display by reducing the extracted original image data, after which the list image data generation process proceeds to step ST68 shown in Figure 28D.

[0189] According to the configurations shown in Figures 33 and 34, even if the frame number counting unit 60I does not calculate the adjustment frame number based on the frame number, the frame number of the image data for adjustment frame display can be made closer to the frame number of the image data for reduced display for the previous year.

[0190] In the above embodiment, a group of image files for the 2019 athletic meet was exemplified, but the image data group for 2019 could also be a group of image data for major events other than athletic meets. Here, as an example, consider a case where the image data group for 2019 includes a first moving image file having a first attribute and a second moving image file having a second attribute. Note that the first moving image file is an example of "first moving image data" according to the technology of the present disclosure, and the second moving image file is an example of "second moving image data" according to the technology of the present disclosure. The first attribute is, for example, a major event such as an athletic meet, and the second attribute is, for example, a major event such as a concert.

[0191] 35, the CPU 60 executes a frame number suppression generation process. A frame number suppression generation process program 84 is stored in the storage 62. The frame number suppression generation process is realized by the CPU 60 reading the frame number suppression generation process program 84 from the storage 62 and executing the frame number suppression generation process program 84 on the memory 64.

[0192] By executing a frame number suppression generation process, when the first frame number of image data for displaying an adjustment frame corresponding to a first moving image file differs from the second frame number of image data for displaying an adjustment frame corresponding to a second moving image file, the CPU 60 generates image data for displaying an adjustment frame from the moving image file with a frame number determined according to the smaller of the first and second frame numbers.

[0193] Figures 36A and 36B show an example of the flow of the frame number suppression generation process. The frame number suppression generation process shown in Figures 36A and 36B is executed by the CPU 60, for example, on the condition that the processing of step ST74 shown in Figure 28D has been completed. Note that, for convenience of explanation, the following description will be given on the assumption that the image file groups for 2018 and 2019 are being processed.

[0194] In the frame number suppression generation process shown in FIG. 36A, first, in step ST100, the CPU 60 calculates the data amount of the image file group for 2018, and then the frame number suppression generation process proceeds to step ST102.

[0195] In step ST102, the CPU 60 calculates the data amount of the moving image files in the image file group for 2018, and then the frame number suppression generation process proceeds to step ST104.

[0196] In step ST104, the CPU 60 calculates the 2018 video data ratio, and then the frame number suppression generation process proceeds to step ST106. The 2018 video data ratio is the ratio of the data amount calculated in step ST102 to the data amount calculated in step ST100.

[0197] In step ST106, the CPU 60 calculates the data amount of the image file group for 2019, and then the frame number suppression generation process proceeds to step ST108.

[0198] In step ST108, the CPU 60 calculates the data amount of the video image files in the image file group for 2019, and then the frame number suppression generation process proceeds to step ST110.

[0199] In step ST110, the CPU 60 calculates the 2019 video data ratio, and then the frame number suppression generation process proceeds to step ST112. The 2019 video data ratio is the ratio of the data amount calculated in step ST108 to the data amount calculated in step ST106.

[0200] In step ST112, the CPU 60 calculates the rate of increase in the video data ratio, and then the frame number suppression generation process proceeds to step ST114. Note that the rate of increase is an example of the "degree of change in ratio over time" according to the technique of the present disclosure.

[0201] In step ST114, it is determined whether the increase rate calculated in step ST112 is greater than a default increase rate (e.g., 30%). The default increase rate is an example of a "default degree" according to the technology of the present disclosure. The default increase rate may be a fixed value or a variable value that changes according to given conditions. In step ST114, if the increase rate calculated in step ST112 is equal to or less than the default increase rate, the determination is negative, and the frame number suppression and generation process proceeds to step ST100. In step ST114, if the increase rate calculated in step ST112 is greater than the default increase rate, the determination is positive, and the frame number suppression and generation process proceeds to step ST116 shown in FIG. 36B.

[0202] 36B, the CPU 60 calculates the number of frames of reduced display image data corresponding to the moving image file for each attribute. Specifically, the number of frames of reduced display image data corresponding to the first moving image file and the number of frames of reduced display image data corresponding to the second moving image file are calculated. Note that in this step ST116, the number of frames of reduced display image data is calculated on the premise that one frame of reduced display image data is generated for one frame of captured image data.

[0203] In the next step ST118, the CPU 60 determines whether the number of frames differs between the attributes. That is, the CPU 60 determines whether the number of frames of the reduced-size display image data corresponding to the first moving image file differs from the number of frames of the reduced-size display image data corresponding to the second moving image file. If the number of frames is the same between the attributes in step ST118, the determination is negative, and the frame number suppression generation process ends. If the number of frames differs between the attributes in step ST118, the determination is positive, and the frame number suppression generation process proceeds to step ST120.

[0204] In step ST120, the CPU 60 sets the number of frames of the reduced-size display image data corresponding to the moving image file, i.e., the number of frames of the adjustment frame display image data included in the list image data generated in step ST74 shown in Fig. 28D, to a minimum number of frames. Here, the minimum number of frames refers to the smaller of the number of frames of the reduced-size display image data corresponding to the first moving image file and the number of frames of the reduced-size display image data corresponding to the second moving image file. After the processing of step ST120 is executed, the frame number suppression generation process ends.

[0205] When the frame number suppression generation process is executed in this manner, image data for displaying adjustment frames is generated from the video file with a frame number determined according to the smaller of the first frame number and the second frame number, thereby preventing the number of frames in the image data for displaying adjustment frames from increasing excessively.

[0206] Furthermore, when the frame number suppression generation process is executed, the number of frames of the image data for displaying the adjustment frame is set to the minimum number of frames, provided that the growth rate exceeds the default growth rate. Therefore, compared to when the number of frames of the image data for displaying the adjustment frame is set to the minimum number of frames regardless of the growth rate, excessive restriction on the number of frames of the image data for displaying the adjustment frame can be suppressed. Note that, while the growth rate across years from 2018 to 2019 is exemplified here, the technology of the present disclosure is not limited to this, and growth rates across seasons (e.g., spring, summer, fall, winter), months, weeks, or days may also be calculated.

[0207] Furthermore, the number of frames of the image data for displaying an adjustment frame may be determined based on the smaller of the first and second frame numbers (e.g., the minimum number of frames) and the rate at which the growth rate exceeds a predetermined growth rate (e.g., the 2018 video data rate). In this case, for example, the CPU 60 derives the number of frames of the image data for displaying an adjustment frame using a frame number derivation table 86 shown in FIG. 37. In the frame number derivation table 86 shown in FIG. 37, the number of frames of the image data for displaying an adjustment frame is associated with the minimum number of frames and the 2018 video data rate. In this way, by adopting a frame number determined based on the minimum number of frames and the 2018 video data rate as the number of frames of the image data for displaying an adjustment frame, it is possible to prevent the number of frames of the image data for displaying an adjustment frame from being excessive or insufficient compared to when the number of frames of the image data for displaying an adjustment frame is determined based solely on the minimum number of frames or the 2018 video data rate.

[0208] In addition, instead of using the frame number derivation table 86, the CPU 60 may calculate the number of frames of the image data for displaying the adjustment frame using an arithmetic formula in which the minimum number of frames and the 2018 video data ratio are independent variables and the number of frames of the image data for displaying the adjustment frame is the dependent variable.

[0209] Furthermore, in the above embodiment, an example was described in which the number of adjustment frames was calculated by adding 1 to the number of missing frames, but the technology of the present disclosure is not limited to this. For example, the number of frames of the adjustment frame display image data may be derived using a frame number derivation table 88 shown in FIG. 38. In the frame number derivation table 88 shown in FIG. 38, the number of frames of the adjustment frame display image data corresponds to the number of missing frames and the imaging time interval. Therefore, the CPU 60 can calculate the number of missing frames and the imaging time interval, and derive the number of frames of the adjustment frame display image data from the frame number derivation table 88 in accordance with the calculated number of missing frames and the imaging time interval. Furthermore, the CPU 60 may calculate the number of frames of the adjustment frame display image data using an arithmetic expression in which the number of missing frames and the imaging time interval are independent variables and the number of frames of the adjustment frame display image data is a dependent variable.

[0210] Alternatively, a frame count determined according to the recording time of the video file may be used as the frame count of the image data for displaying an adjustment frame. In this case, for example, the CPU 60 may derive the frame count of the image data for displaying an adjustment frame using a table that is an application of the frame count derivation table 82 shown in FIG. 19. A table that is an application of the frame number derivation table 82 refers to a table that uses the frame count of the image data for displaying an adjustment frame instead of the frame count for displaying a reduced video in the frame number derivation table 82. By using a frame count determined according to the recording time of the video file as the frame count of the image data for displaying an adjustment frame in this way, it is possible to prevent the number of frames of the image data for displaying an adjustment frame from being excessive or insufficient compared to when the frame count of the image data for displaying an adjustment frame is determined without any consideration of the recording time of the video file.

[0211] Furthermore, the maximum number of frames of the image data for displaying an adjustment frame may be limited according to the recording time of the moving image file, which prevents the number of frames of the image data for displaying an adjustment frame from increasing excessively compared to when the number of frames of the image data for displaying an adjustment frame is determined without taking the recording time of the moving image file into consideration.

[0212] Furthermore, in the above embodiment, an example in which the list image data generation process is executed by the server 14 has been described, but the technology of the present disclosure is not limited to this. For example, the list image data generation process may be executed in a distributed manner by multiple devices. For example, a storage server that stores at least one of the image file DB 74 shown in FIG. 9 and the annual event image file DB 80 shown in FIG. 17, an image analysis server that performs image analysis on the image file DB 74 and the annual event image file DB 80, and a processing execution server that performs processing other than image analysis may be provided. Furthermore, the list image data generation process may be performed by a personal computer and / or a user device, etc., instead of the server 14.

[0213] Furthermore, in the above embodiment, an example was described in which the list image data generation program 72 and the frame number suppression generation processing program 84 (hereinafter, when there is no need to distinguish between these programs, they will be referred to as the "server-side program" without reference numerals) are stored in the storage 62, but the technology of the present disclosure is not limited to this. For example, as shown in FIG. 39, the server-side program may be stored in a storage medium 100. The storage medium 100 is a non-transitory storage medium. An example of the storage medium 100 is any portable storage medium such as an SSD or a USB memory. The server-side program is an example of a "program" according to the technology of the present disclosure.

[0214] The server-side program stored in the storage medium 100 is installed on the computer 50. The CPU 60 executes the list image data generation process in accordance with the list image data generation program 72, and executes the frame number suppression generation process in accordance with the frame number suppression generation process program 84. For ease of explanation, the list image data generation process and the frame number suppression generation process will be referred to as the "server-side process" hereinafter unless there is a need to distinguish between them.

[0215] In addition, the server-side program may be stored in a memory unit of another computer or server device connected to computer 50 via a communication network (not shown), and the server-side program may be downloaded and installed on computer 50 in response to a request from server 14.

[0216] It is not necessary to store all of the server-side programs in the storage unit of another computer or server connected to the computer 50, or in the storage 62; only part of the server-side programs may be stored therein.

[0217] 39, the CPU 60 is a single CPU, but it may be a plurality of CPUs. Also, a GPU may be applied instead of or together with the CPU 60.

[0218] 39 illustrates a computer 50, the technology of the present disclosure is not limited to this, and a device including an ASIC, an FPGA, and / or a PLD may be applied instead of the computer 50. Furthermore, a combination of a hardware configuration and a software configuration may be used instead of the computer 50.

[0219] The hardware resources for executing the server-side processing described in the above embodiments can be various processors, as listed below. Examples of processors include a CPU, which is a general-purpose processor that functions as a hardware resource for executing server-side processing by executing software, i.e., a program. Examples of processors include dedicated electrical circuits, such as FPGAs, PLDs, or ASICs, which are processors with a circuit configuration specifically designed to execute specific processing. Each processor has built-in or connected memory, and each processor uses the memory to execute server-side processing.

[0220] The hardware resource that executes the server-side processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the server-side processing may be a single processor.

[0221] As an example of a system configured with a single processor, there is a first configuration in which one processor is configured with a combination of one or more CPUs and software, and this processor functions as a hardware resource that executes server-side processing. A second configuration is one in which a processor is used that realizes the functions of the entire system, including multiple hardware resources that execute server-side processing, on a single IC chip, as typified by SoCs. In this way, server-side processing is realized using one or more of the above-mentioned various processors as hardware resources.

[0222] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The above server-side processing is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[0223] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0224] In this specification, "A and / or B" is the same as "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0225] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

Claims

1. a processor; The processor: Acquire a first group of image files and a second group of image files; When the number of frames of the first display image data for the first image file group is greater than the number of image files in the second image file group, second display image data is generated from the N moving image files included in the second image file group, with the number of frames being the number of missing frames obtained by subtracting the number of image files from the number of frames and adding N to the number of missing frames. Information processing device.

2. The first image file group and the second image file group are both image file groups related to a common event. The information processing device according to claim 1 .

3. The second image file group includes images obtained by capturing images a certain period after capturing images for obtaining images belonging to the first image file group. The information processing device according to claim 2 .

4. N is 1 The information processing device according to claim 1 .

5. An image of the first display image data; The image of the second display image data and the image of the third display image data corresponding to the file included in the second image file group are displayed in parallel. The information processing device according to claim 1 .

6. The image of the third display image data is displayed so as to be distinguishable from other images. The information processing device according to claim 5 .

7. Acquiring a first group of image files and a second group of image files; and and when the number of frames of the first display image data for the first image file group is greater than the number of image files for the second image file group, generating second display image data for a number of frames obtained by adding N to a number of missing frames obtained by subtracting the number of image files from the number of frames from N moving image files included in the second image file group. Information processing methods.

8. Acquiring a first group of image files and a second group of image files; and A program for causing a computer to execute a process that includes generating, when the number of frames of first display image data for the first image file group is greater than the number of image files in the second image file group, second display image data with a number of frames equal to the number of missing frames obtained by subtracting the number of image files from the number of frames and adding N to that number.

Images