Information processing apparatus, information processing method, and computer program product

Abstract

The present application provides an information processing apparatus, an information processing method, and a computer program product. The information processing apparatus creates a dated image data list by classifying a plurality of dated image data, associates the dated image data list with a specific user, acquires dated image data of a subject similar to a subject of undated image data from the dated image data list, and derives a date to be assigned to the undated image data based on a date assigned to the acquired dated image data. The plurality of dated image data is image data of a plurality of users including the specific user. The dated image data list is created by classifying the plurality of dated image data by each subject. The dated image data list of a subject similar to a subject of the dated image data is associated with the specific user.

Description

[0001] This application is a divisional application of Chinese invention patent application No. 202080098979.4 (PCT / JP2020 / 040103), filed on October 26, 2020, entitled "Information Processing Apparatus, Information Processing Method and Program". Technical Field

[0002] The present invention relates to an information processing device, an information processing method, and a program. Background Technology

[0003] Japanese Patent Application Publication No. 2006-252025 discloses an image management device comprising an extraction mechanism and an estimation mechanism. The extraction mechanism extracts features of an image from image data whose photographing date and time are unknown. The estimation mechanism estimates the photographing date and time of the image data by comparing the extracted features with a time lexicon containing objects used to determine dates and times.

[0004] Furthermore, the time lexicon records the relationship between objects representing faces, hair, body shapes, clothing, and other parts of the subject and dates. The objects are at least one of text data, image data, and video data describing characteristic quantities, or at least one of text data, image data, and video data describing characteristic quantities representing a specific year or season. Moreover, the image management device described in Japanese Patent Application Publication No. 2006-252025 also includes an updating mechanism for updating the time lexicon based on the estimation results of an estimation mechanism. Summary of the Invention

[0005] However, when the variety of objects included in the time lexicon is insufficient, it is difficult to estimate the shooting date and time of image data with unknown shooting dates and times with high accuracy.

[0006] One embodiment of the present invention provides an information processing apparatus, information processing method, and program that can assign appropriate dates to image data without dates, compared to the case where dates are derived solely from date-bearing image data possessed by a specific user.

[0007] means for solving technical problems

[0008] The first aspect of the present invention is an information processing apparatus comprising a processor and memory built into or connected thereto. The processor performs the following processing: creating a list of dated image data by classifying multiple dated image data with assigned dates; associating the list of dated image data with a specific user; obtaining dated image data about subjects similar to those represented by the subject without dated image data of the specific user from the list of dated image data associated with the specific user; and deriving dates assigned to the subject without dated image data based on the dates assigned to the obtained dated image data. The multiple dated image data are image data of multiple users, including the specific user. The multiple dated image data are image data of multiple users, and a list of dated image data is created for each subject by classifying the multiple dated image data according to each subject represented by each of the multiple dated image data. The list of dated image data about subjects similar to those represented by the subject with dated image data of the specific user is associated with the specific user. Attached Figure Description

[0009] Figure 1 This is a conceptual diagram showing the general structure of the information processing system involved in the first and second embodiments.

[0010] Figure 2 This is a block diagram illustrating an example of the hardware structure of the electrical system of the user equipment included in the information processing system according to the first and second embodiments.

[0011] Figure 3 This is a block diagram illustrating an example of the hardware structure of the electrical system of the server included in the information processing system according to the first and second embodiments.

[0012] Figure 4 This is a block diagram illustrating an example of the main functions of the CPU when the CPU of a user device included in the information processing system according to the first embodiment performs date image data creation processing.

[0013] Figure 5 This is a block diagram illustrating an example of the processing content of the CPU of a user device included in the information processing system according to the first embodiment when it operates as a camera control unit, an image data acquisition unit, a GPS information calculation unit, an attribute data creation unit, and a date-based image data creation unit.

[0014] Figure 6 This is a block diagram illustrating an example of the processing content of the CPU of a user device included in the information processing system according to the first embodiment when it operates as a date image data creation unit.

[0015] Figure 7This is a conceptual diagram illustrating an example of a state in which multiple user devices included in the information processing system according to the first embodiment store multiple date image data in their respective memories.

[0016] Figure 8 This is a conceptual diagram illustrating an example of how multiple user devices included in the information processing system according to the first embodiment upload image data sets to a server.

[0017] Figure 9 This is a block diagram illustrating an example of how the CPU of a user device included in the information processing system according to the first embodiment performs the main functions of the CPU when processing a request.

[0018] Figure 10 This is a block diagram illustrating an example of the processing content of the CPU of a user device included in the information processing system according to the first embodiment when the request data creation unit, the request data sending unit, and the display control unit are operating.

[0019] Figure 11 This is a block diagram illustrating an example of the main functions of the CPU when the CPU of the server included in the information processing system according to the first embodiment performs processing to create a list of date image data.

[0020] Figure 12 This is a block diagram illustrating an example of the processing content when the CPU of the server included in the information processing system according to the first embodiment operates as a date image data acquisition unit and a user ID extraction unit.

[0021] Figure 13 This is a block diagram illustrating an example of the processing content of the CPU of the server included in the information processing system according to the first embodiment when it operates as a user ID extraction unit, a determination unit, an image data group acquisition unit, a storage control unit, and a date-bearing image data acquisition unit.

[0022] Figure 14 This is a block diagram illustrating an example of the processing content of the CPU of the server included in the information processing system according to the first embodiment when it operates as an image data acquisition unit, a determination unit, and a person image data extraction unit.

[0023] Figure 15 This is a block diagram illustrating an example of the processing content when the CPU of the server included in the information processing system according to the first embodiment operates as a determination unit and a deletion unit.

[0024] Figure 16This is a block diagram illustrating an example of the processing content when the CPU of the server included in the information processing system according to the first embodiment operates as a determination unit and a deletion unit.

[0025] Figure 17 This is a block diagram illustrating an example of the processing content of the CPU of the server included in the information processing system according to the first embodiment when it operates as an image data acquisition unit, a determination unit, and a GPS information extraction unit.

[0026] Figure 18 This is a block diagram illustrating an example of the processing content of the CPU of the server included in the information processing system according to the first embodiment when it operates as a GPS information extraction unit and a distribution area map creation unit.

[0027] Figure 19 This is a block diagram illustrating an example of the processing content when the CPU of the server included in the information processing system according to the first embodiment operates as a distribution area map creation unit, a repeating area ratio calculation unit, and a determination unit.

[0028] Figure 20 This is a block diagram illustrating an example of the processing content when the CPU of the server included in the information processing system according to the first embodiment operates as a determination unit and a deletion unit.

[0029] Figure 21 This is a block diagram illustrating an example of the processing content when the CPU of the server included in the information processing system according to the first embodiment operates as a distribution area map creation unit, a determination unit, and a deletion unit.

[0030] Figure 22 This is a block diagram illustrating an example of the processing content when the CPU of the server included in the information processing system according to the first embodiment operates as a user ID extraction unit and a user information acquisition unit.

[0031] Figure 23 This is a block diagram illustrating an example of the processing content when the CPU of the server included in the information processing system according to the first embodiment operates as a user information acquisition unit, a user information consistency calculation unit, a determination unit, and a deletion unit.

[0032] Figure 24 This is a block diagram illustrating an example of the processing content of the CPU of the server included in the information processing system according to the first embodiment when it operates as a date image data acquisition unit and a non-person image data extraction unit.

[0033] Figure 25This is a block diagram illustrating an example of the processing content of the CPU of the server included in the information processing system according to the first embodiment when it operates as an image data list creation unit.

[0034] Figure 26 This is a conceptual diagram illustrating an example of image data lists of the first and second subjects created by the CPU of a server included in the information processing system according to the first embodiment, which acts as an image data list creation unit.

[0035] Figure 27 This is a conceptual diagram illustrating an example of image data lists of the third and fourth subjects created by the CPU of the server included in the information processing system according to the first embodiment, which operates as an image data list creation unit.

[0036] Figure 28 This is a conceptual diagram illustrating an example of image data lists of subjects from the 5th to the Nth, created by the CPU of a server included in the information processing system according to the first embodiment, acting as an image data list creation unit.

[0037] Figure 29 This is a block diagram illustrating an example of the processing content of the CPU of the server included in the information processing system according to the first embodiment when it operates as an image data list creation unit and an image data list classification unit.

[0038] Figure 30 This is a conceptual diagram illustrating an example of a state in which a first user establishes an association with a dated image data list by the CPU of a server included in the information processing system according to the first embodiment performing dated image data list creation processing.

[0039] Figure 31 This is a conceptual diagram illustrating an example of a state in which a second user establishes an association with a dated image data list by means of a CPU of a server included in the information processing system according to the first embodiment performing dated image data list creation processing.

[0040] Figure 32 This is a conceptual diagram illustrating an example of a state in which a third user establishes an association with a dated image data list by means of a CPU of a server included in the information processing system according to the first embodiment performing dated image data list creation processing.

[0041] Figure 33 This is a conceptual diagram illustrating an example of a state in which a fourth user is associated with a dated image data list by the CPU of a server included in the information processing system according to the first embodiment performing dated image data list creation processing.

[0042] Figure 34 This is a block diagram illustrating an example of the CPU execution date assigned to the main functions of the CPU during processing in the information processing system according to the first embodiment.

[0043] Figure 35 This is a block diagram illustrating an example of the processing content when the CPU of the server included in the information processing system according to the first embodiment operates as a user ID extraction unit and an image data list acquisition unit.

[0044] Figure 36 This is a block diagram illustrating an example of the processing content when the CPU of the server included in the information processing system according to the first embodiment operates as a user ID extraction unit and an image data list acquisition unit.

[0045] Figure 37 This is a block diagram illustrating an example of the processing content when the CPU of the server included in the information processing system according to the first embodiment operates as an image data list acquisition unit, a dateless image data extraction unit, and a determination unit.

[0046] Figure 38 This is a block diagram illustrating an example of the processing content when the CPU of the server included in the information processing system according to the first embodiment operates as a determination unit, a date export unit, a dateless image data extraction unit, a date assignment unit, and an image data transmission unit.

[0047] Figure 39A This is a flowchart illustrating an example of the process for creating a dated image data list according to the first embodiment.

[0048] Figure 39B yes Figure 39A The following is a continuation of the flowchart shown.

[0049] Figure 39C yes Figure 39A and Figure 39B The following is a continuation of the flowchart shown.

[0050] Figure 39D yes Figure 39C The following is a continuation of the flowchart shown.

[0051] Figure 39E yes Figure 39C The following is a continuation of the flowchart shown.

[0052] Figure 40 This is a flowchart illustrating an example of the date assignment process involved in the first embodiment.

[0053] Figure 41This is a flowchart illustrating an example of the date assignment request processing flow according to the first embodiment.

[0054] Figure 42 This is a block diagram illustrating an example of the main functions of the CPU when the CPU of the server included in the information processing system according to the second embodiment performs date image data list update processing.

[0055] Figure 43 This is a block diagram illustrating an example of the processing content of the CPU of the server included in the information processing system according to the second embodiment when it operates as a date assignment unit, a date export unit, and a date image data generation unit.

[0056] Figure 44 This is a block diagram illustrating an example of the processing content when the CPU of the server included in the information processing system according to the second embodiment operates as a date image data generation unit, a date export unit, a similarity calculation unit, a determination unit, and an image quality determination unit.

[0057] Figure 45 This is a block diagram illustrating an example of the processing content of the CPU of the server included in the information processing system according to the second embodiment when it operates as an image quality determination unit, a judgment unit, an image data appending unit, and a date-bearing image data generation unit.

[0058] Figure 46 This is a flowchart illustrating an example of the process for updating a list of dated image data according to the second embodiment.

[0059] Figure 47A This is a flowchart illustrating an example of the date assignment process involved in the second embodiment.

[0060] Figure 47B yes Figure 47A The following is a continuation of the flowchart shown.

[0061] Figure 48 This is a block diagram representing a variation of the image data list production department.

[0062] Figure 49A This is a flowchart representing a variation of the date assignment process.

[0063] Figure 49B yes Figure 49A The following is a continuation of the flowchart shown.

[0064] Figure 50 This is a conceptual diagram representing an example of assigning priority to multiple lists of dated image data.

[0065] Figure 51This is a flowchart illustrating a variation of the process for creating and processing a list of image data with dates.

[0066] Figure 52 This is a conceptual diagram representing a variation of the attribute data contained in date-bearing image data.

[0067] Figure 53 yes Figure 39E A variation of the flowchart shown.

[0068] Figure 54 This is a block diagram illustrating an example of a method in which a date image data creation program and a date assignment request handler are installed in a computer within a user device from a storage medium containing the date image data creation program and the date assignment request handler.

[0069] Figure 55 This is a block diagram illustrating an example of a method in which a program for creating a date image data list, a program for assigning dates, and a program for updating a date image data list are installed on a computer within a server, using a storage medium containing the program for creating a date image data list, a program for assigning dates, and a program for updating a date image data list. Detailed Implementation

[0070] An example of an embodiment of the information processing apparatus, information processing method, and program related to the technology of the present invention will be described with reference to the accompanying drawings.

[0071] First, let me explain the words and phrases used in the following instructions.

[0072] CPU stands for Central Processing Unit. RAM stands for Random Access Memory. SSD stands for Solid State Drive. HDD stands for Hard Disk Drive. EEPROM stands for Electrically Erasable and Programmable Read Only Memory. ASIC stands for Application Specific Integrated Circuit. PLD stands for Programmable Logic Device. FPGA stands for Field-Programmable Gate Array. SoC stands for System-on-a-chip. CMOS stands for Complementary Metal Oxide Semiconductor. CCD stands for Charge Coupled Device. EL stands for Electro-Luminescence. UI stands for User Interface. USB stands for Universal Serial Bus. GPU stands for Graphics Processing Unit. GPS stands for Global Positioning System. RTC stands for Real Time Clock. ID stands for Identification. Exif stands for Exchangeable Image File Format. WAN stands for Wide Area Network. LAN stands for Local Area Network.

[0073] Furthermore, in this specification, "identical" means not only completely identical, but also includes the meaning of errors generally permissible in the technical field to which the present invention pertains (including errors that do not depart from the spirit of the present invention). Also, in this specification, "identical" in "same date" means not only completely identical, but also includes the meaning of errors generally permissible in the technical field to which the present invention pertains (including errors that do not depart from the spirit of the present invention).

[0074] [First Embodiment] As an example, such as Figure 1 As shown, the information processing system 10 includes multiple user devices 12 and a server 14. The user devices 12 are terminal devices that send and receive input information and / or image information based on user 16 between themselves and the server 14; for example, they are smartphones. Figure 1 In the example shown, user devices 12A, 12B, 12C, and 12D are illustrated as multiple user devices 12. Hereinafter, for ease of explanation, user devices 12A, 12B, 12C, and 12D will be simply referred to as "user device 12" when there is no need to distinguish between them. Furthermore, while four user devices 12 are illustrated here for clarity, the technology of this invention is not limited to this, and the number of user devices 12 can be multiple. Also, while a smartphone is illustrated here as user device 12, the technology of this invention is not limited to this, and it can also be a tablet terminal, a personal computer, a wearable terminal, and / or a photographic terminal such as a digital camera, etc.

[0075] The information processing system 10 is used by multiple users 16. Figure 1 In the example shown, users 16A, 16B, 16C, and 16D are represented as multiple users 16. For ease of explanation, users 16A, 16B, 16C, and 16D will be referred to as "user 16" below.

[0076] Each of the multiple users 16 is assigned a user device 12. User device 12A is assigned to user 16A. User device 12B is assigned to user 16B. User device 12C is assigned to user 16C. User device 12D is assigned to user 16D. For example, user 16A is the owner of user device 12A, user 16B is the owner of user device 12B, user 16C is the owner of user device 12C, and user 16D is the owner of user device 12D. Furthermore, this example illustrates the case where each of the users 16 using user device 12 is a single individual, but one user device 12 can be used by two or more users 16, and one user 16 can also use two or more user devices 12.

[0077] Multiple user equipments 12 are connected to server 14 via network 18. The multiple user equipments 12 and server 14 are communicatively connected to network 18, for example. Furthermore, network 18 is composed of at least one of a WAN or LAN. Moreover, the multiple user equipments 12 and network 18, and server 14 and network 18, can be connected wirelessly or via wired communication. Figure 1 In the example shown, although the illustration is omitted, network 18 includes, for example, a base station. Network 18 establishes communication between multiple user equipments 12 and server 14, and performs various information transmission and reception between the multiple user equipments 12 and server 14. Server 14 receives requests from user equipments 12 via network 18 and provides services corresponding to the requests to the requesting user equipments 12 via network 18. Furthermore, server 14 is an example of an "information processing apparatus" according to the technology of this invention.

[0078] User equipment 12 calculates GPS information using radio waves emitted from GPS satellite 20 as location determination information that can determine the current location of user equipment 12. GPS information includes, for example, latitude and longitude. In this first embodiment, for ease of explanation, latitude and longitude are given as examples of GPS information, but the technology of the present invention is not limited thereto; GPS information can also be latitude, longitude, and altitude. Furthermore, GPS information is an example of the "location determination information" involved in the technology of the present invention.

[0079] As an example, such as Figure 2 As shown, user equipment 12 includes a computer 22, a camera device 24, a clock 26, a communication I / F 28, a GPS receiver 30, a receiving device 32, a display 34, a microphone 36, a speaker 38, and an external I / F 40. The computer 22 includes a CPU 42, a memory 44, and a RAM 46. The CPU 42, memory 44, and RAM 46 are connected to a bus 48. Furthermore, the camera device 24, clock 26, communication I / F 28, GPS receiver 30, receiving device 32, display 34, microphone 36, speaker 38, and external I / F 40 are also connected to the bus 48. Additionally, in... Figure 2 In the example shown, for ease of illustration, a bus 48 is shown as a single bus, but bus 48 includes a data bus, an address bus, and a control bus.

[0080] CPU 42 controls the user equipment 12 as a whole. Memory 44 stores various parameters and programs. Memory 44 is a non-volatile storage device. Here, EEPROM is used as an example of memory 44, but it is not limited to this; it could also be SSD and / or HDD, etc. Memory 46 is a volatile storage device. Memory 46 is used by CPU 42 as working memory and temporarily stores various information. Here, DRAM is used as an example of memory 46, but it is not limited to this; it could also be other types of volatile storage devices such as SRAM.

[0081] The imaging device 24 is a device for generating image data. The imaging device 24 may have, for example, a CMOS image sensor and include a zoom mechanism and a focusing mechanism. While a CMOS image sensor is used as an example of the image sensor in the imaging device 24, it is not limited to this and may also be other types of image sensors such as a CCD image sensor. The imaging device 24 captures an image of a subject according to instructions from the CPU 42. Then, the imaging device 24 generates image data representing the subject by capturing the image. The CPU 42 acquires the image data generated by the imaging device 24 and stores the acquired image data in the memory 44.

[0082] Clock 26 acquires the current time. Clock 26, for example, is an RTC that receives power from a power system separate from the power system of computer 22, and continues to record the current time (year, month, day, hour, minute, second) even when computer 22 is powered off. Clock 26 outputs the current time to CPU 42 each time it updates the current time.

[0083] Communication I / F28 is wirelessly connected to network 18 and manages the sending and receiving of various information between CPU 42 and server 14 via network 18.

[0084] According to the instructions from the CPU 42, the GPS receiver 30 receives radio waves from multiple GPS satellites (not shown), including GPS satellite 20, and outputs reception result information indicating the reception results to the CPU 42. The CPU 42 calculates the aforementioned GPS information based on the reception result information input from the GPS receiver 30.

[0085] The receiving device 32 receives instructions from the user 16, etc. Examples of the receiving device 32 include a touch panel 32A and hard keys. The instructions received by the receiving device 32 are acquired by the CPU 42. The receiving device 32 can receive instructions from the user 16, etc., via audio input through the microphone 36.

[0086] Under the control of CPU 42, display 34 displays various information. As an example of display 34, a liquid crystal display (LCD) can be used. However, it is not limited to LCDs; other types of displays, such as organic EL displays, can also be used as display 34.

[0087] Furthermore, in this first embodiment, an external touch panel 32A is used, which overlaps the display area surface of the display 34. Display. However, external touch panel. Displays are just one example; for instance, embedded or in-wall touch panels can also be used. monitor.

[0088] Microphone 36 converts the collected sound into an electrical signal and outputs the converted electrical signal to CPU 42.

[0089] The speaker 38 converts electrical signals input from a specific device (e.g., CPU 42) into sound and outputs the sound obtained by converting the electrical signals to the outside of the user equipment 12.

[0090] The external I / F40 manages the transmission and reception of various information with devices located outside the user equipment 12. An example of an external I / F40 is a USB interface. User equipment, personal computers, servers, USB memory, memory cards, and / or printers are connected to the USB interface.

[0091] As an example, such as Figure 3 As shown, server 14 includes computer 50, communication I / O 52, receiving device 54, display 56, and external I / O 58. Computer 50 includes CPU 60, memory 62, and RAM 64. CPU 60, memory 62, and RAM 64 are connected to bus 66. Furthermore, communication I / O 52, receiving device 54, display 56, and external I / O 58 are also connected to bus 66. Additionally, in... Figure 3 In the example shown, for ease of illustration, a bus 66 is shown as a single bus, but bus 66 includes a data bus, an address bus, and a control bus.

[0092] CPU 60 controls the entire server 14. Memory 62 stores various parameters and programs. Memory 62 is a non-volatile storage device. Here, an SSD is used as an example of memory 62, but it is not limited to this; it could also be EEPROM and / or HDD, etc. Memory 64 is a volatile storage device. Memory 64 is used by CPU 60 as working memory and temporarily stores various information. Here, DRAM is used as an example of memory 64, but it is not limited to this; it could also be other types of volatile storage devices such as SRAM. Furthermore, CPU 60 is an example of a "processor" according to the technology of this invention, and memory 62 and memory 64 are examples of "memory" according to the technology of this invention.

[0093] The communication I / F52 is able to communicate with the network 18 and manage the sending and receiving of various information between the CPU 60 and the user equipment 12 via the network 18.

[0094] The receiving device 54 receives instructions from administrators or other personnel on the server 14. Examples of receiving devices 54 include voice input via a remote control, touch panel, hard keys, and / or microphone. Instructions received by the receiving device 54 are acquired by the CPU 60.

[0095] Under the control of CPU 60, display 56 displays various information. As an example of display 56, a liquid crystal display (LCD) can be used. However, it is not limited to LCDs; other types of displays, such as EL displays, can also be used as display 56.

[0096] External I / F58 manages the transmission and reception of various information with devices located outside of server 14. An example of an external I / F58 is a USB interface. User devices, personal computers, servers, USB memory, memory cards, and / or printers are connected to the USB interface.

[0097] In the information processing system 10, multiple user devices 12 upload dated image data to a server 14, which manages the uploaded dated image data. Here, dated image data refers to image data with a date attached. The dated image data is created, for example, by the user devices 12.

[0098] In user equipment 12, date-included image data is created by CPU 42 performing date-included image data creation processing. As an example, ... Figure 4As shown, a date-based image data creation program 68 is stored in memory 44. CPU 42 reads the date-based image data creation program 68 from memory 44. Then, CPU 42 executes the date-based image data creation program 68 read from memory 44 on memory 46, operating as a camera control unit 42A, an image data acquisition unit 42B, a GPS information calculation unit 42C, an attribute data creation unit 42D, and a date-based image data creation unit 42E. That is, the date-based image data creation process is implemented by CPU 42 operating as a camera control unit 42A, an image data acquisition unit 42B, a GPS information calculation unit 42C, an attribute data creation unit 42D, and a date-based image data creation unit 42E.

[0099] As an example, such as Figure 5 As shown, if a start shooting instruction (hereinafter also referred to as a "shooting start instruction") is received via receiving device 32, the camera control unit 42A controls the camera device 24 to shoot the subject. The camera device 24 generates image data by shooting the subject. The image data acquisition unit 42B acquires the image data from the camera device 24.

[0100] The GPS information calculation unit 42C calculates GPS information based on the reception result information input from the GPS receiver 30.

[0101] The memory 44 stores a user ID that can identify user 16. The attribute data creation unit 42D creates attribute data representing the attributes of the image data acquired by the image data acquisition unit 42B. The attribute data creation timing is the timing of capturing one frame at a time by the camera device 24. That is, the attribute data is created by the attribute data creation unit 42D each time one frame is captured by the camera device 24.

[0102] The attribute data creation unit 42D obtains GPS information from the GPS information calculation unit 42C. Furthermore, the attribute data creation unit 42D obtains the user ID from the memory 44. Additionally, the attribute data creation unit 42D obtains the current time from the clock 26. Then, the attribute data creation unit 42D creates attribute data containing the user ID, date, and GPS information. GPS information is included in the attribute data as information for determining the camera's location. Furthermore, the attribute data also includes Exif information, etc.

[0103] Furthermore, the date included in the attribute data is the current time obtained from the clock 26 by the attribute data production unit 42D. As described above, the attribute data is produced based on the timing of shooting one frame by the camera device 24, so the date included in the attribute data becomes the date of shooting (hereinafter also referred to as "shooting date").

[0104] The date-included image data production unit 42E acquires image data from the image data acquisition unit 42B and attribute data from the attribute data production unit 42D each time one frame is captured. Then, the date-included image data production unit 42E produces date-included image data by establishing associated image data and attribute data on a one-frame basis.

[0105] As an example, such as Figure 6 As shown, each time date-bearing image data is created by the date-bearing image data creation unit 42E, the date-bearing image data is stored in the memory 44 by the date-bearing image data creation unit 42E. The memory 44 saves multiple date-bearing image data (e.g., multiple frames of date-bearing image data) as an image data group. In this way, date-bearing image data is created each time a shot is taken in each of the multiple user devices 12. For example, Figure 7 As shown, multiple image data with dates are saved as image data groups in the memory 44 of each of the multiple user devices 12.

[0106] As an example, such as Figure 8 As shown, multiple users 16 operate their respective user devices 12, thereby uploading image data sets stored in memory 44 to server 14. Thus, the image data sets uploaded to server 14 are stored and managed by server 14.

[0107] In the information processing system 10, when user 16 carries image data without a date (hereinafter also referred to as "dateless image data"), user equipment 12 can request server 14 to assign a date to the dateless image data. In this case, user equipment 12 requests server 14 to assign a date to the dateless image data by CPU 42 performing date assignment request processing.

[0108] As an example, such as Figure 9 As shown, a date assignment request processing program 70 is stored in memory 44. CPU 42 reads the date assignment request processing program 70 from memory 44. Then, CPU 42 executes the date assignment request processing program 70 read from memory 44 on memory 46, operating as a request data creation unit 42F, a request data transmission unit 42G, and a display control unit 42H. That is, date assignment request processing is implemented by CPU 42 operating as a request data creation unit 42F, a request data transmission unit 42G, and a display control unit 42H.

[0109] As an example, such as Figure 10As shown, if an instruction to create request data (hereinafter also referred to as "request data creation instruction") is received via receiving device 32, the request data creation unit 42F creates request data. Here, request data refers to data indicating a request to server 14 to assign a date to dateless image data. If the request data creation instruction is received via receiving device 32, the request data creation unit 42F first obtains the user ID from memory 44 and obtains dateless image data from an external device (e.g., USB memory or SSD) via external I / F 40. Next, the request data creation unit 42F creates request data by establishing an association between the obtained user ID and the dateless image data. Then, the request data creation unit 42F outputs the created request data to the request data sending unit 42G. In addition, in information processing system 10, user device 12 is provided with attribute data contained in dated image data or dateless image data to assign the user ID, but the technology of the present invention is not limited to this. For example, user authentication processing can be performed between server 14 and user device 12 to determine the user 16 using user device 12, and the uploaded image after authentication can be assigned a user ID corresponding to the authenticated user 16, thereby generating image data with or without a date. Alternatively, user device 12 can be assigned a device ID to identify user device 12 by using attribute data contained in the image data with or without a date instead of the user ID. Server 14 stores a list of users 16 and their corresponding device IDs in memory 62, and determines the user ID from the device ID contained in the image data with or without a date based on the list.

[0110] The request data sending unit 42G sends the request data input from the request data production unit 42F to the server 14 via the communication I / F28.

[0111] Server 14 receives request data sent from request data sending unit 42G, and assigns a date to the dateless image data contained in the received request data, thereby generating date-assigned image data, and provides it to user equipment 12 of the requesting source.

[0112] The display control unit 42H displays an image (hereinafter also referred to as "date-assigned image") represented by date-assigned image data provided from the server 14 on the monitor 34.

[0113] The date has been assigned to the image data, which is processed by CPU 60 of server 14 to create a list of image data with the date (see reference). Figure 11 ) and date assignment processing (see reference) Figure 34 It is generated by ).

[0114] As an example, such as Figure 11As shown, a date-based image data list creation program 72 is stored in memory 62. CPU 60 reads the date-based image data list creation program 72 from memory 62. Then, by executing the date-based image data list creation program 72 read from memory 62 on memory 64, CPU 60 functions as a date-based image data acquisition unit 60A, a user ID extraction unit 60B, a storage control unit 60C, an image data group acquisition unit 60D, a person image data extraction unit 60E, a deletion unit 60F, a GPS information extraction unit 60G, a distribution area map creation unit 60H, a duplicate area ratio calculation unit 60I, a user information acquisition unit 60J, a user information consistency calculation unit 60K, a non-person image data extraction unit 60L, an image data list creation unit 60M, an image data list classification unit 60N, and a determination unit 60P. That is, the creation and processing of the image data list with date is achieved by the CPU 60 as the image data acquisition unit 60A with date, the user ID extraction unit 60B, the storage control unit 60C, the image data group acquisition unit 60D, the person image data extraction unit 60E, the deletion unit 60F, the GPS information extraction unit 60G, the distribution area map creation unit 60H, the duplicate area ratio calculation unit 60I, the user information acquisition unit 60J, the user information consistency calculation unit 60K, the non-person image data extraction unit 60L, the image data list creation unit 60M, the image data list classification unit 60N, and the judgment unit 60P.

[0115] CPU 60 creates a dated image data list by performing a dated image data list creation process. The dated image data list is created for each subject by classifying the multiple dated image data into multiple subjects represented by each of the multiple dated image data. Here, "subject" refers to a subject whose visual changes over time can be determined. Furthermore, the "multiple dated image data" defined as the classification object here refers to multiple dated image data with different dates, and includes image data from multiple users 16 who are specific users. A specific user, for example, refers to user 16 among the multiple users 16 who has been assigned user device 12 that sent request data to server 14 (e.g., the owner of user device 12).

[0116] Furthermore, the CPU60 associates the dated image data list with a specific user by performing a dated image data list creation process. This associates the dated image data list of subjects similar to those represented by the dated image data of the specific user with the specific user.

[0117] As an example, such as Figure 12As shown, if an image data set sent from user equipment 12 is received via communication I / F 52, the date-bearing image data acquisition unit 60A acquires the image data set received via communication I / F 52. Then, the date-bearing image data acquisition unit 60A acquires date-bearing image data from the image data set. The user ID extraction unit 60B extracts the user ID from the attribute data contained in the date-bearing image data acquired by the date-bearing image data acquisition unit 60A. Alternatively, while the user ID extraction unit 60B extracts the user ID from the attribute data contained in the date-bearing image data, it is not particularly limited to this. The server 14 may also store a list in memory 62 that associates user IDs of multiple users with date-bearing image data associated with multiple users 16. The user ID extraction unit 60B may also select a preset condition or the user ID of any user 16 from the stored list of multiple users 16. The preset condition may, for example, be a timing condition where user 16 uploads date-bearing image data via user equipment 12.

[0118] As an example, such as Figure 13 As shown, a list of logged-in users is stored in memory 62. The list of logged-in users is a list of multiple user IDs that can be identified as a user group that meets the condition of agreeing to share information containing dated image data and has logged in. That is, multiple users 16 identified by the multiple user IDs represented by the list of logged-in users are a user group that agrees to share information containing dated image data and has logged in (hereinafter also referred to as "logged-in user group").

[0119] The CPU 60 performs the date-based image data list creation process shown below to further narrow down the logged-in user group. As described above, a corresponding association is established between the image data group and each of the multiple users 16, and the logged-in user group is narrowed down to users who meet the condition that the image data groups are similar to each other. Moreover, the logged-in user group is narrowed down to users who meet the condition that the logged-in user information is similar. A more detailed explanation follows.

[0120] The determination unit 60P refers to the list of logged-in users in the memory 62 and determines whether the user ID extracted by the user ID extraction unit 60B is a logged-in user ID. When the determination unit 60P determines that the user ID is a logged-in user ID, the storage control unit 60C stores the image data group acquired by the date-inclusive image data acquisition unit 60A in the memory 62.

[0121] When the determination unit 60P determines that the user ID retrieved by the user ID extraction unit 60B is not a logged-in user ID, the determination unit 60P then determines whether there are multiple groups of image data stored in the memory 62. If there are not multiple groups of image data stored in the memory 62, the determination unit 60P stands still until the next determination timer arrives.

[0122] When it is determined that there are multiple groups of image data stored in memory 62, the determination unit 60P instructs the image data acquisition unit 60D to acquire the image data groups.

[0123] As an example, such as Figure 14 As shown, if the determination unit 60P instructs the acquisition of image data groups, the image data group acquisition unit 60D acquires the image data groups from the memory 62. The person image data extraction unit 60E performs image recognition processing on the date-bearing image data contained in the image data groups, extracts person image data representing people from the date-bearing image data, and establishes an association between the extracted person image data and the date-bearing image data of the extraction source. Then, the person image data extraction unit 60E stores the date-bearing image data with the established person image data in the memory 62 for each image data group, and returns the date-bearing image data to the memory 62 for each image data group.

[0124] Furthermore, in this first embodiment, image analysis using a cascaded classifier is applied as the image recognition process. However, this is just one example; other image recognition processes such as pattern matching can also be used. Any process can be used as long as it can identify a subject image data representing a specific subject from the date-bearing image data.

[0125] As an example, such as Figure 15 As shown, the determination unit 60P determines, for each group of image data stored in the memory 62, whether the number of frames for image data of the same person (hereinafter also referred to as "same person image data") is a first predetermined number of frames (e.g., 10) or more. Here, the determination of whether the image data is of the same person is based on the image recognition result obtained by performing image recognition processing on image data of the person associated with image data with date. In addition, the first predetermined number of frames is a fixed value here. However, this is just one example; the first predetermined number of frames can be a fixed value or a variable value as long as it is a natural number of 2 or more. As a variable value, examples include a value that can be changed according to an instruction received by the receiving device 54 or a value that changes periodically.

[0126] When the number of frames that are determined to be image data of the same person is less than a first predetermined number of frames, the determination unit 60P instructs the deletion unit 60F to delete the image data group designated as the determination object from the image data group of the image data list creation object. If the determination unit 60P instructs the deletion of the image data group, the deletion unit 60F deletes the image data group designated as the determination object from the image data group of the image data list creation object stored in the memory 62. In addition, in this embodiment, the CPU 60 is provided to have a deletion unit 60F that deletes the image data group designated as the determination object from the image data group of the image data list creation object by the determination unit 60P, but it is not limited to this. For example, the determination unit 60P may have an extraction unit that extracts the image data group designated as the determination object from the memory 44 as the image data group of the image data list creation object.

[0127] On the other hand, as an example, such as Figure 16 As shown, when the number of frames that are determined to be identical image data is a first predetermined frame number or more, the determination unit 60P determines whether there are multiple image data groups stored in the memory 62 whose number of frames that are determined to be identical image data is a first predetermined frame number or more. Here, if there are not multiple image data groups, the determination unit 60P stands still until the next determination time arrives. On the other hand, when there are multiple image data groups, the determination unit 60P determines for each image data group whose number of frames that are determined to be identical image data is a first predetermined frame number or more, whether the number of frames that are common to the identical image data is a second predetermined frame number (e.g., 5) or more.

[0128] Here, when there are image data sets with the same character image data having fewer than a second predetermined frame number of frames, the determination unit 60P instructs the deletion unit 60F to delete the image data set with fewer than a second predetermined frame number of frames from the image data set of the image data list creation object. If the determination unit 60P instructs the deletion of the image data set, the deletion unit 60F deletes the image data set with fewer than a second predetermined frame number of frames of images of the same character image data having more than a second predetermined frame number from the image data set of the image data list creation object in the memory 62. On the other hand, when there are image data sets with more than a second predetermined frame number of frames of images of the same character image data having more than a second predetermined frame number of frames, the determination unit 60P instructs the image data set acquisition unit 60D to acquire the image data set with more than a second predetermined frame number of frames of images of the same character image data set from the memory 62.

[0129] As an example, such as Figure 17As shown, if the determination unit 60P instructs the acquisition of image data groups, the image data group acquisition unit 60D acquires from the memory 62 image data groups containing the same person image data for a second predetermined number of frames or more. The GPS information extraction unit 60G extracts GPS information from the attribute data including the date image data contained in the image data groups acquired by the image data group acquisition unit 60D.

[0130] As an example, such as Figure 18 As shown, the distribution area map creation unit 60H acquires GPS information extracted by the GPS information extraction unit 60G for each image data group. Then, the distribution area map creation unit 60H creates a camera location distribution area map for each image data group based on the GPS information. The camera location distribution area map is a map showing the areas where cameras are taken (hereinafter also referred to as "camera locations") as indicated by GPS information. Figure 18 The example shown illustrates a distribution map of camera positions for two image data groups, but the technology of this invention is not limited to this; a distribution map of camera positions for the number of image data groups can be created.

[0131] As an example, such as Figure 19 As shown, the repetition area ratio calculation unit 60I calculates the ratio of repetitive areas (hereinafter also referred to as "repetition areas") in the image data group's camera position distribution area map. The determination unit 60P determines whether the ratio of the repetition areas is a predetermined ratio (e.g., 60%) or higher. Here, the predetermined ratio is a fixed value. However, this is just one example; the predetermined ratio can also be a variable value. Examples of variable values ​​include values ​​that can be changed according to instructions received via the receiving device 54 or values ​​that change periodically. Furthermore, in... Figure 19 In the example shown, the areas marked with shaded lines represent repeated areas, which are areas exceeding a certain proportion.

[0132] As an example, such as Figure 20 As shown, when the proportion of areas deemed to be duplicates is less than a predetermined proportion, the determination unit 60P instructs the deletion unit 60F to delete the image data group designated as the object of determination. For example, when the proportion of duplicate areas in the camera position distribution area map between two image data groups is less than a predetermined proportion, the determination unit 60P instructs the deletion unit 60F to delete both image data groups.

[0133] The deletion unit 60F restricts the date-bearing image data of the production object set in the date-bearing image data list from a plurality of date-bearing image data to image data acquired by shooting within a range set according to GPS information. That is, if the determination unit 60P instructs the production object's image data group to be deleted from the image data list of the image data group, the deletion unit 60F deletes the image data group set as the determination object from the image data group of the production object set in the image data list of the image data group in the memory 62, thus restricting the date-bearing image data of the production object set in the date-bearing image data list.

[0134] On the other hand, when it is determined that the proportion of overlapping areas among all image data groups is greater than a predetermined proportion, the determination unit 60P determines whether the number of image data groups stored in the memory 62 is multiple.

[0135] As an example, such as Figure 21 As shown, when the number of image data groups stored in memory 62 is not multiple, the determination unit 60P stands by until the next determination time arrives. On the other hand, when the determination unit 60P determines that the number of image data groups stored in memory 62 is multiple, the deletion unit 60F obtains GPS information outside the overlapping areas between the image data groups stored in memory 62 from the distribution area map creation unit 60H. Then, referring to the GPS information obtained from the distribution area map creation unit 60H, the deletion unit 60F deletes the dated image data containing GPS information outside the overlapping areas from the dated image data included in the image data groups stored in memory 62, from the dated image data of the image data list creation object.

[0136] As an example, such as Figure 22 As shown, memory 62 associates user information with logged-in user IDs. This user information is related to the user 16 identified based on the corresponding user ID. Examples of user information include, for instance, age, family member information, address information, gender information, occupation information, and interest information.

[0137] The date information indicates the year of User 16. For example, if User 16 was born in 1975, then "1970s-1980s" would be used as the date information. Family member information indicates the family members of User 16. Examples of family member information include whether the user is married, unmarried, the number of brothers, sisters, and siblings, the age difference between brothers, and / or the ages of parents. Address information indicates the address of User 16. Examples of address information include the country name, province, city / county name, and / or town / village name. Gender information indicates the gender of User 16. Examples of gender information include male and female. Occupation information indicates the occupation of User 16. Examples of occupation information include sales, technical, teacher, unemployed, and / or housewife. Interest information indicates the interests of User 16. As examples of interest-based information, one could include outdoor activities, indoor activities, golf, soccer, baseball, fishing, watching movies, reading, and / or online games.

[0138] The user ID extraction unit 60B extracts the user ID from each image data group stored in the memory 62. The user information acquisition unit 60J acquires the user information corresponding to the user ID extracted by the user ID extraction unit 60B, and establishes an association between the acquired user information and the corresponding user ID.

[0139] As an example, such as Figure 23 As shown, the user information consistency calculation unit 60K obtains user information for each user ID from the user information acquisition unit 60J and calculates the consistency of user information between user IDs (hereinafter also referred to as "user information consistency"). The determination unit 60P determines whether to store image data groups with a user information consistency of less than a predetermined consistency (e.g., 5%) with any user ID in the memory 62. Furthermore, a fixed value is used here as the predetermined consistency. However, this is just one example; the predetermined consistency can also be a variable value. Examples of variable values ​​include values ​​that can be changed according to instructions received via the receiving device 54 or values ​​that change periodically.

[0140] When image data groups with user information consistency less than a predetermined consistency among any user IDs are not stored in memory 62, the determination unit 60P stands still until the next determination timer arrives. When image data groups with user information consistency less than a predetermined consistency among any user IDs are stored in memory 62, the determination unit 60P instructs the deletion unit 60F to delete image data groups with user information consistency less than the predetermined consistency from the image data groups that are the objects of image data list creation. Accordingly, the deletion unit 60F deletes image data groups with user information consistency less than the predetermined consistency from the image data groups that are the objects of image data list creation in memory 62.

[0141] As an example, such as Figure 24 As shown, the date-attached image data acquisition unit 60A acquires date-attached image data from each image data group stored in the memory 62. The non-human image data extraction unit 60L performs image recognition processing on the date-attached image data acquired by the date-attached image data acquisition unit 60A, extracts non-human image data representing non-human figures from the date-attached image data, and establishes an association between the extracted non-human image data and the date-attached image data of the extraction source. The non-human image data extraction unit 60L stores the date-attached image data with the associated non-human image data in the memory 62 for each image data group, and returns the date-attached image data to the memory 62 for each image data group.

[0142] Here, "non-human" refers to objects other than people that can be visually described as changing over time. For example, a way of changing over time can be exemplified by objects that symbolize an era. Examples of objects that symbolize an era include buildings, roads, streets, signs, posters, and food.

[0143] As an example, such as Figure 25 As shown, the image data list creation unit 60M retrieves date-attached image data (hereinafter also referred to as "date-attached image data with person image data") associated with person image data from the memory 62. Furthermore, the image data list creation unit 60M retrieves date-attached image data (hereinafter also referred to as "date-attached image data with non-person image data") associated with non-person image data from the memory 62.

[0144] Then, the image data list creation unit 60M determines whether the image data containing person images and date data are similar. Furthermore, the image data list creation unit 60M determines whether the image data containing non-person images and date data are similar.

[0145] In addition, Figure 25 In the example shown, for ease of explanation, only one corresponding image data of a person or one non-person image data is established for a date-bearing image data, but the technology of the present invention is not limited to this. For example, there may also be a way to establish a corresponding association between a date-bearing image data and at least one image data of a person and at least one image data of a non-person.

[0146] As an example, such as Figures 26-28As shown, the image data list creation unit 60M creates a dated image data list for each subject by classifying the dated image data according to each representation of all dated image data stored in memory 62 as the image data list creation object. The dated image data in the image data list creation object includes, for example, dated image data that has been associated with multiple users. Figures 26-28 The example shown presents a list of image data for subjects 1 through N.

[0147] exist Figure 26 The example shown illustrates a first subject image data list and a second subject image data list. The first subject image data list is a list of date-bearing image data composed of multiple date-bearing image data mapped onto the first subject. Furthermore, the first subject image data list may be, for example, a list formed by date-bearing image data contained in multiple image data groups A to D, which are respectively associated with users 16A to 16D. The second subject image data list is a list of date-bearing image data composed of multiple date-bearing image data mapped onto the second subject. Furthermore, the second subject image data list may be, for example, a list formed by date-bearing image data contained in multiple image data groups A and C, which are respectively associated with users 16A and 16C. Figure 26 In the example shown, a radio tower is shown as the first subject, and "×× soft-serve ice cream" is shown as the second subject. Both the radio tower and the "×× soft-serve ice cream" are non-human figures. Furthermore, the radio tower is an example of the aforementioned building, and the "×× soft-serve ice cream" is an example of the aforementioned food.

[0148] exist Figure 27 The example shown illustrates a third subject image data list and a fourth subject image data list. The third subject image data list is a list of date-bearing image data composed of multiple date-bearing image data mapped onto the third subject. Furthermore, the third subject image data list may be, for example, a list formed by date-bearing image data contained in multiple image data groups A to C, which are respectively associated with users 16A to 16C. The fourth subject image data list is a list of date-bearing image data composed of multiple date-bearing image data mapped onto the fourth subject. Furthermore, the fourth subject image data list may be, for example, a list formed by date-bearing image data contained in multiple image data groups A and C, which are respectively associated with users 16A and 16C. Figure 27 In the examples shown, a football player is shown as one of the third subjects, and a woman is shown as one of the fourth subjects. The football player and the woman are human figures.

[0149] exist Figure 28The example shown illustrates a list of image data for the fifth subject. This list is a list of dated image data consisting of multiple dated image data images of the fifth subject. Furthermore, the list of image data for the fifth subject can be, for example, a list of dated image data contained in multiple image data groups A and D that are respectively associated with user 16A and user 16D. Figure 28 In the examples shown, signs and posters are presented as one of the fifth subjects. These signs and posters are not depicting people.

[0150] As an example, such as Figure 29 As shown, the image data list classification unit 60N classifies the dated image data list created by the image data list creation unit 60M according to each user ID, and associates the dated image data list with each of the multiple users 16. In this case, firstly, the image data list classification unit 60N extracts the user ID from each of the multiple dated image data contained in the dated image data list. Then, the image data list classification unit 60N associates the dated image data list containing the extracted user IDs with the user IDs that match the extracted user IDs from the multiple user IDs stored in the memory 62.

[0151] Thus, by associating a list of date-bearing image data with each of the multiple user IDs stored in memory 62, for example, Figures 30-33 As shown, the list of dated image data is associated with each of the multiple users 16.

[0152] exist Figure 30 In the example shown, the image data list of subjects 1 to 5 is associated with user 16A. The image data group uploaded from user device 12A assigned to user 16A to server 14 includes dated image data of subjects similar to subjects 1 to 5. Therefore, through the image data list classification unit 60N, the image data list of subjects 1 to 5 associated with subjects 1 to 5 is associated with user 16A.

[0153] exist Figure 31 The example shown illustrates the association between the first subject image data list and the third subject image data list and user 16B. The image data group uploaded from user device 12B assigned to user 16B to server 14 includes dated image data of subjects similar to the first and third subjects. Therefore, through the image data list classification unit 60N, the first subject image data list and the third subject image data list associated with the first and third subjects are associated with user 16B.

[0154] exist Figure 32 In the example shown, the image data list of subjects 1 to 4 is associated with user 16C. The image data group uploaded from user device 12C assigned to user 16C to server 14 includes dated image data of subjects similar to subjects 1 and 4. Therefore, through the image data list classification unit 60N, the image data list of subjects 1 to 4 associated with subjects 1 to 4 is associated with user 16C.

[0155] exist Figure 33 The example shown illustrates the association between the first subject image data list and the fifth subject image data list with the user 16D. The image data group uploaded from the user device 12D assigned to the user 16D to the server 14 includes dated image data of subjects similar to the first and fifth subjects. Therefore, the image data list classification unit 60N associates the first subject image data list and the fifth subject image data list with the first and fifth subjects with the user 16D.

[0156] As an example, such as Figure 34 As shown, a date assignment processing program 74 is stored in memory 62. CPU 60 reads the date assignment processing program 74 from memory 62. Then, CPU 60 executes the date assignment processing program 74 read from memory 62 on memory 64, functioning as a user ID extraction unit 60B, a determination unit 60P, an image data list acquisition unit 60Q, a dateless image data extraction unit 60R, a date export unit 60S, a date assignment unit 60T, and an image data transmission unit 60U. In other words, the date assignment processing is implemented by CPU 60 operating as a user ID extraction unit 60B, a determination unit 60P, an image data list acquisition unit 60Q, a dateless image data extraction unit 60R, a date export unit 60S, a date assignment unit 60T, and an image data transmission unit 60U.

[0157] CPU 60 performs date assignment processing to obtain dated image data of subjects similar to those represented by dateless image data of the specific user from a list of dated image data associated with a specific user. Furthermore, CPU 60 performs date assignment processing to derive dates for the dateless image data based on the dates assigned to the obtained dated image data, and assigns the derived dates to the dateless image data. A more detailed explanation follows.

[0158] As an example, such as Figure 35 As shown, if the request is assigned a date by the CPU 42 of user equipment 12 (see reference) Figure 9), and sends request data to server 14 through request data sending unit 42G (see reference). Figure 10 If the request data is received via the communication I / F52 of server 14, the user ID extraction unit 60B extracts the user ID from the request data received via the communication I / F52 and outputs the extracted user ID to the image data list acquisition unit 60Q.

[0159] As an example, such as Figure 36 As shown, the image data list acquisition unit 60Q acquires from the memory 62 a list of date-bearing image data corresponding to the user ID input from the user ID extraction unit 60B.

[0160] As an example, such as Figure 37 As shown, the dateless image data extraction unit 60R extracts dateless image data from request data received via communication I / F52 and outputs the extracted dateless image data to the determination unit 60P. The determination unit 60P determines whether there exists dated image data in the dated image data list acquired by the image data list acquisition unit 60Q that is similar to the dateless image data input from the dateless image data extraction unit 60R. Here, "similar" means consistent within a predetermined error range. The predetermined error can be a fixed value or a variable value.

[0161] Examples of variable values ​​include values ​​that can be changed according to instructions received by receiving device 54, values ​​that change according to the number of frames of dated image data included in the dated image data list, values ​​that are set according to the degree of deviation (e.g., dispersion or standard deviation) of the dates assigned to the dated image data included in the dated image data list, values ​​that change according to user 16 who provides dateless image data, and / or values ​​that change periodically.

[0162] As an example, such as Figure 38 As shown, when it is determined that there is no dated image data similar to the dateless image data input from the dateless image data extraction unit 60R in the dated image data list acquired by the image data list acquisition unit 60Q, the determination unit 60P stands still until the next determination time arrives. On the other hand, when it is determined that there is dated image data similar to the dateless image data input from the dateless image data extraction unit 60R in the dated image data list acquired by the image data list acquisition unit 60Q, the determination unit 60P determines whether the dated image data similar to the dateless image data is multiple frames.

[0163] When the determination unit 60P determines that the date-bearing image data similar to the dateless image data is not multi-frame, the date export unit 60S extracts the date from the date-bearing image data set as the determination object, i.e., the date-bearing image data similar to the dateless image data. When the determination unit 60P determines that the date-bearing image data similar to the dateless image data is multi-frame, the date export unit 60S extracts the date from each of the multiple date-bearing image data set set as the determination object, i.e., the multiple date-bearing image data similar to the dateless image data. Then, the date export unit 60S exports the date of the dateless image data based on the multiple dates extracted from each of the multiple date-bearing image data sets. The date exported by the date export unit 60S can be only the year, month, day, hour, minute, and second, or only the year.

[0164] Here, the date exported by the date export unit 60S as a date without date image data is, for example, a date based on the average of multiple dates extracted from each of multiple date image data. A date based on the average of multiple dates refers, for example, a date that is a rounded average of multiple dates.

[0165] Furthermore, while an example here is based on the average of multiple dates, the technology of the present invention is not limited to this, and the date can also be the date of the most frequent value or the median value among multiple dates. When the date derived from dateless image data is set as the date of the most frequent value or the median value among multiple dates, it can be set as the date of the most frequent value or the median value of the period with the highest date density among the period of multiple dates. Moreover, the date of dateless image data with the highest similarity to dateless image data can also be derived by the date derivation unit 60S as the date of dateless image data.

[0166] The date export unit 60S outputs the date derived as the date of the dateless image data to the date assignment unit 60T. The date assignment unit 60T assigns the date input from the date export unit 60S to the dateless image data extracted from the request data by the dateless image data extraction unit 60R (i.e., the dateless image data compared with the date-bearing image data by the determination unit 60P). Thus, by assigning a date to the dateless image data, a date is assigned to the dateless image data. In this way, by assigning a date to the dateless image data by the date assignment unit 60T, date-assigned image data is generated.

[0167] The image data transmission unit 60U transmits date-assigned image data generated by the date assignment unit 60T to the user equipment 12A, the source of the requested data, via communication I / F 52. Thus, date-assigned image data is provided to the user 16, who is assigned to the user equipment 12A, the source of the requested data.

[0168] Next, the function of the information processing system 10 will be explained.

[0169] First, refer to Figures 39A-39E The process of creating a list of image data with dates, performed by CPU 60 of server 14, is described. Additionally, Figures 39A-39E The process of creating and processing the date-bearing image data list shown is an example of the "information processing method" involved in the technology of this invention.

[0170] exist Figure 39A In the date-included image data list creation process shown, firstly, in step ST10, the determination unit 60P determines whether an image data group sent from the user equipment 12 has been received via communication I / F 52. If, in step ST10, an image data group sent from the user equipment 12 has not yet been received via communication I / F 52, the determination is negative, and the date-included image data list creation process proceeds to... Figure 39E The step ST80 is shown. In step ST10, when an image data group sent from user equipment 12 is received via communication I / F52, the determination is positive, and the process of creating a dated image data list proceeds to step ST12.

[0171] In step ST12, the date-bearing image data acquisition unit 60A acquires date-bearing image data from the image data group received via communication I / F52, and then the date-bearing image data list creation process proceeds to step ST14.

[0172] In step ST14, the user ID extraction unit 60B extracts the user ID from the date-bearing image data obtained in step ST12, and then the date-bearing image data list creation process proceeds to step ST16.

[0173] In step ST16, the determination unit 60P determines whether the user ID extracted in step ST14 has already logged in. The determination of whether the user ID has logged in is performed by checking whether the user ID is included in the list of logged-in users stored in memory 62. In step ST16, if the user ID extracted in step ST14 has not yet logged in, the determination is negative, and the process of creating the date image data list proceeds to step ST20. In step ST16, if the user ID extracted in step ST14 has already logged in, the determination is positive, and the process of creating the date image data list proceeds to step ST18.

[0174] In step ST18, the storage control unit 60C stores the image data group set as the acquisition source with date image data in the memory 62 in step ST12, and then the process of creating the date image data list proceeds to step ST20.

[0175] In step ST20, the determination unit 60P determines whether two or more sets of image data are stored in the memory 62. If, in step ST20, two or more sets of image data are not stored in the memory 62, the determination is negative, and the image data list creation process with date proceeds to step ST10. If, in step ST20, two or more sets of image data are stored in the memory 62, the determination is positive, and the image data list creation process with date proceeds to step ST22.

[0176] In step ST22, the image data acquisition unit 60D acquires an unprocessed image data group from the memory 62. In step ST22, an unprocessed image data group refers to an image data group that has not yet undergone the processing described in steps ST24 to ST30. After the processing in step ST22 is performed, the process of creating a dated image data list proceeds to step ST24.

[0177] In step ST24, the person image data extraction unit 60E extracts person image data from each of the date-bearing image data included in the image data group acquired in step ST22, and establishes an association between the extracted person image data and the date-bearing image data of the extraction source. Then, the person image data extraction unit 60E returns the date-bearing image data to the memory 62 by storing the date-bearing image data with the established association in each image data group. After performing the processing in step ST24, the date-bearing image data list creation process proceeds to step ST26.

[0178] In step ST26, the determination unit 60P determines whether the number of frames for the latest image data group stored in memory 62 in step ST24 that contains images of the same person is greater than or equal to the first predetermined number of frames. In step ST26, if the number of frames for the same image data is less than the first predetermined number of frames, the determination is negative, and the process of creating the date-bearing image data list proceeds to... Figure 39B The step ST34 is shown. In step ST26, when the number of frames of the same person image data is more than the first predetermined frame number, the determination is affirmative, and the process of creating the image data list with date proceeds to step ST28.

[0179] In step ST28, the determination unit 60P determines whether an image data group that has been affirmed is stored in the memory 62. Here, an image data group that has been affirmed refers to the image data group that was affirmed in step ST26. In step ST28, if no image data group that has been affirmed is stored in the memory 62, the determination is negative, and the process of creating the date-based image data list proceeds to... Figure 39EThe step ST80 is shown. In step ST28, when there is a group of image data that has been affirmed, the affirmation is confirmed, and the process of creating a list of image data with dates proceeds to step ST30.

[0180] In step ST30, the determination unit 60P determines whether the latest image data group, which was affirmed in step ST26, contains image data of the same person that is common to the image data groups that have been affirmed in step ST26 or later. In step ST30, if the latest image data group, which was affirmed in step ST26, does not contain image data of the same person that is common to the image data groups that have been affirmed in step ST26, the determination is negative, and the process of creating the dated image data list proceeds to... Figure 39B The step ST34 is shown. In step ST30, when the latest image data group that has been confirmed in step ST26 contains image data of the same person that is common to the image data group that has been confirmed in step ST26, the confirmation is confirmed, and the image data list creation process with date proceeds to step ST32.

[0181] In step ST32, the determination unit 60P determines whether steps ST24 to ST30 have been performed on all image data groups stored in memory 62. In step ST32, if steps ST24 to ST30 have not yet been performed on all image data groups stored in memory 62, the determination is negative, and the process of creating the date image data list proceeds to step ST20. In step ST32, if steps ST24 to ST30 have been performed on all image data groups stored in memory 62, the determination is positive, and the process of creating the date image data list proceeds to step ST20. Figure 39C Step ST38 is shown.

[0182] exist Figure 39B In step ST34, the deletion unit 60F deletes the image data group of the processing object from the memory 62, and then the image data list creation process with date proceeds to step ST36.

[0183] In this step ST34, the image data group to be processed refers to the image data group whose frame count of the same person image data determined by the determination unit 60P is less than the first predetermined frame count, and the latest image data group that does not contain the second predetermined frame count or more of the same person image data determined by the determination unit 60P to be common to the image data group that has been affirmatively determined.

[0184] In step ST34, the image data group deleted by the deletion unit 60F is an image data group that does not meet the conditions set in step ST26 or step ST30. This means that the image data group deleted by the deletion unit 60F is an image data group that is not similar to other image data groups stored in memory 62. Furthermore, each image data group corresponds one-to-one with the registered user 16. Therefore, in step ST34, the image data group is deleted by the deletion unit 60F, thereby narrowing down the image data groups provided by the registered user 16 that meet the condition of being similar to each other to the production target candidates with a date image data list.

[0185] In step ST36, the determination unit 60P determines whether steps ST24 to ST30 have been performed on all image data groups stored in memory 62. If steps ST24 to ST30 have not been performed on all image data groups stored in memory 62 in step ST36, the determination is negative, and the image data list creation process proceeds to step ST20. If steps ST24 to ST30 have been performed on all image data groups stored in memory 62 in step ST36, the determination is positive, and the image data list creation process proceeds to step ST37.

[0186] In step ST37, the determination unit 60P determines whether two or more sets of image data are stored in the memory 62. If, in step ST37, two or more sets of image data are not stored in the memory 62, the determination is negative, and the process of creating the date-based image data list proceeds to... Figure 39E The step ST80 is shown. In step ST37, when two or more sets of image data are stored in memory 62, the determination is affirmative, and the process of creating the image data list with date proceeds to step ST80. Figure 39C Step ST38 is shown.

[0187] exist Figure 39C In step ST38, the image data acquisition unit 60D acquires an unprocessed image data group from the memory 62. In step ST38, an unprocessed image data group refers to an image data group that has not yet undergone the processing described in steps ST40 to ST46. After the processing in step ST38 is performed, the process of creating a dated image data list proceeds to step ST40.

[0188] In step ST40, the GPS information extraction unit 60G extracts GPS information from the attribute data in the date-bearing image data included in the image data group acquired in step ST38. Then, the date-bearing image data list creation process proceeds to step ST42.

[0189] In step ST42, the distribution area map making unit 60H makes a camera location distribution area map of the image data group based on the GPS information extracted in step ST40. Then, the image data list making process with date proceeds to step ST44.

[0190] In step ST44, the determination unit 60P determines whether there exists a camera location distribution area map, i.e., another camera location distribution area map, that is, a comparison object set as the camera location distribution area map created in step ST42. In step ST44, if there is no camera location distribution area map set as a comparison object set as the camera location distribution area map created in step ST42, the determination is negative, and the process of creating the date image data list proceeds to step ST50. In step ST44, if there is a camera location distribution area map set as a comparison object set as the camera location distribution area map created in step ST42, the determination is negative, and the process of creating the date image data list proceeds to step ST46.

[0191] In step ST46, the repeating region ratio calculation unit 60I calculates the ratio of the repeating regions between the camera position distribution area map created in step ST42 and other camera position distribution area maps. Then, the image data list creation process with date proceeds to step ST48. Here, "other camera position distribution area maps" refers to existing camera position distribution area maps, that is, all camera position distribution area maps created before the camera position distribution area map created in step ST42 (hereinafter also referred to as "all camera position distribution area maps").

[0192] In step ST48, the determination unit 60P determines whether the scale calculated for the distribution area map of all camera positions in step ST46 is greater than or equal to a predetermined scale. In step ST48, if the scale calculated for the distribution area map of all camera positions in step ST46 is not greater than or equal to a predetermined scale, the determination is negative, and the process of creating the dated image data list proceeds to... Figure 39D The step ST60 is shown. In step ST48, when the ratio calculated in step ST46 for the distribution area map of all camera positions is above a predetermined ratio, the determination is affirmative, and the processing of creating the image data list with date proceeds to step ST50.

[0193] In step ST50, the determination unit 60P determines whether steps ST40 to ST48 have been performed on all image data groups stored in memory 62. If steps ST40 to ST48 have not been performed on all image data groups stored in memory 62 in step ST50, the determination is negative, and the image data list creation process proceeds to step ST38. If steps ST40 to ST48 have been performed on all image data groups stored in memory 62 in step ST50, the determination is positive, and the image data list creation process proceeds to step ST52.

[0194] exist Figure 39D In step ST60 shown, the deletion unit 60F deletes the image data group of the processing object from the memory 62, and then the image data list creation process with date proceeds to step ST62.

[0195] In this step ST60, the image data group being processed refers to the image data group that is determined by the determination unit 60P to have a scale calculated for the distribution area map of all camera positions that is not above a predetermined scale.

[0196] In step ST60, the image data group deleted by the deletion unit 60F is an image data group that does not meet the conditions set in step ST48. This means that the image data group deleted by the deletion unit 60F is an image data group that is not similar to other image data groups stored in memory 62. Furthermore, each image data group corresponds one-to-one with the registered user 16. Therefore, in step ST60, the image data group deleted by the deletion unit 60F narrows down the image data groups provided by the registered user 16 from those that meet the condition that the image data groups are similar to each other (e.g., the geographical distribution of the camera locations between the image data groups is similar) to the production object candidates with a list of dated image data.

[0197] In step ST62, the determination unit 60P determines whether steps ST40 to ST48 have been performed on all image data groups stored in memory 62. If steps ST40 to ST48 have not been performed on all image data groups stored in memory 62 in step ST62, the determination is negative, and the process of creating the date-bearing image data list proceeds to... Figure 39C The step ST38 is shown. In step ST62, when all image data groups stored in memory 62 have been processed by steps ST40 to ST48, the determination is affirmative, and the process of creating the image data list with date proceeds to step ST38. Figure 39C Step ST52 is shown.

[0198] exist Figure 39CIn step ST52, the determination unit 60P determines whether two or more sets of image data are stored in the memory 62. If, in step ST52, two or more sets of image data are not stored in the memory 62, the determination is negative, and the process of creating the date-based image data list proceeds to... Figure 39E The step ST80 is shown. In step ST52, when two or more sets of image data are stored in memory 62, the determination is affirmative, and the process of creating a dated image data list proceeds to step ST54.

[0199] In step ST54, the deletion unit 60F deletes date-bearing image data that are distributed outside the overlapping areas of the camera position distribution area map of each image data group stored in memory 62. Then, the date-bearing image data list creation process proceeds to step ST56.

[0200] In step ST56, the user ID extraction unit 60B extracts the user ID from all image data groups stored in memory 62, and then the image data list creation process with date proceeds to step ST58.

[0201] In step ST58, the user information acquisition unit 60J retrieves user information corresponding to the user ID extracted from all image data groups in step ST56 from the memory 62, and then the image data list creation process with date is transferred to the next step. Figure 39E Step ST64 is shown.

[0202] Figure 39E In step ST64, the user information consistency calculation unit 60K uses all the user information obtained in step ST58 to calculate the user information consistency between the image data groups. Then, the image data list creation process with date proceeds to step ST66.

[0203] In step ST66, the determination unit 60P determines whether there exists an image data group whose user information consistency is less than a predetermined consistency, as calculated in step ST64. If, in step ST66, there is no image data group whose user information consistency is less than the predetermined consistency, the determination is negative, and the image data list creation process proceeds to step ST70. If, in step ST66, there is an image data group whose user information consistency is less than the predetermined consistency, the determination is positive, and the image data list creation process proceeds to step ST68.

[0204] In step ST68, the deletion unit 60F deletes the image data group whose user information consistency is less than the predetermined consistency calculated in step ST64 from the memory 62. Then, the image data list creation process with date proceeds to step ST70.

[0205] In step ST68, the image data group deleted by the deletion unit 60F is the image data group whose user information consistency calculated in step ST64 is less than a predetermined consistency. This means that the image data group deleted by the deletion unit 60F is an image data group that is not similar to other image data groups stored in memory 62. Furthermore, each image data group corresponds one-to-one with the logged-in user 16. Therefore, in step ST68, the image data group is deleted by the deletion unit 60F, thereby narrowing down the image data groups provided by the logged-in user 16 that meet the condition of being similar to each other to the creation target of a dated image data list.

[0206] In step ST70, the image data acquisition unit 60D acquires an unprocessed image data with a date from the memory 62. An unprocessed image data with a date refers to image data that has not yet undergone the processing steps ST72 to ST78. After performing the processing in step ST70, the process of creating the image data list with the date proceeds to step ST72.

[0207] In step ST72, the non-person image data extraction unit 60L extracts non-person image data from the date-bearing image data acquired in step ST70, and establishes an association between the extracted non-person image data and the date-bearing image data from the extraction source. Then, the non-person image data extraction unit 60L returns the date-bearing image data to memory 62 by storing the date-bearing image data with the associated non-person image data in memory 62 for each image data group. After performing the processing in step ST72, the date-bearing image data list creation process proceeds to step ST74.

[0208] In step ST74, the determination unit 60P determines whether step ST72 has been performed on all date-bearing image data stored in memory 62. If step ST72 has not been performed on all date-bearing image data stored in memory 62 in step ST74, the determination is negative, and the date-bearing image data list creation process proceeds to step ST70. If step ST72 has been performed on all date-bearing image data stored in memory 62 in step ST74, the determination is positive, and the date-bearing image data list creation process proceeds to step ST76.

[0209] In step ST76, the image data list creation unit 60M retrieves date-bearing image data containing person image data from the memory 62 and determines whether the person image data is similar among the date-bearing image data containing person image data. Furthermore, the image data list creation unit 60M retrieves date-bearing image data containing non-person image data from the memory 62 and determines whether the non-person image data is similar among the date-bearing image data containing non-person image data. Then, the image data list creation unit 60M creates a date-bearing image data list for each subject by classifying the date-bearing image data according to each subject represented by each of all date-bearing image data stored in the memory 62. The date-bearing image data list creation process then proceeds to step ST78.

[0210] In step ST78, the image data list classification unit 60N classifies the date-bearing image data list created by the image data list creation unit 60M according to each user ID, and establishes an associated date-bearing image data list for each of the multiple users 16. Then, the date-bearing image data list creation process proceeds to step ST80.

[0211] In step ST80, the determination unit 60P determines whether the condition for ending the date-based image data list creation process (hereinafter also referred to as the "image data list creation process end condition") is met. One example of the image data list creation process end condition is that the server 14 has been given an instruction to end the date-based image data list creation process. This instruction is received, for example, by the receiving device 54. In step ST80, if the image data list creation process end condition is not met, the determination is negative, and the date-based image data list creation process proceeds to... Figure 39A The step ST10 is shown. In step ST80, when the conditions for ending the image data list creation process are met, the determination is affirmative, and the image data list creation process with date ends.

[0212] Next, refer to Figure 40 The date assignment process executed by CPU 60 of server 14 is explained. Additionally, Figure 40 The date assignment process shown is an example of the "information processing method" involved in the technology of this invention.

[0213] exist Figure 40In the date assignment process shown, firstly, in step ST100, the determination unit 60P determines whether request data sent from user equipment 12 has been received via communication I / F 52. If, in step ST100, request data sent from user equipment 12 has not been received via communication I / F 52, the determination is negative, and the date assignment process proceeds to step ST120. If, in step ST100, request data sent from user equipment 12 has been received via communication I / F 52, the determination is positive, and the date assignment process proceeds to step ST102.

[0214] In step ST102, the user ID extraction unit 60B extracts the user ID from the request data received via communication I / F52, and then the date assignment process proceeds to step ST104.

[0215] In step ST104, the image data list acquisition unit 60Q acquires the image data list with date corresponding to the user ID extracted in step ST102 from the memory 62, and then the date assignment process proceeds to step ST106.

[0216] In step ST106, the dateless image data extraction unit 60R extracts dateless image data from the request data of the extraction source from which the user ID was extracted in step ST102, and then the date assignment process proceeds to step ST108.

[0217] In step ST108, the determination unit 60P determines whether there is any date-bearing image data similar to the dateless image data extracted in step ST106 within the date-bearing image data list obtained in step ST104. In step ST108, if there is no date-bearing image data similar to the dateless image data extracted in step ST106 within the date-bearing image data list obtained in step ST104, the determination is negative, and the date assignment process proceeds to step ST120. In step ST108, if there is date-bearing image data similar to the dateless image data extracted in step ST106 within the date-bearing image data list obtained in step ST104, the determination is positive, and the date assignment process proceeds to step ST110.

[0218] In step ST110, the determination unit 60P determines whether the date-bearing image data similar to the dateless image data extracted in step ST106 consists of multiple frames. In step ST110, if the date-bearing image data similar to the dateless image data extracted in step ST106 is a single frame, the determination is negative, and the date assignment process proceeds to step ST114. In step ST110, if the date-bearing image data similar to the dateless image data extracted in step ST106 consists of multiple frames, the determination is positive, and the date assignment process proceeds to step ST112.

[0219] In step ST112, the date export unit 60S extracts a date from each of a plurality of date-bearing image data that is similar to the dateless image data extracted in step ST106. Then, the date export unit 60S exports the date of the dateless image data based on the plurality of dates extracted from each of the plurality of date-bearing image data, and then the date assignment process proceeds to step ST116.

[0220] In step ST114, the date extraction unit 60S extracts the date from the date-bearing image data similar to the dateless image data extracted in step ST106, and then the date assignment process proceeds to step ST116.

[0221] In step ST116, the date assignment unit 60T generates date-assigned image data by assigning the date derived in step ST112 or the date extracted in step ST114 to the dateless image data extracted in step ST106. Then, the date assignment process proceeds to step ST118.

[0222] In step ST118, the image data sending unit 60U sends the date-assigned image data generated in step ST116 to the user equipment 12 of the data sending source via communication I / F52. Then, the date assignment process proceeds to step ST120.

[0223] In step ST120, the determination unit 60P determines whether the condition for ending the date assignment process (hereinafter also referred to as the "date assignment process end condition") is met. One example of a date assignment process end condition is that the server 14 has been given an instruction to end the date assignment process. This instruction is received, for example, by the receiving device 54. In step ST120, if the date assignment process end condition is not met, the determination is negative, and the date assignment process proceeds to step ST100. In step ST120, if the date assignment process end condition is met, the determination is positive, and the date assignment process ends.

[0224] Next, refer to Figure 41 The date assignment request processing executed by the CPU 42 of user equipment 12 will be explained. Furthermore, this explanation assumes that request data has already been generated by the request data generation unit 42F.

[0225] exist Figure 41 In the date assignment request processing shown, firstly, in step ST150, the request data sending unit 42G sends the request data generated by the request data generating unit 42F to the server 14 via the communication I / F28. Then, the date assignment request processing proceeds to step ST152.

[0226] If data is requested to be sent to server 14 by performing step ST150, then accordingly, as described above, server 14 generates date-assigned image data and sends the generated date-assigned image data to user equipment 12, the source of the requested data.

[0227] Therefore, in step ST152, the display control unit 42H determines whether date-assigned image data has been received via communication I / F28. If, in step ST152, date-assigned image data has not yet been received via communication I / F28, the determination is negative, and the date assignment process proceeds to step ST156. If, in step ST152, date-assigned image data has been received via communication I / F28, the determination is positive, and the date assignment process proceeds to step ST154.

[0228] In step ST154, the display control unit 42H displays a date-assigned image, represented by the date-assigned image data received via communication I / F28, on the display 34. Then, the date assignment request processing proceeds to step ST156. Because the date-assigned image is displayed on the display 34 after this step ST154 is executed, the date derived by the date derivation unit 60S is displayed to a specific user via the display 34. Furthermore, the display 34 is an example of a "prompt device" according to the technology of this invention.

[0229] In step ST156, the determination unit 60P determines whether the condition for ending the date assignment request processing (hereinafter also referred to as the "date assignment request processing end condition") is met. One example of the date assignment request processing end condition is that the user equipment 12 has been given an instruction to end the date assignment request processing. This instruction is received, for example, by the receiving device 32. In step ST156, if the date assignment request processing end condition is not met, the determination is negative, and the date assignment request processing proceeds to step ST150. In step ST156, if the date assignment request processing end condition is met, the determination is positive, and the date assignment request processing ends.

[0230] As explained above, in this first embodiment, in server 14, a list of dated image data is created by classifying multiple dated image data sets using CPU 60, and this list is associated with a specific user. The multiple dated image data sets are image data sets belonging to multiple users 16, including the specific user. A list of dated image data sets is created for each subject by classifying the multiple dated image data sets according to each subject represented by each of the multiple dated image data sets. A list of dated image data sets for subjects similar to those represented by the dated image data set of the specific user is associated with the specific user. Furthermore, from the list of dated image data sets associated with the specific user, the CPU 60 obtains dated image data sets for subjects similar to those represented by dateless image data provided by the specific user. Then, based on the date assigned to the obtained dated image data sets by CPU 60, the date assigned to the dateless image data sets is derived.

[0231] Therefore, according to this structure, it is possible to assign an appropriate date to image data without a date, compared to deriving a date from date-bearing image data only from date-bearing image data possessed by a specific user.

[0232] Furthermore, in this first embodiment, in the server 14, a list of dated image data is created by classifying multiple dated image data for each subject according to a manner in which visual changes over time can be determined. Therefore, according to this structure, compared to the case where multiple dated image data are classified for each subject according to a manner in which visual changes over time cannot be determined, a list of dated image data can be created for each subject that can be visually distinguished.

[0233] Furthermore, in this first embodiment, the server 14 creates a list of multiple date-bearing image data sets containing different dates. The date assigned to the date-bearing image data set is the date of capture. Therefore, according to this structure, compared to the case where the dates assigned to the multiple date-bearing image data sets contained in the date-bearing image data set are all the same, it is possible to assign an appropriate date to dateless image data sets.

[0234] Furthermore, in this first embodiment, the multiple users 16 constitute a group of users who have logged in and agree to share information containing dated image data. Therefore, according to this structure, compared to the case where all dated image data provided to the server 14 is processed regardless of whether the user has logged in and agrees to share the information containing dated image data, it is possible to suppress the use of dated image data provided to the server 14 by individuals who do not wish to share such information. In other words, it is possible to facilitate the inclusion of personal information.

[0235] Furthermore, in this first embodiment, an image data set is associated with each of a plurality of users 16, which are user groups that satisfy the condition that their image data sets are similar to each other. Therefore, according to this structure, compared to deriving dates from dated image data provided to the server 14 by persons who do not satisfy the condition that their image data sets are similar to each other with user 16, the processing load required to derive appropriate dates for dateless image data can be reduced. Moreover, by using only image data sets provided by user groups that satisfy the condition that their image data sets are similar to each other, personal information can be protected.

[0236] Furthermore, in this first embodiment, the multiple users 16 are a group of users who meet the condition that their registered user information is similar. Therefore, according to this structure, compared to the case where date-bearing image data provided to the server 14 from individuals whose registered user information is dissimilar is also used to derive the date, the processing load required to derive the appropriate date assigned to the dateless image data can be reduced. Moreover, by using only image data groups provided from user groups that meet the condition of similar registered user information, it is possible to help protect personal information.

[0237] Furthermore, in this first embodiment, in the server 14, the date-bearing image data of the production target set as a list of date-bearing image data from a plurality of date-bearing image data is limited to image data obtained by shooting within a range set according to GPS information. Therefore, according to this structure, compared to the case where the date is also derived by referring to date-bearing image data obtained by shooting outside the range set according to GPS information, the processing load required to derive the appropriate date assigned to the dateless image data can be reduced.

[0238] Furthermore, in this first embodiment, the date image is displayed on the display 34. That is, the date without date image data is displayed on the display 34. Therefore, according to this structure, the date without date image data can be visually perceived.

[0239] Furthermore, while no specific example of updating the date-bearing image data list was given in the first embodiment described above, the technology of the present invention is not limited thereto. For example, the CPU 42 may also update the date-bearing image data list associated with a specific user according to instructions received via the receiving device 32 or 54. In this case, the date-bearing image data list can be updated simply by removing several of the multiple date-bearing image data from the date-bearing image data list according to instructions received via the receiving device 32 or 54, or by appending the latest date-bearing image data uploaded from the user device 12 to the specific date-bearing image data list. Moreover, new date-bearing image data can also be generated based on image data already assigned a date (see reference). Figure 43To update a specific dated image data list, append it to that list.

[0240] Thus, by updating the list of dated image data associated with a specific user according to instructions provided from the outside, the content of the list of dated image data can be set to reflect the wishes of user 16.

[0241] Furthermore, while the first embodiment described above exemplifies uploading date-bearing image data from user device 12 to server 14, the technology of the present invention is not limited thereto. For example, image data sets associated with multiple users 16 may be pre-stored in memory 62. Alternatively, server 14 may retrieve image data sets associated with multiple users 16 from other devices (USB memory or memory card, etc.) via external I / F 58. In this case, the image data sets are also stored in memory 62 in a state where they are associated with multiple users 16.

[0242] Furthermore, in the first embodiment described above, an example of displaying a date on the display 34 without date image data was given, but the technology of the present invention is not limited thereto. For example, instead of a visible prompt based on the date on the display 34, or together with a visible prompt based on the date on the display 34, a prompt may also be displayed from the speaker 38 (see reference 38). Figure 2 It can output a sound indicating the date. Furthermore, the date can also be printed on recording media (e.g., paper) using a printer (illustration omitted).

[0243] Furthermore, in the first embodiment described above, an example was given of performing date-based image data creation processing and date assignment request processing via user equipment 12, and performing date-based image data list creation processing and date assignment processing via server 14. However, the technology of the present invention is not limited to this. For example, date-based image data creation processing, date assignment request processing, date-based image data list creation processing, and date assignment processing can be performed by a single device (e.g., user equipment 12, server 14, or personal computer). Also, at least one of date-based image data creation processing, date assignment request processing, date-based image data list creation processing, and date assignment processing can be performed by multiple devices. For example, date-based image data list creation processing and date assignment processing can be performed by different devices. Furthermore, for example, various processes can be performed on multiple servers including a server for storing image data with and without dates provided by multiple users 16, an image analysis server for performing image recognition processing, and a server for storing a list of date-based image data.

[0244] [Second Embodiment] In this second embodiment, an example of a method for updating a list of image data with dates will be described. Furthermore, in this second embodiment, components that are identical to those described in the first embodiment will be labeled with the same symbols, and their descriptions will be omitted. Structures and functions different from those in the first embodiment will be described.

[0245] The update of the image data list with date is performed by CPU 60 of server 14 (see reference). Figure 42 This is achieved through [method / method]. For example, such as... Figure 42 As shown, a date-based image data list update program 76 is stored in memory 62. CPU 60 reads the date-based image data list update program 76 from memory 62. Then, CPU 60 executes the date-based image data list update program 76 read from memory 62 on memory 64, functioning as a determination unit 60P, a date derivation unit 60S, a date assignment unit 60T, a date-based image data generation unit 60V, a similarity calculation unit 60W, an image quality determination unit 60X, and an image data appending unit 60Y. In other words, the date-based image data list update process is implemented by CPU 60 functioning as a determination unit 60P, a date derivation unit 60S, a date assignment unit 60T, a date-based image data generation unit 60V, a similarity calculation unit 60W, an image quality determination unit 60X, and an image data appending unit 60Y.

[0246] CPU60 updates the dated image data list by performing dated image data list update processing. When condition 1 is met, new image data, provided as new image data, is added to the dated image data list, thereby updating the dated image data list. Here, condition 1 refers to the condition that the image quality of the new image data is at or above the baseline image quality.

[0247] Furthermore, the CPU 60 updates the dated image data list by adding new image data to the dated image data list associated with a specific user when the second condition is met, by performing dated image data list update processing. Here, the second condition refers to the condition that the subject represented by the newly provided dated image data is dissimilar to the subject represented by the dated image data included in the dated image data list associated with the specific user. Additionally, the new image data is an example of the "first new image data" and "second new image data" involved in the technology of this invention.

[0248] The following provides a more detailed explanation of how to update a list of image data with dates. For example, as follows... Figure 43As shown, the date-assigned image data generation unit 60V generates new date-assigned image data based on the date-assigned image data generated by the date assignment unit 60T and the date-assigned image data used in the derivation of the dates contained in the date-assigned image data. This new date-assigned image data is an example of the "first new image data" and "second new image data" according to the technology of this invention, and is generated by updating the date-assigned image data used in the derivation of the dates contained in the date-assigned image data based on the date-assigned image data. More specifically, the image data contained in the date-assigned image data used in the derivation of the dates contained in the date-assigned image data is replaced with dateless image data contained in the date-assigned image data. Furthermore, the date in the attribute data (hereinafter also referred to as "attribute data for dateless images") contained in the date-assigned image data used in the derivation of the dates (dates contained in the date-assigned image data) assigned by the date assignment unit 60T to the date is replaced with the date contained in the date-assigned image data.

[0249] Data other than the date contained in the attribute data for dateless images is generated, for example, based on the user ID of the source providing the dateless image data and one or more attribute data of the exported object with date image data set by the date export unit 60S.

[0250] More specifically, the user ID included in the attribute data of the new date-bearing image data is the user ID of the source providing the date-bearing image data. Furthermore, when there is only one date-bearing image data for an export object set to date by the date export unit 60S, the data for each item other than the user ID and date (e.g., GPS information and Exif information) included in the attribute data of the date-bearing image data are the data included in the attribute data of the date-bearing image data. When there are multiple date-bearing image data for an export object set to date by the date export unit 60S, the data for each item other than the user ID and date are the average of the data included in the attribute data of the multiple date-bearing image data.

[0251] Furthermore, this example illustrates data based on the average of data contained in attribute data of multiple date-image data sets, but the technology of the present invention is not limited to this; it can also be based on the most frequent or median value of data contained in attribute data of multiple date-image data sets. Moreover, data other than the user ID and date contained in the attribute data of dateless image data that has the highest similarity to dateless image data can be used as part of the attribute data contained in new date-image data sets.

[0252] As an example, such as Figure 44As shown, the similarity calculation unit 60W calculates the similarity between the new date-bearing image data generated by the date-bearing image data generation unit 60V and the date-bearing image data of the exported object set to date by the date export unit 60S. The similarity is, for example, the average of the similarity between image data and the similarity between attribute data.

[0253] Image data similarity refers to the similarity between the image data contained in the new dated image data and the image data contained in the dated image data of the exported object set to date by the date export unit 60S. Attribute data similarity refers to the similarity between the attribute data contained in the new dated image data and the attribute data contained in the dated image data of the exported object set to date by the date export unit 60S.

[0254] Different weight values ​​can be assigned to the similarity between image data and the similarity between attribute data. The weight values ​​can be fixed or variable. Examples of variable values ​​include values ​​that can be changed according to instructions received by the receiving device 54, the number of frames of date-bearing image data of the exported object set by the date exporting unit 60S, values ​​set according to the degree of deviation (e.g., dispersion or standard deviation) of the date assigned to the date-bearing image data of the exported object set by the date exporting unit 60S, values ​​that change according to the user 16 who provides dateless image data, and / or values ​​that change periodically.

[0255] Furthermore, when the date-bearing image data of the exported object set to date by the date export unit 60S consists of multiple frames, for example, the similarity between the synthesized image data obtained by performing an arithmetic average of the multiple image data contained in the multiple frames of date-bearing image data of the exported object set to date by the date export unit 60S in pixel units and the image data contained in the new date-bearing image data can be used as the similarity between the image data. Also, when the date-bearing image data of the exported object set to date by the date export unit 60S consists of multiple frames, the similarity between the image data contained in one frame of date-bearing image data from the multiple frames of date-bearing image data and the image data contained in the new date-bearing image data can be used as the similarity between the image data.

[0256] The determination unit 60P determines whether the similarity calculated by the similarity calculation unit 60W is outside a predetermined range. This predetermined range can be a fixed value or a variable value. Examples of variable values ​​include values ​​that can be changed based on instructions received by the receiving device 54, the number of frames of date-bearing image data of the exported object set by the date export unit 60S, values ​​set based on the degree of deviation (e.g., dispersion or standard deviation) of the date assigned to the date-bearing image data of the exported object set by the date export unit 60S, values ​​that change based on the user 16 who provides dateless image data, and / or values ​​that change periodically.

[0257] When it is determined that the similarity calculated by the similarity calculation unit 60W is within a predetermined range, the determination unit 60P stands still until the next determination time arrives. When it is determined that the similarity calculated by the similarity calculation unit 60W is outside the predetermined range, the determination unit 60P determines the image quality of the image data indicated by the image quality determination unit 60X.

[0258] The image quality determination unit 60X determines the image quality of the image data contained in the new date-bearing image data generated by the date-bearing image data generation unit 60V, based on instructions from the determination unit 60P. Here, image quality refers to, for example, resolution and noise levels. Lower resolution results in lower image quality, and higher noise levels also result in lower image quality.

[0259] As an example, such as Figure 45 As shown, the determination unit 60P determines whether the image quality determined by the image quality determination unit 60X is at or above the reference image quality. Here, a fixed value is used as the reference image quality. However, this is just one example, and it could also be a variable value. Examples of such variable values ​​include values ​​that can be changed according to instructions received by the receiving device 54, the number of frames of date-bearing image data of the export object set by the date export unit 60S, a value set according to the degree of deviation (e.g., dispersion or standard deviation) of the date assigned to the date-bearing image data of the export object set by the date export unit 60S, a value that changes according to the user 16 who provides dateless image data, and / or a value that changes periodically.

[0260] When it is determined that the image quality determined by the image quality determination unit 60X is lower than the reference image quality, the determination unit 60P stands still until the next determination time arrives. When it is determined that the image quality determined by the image quality determination unit 60X is higher than the reference image quality, the determination unit 60P instructs the image data appending unit 60Y to append new image data with the date to the memory 62.

[0261] The image data appending unit 60Y, based on the instruction from the determination unit 60P, appends new date-bearing image data generated by the date-bearing image data generation unit 60V to a specific date-bearing image data list stored in the memory 62. As a result, the specific date-bearing image data list is updated. Here, the specific date-bearing image data list refers to the list of date-bearing image data containing date-bearing images of exported objects whose dates are set by the date-exporting unit 60S.

[0262] Next, refer to Figure 46 The process of updating the list of image data with dates, performed by CPU 60 of server 14, is described.

[0263] exist Figure 46 In the date-assigned image data list update process shown, firstly, in step ST200, the determination unit 60P determines whether date-assigned image data has been generated by the date assignment unit 60T. If, in step ST200, date-assigned image data has not yet been generated by the date assignment unit 60T, the determination is negative, and the date-assigned image data list update process proceeds to step ST202. If, in step ST200, date-assigned image data has been generated by the date assignment unit 60T, the determination is positive, and the date-assigned image data list update process proceeds to step ST214.

[0264] In step ST202, the date-assigned image data generation unit 60V generates new date-assigned image data based on the date-assigned image data generated by the date assignment unit 60T and the date-assigned image data used in the derivation of the dates contained in the date-assigned image data. Then, the date-assigned image data list update process proceeds to step ST204.

[0265] In step ST204, the similarity calculation unit 60W calculates the similarity between the new date-bearing image data generated in step ST202 and the date-bearing image data of the exported object set to date by the date export unit 60S. Then, the date-bearing image data list update process proceeds to step ST206.

[0266] In step ST206, the determination unit 60P determines whether the similarity calculated in step ST204 is outside a predetermined range. In step ST206, if the similarity calculated in step ST204 is within the predetermined range, the determination is negative, and the date image data list update process proceeds to step ST214. In step ST206, if the similarity calculated in step ST204 is outside the predetermined range, the determination is positive, and the date image data list update process proceeds to step ST208.

[0267] In step ST208, the image quality determination unit 60X determines the image quality of the image data contained in the new dated image data generated by the dated image data generation unit 60V, and then the dated image data list update process proceeds to step ST210.

[0268] In step ST210, the determination unit 60P determines whether the image quality determined in step ST208 is at or above the reference image quality. In step ST210, if the image quality determined in step ST208 is lower than the reference image quality, the determination is negative, and the image data list update process proceeds to step ST214. In step ST210, if the image quality determined in step ST208 is at or above the reference image quality, the determination is positive, and the image data list update process proceeds to step ST212.

[0269] In step ST212, the image data appending unit 60Y updates the specific dated image data list by appending new dated image data generated by the dated image data generation unit 60V to the specific dated image data list, and then the dated image data list update process proceeds to step ST214.

[0270] In step ST214, the determination unit 60P determines whether the condition for ending the date-based image data list update process (hereinafter also referred to as the "list update process end condition") is met. One example of the list update process end condition is that the server 14 has been given an instruction to end the date-based image data list update process. This instruction is received, for example, by the receiving device 54. In step ST214, if the list update process end condition is not met, the determination is negative, and the date-based image data list update process proceeds to step ST200. In step ST214, if the list update process end condition is met, the determination is positive, and the date-based image data list update process ends.

[0271] Next, refer to Figure 47A and Figure 47B The date assignment process involved in this second embodiment will be explained. Figure 47A and Figure 47B The flowchart shown is Figure 40 Compared to the flowchart shown, the difference lies in the fact that step ST108A is replaced by step ST108, and steps ST122 and ST124 are also present. Additionally, in Figure 47A In the middle, to and Figure 40 The flowcharts shown use the same step numbers for the same steps, and their descriptions are omitted.

[0272] exist Figure 47AIn step ST108A shown, the determination unit 60P determines whether to proceed in step ST104 or step ST124 (see reference). Figure 47B In step ST108A, if the list of dated image data obtained in step ST104 or ST124 contains dated image data similar to the dateless image data extracted in step ST106, the determination is affirmative, and the date assignment process proceeds to step ST110. In step ST108A, if the list of dated image data obtained in step ST104 or ST124 does not contain dated image data similar to the dateless image data extracted in step ST106, the determination is negative, and the date assignment process proceeds to step ST110. Figure 47B The step ST122 is shown.

[0273] exist Figure 47B In step ST122, the determination unit 60P determines whether a specific date-bearing image data list has been updated by performing date-bearing image data list update processing. In step ST122, if the specific date-bearing image data list has not yet been updated, the determination is negative, and the date assignment processing proceeds to... Figure 47A The step ST120 is shown. In step ST122, when a specific list of image data with dates is updated, the determination is affirmative, and the date assignment process proceeds to step ST124.

[0274] In step ST124, the image data list acquisition unit 60Q acquires the image data list with date corresponding to the user ID extracted in step ST102 from the memory 62, and then the date assignment process proceeds to step ST108A.

[0275] Then, CPU60 executes... Figure 47A The processes shown in steps ST108A to ST114 acquire dated image data of subjects similar to those represented by dateless image data of a specific user. Specifically, the CPU 60, by executing the date assignment process according to this second embodiment, acquires dated image data of subjects similar to those represented by dateless image data of a specific user from the updated dated image data list associated with that specific user, i.e., the specific dated image data list, provided that the dated image data list associated with that specific user has been updated.

[0276] Furthermore, here is an example of a method in which the image data list acquisition unit 60Q acquires dated image data of a subject similar to a subject represented by dateless image data from a specific dated image data list, provided that a specific dated image data list has been updated by performing dated image data list update processing. However, the technology of the present invention is not limited to this. Alternatively, new dated image data may be provided from the user equipment 12 to the server 14, and an updated dated image data list may be obtained by appending the newly provided dated image data to the dated image data list, and the newly provided dated image data may be acquired by the image data list acquisition unit 60Q.

[0277] As explained above, in this second embodiment, in server 14, when the quality of new date-bearing image data is above the baseline quality, the date-bearing image data list is updated by adding the new date-bearing image data to the date-bearing image data list. Therefore, according to this structure, compared to the case where new date-bearing image data is added to the date-bearing image data list regardless of its quality, it is possible to suppress the addition of date-bearing image data that is unsuitable for image recognition processing (e.g., date-bearing image data that cannot be identified as people and / or non-people by performing image recognition processing) to the date-bearing image data list.

[0278] Furthermore, in this second embodiment, in server 14, when a subject represented by new dated image data is dissimilar to a subject represented by dated image data included in a dated image data list associated with a specific user, new dated image data is added to the specific dated image data list. Therefore, according to this structure, compared to the case where new dated image data is added to the specific dated image data list regardless of whether the subject represented by new dated image data is similar to a subject represented by dated image data included in a dated image data list associated with a specific user, the increase in the amount of data in the dated image data list can be suppressed.

[0279] Furthermore, in this second embodiment, in server 14, subject-specific dated image data is retrieved from the specific dated image data list, provided that the list has been updated. This retrieves dated image data for subjects similar to those represented by dateless image data provided by a specific user. Therefore, according to this structure, compared to the case where, even though the specific dated image data list has been updated, it is impossible to retrieve dated image data for subjects similar to those represented by dateless image data provided by a specific user, the appropriate date assigned to the dateless image data can be exported instantly.

[0280] Furthermore, in the second embodiment described above, an example was given of adding new date-bearing image data to a specific list of date-bearing image data when the image quality determined by the image quality determination unit 60X is a reference image quality or higher. However, the technology of the present invention is not limited to this. For example, when the image quality of the new date-bearing image data exceeds the image quality of similar date-bearing image data (hereinafter also referred to as "similar image data") within a predetermined similarity range among multiple date-bearing image data in the specific list of date-bearing image data, the similar image data can be deleted from the specific list of date-bearing image data and the new date-bearing image data can be added.

[0281] Furthermore, in the above embodiments, examples of including multiple date-bearing image data in a date-bearing image data list have been described. However, as one example, such as Figure 48 As shown, the image data list creation unit 60M can also include feature data representing image data groups with the same date in a dated image data list, replacing the image data groups with the same date. Figure 48 In the example shown, as an example of a group of image data with the same date, multiple image data with the date May 19, 1985 are shown. However, the image data list production unit 60M extracts feature data from the multiple image data with the date May 19, 1985 and replaces the multiple image data with the date May 19, 1985 with feature data.

[0282] Here, as a first example of feature data, one could be data that is pre-set to be the minimum data capable of determining the summary of date-bearing image data included in a group of date-bearing image data (e.g., spatial frequency, contrast value, and brightness of the image data included in each group of date-bearing image data). As a second example of feature data, one could be human image data and non-human image data that are associated with the date-bearing image data included in the group of date-bearing image data. As a third example of feature data, one could be data that is pre-set to be the minimum data capable of determining the summary of human image data and non-human image data that are associated with the date-bearing image data (e.g., spatial frequency, contrast value, and brightness of the image data included in each group of date-bearing image data).

[0283] Thus, instead of groups of images with the same date, by including feature data representing the characteristics of groups of images with the same date in the list of images with the same date, the amount of data in the list of images with the same date can be reduced compared to the case where the list of images with the same date consists of only a number of images with the same date.

[0284] Furthermore, in the above embodiments, an example is given of obtaining a list of date-assigned image data corresponding to a user ID by performing a date assignment process (see reference). Figure 40 Step ST104 is illustrated, but it is also possible to assign a priority order to multiple lists of dated image data and retrieve them sequentially from the list of dated image data with the highest priority.

[0285] In this case, CPU60 executes Figure 49A and Figure 49B The date assignment process shown involves retrieving dated image data of subjects similar to those represented by undateable image data from multiple dated image data lists associated with a specific user, based on the priority of the dated image data contained in each list.

[0286] Furthermore, in the above embodiments, by performing date assignment processing, when there is no dated image data similar to dateless image data in the dated image data list associated with a specific user, the date is not assigned to the dateless image data. However, the technology of the present invention is not limited to this. For example, when there is no dated image data similar to dateless image data in the dated image data list associated with a specific user, dated image data similar to dateless image data can be obtained from the dated image data list associated with user 16 other than the specific user.

[0287] In this case, CPU60 executes Figure 49A and Figure 49B The date assignment process shown above, when the list of dated image data associated with a specific user does not contain dated image data for a subject similar to a subject represented by dateless image data, retrieves dated image data for a subject similar to a subject represented by dateless image data from a group of image data associated with at least one user other than the specific user.

[0288] Here, for reference Figure 49A and Figure 49B A more detailed explanation is given of an example of assigning priority to multiple lists of dated image data and retrieving them sequentially from the list of dated image data with higher priority, and an example of retrieving dated image data similar to that without dated image data from a list of dated image data associated with user 16 other than a specific user.

[0289] Figure 49A and Figure 49B The flowchart shown is Figure 40Compared to the flowchart shown, the difference lies in the fact that step ST104A is replaced by step ST104, step ST108B is replaced by step ST108, and steps ST130 to ST136 are also included. Additionally, in Figure 49A In the middle, to and Figure 40 The flowcharts shown use the same step numbers for the same steps, and their descriptions are omitted.

[0290] exist Figure 49A In step ST104A, the image data list acquisition unit 60Q retrieves from memory 62 the list of unprocessed dated image data with the highest priority corresponding to the user ID extracted in step ST102. "Highest priority" indicates the highest priority order. Furthermore, in this step ST104A, the unprocessed dated image data list refers to the list of dated image data that has not yet undergone the processing described in step ST108B.

[0291] In step ST108B, the determination unit 60P determines whether there is any date-bearing image data similar to the dateless image data extracted in step ST106 within the date-bearing image data list obtained in step ST104A. In step ST108B, if date-bearing image data similar to the dateless image data extracted in step ST106 exists within the date-bearing image data list obtained in step ST104A, the determination is affirmative, and the date assignment process proceeds to step ST110. In step ST108B, if date-bearing image data similar to the dateless image data extracted in step ST106 does not exist within the date-bearing image data list obtained in step ST104A, the determination is negative, and the date assignment process proceeds to step ST110. Figure 49B Step ST130 is shown.

[0292] In step ST130, the determination unit 60P determines whether a list of all date-bearing image data corresponding to the user ID extracted in step ST102 has been obtained in step ST104. If, in step ST130, a list of all date-bearing image data corresponding to the user ID extracted in step ST102 has not yet been obtained in step ST104, the determination is negative, and the date assignment process proceeds to... Figure 49A The step ST104A is shown. In step ST130, when a list of all date-bearing image data corresponding to the user ID extracted in step ST102 is obtained in step ST104, the determination is affirmative, and the date assignment process proceeds to step ST132.

[0293] In step ST132, the image data list acquisition unit 60Q acquires from the memory 62 an unprocessed list of dated image data corresponding to user IDs other than the user IDs extracted in step ST102 (hereinafter also referred to as "other user IDs"), and then proceeds to step ST134. Furthermore, in this step ST132, the unprocessed list of dated image data refers to the list of dated image data that has not yet undergone the processing in step ST134.

[0294] In step ST134, the determination unit 60P determines whether there is date-bearing image data similar to the dateless image data extracted in step ST106 within the date-bearing image data list obtained in step ST132. In step ST134, if date-bearing image data similar to the dateless image data extracted in step ST106 exists within the date-bearing image data list obtained in step ST132, the determination is affirmative, and the date assignment process proceeds to... Figure 49A The step ST110 is shown. In step ST134, if there is no date-bearing image data similar to the dateless image data extracted in step ST106 in the date-bearing image data list obtained in step ST132, the determination is negative, and the date assignment process proceeds to step ST136.

[0295] In step ST136, it is determined whether the number of times the list of dated image data was obtained in step ST132 (hereinafter also referred to as "number of lists obtained") has reached the upper limit. The upper limit can be a fixed value or a variable value. Examples of variable values ​​at this time include values ​​that can be changed according to instructions received through receiving device 54, the number of dated image data lists corresponding to other user IDs, values ​​that change according to user 16 who provided no dated image data, and / or values ​​that change periodically.

[0296] In step ST136, if the number of items retrieved has not yet reached the upper limit, the determination is negative, and the date assignment process proceeds to step ST132. In step ST136, if the number of items retrieved reaches the upper limit, the determination is positive, and the date assignment process proceeds to step ST132. Figure 49A The step ST120 is shown.

[0297] exist Figure 50 The image shows an example of multiple lists of dated image data assigned a priority order. Figure 50In the example shown, as multiple dated image data lists assigned a priority order, the 6th to 9th subject image data lists are shown, each associated with a user ID related to a specific user. The 6th subject image data list is a list of dated image data consisting of multiple dated image data containing the 6th subject. The 7th subject image data list is a list of dated image data consisting of multiple dated image data containing the 7th subject. The 8th subject image data list is a list of dated image data consisting of multiple dated image data containing the 8th subject. The 9th subject image data list is a list of dated image data consisting of multiple dated image data containing the 9th subject.

[0298] exist Figure 50 In the example shown, a young male is depicted as the sixth subject. Furthermore, in... Figure 50 In the examples shown, a prime-aged male is presented as the seventh subject. Furthermore, in... Figure 50 In the examples shown, one of the eighth subjects is a middle-aged or elderly man. Furthermore, in... Figure 50 In the examples shown, one of the subjects in the 9th subject is a late-stage elderly male.

[0299] The image data list production unit 60M produces subject image data lists 6 through 9 by performing image recognition processing on human image data associated with dated image data, classifying them into young males, middle-aged males, middle-aged and elderly males, and late-aged males. Typically, the physical conditions of young males, middle-aged males, middle-aged and elderly males, and late-aged males differ. Physical condition refers, for example, to the condition of the head (e.g., at least one of the face and hair). Generally, as humans age, the roundness of the face, the amount of hair, and the hair color change; therefore, image recognition processing is performed based on these characteristics, thereby enabling the classification of young males, middle-aged males, middle-aged and elderly males, and late-aged males.

[0300] The magnitude of changes in appearance with age in young men, middle-aged men, older men, and late-advanced men is generally in the order of "young men > middle-aged men > older men > late-advanced men." That is, the younger the age, the greater the changes in appearance with age. For example, the younger the age, the rounder the facial contours, and the greater the change in roundness compared to older individuals.

[0301] Therefore, dated image data featuring men from periods of significant age-related physical changes are more likely to show a greater degree of subject bias compared to dated image data featuring men from periods of less age-related physical changes. This means that obtaining dates from dated image data featuring men from periods of significant age-related physical changes is more likely to accurately and quickly determine dates assigned to dateless image data compared to obtaining dates from dated image data featuring men from periods of less age-related physical changes. Therefore, in Figure 26 In the example shown, the image data list of the 6th subject is assigned a priority number of 1, the image data list of the 7th subject is assigned a priority number of 2, the image data list of the 8th subject is assigned a priority number of 3, and the image data list of the 9th subject is assigned a priority number of 4.

[0302] In addition, Figure 26 The example shown illustrates a method of creating a list of dated image data for each decade of a man's life, but the technology of the present invention is not limited thereto. For example, a list of dated image data could also be created for each decade of a woman's life. Furthermore, the decades could be set in more detail or in a more coarse manner.

[0303] Furthermore, the technology of this invention is not limited to creating dated image data lists by each era. For example, dated image data lists with different priority orders can also be created based on each characteristic of user 16. Characteristics of user 16 may include, for example, family members, address, occupation, and interests. Moreover, the priority order can be set higher for dated image data lists with more frames of dated image data. Furthermore, the priority order can be set higher for dated image data lists with more frames of dated image data assigned different dates.

[0304] Thus, through execution Figure 49A and Figure 49B The date assignment process shown retrieves dated image data for subjects similar to those represented by undateable image data from multiple dated image data lists, based on the priority of the dated image data included in each dated image data list. Therefore, according to this structure, compared to the case where dated image data is retrieved from the dated image data lists regardless of the priority of the dated image data included in each dated image data list, the date assigned to undateable image data can be determined with high accuracy and speed.

[0305] Furthermore, through execution Figure 49A and Figure 49BThe date assignment process shown retrieves dated image data similar to dateless image data from a list of dated image data associated with user 16 (excluding the specific user). Therefore, according to this structure, compared to retrieving dated image data of exported objects set to dates only from a list of dated image data associated with a specific user, the possibility of assigning appropriate dates to dateless image data is improved.

[0306] Furthermore, in the above embodiments, examples of creating a dated image data list that mixes dated image data with images of people and dated image data with images of non-people have been described, but the technology of the present invention is not limited to this. For example, the dated image data can be divided into dated image data with images of people and dated image data with images of non-people, and the image data list creation unit 60M can only acquire the dated image data with images of non-people and use the acquired dated image data with images of non-people to create a dated image data list. According to this structure, the date assigned to the dated image data with images of people will not be assigned to the dateless image data.

[0307] Furthermore, in the above embodiments, examples have been given of limiting the dated image data to be used as the production object of the dated image data list, based on the similarity of the people reflected as subjects in the image data, the similarity of the geographical distribution of the camera locations, and the similarity of user information. However, the technology of the present invention is not limited thereto. For example, as an example, the CPU 60 can execute as follows: Figure 51 The process of creating a list of dated image data shows that the dated image data of the object set in the list of dated image data is restricted to image data of people with specific relationships.

[0308] Here, "persons with a specific relationship" refers to friends, family members, relatives, or employees of a specific organization. These individuals can be logged into the server 14 by each of multiple users 16 via user device 12. Furthermore, image recognition processing can be performed on image data sets stored by multiple user devices 12 via CPU 42 and / or 60 to determine image data with individuals with a specific relationship as subjects, and the determined image data is logged into the memory 62 of the server 14. Moreover, when the number of dated image data frames with the same person as a subject in a folder within a user device 12 is more than a specified number (e.g., 10 or more), the dated image data frames with the same person as a subject in the folder can be logged into the server 14 as image data containing individuals with a specific relationship.

[0309] Figure 51 The flowchart shown is Figure 39A The difference between the flowchart shown and the one presented is the presence of step ST19 between steps ST18 and ST20. Additionally, in... Figure 51 In the middle, to and Figure 39A The flowcharts shown use the same step numbers for the same steps, and their descriptions are omitted.

[0310] exist Figure 51 In step ST19, the deletion unit 60F deletes dated image data from the image data group stored in memory 62 that does not include a person with a specific relationship as a subject. Thus, the dated image data of the production object set in the multiple dated image data list is limited to image data containing a person with a specific relationship. Therefore, according to this structure, compared to the case where the dated image data of the production object set in the dated image data list is not limited to image data containing a person with a specific relationship, when a person with a specific relationship is included as a subject in the absence of dated image data, the appropriate date can be quickly derived.

[0311] Furthermore, in the above embodiments, the attribute data included in the date-bearing image data are exemplified as data including user ID, date, and GPS information, but the technology of the present invention is not limited to this. For example, multiple date-bearing image data may be assigned age determination information, and the CPU 60 may limit the date-bearing image data set as the creation object in the multiple date-bearing image data to the age determined by the age determination information.

[0312] In this case, as an example, such as Figure 52 As shown, this example demonstrates how to include age-specific information in attribute data. Figure 53 The image data list with dates shown can be processed. Figure 53 The flowchart shown is Figure 39E Compared to the flowchart shown, the difference lies in the presence of steps ST69A to ST69E between steps ST68 and ST70. Additionally, in Figure 53 In the middle, to and Figure 39E The flowcharts shown use the same step numbers for the same steps, and their descriptions are omitted.

[0313] exist Figure 53 In step ST69A, CPU 60 determines whether two or more sets of image data are stored in memory 62. If, in step ST69A, two or more sets of image data are not stored in memory 62, the determination is negative, and the process of creating the date-based image data list proceeds to step ST80 (see reference). Figure 39EIn step ST69A, when two or more sets of image data are stored in memory 62, the determination is affirmative, and the process of creating a dated image data list proceeds to step ST69B.

[0314] In step ST69B, CPU 60 extracts the age determination information from all image data groups stored in memory 62, and then the image data list creation process with date proceeds to step ST69C.

[0315] In step ST69C, CPU 60 uses all the date determination information extracted in step ST69B to calculate the date consistency between image data groups. Then, the image data list creation process with dates proceeds to step ST69D. Date consistency refers to the degree of consistency between date determination information. For example, the consistency between the dates 1970 and 1980 is 100%, the consistency between 1970 and 1980 and 1975 and 1985 is 50%, and the consistency between 1970 and 1980 and 1930 and 1940 is 0%.

[0316] In step ST69D, CPU 60 determines whether image data sets with an age consistency of 50% or higher are stored in memory 62. Furthermore, in this step ST69D, a fixed value is used as the predetermined consistency. However, this is only one example; the predetermined consistency used in step ST69D can also be a variable value. Examples of variable values ​​include values ​​that can be changed based on instructions received via receiving device 54 or values ​​that change periodically.

[0317] In step ST69D, if no image data set with a date consistency of at least a predetermined degree is stored in memory 62, the determination is negative, and the process of creating the image data list with date proceeds to step ST70. In step ST69D, if an image data set with a date consistency of at least a predetermined degree is stored in memory 62, the determination is positive, and the process of creating the image data list with date proceeds to step ST69E.

[0318] In step ST69E, CPU 60 deletes image data groups with an age consistency less than a predetermined consistency from memory 62, and then the image data list creation process with date proceeds to step ST70.

[0319] Thus, by performing steps ST69A to ST69E, the date-bearing image data of the production object set as part of the date-bearing image data list is restricted based on the date determined by the date determination information. Therefore, according to this structure, compared to the case where the date-bearing image data of the production object set as part of the date-bearing image data list is not restricted by date, the possibility of assigning appropriate dates to dateless image data can be improved.

[0320] Furthermore, in the above embodiments, an example has been described where a date image data creation program 68 and a date assignment request processing program 70 (hereinafter referred to as "terminal-side program" without further distinction when it is not necessary to distinguish these programs) are stored in the memory 44. However, the technology of the present invention is not limited thereto. For example, such as Figure 54 As shown, the terminal-side program can be stored on storage medium 100. Storage medium 100 is a non-transitory storage medium. As an example of storage medium 100, any portable storage medium such as an SSD or USB memory can be cited.

[0321] The terminal-side program stored in storage medium 100 is installed on computer 22. CPU 42 executes date image data creation processing according to date image data creation program 68, and executes date assignment request processing according to date assignment request processing program 70. Furthermore, for ease of explanation, the date image data creation processing and date assignment request processing will be referred to as "terminal-side processing" below, unless it is necessary to distinguish between them.

[0322] Furthermore, the terminal-side program can also be stored in the storage unit of other computers or servers connected to computer 22 via a communication network (not shown), and the terminal-side program can be downloaded and installed on computer 22 upon request from user equipment 12.

[0323] In addition, it is not necessary to store all terminal-side programs in the storage section or memory 44 of other computers or servers connected to computer 22; only a portion of the terminal-side programs can be stored.

[0324] exist Figure 54 In the example shown, CPU42 is a single CPU, but it could also be multiple CPUs. Furthermore, a GPU can be used instead of or alongside CPU42.

[0325] exist Figure 54 The example shown illustrates a computer 22, but the technology of the present invention is not limited thereto. Instead of computer 22, devices including ASICs, FPGAs, and / or PLDs can be used. Furthermore, instead of computer 22, a combination of hardware and software structures can also be used.

[0326] As the hardware resource for performing the terminal-side processing described in the above embodiments, various processors can be used, as shown below. For example, a general-purpose processor, i.e., a CPU, can function as a hardware resource that performs terminal-side processing by executing software, i.e., a program. Furthermore, as a processor, a dedicated circuit, i.e., a processor with a circuit structure specifically designed for performing a particular process, such as an FPGA, PLD, or ASIC, can be used. Memory is built into or connected to any processor, and terminal-side processing is performed by also using memory in any processor.

[0327] The hardware resources for performing terminal-side processing can consist of one of these various processors, or a combination of two or more processors of the same or different types (e.g., a combination of multiple FPGAs or a combination of a CPU and an FPGA). Furthermore, the hardware resources for performing terminal-side processing can be a single processor.

[0328] As examples of processors, firstly, there are processors composed of a combination of one or more CPUs and software, which function as hardware resources for performing terminal-side processing. Secondly, there are processors, such as SoCs, that use a single IC chip to implement the functionality of the entire system, which includes multiple hardware resources for performing terminal-side processing. Thus, terminal-side processing is implemented as a hardware resource using one or more of the aforementioned processors.

[0329] Furthermore, more specifically, the hardware architecture of these various processors can utilize electrical circuits that combine circuit elements such as semiconductor components. And the aforementioned terminal-side processing is merely one example. Therefore, it goes without saying that, without departing from the main point, unnecessary steps can be deleted, new steps can be added, or the processing order can be changed.

[0330] Furthermore, in the above embodiments, examples have been given of storing a date-image data list creation program 72, a date assignment processing program 74, and a date-image data list update program 76 (hereinafter referred to as "server-side program" without distinction when describing these programs) in the memory 62; however, the technology of the present invention is not limited thereto. For example, such as Figure 55 As shown, the server-side program can also be stored on storage medium 200. Storage medium 200 is a non-transitory storage medium. Examples of storage medium 200 include any portable storage medium such as an SSD or USB memory. Furthermore, the server-side program is an example of a "program" as understood in this invention.

[0331] The server-side program stored in storage medium 200 is installed on computer 50. CPU 60 executes date image data list creation processing according to date image data list creation program 72, date assignment processing according to date assignment processing program 74, and date image data list update processing according to date image data list update program 76. Furthermore, for ease of explanation, the date image data creation processing and date assignment request processing will be referred to as "server-side processing" below when there is no need to distinguish between them.

[0332] Furthermore, the server-side program can also be stored in the storage unit of other computers or servers connected to computer 50 via a communication network (not shown), and can be downloaded and installed on computer 50 upon request from server 14.

[0333] In addition, it is not necessary to store all server-side programs in the storage section or memory 62 of other computers or servers connected to computer 50; only a portion of the server-side programs can be stored.

[0334] exist Figure 55 In the example shown, CPU60 is a single CPU, but it can also be multiple CPUs. Furthermore, a GPU can be used instead of or alongside CPU60.

[0335] exist Figure 55 The example shown illustrates a computer 50, but the technology of the present invention is not limited thereto. Instead of computer 50, devices including ASICs, FPGAs, and / or PLDs can be used. Furthermore, instead of computer 50, a combination of hardware and software structures can also be used.

[0336] As the hardware resource for performing the server-side processing described in the above embodiments, various processors can be used, as shown below. For example, a general-purpose processor, i.e., a CPU, can function as a hardware resource that performs server-side processing by executing software, i.e., a program. Furthermore, as a processor, a dedicated circuit, i.e., an FPGA, PLD, or ASIC, can be used, which has a circuit structure specifically designed for performing a particular process. Memory is also built into or connected to any processor, and server-side processing is performed by using memory in any processor.

[0337] The hardware resources for performing server-side processing can consist of one of these various processors, or a combination of two or more processors of the same or different types (e.g., a combination of multiple FPGAs or a combination of a CPU and an FPGA). Furthermore, the hardware resources for performing server-side processing can be a single processor.

[0338] As examples of processors, firstly, there are processors composed of a combination of one or more CPUs and software, which function as hardware resources for performing server-side processing. Secondly, there are processors, such as SoCs, that use a single IC chip to implement the functionality of the entire system, which includes multiple hardware resources for performing server-side processing. In these cases, server-side processing is implemented as a hardware resource using one or more of the aforementioned processors.

[0339] Furthermore, and more specifically, the hardware architecture of these various processors can utilize electrical circuits that combine circuit elements such as semiconductor components. Moreover, the server-side processing described above is merely one example. Therefore, it goes without saying that, without departing from the main point, unnecessary steps can be removed, new steps can be added, or the processing order can be changed.

[0340] The descriptions and illustrations shown above are detailed explanations of the parts involved in the technology disclosed herein, and are merely one example of the technology disclosed herein. For example, the descriptions related to the structure, function, role, and effect described above are only one example of the structure, function, role, and effect of the parts involved in the technology disclosed herein. Therefore, it is self-evident that unnecessary parts may be deleted, new elements may be added, or substitutions may be made to the descriptions and illustrations shown above without departing from the spirit of the technology disclosed herein. Furthermore, in order to avoid complexity and facilitate understanding of the parts involved in the technology disclosed herein, descriptions related to common technical knowledge that do not require special explanation have been omitted from the descriptions and illustrations shown above, provided that the technology disclosed herein can be implemented.

[0341] In this specification, "A and / or B" has the same meaning as "at least one of A and B". That is, "A and / or B" means that it can be only A, only B, or a combination of A and B. Furthermore, in this specification, when three or more things are connected and expressed using "and / or", the same way of thinking applies as with "A and / or B".

[0342] All documents, patent applications and technical standards recorded in this specification are incorporated herein by reference to the same extent as those specifically and individually described herein.

[0343] The following notes further disclose the above implementation methods.

[0344] (Note 1)

[0345] An information processing apparatus includes a processor and memory built into or connected to the processor, wherein... The processor described above performs the following processing: Create a list of dated image data by classifying and assigning dates to multiple dated image data; Associate the above list of dated image data with specific users. Obtain dated image data of subjects similar to those represented by dateless image data of the aforementioned specific user from the aforementioned dated image data list associated with the aforementioned specific user; and Based on the date assigned to the acquired date-bearing image data, derive the date assigned to the dateless image data. The aforementioned multiple date-bearing image data sets comprise image data from multiple users, including the specific user mentioned above. The above-mentioned dated image data list is created by classifying the above-mentioned multiple dated image data into each subject according to each subject represented by each of the above-mentioned multiple dated image data. The list of dated image data for subjects similar to those represented by the dated image data of the aforementioned specific user is associated with the aforementioned specific user.

[0346] (Note 2)

[0347] According to the information processing apparatus described in Appendix 1, the aforementioned list of dated image data is created by classifying the aforementioned plurality of dated image data according to each subject in a manner that can determine visual changes over time.

[0348] (Note 3)

[0349] According to the information processing apparatus described in Appendix 1 or Appendix 2, the date assigned to the aforementioned plurality of image data with date data is the date of image capture. The above list of dated image data contains multiple dated image data sets with different shooting dates.

[0350] (Note 4)

[0351] According to any of the appendices 1 to 3, the information processing apparatus refers to a group of users who have logged in and agree to share information containing the aforementioned dated image data.

[0352] (Note 5)

[0353] According to any one of the appendices 1 to 4, the information processing apparatus wherein the image data set is associated with each of the aforementioned plurality of users. The aforementioned users are user groups that meet the condition that the image data groups are similar to each other.

[0354] (Note 6)

[0355] According to any of the appendices 1 to 5, the information processing apparatus, wherein the aforementioned plurality of users constitute a user group that satisfies the condition that the logged-in user information is similar.

[0356] (Note 7)

[0357] According to any one of the appendices 1 to 6, the aforementioned date-bearing image data is broadly divided into image data with people as the subject and image data without people as the subject. The processor described above performs the following processing: As mentioned above, only the image data without people is obtained as the date-bearing image data; The above-mentioned list of dated image data was created using the acquired image data without people.

[0358] (Postscript 8)

[0359] According to any one of Appendix 1 to Appendix 7, the information processing apparatus wherein the processor restricts the dated image data that is set as the production object of the dated image data list among the plurality of dated image data to image data containing persons with a specific relationship.

[0360] (Note 9)

[0361] According to any one of Appendix 1 to Appendix 8, the aforementioned plurality of date-bearing image data includes image data that provides location determination information for determining the camera position. The processor restricts the dated image data that is set as the production object in the above-mentioned dated image data list from the above-mentioned plurality of dated image data to image data obtained by shooting within the range set according to the above-mentioned location determination information.

[0362] (Postscript 10)

[0363] According to any one of Appendix 1 to Appendix 9, the information processing apparatus is provided with date determination information capable of determining the year of the user, which applies to the aforementioned plurality of date-bearing image data. The processor limits the date image data of the above-mentioned multiple date image data to the production object set in the above-mentioned date image data list based on the date determined according to the above-mentioned date determination information.

[0364] (Postscript 11)

[0365] According to any one of Appendix 1 to Appendix 10, in an information processing apparatus, when the dated image data list associated with the specific user does not contain dated image data relating to a subject similar to the subject represented by the dateless image data, the processor obtains the dated image data relating to a subject similar to the subject represented by the dateless image data from an image data group associated with at least one user other than the specific user among the plurality of users.

[0366] (Postscript 12)

[0367] According to any one of Appendix 1 to Appendix 11, the processor updates the dated image data list by adding the first new image data to the dated image data list, provided that the first new image data is provided as new image data and the image quality of the first new image data is at least as a baseline image quality.

[0368] (Postscript 13)

[0369] According to any one of Appendix 1 to Appendix 12, in an information processing apparatus, when a subject represented by a second new image data provided as the aforementioned date-bearing image data is dissimilar to a subject represented by the aforementioned date-bearing image data included in the aforementioned date-bearing image data list associated with the aforementioned specific user, the processor updates the aforementioned date-bearing image data list by adding the aforementioned second new image data to the aforementioned date-bearing image data list associated with the aforementioned specific user.

[0370] (Postscript 14)

[0371] According to the information processing apparatus described in Appendix 12 or Appendix 13, the processor obtains dated image data about a subject similar to a subject represented by dateless image data of the aforementioned specific user from the updated dated image data list associated with the aforementioned specific user, provided that the dated image data list associated with the aforementioned specific user has been updated.

[0372] (Postscript 15)

[0373] According to any one of Appendix 1 to Appendix 14, the processor includes feature data representing the characteristics of a group of identical date image data with the same date in the plurality of date image data in the list of date image data, replacing the group of identical date image data.

[0374] (Postscript 16)

[0375] According to any one of Appendix 1 to Appendix 15, in the information processing apparatus, when a plurality of the aforementioned dated image data lists are associated with the aforementioned specific user, the processor sequentially obtains dated image data about a subject similar to the subject represented by the aforementioned dateless image data from the plurality of dated image data lists associated with the aforementioned specific user, based on the dated image data list with the highest priority of the image data contained in each of the dated image data lists.

[0376] (Postscript 17)

[0377] According to any of the appendices 1 to 16, the information processing apparatus wherein the processor displays the derived date on the display device.

[0378] (Postscript 18)

[0379] According to any one of Appendix 1 to Appendix 17, the information processing apparatus wherein the processor updates the list of dated image data associated with the specific user in accordance with instructions received by the receiving device.

[0380] (Postscript 19)

[0381] According to any one of Annexes 1 to 18, the information processing apparatus wherein the processor creates a list of dated image data for each subject by classifying the plurality of dated image data for each subject according to each subject, which includes physical conditions that arise with age.

[0382] (Postscript 20)

[0383] According to the information processing device described in Appendix 19, the aforementioned physical condition includes the condition of the head, which comprises at least one of the face and hair.

[0384] (Postscript 21)

[0385] According to any one of Appendix 1 to Appendix 20, the information processing apparatus wherein the processor performs the following processing: when a plurality of the aforementioned date-bearing image data lists are associated with the aforementioned specific user, the processor sequentially acquires date-bearing image data about a subject similar to the subject represented by the aforementioned dateless image data from the date-bearing image data list with the most frames of date-bearing image data among the plurality of date-bearing image data lists associated with the aforementioned specific user.

[0386] (Postscript 22)

[0387] According to the information processing apparatus described in Appendix 21, the number of images with date data is the number of images with date data assigned different dates.

[0388] (Postscript 23)

[0389] The information processing apparatus described in any one of Annexes 1 to 22, wherein the image data set is associated with each of the plurality of users. The aforementioned users constitute a group of users that meet the condition that the image data sets are similar to each other. The condition that the aforementioned image data sets are similar to each other includes the condition that the aforementioned image data sets contain a predetermined number of images with the same person.

[0390] (Postscript 24)

[0391] According to the information processing apparatus described in Appendix 23, the aforementioned image data set is assigned location determination information that determines the camera position. The condition that the aforementioned image data sets are similar to each other includes the condition that the geographical distribution of the aforementioned camera locations among the aforementioned image data sets is similar, based on the location determination information.

[0392] (Postscript 25)

[0393] An information processing method includes the following steps: Create a list of dated image data by classifying and assigning dates to multiple dated image data; Associate the above list of dated image data with a specific user; Obtain dated image data of subjects similar to those represented by dateless image data of the aforementioned specific user from the aforementioned dated image data list associated with the aforementioned specific user; and Based on the date assigned to the acquired date-bearing image data, derive the date assigned to the dateless image data. The aforementioned multiple date-bearing image data sets comprise image data from multiple users, including the specific user mentioned above. The above-mentioned dated image data list is created by classifying the above-mentioned multiple dated image data into each subject according to each subject represented by each of the above-mentioned multiple dated image data. The list of dated image data for subjects similar to those represented by the dated image data of the aforementioned specific user is associated with the aforementioned specific user.

[0394] (Postscript 26)

[0395] A program that causes a computer to perform the following processes: The above processing will be carried out as follows: Create a list of dated image data by classifying and assigning dates to multiple dated image data; Associate the above list of dated image data with specific users. Obtain dated image data of subjects similar to those represented by dateless image data of the aforementioned specific user from the aforementioned dated image data list associated with the aforementioned specific user; and Based on the date assigned to the acquired date-bearing image data, derive the date assigned to the dateless image data. The aforementioned multiple date-bearing image data sets comprise image data from multiple users, including the specific user mentioned above. The above-mentioned dated image data list is created by classifying the above-mentioned multiple dated image data into each subject according to each subject represented by each of the above-mentioned multiple dated image data. The list of dated image data for subjects similar to those represented by the dated image data of the aforementioned specific user is associated with the aforementioned specific user.

Claims

1. An information processing device comprising: Processor; and Memory, which is built into or connected to the processor. The processor performs the following processing: A list of dated image data is created by classifying multiple dated image data sets; and Associate the list of dated image data with a specific user. The multiple date-bearing image data are image data of multiple users, including the specific user. The list of dated image data is created for each subject by classifying the plurality of dated image data according to each subject represented by the individual dated image data. The list of dated image data associated with the following subjects is associated with the specific user, and the subjects are similar to those represented by the dated image data of the specific user.

2. The information processing apparatus according to claim 1, wherein, The list of dated image data is created by classifying the plurality of dated image data according to each subject whose appearance changes over time in a visually determinable manner.

3. The information processing apparatus according to claim 1, wherein, The dates assigned to the multiple date-bearing image data are the dates of the images taken. The dated image data list contains multiple dated image data sets with different shooting dates.

4. The information processing apparatus according to any one of claims 1 to 3, wherein, The multiple users are a group of users who meet the following condition: they have registered to agree to share information containing the dated image data.

5. The information processing apparatus according to any one of claims 1 to 3, wherein, Each of the multiple users corresponds to a group of image data. The multiple users are user groups that satisfy the condition that the image data groups are similar to each other.

6. The information processing apparatus according to any one of claims 1 to 3, wherein, The multiple users are a group of users who meet the condition that their registered user information is similar.

7. The information processing apparatus according to any one of claims 1 to 3, wherein, The date-bearing image data is divided into image data with people as the subject and image data without people as the subject. The processor performs the following processing: As the dated image data, only the image data without people is obtained; and The acquired image data without people is used to create the dated image data list.

8. The information processing apparatus according to any one of claims 1 to 3, wherein, The processor restricts the dated image data of the production object set in the list of dated image data to image data of people with specific relationships.

9. The information processing apparatus according to any one of claims 1 to 3, wherein, The plurality of date-bearing image data includes image data for which location determination information is assigned to determine the camera's position. The processor restricts the dated image data of the production object set in the list of dated image data to image data obtained by shooting within a range specified according to the location determination information.

10. The information processing apparatus according to any one of claims 1 to 3, wherein, The multiple date-bearing image data are assigned date-determination information that can identify the user's era. The processor limits the dated image data of the production object set in the dated image data list to the date determined according to the date determination information.

11. The information processing apparatus according to any one of claims 1 to 3, wherein, When the list of dated image data associated with the specific user does not contain dated image data relating to a subject, the processor obtains the dated image data relating to a subject similar to the subject represented by the dateless image data of the specific user from the image data group corresponding to at least one user other than the specific user among the plurality of users.

12. The information processing apparatus according to any one of claims 1 to 3, wherein, The processor updates the dated image data list by adding the first new image data to the dated image data list, provided that the first new image data is provided as new image data and the image quality of the first new image data is at least as a baseline image quality.

13. The information processing apparatus according to any one of claims 1 to 3, wherein, When a subject represented by a second new image data provided as dated image data is dissimilar to a subject represented by dated image data included in the dated image data list associated with the specific user, the processor updates the dated image data list by adding the second new image data to the dated image data list associated with the specific user.

14. The information processing apparatus according to claim 12, wherein, The processor, conditional upon updating the list of dated image data associated with the specific user, retrieves dated image data from the updated list of dated image data associated with the specific user that relates to a subject similar to a subject represented by dateless image data of the specific user.

15. The information processing apparatus according to any one of claims 1 to 3, wherein, The processor includes feature data representing the characteristics of a group of identical date image data that has been assigned the same date in the list of date image data, replacing the identical date image data group.

16. The information processing apparatus according to any one of claims 1 to 3, wherein, When multiple lists of dated image data are associated with a specific user, the processor sequentially retrieves dated image data related to a subject from the higher-priority list of the multiple lists of dated image data associated with the specific user. This priority is based on the priority of the image data contained in each list of dated image data, and the subject is similar to a subject represented by the dateless image data of the specific user.

17. The information processing apparatus according to any one of claims 1 to 3, wherein, The processor updates the list of dated image data associated with the specific user according to instructions received via the receiving device.

18. An information processing method, comprising the following steps: A list of dated image data is created by classifying multiple dated image data sets; and Associate the list of dated image data with a specific user. The multiple date-bearing image data are image data of multiple users, including the specific user. The list of dated image data is created for each subject by classifying the plurality of dated image data according to each subject represented by the individual dated image data. The list of dated image data associated with the following subjects is linked to the specific user, which is similar to the subject represented by the dated image data of the specific user.

19. A computer program product for causing a computer to perform the following processes: The process includes the following steps: A list of dated image data is created by classifying multiple dated image data sets; and Associate the list of dated image data with a specific user. The multiple date-bearing image data are image data of multiple users, including the specific user. The list of dated image data is created for each subject by classifying the plurality of dated image data according to each subject represented by the individual dated image data. The list of dated image data associated with the following subjects is linked to the specific user, which is similar to the subject represented by the dated image data of the specific user.

Images