System

Abstract

A system is provided.SOLUTION: A system comprising: means for capturing film photographs as digital images with a scanner; means for automatically sorting digital images with date information; means for predicting and sorting the age and time of photography of digital images without date information; means for selecting the most appropriate from among similar digital images; and means for automatically displaying digital images based on a designated person or theme.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Nowadays, many people own a large number of film photographs taken in the past. However, film photographs have physical limitations, are at high risk of being scattered or deteriorated, and efficiently organizing large quantities of photographs is difficult. Organizing photographs in the proper chronological order is particularly time-consuming when there are no dates. Furthermore, when there are many similar photographs, determining which one to select takes time and effort. Therefore, there is a need for a system that can efficiently digitize film photographs and automatically sort them chronologically. [Means for solving the problem]

[0005] The present invention provides a means for automatically sorting date-bearing digital images by scanning film photographs. It also includes a means for predicting the year and time of photographing for digital images without date information using a generative AI model and sorting them accordingly. It also includes a means for selecting the best image from among similar digital images, which is achieved by using an evaluation module that evaluates factors such as resolution, focus, lighting, facial expression, and composition. It also provides a means for automatically displaying digital images based on specified people or themes. This invention enables users to efficiently digitize film photographs and automatically generate organized albums.

[0006] A "scanner" is a device that converts analog film photographs into digital images.

[0007] "Digital images" are image data of film photographs stored in a digital format.

[0008] "Date information" is data indicating the date and time a photo was taken, and may exist as a date stamp or metadata on the image.

[0009] "Sorting" is the process of arranging data in order according to specific criteria.

[0010] A "generative AI model" is an algorithm that uses artificial intelligence technology to generate and analyze information from specific data.

[0011] "Photography year" is information that indicates the specific year or time when the photograph was taken.

[0012] "Prediction" is the act of making reasonable estimates or hypotheses based on unknown information or data.

[0013] The "optimal" is the object that has the most desirable characteristics and conditions among multiple options.

[0014] An "evaluation module" is a software component that evaluates the quality or characteristics of data based on specific criteria.

[0015] A "person recognition module" is an algorithm for identifying and identifying people in an image.

[0016] A "theme analysis module" is a software component for identifying related images for a specified theme. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] This invention relates to a system for efficiently digitizing film photographs and automatically organizing and displaying them.

[0039] Photo scanning process

[0040] The user first places a film photo in the scanner and performs the scan operation. The scanner scans the photo and generates a digital image. This digital image is saved in the device's local storage and then sent to the server.

[0041] Sorting photos with dates

[0042] The server extracts date information from the received digital images. If date information is included, it identifies the specific date of the photo by analyzing the date stamp and metadata in the photo. The digital images with this date information are sorted and stored in chronological order.

[0043] Dating and sorting undated photos

[0044] The server analyzes digital images that do not contain date information using a generative AI model. The generative AI model evaluates features such as facial expressions, backgrounds, clothing, and buildings to predict the age and time period in which the photo was taken. This allows digital images without date information to be sorted based on their estimated age.

[0045] Selecting the best photo

[0046] The server uses a clustering algorithm to group multiple digital images taken at the same event or scene. Within each group, an evaluation module evaluates resolution, focus, lighting, facial expression, composition, and other factors to select the best quality photos. The selected best photos are then stored in a database.

[0047] Display albums based on specified people or themes

[0048] Users specify themes and people they want to include in their albums, such as "family vacations" or "children growing up," or specific people. The server searches the database for relevant photos based on the specified criteria. The person recognition module identifies photos containing the specified people, and the theme analysis module identifies photos that match the specified theme.

[0049] Photos selected from the search results are laid out in an album format and displayed to the user, allowing users to easily digitize their old film photos, efficiently organize them, and immediately use them as albums tailored to their needs.

[0050] Specific examples

[0051] Let's say a user wants to create an album themed around a family trip in the 1980s. First, the user places film photos stored at home in a scanner and scans all of the photos. The digital images obtained by the scanner are sent to a server. The server sorts photos that contain date information by date, and for photos without date information, it uses a generative AI model to identify photos from around the 1980s.

[0052] The server then uses a clustering algorithm to group photos taken at the same event or scene and select the best photos from each group.The user selects "family vacations in the 1980s," and the server uses a person recognition module and a theme analysis module to search for and select related photos, lay them out in an album format, and display them to the user.

[0053] Through the above process, users can efficiently digitize old film photographs without much effort and easily create albums organized chronologically or by theme.

[0054] The processing flow will be explained below.

[0055] Step 1:

[0056] The user places the film photo in the scanner and presses the start scan button.

[0057] Step 2:

[0058] The device operates a scanner to scan film photographs and generate digital images.

[0059] Step 3:

[0060] The device stores the generated digital images in local storage.

[0061] Step 4:

[0062] The terminal transmits the stored digital images to a server.

[0063] Step 5:

[0064] The server analyzes the received digital images and extracts date information by analyzing the date stamps and metadata contained in the images.

[0065] Step 6:

[0066] The server classifies the digital images into those with date information and those without date information.

[0067] Step 7:

[0068] The server sorts the digital images with date information in chronological order.

[0069] Step 8:

[0070] The server uses a generative AI model to predict the age and time of a photo taken for digital images that do not have date information.

[0071] Step 9:

[0072] The server sorts undated digital images based on their predicted age.

[0073] Step 10:

[0074] The server uses a clustering algorithm to group digital images taken at the same event or scene.

[0075] Step 11:

[0076] For each group, the server selects the highest quality photo using an evaluation module that evaluates resolution, focus, lighting, facial expression, composition, etc.

[0077] Step 12:

[0078] The server stores the selected best photos in a database.

[0079] Step 13:

[0080] Users specify themes or people they want to include in their albums (e.g., "Family Vacations in the 1980s" or "Children Growing Up").

[0081] Step 14:

[0082] The server searches the database for relevant photos based on the specified criteria.

[0083] Step 15:

[0084] The server uses a people recognition module to identify photos that contain the specified people.

[0085] Step 16:

[0086] The server uses a theme analysis module to identify photos that match a specified theme.

[0087] Step 17:

[0088] The server lays out the photos selected from the search results in an album format.

[0089] Step 18:

[0090] The server generates the album and displays it to the user.

[0091] By following these steps, users can efficiently digitize their old film photos and easily create albums organized chronologically or thematically.

[0092] Example 1

[0093] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0094] Digitizing film photographs is time-consuming, and organizing and searching for specific photographs is complicated. Furthermore, it is difficult to identify the age or time period of photographs that do not have date information, and it is also extremely difficult to select the most suitable photographs from a large number of photographs. A system that can solve these problems is needed.

[0095] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0096] In this invention, the server includes means for capturing film photographs as digital images, means for saving the digital images in local storage, means for transmitting the saved digital images to the server, means for automatically sorting digital images with date information, means for predicting and sorting digital images without date information by the year or time they were taken, means for clustering digital images taken in similar scenes, means for selecting the most suitable digital image from the clustering results, and means for searching and displaying digital images based on a specified person or theme. This enables users to efficiently digitize film photographs and easily organize and search for them based on year or theme.

[0097] "Film photography" is an analogue form of photography taken using photographic film.

[0098] "Digital image" refers to an image file represented in an electronic data format.

[0099] "Local storage" refers to a storage medium, either internal or external to a computer, that is used to store data.

[0100] A "server" is a computer system that stores, processes, and manages data over a network.

[0101] "Date information" is data indicating the date and time a photo was taken, and is stored as metadata or text information included in the image.

[0102] "Clustering" is a method of grouping data based on similarity.

[0103] A "generative AI model" is an algorithm that uses artificial intelligence technology to analyze data and generate new information.

[0104] A "clustering algorithm" is a computational method for classifying data into groups based on similarity.

[0105] The "optimal digital image" is the digital image that is judged to be of the highest quality based on multiple evaluation criteria, including resolution, focus, lighting, facial expression, and composition.

[0106] A "person recognition module" is a software component for identifying specific people in an image.

[0107] A "theme analysis module" is a software component that analyzes the content of images and identifies matches to specific themes.

[0108] This invention is a system for efficiently digitizing film photographs and automatically organizing and displaying them. This system performs a series of steps, from scanning film photographs to generating, saving, and processing digital images. Specific embodiments are described below.

[0109] 1. Scanning film photos

[0110] First, the user places the film photograph into the scanner and performs the scan operation. The scanner used can be a general photo scanner (e.g., EPSON Perfection V600). The scanner converts the film photograph into a digital image.

[0111] 2. Local storage of digital images

[0112] The device temporarily stores digital images acquired from the scanner in local storage, which is implemented using the Windows or macOS file system.

[0113] 3. Sending digital images to the server

[0114] The device sends the stored digital images to a server using the HTTP protocol, for example using the Python requests library.

[0115] 4. Extracting Date Information

[0116] The server extracts date information from received digital images, using an image processing library (e.g., OpenCV) to detect date stamps in the images and the Pillow library to parse metadata, e.g., EXIF ​​information.

[0117] 5. Sorting photos with dates

[0118] The server sorts the digital images with date information in the order of the extracted dates, using Python's standard sort function.

[0119] 6. Dating undated photographs

[0120] The server uses generative AI models to estimate the age of undated digital images, for example, by using image classification libraries such as TensorFlow or PyTorch to evaluate features of the photo (such as facial expression, background, clothing, and buildings) to estimate the age.

[0121] 7. Photo Clustering

[0122] The server uses clustering algorithms to group digital images taken during the same event or scene, such as k-means clustering or DBSCAN (Density-Based Spatial Clustering of Applications with Noise).

[0123] 8. Selecting the best photo

[0124] The server evaluates each group's photos based on their resolution, focus, lighting, facial expression, composition, etc., and selects the highest quality photo. The evaluation module scores each item using an image evaluation library.

[0125] 9. Enter album conditions

[0126] Users can select themes and people they want to include in their albums, such as "family vacations" or "children growing up," through their device, either via a web interface or a mobile app.

[0127] 10. Search by people or topics

[0128] The server searches the database for relevant photos based on the specified criteria. The person recognition module performs face recognition processing using, for example, the dlib library, and the theme analysis module identifies photos that match a theme using an image captioning model.

[0129] 11. Album Creation and Display

[0130] The server then selects photos from the search results, lays them out in an album format, and displays them to the user. The layout process can be performed using an HTML template engine (e.g., Jinja2) to display them as a web page, or using a PDF generation library (e.g., ReportLab) to output them as PDF.

[0131] Specific examples

[0132] For example, if a user wants to create an album themed around a family trip in the 1980s, they first load the film photos stored at home into a scanner and scan all of the photos. The digital images obtained by the scanner are sent to a server, which sorts photos with date information by date and uses a generative AI model to identify photos without date information as being from around the 1980s. The server then uses a clustering algorithm to group photos taken at the same event or scene and selects the most suitable photo from each group. The user selects "family trip in the 1980s," and the server uses a person recognition module and theme analysis module to search for and select related photos, lay them out in an album format, and displays them to the user.

[0133] Prompt Sentence Examples

[0134] Example 1: "Create a family vacation album from the 1980s. Scan and digitize film photos, parse the date information, and arrange them chronologically. For undated photos, use a generative AI model to estimate the age and sort them appropriately."

[0135] Example 2: "Scan film photos from a family vacation. Send the resulting digital images to a server, group photos from the same event, and select and store the best ones."

[0136] By processing in this manner, users can efficiently digitize film photographs and easily organize and search them by era or theme.

[0137] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0138] Step 1:

[0139] The user places the film photograph in the scanner and performs the scanning operation.

[0140] Input: Film photo

[0141] Output: Digital image

[0142] Specific operation: Scan film photographs using a scanner (e.g., EPSON Perfection V600) and convert them into digital images.

[0143] Step 2:

[0144] The device stores the scanned digital images in local storage.

[0145] Input: Digital image

[0146] Output: Digital image saved to local storage

[0147] What it does: It uses the Windows or macOS file system to temporarily store scanned digital images.

[0148] Step 3:

[0149] The terminal transmits the stored digital images to a server.

[0150] Input: Digital image stored in local storage

[0151] Output: Digital image sent to server

[0152] Specific operation: Uploads a file to the server using the HTTP protocol and, for example, the Python requests library.

[0153] Step 4:

[0154] The server extracts date information from the received digital images.

[0155] Input: Digital image sent to the server

[0156] Output: Digital image with extracted date information

[0157] Specific operation: Detects date stamps in images using an image processing library (OpenCV), and analyzes metadata (EXIF information) using the Pillow library.

[0158] Step 5:

[0159] The server sorts the digital images containing date information in chronological order.

[0160] Input: Digital image with extracted date information

[0161] Output: Digital images sorted by date

[0162] What it does: Organizes digital images by date using Python's standard sorting functions.

[0163] Step 6:

[0164] The server uses generative AI models to estimate the age of undated digital images.

[0165] Input: undated digital image

[0166] Output: Dated digital image

[0167] Specific operation: Using image classification libraries such as TensorFlow and PyTorch, the features of the photo (facial expression, background, clothing, buildings, etc.) are evaluated to estimate the age.

[0168] Step 7:

[0169] The server uses a clustering algorithm to group digital images taken at the same event or scene.

[0170] Input: Dated digital image

[0171] Output: Clustered digital image groups

[0172] What it does: Group digital images using algorithms such as k-means clustering and DBSCAN.

[0173] Step 8:

[0174] The server evaluates the resolution, focus, lighting, facial expression, composition, etc. within each group and selects the highest quality photo.

[0175] Input: Clustered digital image groups

[0176] Output: Optimal digital images

[0177] Specific operation: Using an image evaluation library, each item (resolution, focus, lighting, facial expression, composition, etc.) is scored and the highest quality photo is selected.

[0178] Step 9:

[0179] The user specifies the themes and people they want to include in the album through the terminal.

[0180] Input: Album theme or person

[0181] Output: specified conditions

[0182] What it does: Enter a topic (e.g., "family vacation") and people using the web interface or mobile app.

[0183] Step 10:

[0184] The server searches the database for relevant photos based on the specified criteria.

[0185] Input: Specified conditions (theme or person)

[0186] Output: Related photos found

[0187] What it does: It uses a facial recognition library (dlib) to identify photos containing specified people, and a theme analysis module to identify photos that match specified themes.

[0188] Step 11:

[0189] The server lays out the photos selected from the search results in an album format and displays them to the user.

[0190] Input: Related photos found

[0191] Output: Layout album

[0192] Specific behavior: Display as a web page using an HTML template engine (Jinja2), or output as PDF using a PDF generation library (ReportLab).

[0193] (Application example 1)

[0194] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0195] It is difficult to efficiently digitize film photographs and automatically organize and display them, and it takes a great deal of effort, especially when digitizing and providing old film photographs in brick-and-mortar stores. Organizing photographs with missing date information and selecting the most suitable photograph from multiple similar photographs are also difficult challenges.

[0196] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0197] In this invention, the server includes means for capturing film images as digital data using a scanner, means for automatically sorting digital data with date information, means for predicting and sorting the age or period of digital data without date information, means for selecting the most suitable digital data from similar digital data, means for automatically displaying digital data based on a specified target person or theme, means for use with a smartphone in a physical store, means for estimating the age of digital data without date information using an AI model, means for identifying target people using a face recognition module, and means for identifying photos that match a specified theme using a theme recognition module.This makes it possible to efficiently digitize old film photos and easily organize and display them in a physical store using a smartphone.

[0198] A "scanner" is a device that captures film images as digital data.

[0199] "Digital data" refers to digital image files captured from film images.

[0200] "Date information" refers to data related to the date and time of shooting, such as date labels and meta information that appear on the photo.

[0201] "Sorting" is the process of organizing and rearranging digital data based on specific criteria.

[0202] "Decade" refers to a specific range that indicates when a photo was taken, such as the 1980s.

[0203] "Time" refers to the specific period of time when the photograph was taken, such as a particular season or year.

[0204] An "AI model" refers to a model that uses artificial intelligence to analyze multiple factors and infer specific information from the data.

[0205] "Facial Recognition Module" refers to technology for identifying and recognizing human faces contained in digital data.

[0206] "Theme recognition module" refers to technology that analyzes the image content and themes contained in digital data and identifies information that matches a specified theme.

[0207] "Brick and mortar store" refers to a commercial establishment that offers goods or services in a physical location, such as a photo studio or retail store.

[0208] A "smartphone" refers to a mobile device that not only has the functions of a mobile phone but also has advanced computing power and multifunctionality.

[0209] This invention builds a system that uses a scanner, a server, a smartphone terminal, and multiple software modules to efficiently digitize, organize, and display film photographs.

[0210] The user first places the film image on the scanner and operates the scanner to capture the film image as digital data. The captured digital data is saved in local storage and then sent to the server.

[0211] The server analyzes the digital data and automatically sorts it using the date information if it contains it, and if it does not, it uses a generative AI model to predict the age and time period of the photo and sorts it accordingly.

[0212] The server then uses a clustering algorithm to group digital images taken at the same event or scene. For each cluster, the server evaluates resolution, focus, lighting, facial expression, composition, and other factors to select the highest quality images. This selected data is then stored in a database.

[0213] Furthermore, users can use their smartphones to search and display digital data based on specified people or themes. The face recognition module identifies people, and the theme recognition module identifies photos that match the specified theme. Digital data selected from the search results is laid out and displayed to the user in an album format.

[0214] For example, let's say a user wants to create an album themed around a family trip in the 1980s. First, the user places all the film photos stored at home in a scanner and scans them all. The digital data obtained by the scanner is sent to a server. The server sorts photos that contain date information by date, and for photos without date information, it uses a generative AI model to identify photos from around the 1980s.

[0215] The server then uses a clustering algorithm to group digital data taken at the same event or scene and selects the most suitable data from each group.The user selects "1980s family vacation" on their smartphone, and the server uses face and theme recognition modules to search for and select related digital data, lays it out in an album format, and displays it to the user.

[0216] Example prompts for generative AI models:

[0217] To estimate the age of a photograph, pay attention to the following:

[0218] The clothing and hairstyle of the people in the photo

[0219] Background and buildings

[0220] Photo Tone

[0221] For example, characteristics of the 1980s include brightly colored clothing, big hairstyles, and a particular architectural style.

[0222] Photo: [Link to photo data]

[0223] What era do you estimate the photo was taken in?

[0224] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0225] Step 1:

[0226] A user places a film image on a scanner and operates the scanner to capture the film image as digital data.

[0227] Input: Film images, scanner

[0228] Processing: The scanner scans the film images and generates digital data, which is then stored in local storage.

[0229] Output: Digital data stored in local storage

[0230] Step 2:

[0231] The terminal transmits the digital data captured by the scanner to the server.

[0232] Input: Digital data stored in local storage, terminal, server

[0233] Processing: The device reads the digital data from local storage and sends it to the server over the network.

[0234] Output: Digital data received by the server

[0235] Step 3:

[0236] The server extracts date information from the received digital data and sorts the data having the date information.

[0237] Input: Digital data received by the server

[0238] Processing: The server uses an image analysis engine (e.g., pytesseract) to extract date information from the digital data. Based on the extracted date information, the digital data is sorted in chronological order.

[0239] Output: Digital data sorted by date

[0240] Step 4:

[0241] The server uses a generative AI model to predict and sort the age and time period of digital data that does not have date information.

[0242] Input: Digital data without date information

[0243] Processing: The server passes the input data to the generative AI model to obtain a predicted age and time period. The digital data is then sorted based on the obtained age and time period information.

[0244] Output: Digital data sorted and dated

[0245] Step 5:

[0246] The server uses a clustering algorithm to group digital data captured at the same event or scene.

[0247] Input: Sorted digital data stored on the server

[0248] Processing: The server applies a clustering algorithm to group the digital data based on the features.

[0249] Output: Digital data grouped into clusters

[0250] Step 6:

[0251] The server selects the most suitable digital data within each cluster.

[0252] Input: Digital data grouped into clusters

[0253] Processing: The server uses an evaluation module to evaluate resolution, focus, lighting, facial expression, composition, etc. within each cluster and select the most suitable digital data.

[0254] Output: Optimal digital data selected for each cluster

[0255] Step 7:

[0256] The server searches and displays digital data based on the target person or theme specified by the user.

[0257] Input: Specified target person or theme, selected and sorted digital data

[0258] Processing: The server uses a face recognition module to identify the target person, and a theme recognition module to search for digital data matching the specified theme. The search results are then laid out in an album format and displayed to the user on their smartphone.

[0259] Output: Digital data in album format displayed on a smartphone

[0260] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0261] This invention is a system that efficiently digitizes film photographs and automatically organizes and displays them, and also has the ability to recognize the user's emotions and select and display the most appropriate images.

[0262] Photo scanning process

[0263] The user first places a film photo in the scanner and performs the scan operation. The scanner scans the photo and generates a digital image. This digital image is saved in the device's local storage and then sent to the server.

[0264] Sorting photos with dates

[0265] The server extracts date information from the received digital images. If date information is included, it identifies the specific date of the photo by analyzing the date stamp and metadata in the photo. The digital images with this date information are sorted and stored in chronological order.

[0266] Dating and sorting undated photos

[0267] The server analyzes digital images that do not contain date information using a generative AI model. The generative AI model evaluates features such as facial expressions, backgrounds, clothing, and buildings to predict the age and time period in which the photo was taken. This allows digital images without date information to be sorted based on their estimated age.

[0268] Selecting the best photo

[0269] The server uses a clustering algorithm to group multiple digital images taken at the same event or scene. Within each group, an evaluation module evaluates resolution, focus, lighting, facial expression, composition, and other factors to select the best quality photos. The selected best photos are then stored in a database.

[0270] Implementing the Emotion Engine

[0271] The server includes an emotion engine that recognizes the user's emotions. The emotion engine includes means for analyzing emotions from the user's facial expressions, voice, and actions. The analyzed emotion data is reflected in the selection and display order of photos.

[0272] Display albums based on specified people or themes

[0273] Users specify themes and people they want to include in their albums, such as "family vacations" or "children growing up," or specific people. The server searches the database for relevant photos based on the specified criteria. The person recognition module identifies photos containing the specified people, and the theme analysis module identifies photos that match the specified theme.

[0274] Emotion-based selection and display of optimal photos

[0275] The album layout and display order are dynamically changed based on the user's emotions as recognized by the emotion engine. For example, if the user feels nostalgic for the past, photos that best match that emotion will be displayed preferentially. Furthermore, the system will display more photos of fun events that reflect the user's joy and happiness.

[0276] Specific examples

[0277] Let's say a user wants to create an album themed around a family trip in the 1980s. First, the user places film photos stored at home in a scanner and scans all of the photos. The digital images obtained by the scanner are sent to a server. The server sorts photos that contain date information by date, and for photos without date information, it uses a generative AI model to identify photos from around the 1980s.

[0278] The server then uses a clustering algorithm to group photos taken at the same event or scene and select the best photos from each group.The user selects "family vacations in the 1980s," and the server uses a person recognition module and a theme analysis module to search for and select related photos, lay them out in an album format, and display them to the user.

[0279] Furthermore, while a user is browsing an album, the emotion engine recognizes the user's emotions and dynamically adjusts the photos and album layout based on those emotions. For example, if the user smiles, photos with lots of happy family memories will be prioritized.

[0280] Through this process, users can efficiently digitize their old film photos without much effort and create albums organized chronologically or by theme. Furthermore, by displaying photos optimally according to their emotions, users can enjoy even greater satisfaction.

[0281] The processing flow will be explained below.

[0282] Step 1:

[0283] The user places the film photo in the scanner and presses the start scan button.

[0284] Step 2:

[0285] The device operates a scanner to scan film photographs and generate digital images.

[0286] Step 3:

[0287] The device stores the generated digital images in local storage.

[0288] Step 4:

[0289] The terminal transmits the stored digital images to a server.

[0290] Step 5:

[0291] The server analyzes the received digital images and extracts date information by analyzing the date stamps and metadata contained in the images.

[0292] Step 6:

[0293] The server classifies the digital images into those with date information and those without date information.

[0294] Step 7:

[0295] The server sorts the digital images with date information in chronological order.

[0296] Step 8:

[0297] The server uses a generative AI model to predict the age and time of a photo taken for digital images that do not have date information.

[0298] Step 9:

[0299] The server sorts undated digital images based on their predicted age.

[0300] Step 10:

[0301] The server uses a clustering algorithm to group digital images taken at the same event or scene.

[0302] Step 11:

[0303] For each group, the server selects the highest quality photo using an evaluation module that evaluates resolution, focus, lighting, facial expression, composition, etc.

[0304] Step 12:

[0305] The server stores the selected best photos in a database.

[0306] Step 13:

[0307] Users specify themes or people they want to include in their albums (e.g., "Family Vacations in the 1980s" or "Children Growing Up").

[0308] Step 14:

[0309] The server searches the database for relevant photos based on the specified criteria.

[0310] Step 15:

[0311] The server uses a people recognition module to identify photos that contain the specified people.

[0312] Step 16:

[0313] The server uses a theme analysis module to identify photos that match a specified theme.

[0314] Step 17:

[0315] The server lays out the photos selected from the search results in an album format.

[0316] Step 18:

[0317] The server generates the album and displays it to the user.

[0318] Step 19:

[0319] The server uses an emotion engine to recognize the user's emotions, which analyzes emotions from the user's facial expressions, voice, and behavior.

[0320] Step 20:

[0321] The server dynamically adjusts the display order and layout of the albums based on the analysis results of the emotion engine. For example, if a user is feeling nostalgic, photos that best match that emotion will be displayed first.

[0322] Step 21:

[0323] The server continuously monitors the user's emotional data and updates the displayed content in real time according to changes in the user's emotions. For example, if the user shows a happy expression, the server displays many photos of fun events that emphasize that emotion.

[0324] Specific examples

[0325] Consider a specific scenario where a user wants to create an album themed around a family trip in the 1980s.

[0326] 1. The user places the film photos stored at home into a scanner and scans all of the photos.

[0327] 2. The device performs the scan, generates and saves a digital image, and sends it to the server.

[0328] 3. The server analyzes the received images to extract date information, and if date information is included, sorts them by date.

[0329] 4. For photos without date information, a generative AI model identifies photos from around the 1980s.

[0330] 5. The server uses a clustering algorithm to group photos taken at the same event and selects and stores the best photos.

[0331] 6. The user specifies the theme "Family Vacations in the 1980s," and the server searches for related photos based on the specified criteria.

[0332] 7. The server lays out the photos in an album format and displays them to the user.

[0333] 8. While a user is browsing an album, the emotion engine recognizes the user's emotions and changes the layout of the photos and albums displayed based on those emotions.

[0334] In this way, the user can efficiently digitize past film photographs and realize optimal photo display according to his / her emotions.

[0335] Example 2

[0336] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0337] In today's digital society, there is a need to efficiently digitize, organize, and display old film photographs, but doing this manually requires a great deal of time and effort.In addition to automatically organizing extracted digital images by theme or era, there is also a need to provide a more satisfying user experience by selecting and displaying the most appropriate images based on the user's emotions.

[0338] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0339] In this invention, the server includes means for capturing film photographs as digital images using a scanner, means for automatically sorting digital images with date information, means for predicting and sorting digital images without date information by the year or time they were taken, means for selecting the most suitable image from similar digital images, means for automatically displaying digital images based on a specified person or theme, and means for recognizing a user's emotion and selecting and optimizing the display order of digital images based on that emotion. This allows for efficient digitization and organization of film photographs, and also enables optimal image display according to the user's emotion.

[0340] "Means for capturing film photographs as digital images using a scanner" refers to the device and its operating process for converting film photographs into high-resolution digital format.

[0341] "Means for automatically sorting date-identified digital images" refers to algorithms and software that analyze date labels and metadata contained in digital images and organize the images chronologically based on that information.

[0342] "Means for predicting and sorting the age and period of undated digital images" refers to a process and system that uses a generative AI model to analyze digital images that lack information about the date and time of their capture and estimate the age and period of their capture.

[0343] "Means for selecting optimal images from among similar digital images" refers to a process and system that uses a clustering algorithm to group multiple images taken of the same event or scene, and then evaluates the image quality and selects the optimal image from the group.

[0344] "Means for automatically displaying digital images based on a specified person or subject" means systems and algorithms that search, select, and automatically display relevant images from a database based on user-specified subject or person-specific input.

[0345] "Means for recognizing a user's emotions and selecting and optimizing the display order of digital images based on those emotions" refers to an emotion engine and related technologies that analyzes emotions from a user's facial expressions, voice, movements, etc., and dynamically adjusts the images to be displayed and their order based on the results of that analysis.

[0346] This invention provides a system that allows users to efficiently digitize film photographs and automatically organize and display them. It also has the ability to recognize the user's emotions and select and display the most suitable images.

[0347] Hardware and Software

[0348] The user places a film photograph into a scanner (e.g., a general-purpose film scanner), launches the scanner software, performs a scan, and generates a high-resolution digital image file (JPEG or TIFF format), which is temporarily stored in the device's local storage.

[0349] When a digital image is saved, the device detects it and transmits it over the Internet to a server using a secure channel (e.g., SSL / TLS).

[0350] The server processes the received digital images. For images with date information, it uses open source tools (e.g., OpenCV and ExifTool) to analyze the date labels and metadata to identify the date of capture. Based on the identified date, the images are sorted in chronological order and stored in a database.

[0351] For images that do not contain date information, the server uses a generative AI model (for example, a model built using TensorFlow or PyTorch) to analyze features in the image (for example, facial expressions, backgrounds, clothing, buildings, etc.) to estimate the age and time period in which the image was taken. Based on this information, the undated images are sorted chronologically and stored in a database.

[0352] The server then uses a clustering algorithm (e.g., K-means or DBSCAN) to group multiple images taken at the same event or scene. Within each group, an evaluation module evaluates resolution, focus, lighting, facial expression, composition, etc. to select the highest-quality image. The selected images are then stored in a database.

[0353] Furthermore, the server uses an emotion engine (e.g., Microsoft Azure Cognitive Services or Amazon Rekognition) to recognize the user's emotions. This emotion analysis engine analyzes the user's facial expressions, voice, and behavioral data in real time to estimate their current emotional state. The estimated emotion data is used to customize the image selection and display order.

[0354] The user logs into the system and specifies the theme and people for the album. For example, they can specify themes such as "family trips" or "children growing up" or a specific person. The server uses the person recognition module and theme analysis module to search and select photos from the database that match the specified criteria. The selected photos are laid out in an album format and displayed to the user.

[0355] As a concrete example, let's consider a case where a user creates a family travel album from the 1980s. First, the user places film photos from home in a scanner and scans all the photos. The digital images obtained by the scanner are sent to a server via the device. The server receives the images, analyzes photos with date information, and sorts them by date. For photos without date information, a generative AI model is used to identify photos from around the 1980s.

[0356] The server then uses a clustering algorithm to group photos taken at the same event or scene and select the most suitable photos.When a user selects a theme such as "1980s family vacation," the server uses a person recognition module and a theme analysis module to search for related photos, lays them out in an album format, and displays them to the user.

[0357] Furthermore, the emotion engine recognizes the user's emotions while browsing the album and dynamically adjusts the displayed photos and album layout based on those emotions. For example, when the user smiles, photos that evoke happy memories are displayed first. This system allows users to efficiently digitize film photos and effortlessly create albums organized chronologically or by theme. Displaying optimal photos based on emotions also enhances user satisfaction.

[0358] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0359] Step 1: Scan your film photos

[0360] The user places the film photograph in the scanner and performs the scan operation. Specifically, the user starts the scanner software and presses the scan button to capture the film as a digital image. The input is the film photograph, and the output is a high-resolution digital image file (JPEG or TIFF format).

[0361] Step 2: Save digital images locally and send them to the server

[0362] The device stores the digital image generated by the scanner in its local storage, then detects the image data and transmits it to a server over the Internet using a secure channel. The input is the digital image file, and the output is the digital image data transmitted to the server.

[0363] Step 3: Extract and sort date information

[0364] The server analyzes the received digital images. Specifically, it extracts date labels and metadata using analysis tools such as OpenCV and ExifTool. Photos containing date information are sorted chronologically based on the date they were taken and stored in a database. The input is the digital images and their metadata, and the output is the sorted digital images.

[0365] Step 4: Dating and Sorting Undated Photos

[0366] The server uses a generative AI model to estimate the age and time of a photo taken for digital images that do not contain date information. The model analyzes the image's features and assigns estimated age information. The images are then sorted based on the estimated age information and stored in a database. The input is digital images that do not contain date information, and the output is digital images sorted chronologically.

[0367] Step 5: Clustering and selecting the best photos

[0368] The server uses a clustering algorithm to group multiple images taken at the same event or scene. It forms clusters using K-means or DBSCAN, and within each group, it selects the best photo based on evaluation criteria such as resolution, focus, lighting, facial expression, and composition. The selected best photo is stored in a database. The input is multiple digital images, and the output is the selected best digital image.

[0369] Step 6: Sentiment analysis and database update

[0370] The server uses an emotion engine to analyze the user's emotions. It captures the user's facial expressions, voice, and actions in real time to estimate the user's emotional state. This emotion data is stored in a database for the selection and display order of digital images. The input is the user's real-time data, and the output is the analyzed emotion data.

[0371] Step 7: Album Settings and Search

[0372] The user inputs the album theme or a specific person into the system. The server uses the person recognition module and the theme analysis module to search and select relevant digital images from the database. The input is a prompt about the theme or person, and the output is the selected digital image.

[0373] Step 8: Dynamic optimization of album display based on user sentiment

[0374] The server uses an emotion engine to continuously monitor the user's emotions and dynamically optimize the displayed images and album layout based on those emotions. For example, when the user smiles, photos that reflect happy memories, such as family trips, are displayed preferentially. The input is real-time emotion data, and the output is a dynamically updated album layout.

[0375] (Application example 2)

[0376] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0377] While systems for efficiently digitizing film photographs already exist, these systems typically require a significant amount of effort to organize and select the digitized photographs. Furthermore, systems for suggesting entertainment and scenery based on passengers' emotions in autonomous vehicles are underdeveloped, necessitating a method for optimizing the riding experience. To address these challenges, a more advanced and efficient system is needed that automatically suggests and displays the most appropriate images and entertainment based on the user's emotions.

[0378] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for capturing film photographs as digital images using a scanner, means for automatically sorting digital images with date information, means for predicting and sorting digital images without date information by the year or time they were taken, means for selecting the most suitable image from similar digital images, means for automatically displaying digital images based on a specified person or theme, and means for analyzing images captured by a camera in an autonomous vehicle and suggesting the most suitable scenic image or entertainment based on the user's emotions. This enables efficient digitization and organization of film photographs and also makes it possible to provide an optimal entertainment experience tailored to the emotions of passengers in autonomous vehicles.

[0379] A "scanner" is a device that captures film photographs as digital images.

[0380] A "digital image" is image data that is acquired by a scanner, camera, or the like and can be stored and processed electronically.

[0381] "Sorting" is the process of rearranging digital images based on specific criteria.

[0382] "Date information" refers to the date and time of photography included in a photograph or time information embedded as metadata.

[0383] "Era and time period prediction" is the process of analyzing the characteristics of a digital image and estimating the age and time period when it was taken.

[0384] A "generative AI model" is an artificial intelligence that can analyze digital images and other data to recognize and predict various features and patterns.

[0385] "Best selection" means selecting high-quality images from similar digital images based on evaluation criteria such as resolution, focus, lighting, facial expression, and composition.

[0386] A "specified person or theme" refers to a particular person or theme that the user wants to include in the album.

[0387] "Display" means that the digital image is visually represented on a screen such as a display.

[0388] An "autonomous vehicle" is a vehicle that automatically performs driving operations under computer control.

[0389] A "camera" is a device for taking still or video images.

[0390] "Video analysis" refers to the analysis of video captured by cameras in self-driving vehicles using algorithms and AI.

[0391] "User's emotion" refers to the emotional state recognized from the user's facial expression, voice, behavior, etc.

[0392] A "landscape image" is a digital image of a natural or urban landscape.

[0393] "Entertainment" is content such as music, videos, podcasts, etc. that users can enjoy.

[0394] "Suggestion" means recommending certain content or images to the user.

[0395] This invention provides a system that efficiently digitizes film photographs and automatically organizes and displays them, as well as a system that analyzes images captured by cameras in autonomous vehicles and suggests optimal scenic images and entertainment based on the user's emotions.

[0396] First, the user places a film photo into a scanner to generate a digital image. The digital image captured by the scanner is saved to the device's local storage and then sent to the server. The server extracts date information from the digital image and automatically sorts photos with date information. For photos without date information, a generative AI model is used to analyze features such as facial expressions, backgrounds, clothing, and buildings to predict the era and time period and sort them.

[0397] The server then selects the best photos from among similar digital images, using criteria such as resolution, focus, lighting, facial expression, and composition to select the highest quality photos. Based on people or themes specified by the user, related digital images are searched and automatically displayed in an album format.

[0398] In autonomous vehicles, images captured by onboard cameras are analyzed in real time. The server uses an emotion engine to analyze the user's facial expressions and voice to recognize their emotions. Based on this emotion data, the server uses a generative AI model to select optimal scenery images and entertainment to display on the passenger's screen. For example, if the driver or passenger appears bored, the server will suggest relaxing music or scenery images.

[0399] The hardware required for the server to realize this system includes a scanner, an on-board camera, and an on-board computer with high-performance processing power. The software used also includes a generative AI model, an emotion recognition engine, and an entertainment recommendation engine. The specific technology stack includes OpenCV for image analysis, Amazon Rekognition and Google Cloud Vision for emotion recognition, and Spotify API and YouTube API for entertainment recommendations.

[0400] As a concrete example, a user can use a scanner to scan film photos of family trips from the 1980s stored at home and send them to the server. The server then analyzes the date information, sorts them, and selects the most suitable photos. If the user then selects the theme "Family trips in the 1980s," related photos are displayed in an album format. Furthermore, during long-distance drives in an autonomous vehicle, the system analyzes footage captured by the onboard camera and suggests the most suitable entertainment or scenic images depending on whether the user is relaxed or bored.

[0401] Example prompt sentence:

[0402] 1. "What is the best entertainment option for a bored passenger on a long drive?"

[0403] 2. "Based on the emotions you experience on a family trip, suggest a photo of the scenery seen from the car window."

[0404] This enables the efficient digitization, organization, and display of film photographs, providing an optimal entertainment experience for passengers in autonomous vehicles.

[0405] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0406] Step 1:

[0407] The user places the film photo in the scanner, which then captures the photo as a digital image, which is then saved in the device's local storage.

[0408] (Input) Film photo

[0409] (Output) Digital image

[0410] (Specific operation) The user operates the scanner software and starts scanning a film photograph. The scanner digitizes the photograph at high resolution and saves the resulting digital image on the device.

[0411] Step 2:

[0412] The terminal transmits the digital image stored in the local storage to the server.

[0413] (Input) Digital image

[0414] (Output) Digital image sent to server

[0415] (Specific operation) The terminal uploads the digital image to the server via the network, using a secure file transfer protocol (such as FTP).

[0416] Step 3:

[0417] The server extracts date information from the received digital images and automatically sorts the photos that have date information.

[0418] (Input) Digital image

[0419] (Output) A set of images sorted by date

[0420] (Specific operation) The server uses an algorithm (e.g., OpenCV) to analyze the metadata of image files and the date stamps on the photos to extract date information, and then sorts the digital images based on the extracted date information.

[0421] Step 4:

[0422] For digital images that do not have date information, the server uses a generative AI model to predict and sort the year and time of the photo.

[0423] (Input) Digital image with no date information

[0424] (Output) A set of images sorted based on age and time period

[0425] (Specific operation) The server uses a generative AI model (e.g., TensorFlow) to analyze features such as facial expressions, backgrounds, clothing, and buildings. Based on the analysis results, it predicts the age and time period in which the photos were taken and sorts these digital images.

[0426] Step 5:

[0427] The server groups digital images taken at the same event or scene and selects the highest quality photo based on criteria such as resolution, focus, lighting, facial expression, and composition.

[0428] (Input) Sorted digital images

[0429] (Output) Optimal photo set

[0430] (Specific operation) Uses a clustering algorithm (e.g., k-means) to group images by event or scene, then selects the best photo within each group based on quality criteria.

[0431] Step 6:

[0432] Based on people or themes specified by the user, the server searches for related digital images and automatically displays them in an album format.

[0433] (Input) Specify the person or theme

[0434] (Output) A set of images displayed in album format

[0435] (Specific operation) The server uses the person recognition module and the theme analysis module to search for images that match the specifications (e.g., face recognition algorithm and theme analysis algorithm). The found images are laid out in an album format and displayed to the user.

[0436] Step 7:

[0437] The server analyzes the video captured by the camera of the self-driving vehicle in real time and recognizes the user's emotions.

[0438] (Input) Video captured by an in-vehicle camera

[0439] (Output) Emotion data

[0440] (Specific operation) The server uses an emotion recognition engine (e.g., Amazon Rekognition) to analyze the user's facial expressions and voice in the video to recognize their emotions.

[0441] Step 8:

[0442] The server uses a generative AI model to suggest optimal landscape images and entertainment to the user based on the recognized user emotions.

[0443] (Input) Emotion data

[0444] (Output) Suggested landscape images and entertainment

[0445] (Specific operation) The server uses the generative AI model to select appropriate scenery images and entertainment based on emotional data and display them on the in-car display. For example, if it recognizes that the user is in a relaxing mood, it will suggest relaxing scenery images and music.

[0446] These steps will enable users to efficiently digitize and archive their film photos, while also optimizing the riding experience in autonomous vehicles.

[0447] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0448] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0449] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0450] [Second embodiment]

[0451] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0452] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0453] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0454] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0455] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0456] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0457] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0458] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0459] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0460] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0461] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0462] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0463] This invention relates to a system for efficiently digitizing film photographs and automatically organizing and displaying them.

[0464] Photo scanning process

[0465] The user first places a film photo in the scanner and performs the scan operation. The scanner scans the photo and generates a digital image. This digital image is saved in the device's local storage and then sent to the server.

[0466] Sorting photos with dates

[0467] The server extracts date information from the received digital images. If date information is included, it identifies the specific date of the photo by analyzing the date stamp and metadata in the photo. The digital images with this date information are sorted and stored in chronological order.

[0468] Dating and sorting undated photos

[0469] The server analyzes digital images that do not contain date information using a generative AI model. The generative AI model evaluates features such as facial expressions, backgrounds, clothing, and buildings to predict the age and time period in which the photo was taken. This allows digital images without date information to be sorted based on their estimated age.

[0470] Selecting the best photo

[0471] The server uses a clustering algorithm to group multiple digital images taken at the same event or scene. Within each group, an evaluation module evaluates resolution, focus, lighting, facial expression, composition, and other factors to select the best quality photos. The selected best photos are then stored in a database.

[0472] Display albums based on specified people or themes

[0473] Users specify themes and people they want to include in their albums, such as "family vacations" or "children growing up," or specific people. The server searches the database for relevant photos based on the specified criteria. The person recognition module identifies photos containing the specified people, and the theme analysis module identifies photos that match the specified theme.

[0474] Photos selected from the search results are laid out in an album format and displayed to the user, allowing users to easily digitize their old film photos, efficiently organize them, and immediately use them as albums tailored to their needs.

[0475] Specific examples

[0476] Let's say a user wants to create an album themed around a family trip in the 1980s. First, the user places film photos stored at home in a scanner and scans all of the photos. The digital images obtained by the scanner are sent to a server. The server sorts photos that contain date information by date, and for photos without date information, it uses a generative AI model to identify photos from around the 1980s.

[0477] The server then uses a clustering algorithm to group photos taken at the same event or scene and select the best photos from each group.The user selects "family vacations in the 1980s," and the server uses a person recognition module and a theme analysis module to search for and select related photos, lay them out in an album format, and display them to the user.

[0478] Through the above process, users can efficiently digitize old film photographs without much effort and easily create albums organized chronologically or by theme.

[0479] The processing flow will be explained below.

[0480] Step 1:

[0481] The user places the film photo in the scanner and presses the start scan button.

[0482] Step 2:

[0483] The device operates a scanner to scan film photographs and generate digital images.

[0484] Step 3:

[0485] The device stores the generated digital images in local storage.

[0486] Step 4:

[0487] The terminal transmits the stored digital images to a server.

[0488] Step 5:

[0489] The server analyzes the received digital images and extracts date information by analyzing the date stamps and metadata contained in the images.

[0490] Step 6:

[0491] The server classifies the digital images into those with date information and those without date information.

[0492] Step 7:

[0493] The server sorts the digital images with date information in chronological order.

[0494] Step 8:

[0495] The server uses a generative AI model to predict the age and time of a photo taken for digital images that do not have date information.

[0496] Step 9:

[0497] The server sorts undated digital images based on their predicted age.

[0498] Step 10:

[0499] The server uses a clustering algorithm to group digital images taken at the same event or scene.

[0500] Step 11:

[0501] For each group, the server selects the highest quality photo using an evaluation module that evaluates resolution, focus, lighting, facial expression, composition, etc.

[0502] Step 12:

[0503] The server stores the selected best photos in a database.

[0504] Step 13:

[0505] Users specify themes or people they want to include in their albums (e.g., "Family Vacations in the 1980s" or "Children Growing Up").

[0506] Step 14:

[0507] The server searches the database for relevant photos based on the specified criteria.

[0508] Step 15:

[0509] The server uses a people recognition module to identify photos that contain the specified people.

[0510] Step 16:

[0511] The server uses a theme analysis module to identify photos that match a specified theme.

[0512] Step 17:

[0513] The server lays out the photos selected from the search results in an album format.

[0514] Step 18:

[0515] The server generates the album and displays it to the user.

[0516] By following these steps, users can efficiently digitize their old film photos and easily create albums organized chronologically or thematically.

[0517] Example 1

[0518] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0519] Digitizing film photographs is time-consuming, and organizing and searching for specific photographs is complicated. Furthermore, it is difficult to identify the age or time period of photographs that do not have date information, and it is also extremely difficult to select the most suitable photographs from a large number of photographs. A system that can solve these problems is needed.

[0520] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0521] In this invention, the server includes means for capturing film photographs as digital images, means for saving the digital images in local storage, means for transmitting the saved digital images to the server, means for automatically sorting digital images with date information, means for predicting and sorting digital images without date information by the year or time they were taken, means for clustering digital images taken in similar scenes, means for selecting the most suitable digital image from the clustering results, and means for searching and displaying digital images based on a specified person or theme. This enables users to efficiently digitize film photographs and easily organize and search for them based on year or theme.

[0522] "Film photography" is an analogue form of photography taken using photographic film.

[0523] "Digital image" refers to an image file represented in an electronic data format.

[0524] "Local storage" refers to a storage medium, either internal or external to a computer, that is used to store data.

[0525] A "server" is a computer system that stores, processes, and manages data over a network.

[0526] "Date information" is data indicating the date and time a photo was taken, and is stored as metadata or text information included in the image.

[0527] "Clustering" is a method of grouping data based on similarity.

[0528] A "generative AI model" is an algorithm that uses artificial intelligence technology to analyze data and generate new information.

[0529] A "clustering algorithm" is a computational method for classifying data into groups based on similarity.

[0530] The "optimal digital image" is the digital image that is judged to be of the highest quality based on multiple evaluation criteria, including resolution, focus, lighting, facial expression, and composition.

[0531] A "person recognition module" is a software component for identifying specific people in an image.

[0532] A "theme analysis module" is a software component that analyzes the content of images and identifies matches to specific themes.

[0533] This invention is a system for efficiently digitizing film photographs and automatically organizing and displaying them. This system performs a series of steps, from scanning film photographs to generating, saving, and processing digital images. Specific embodiments are described below.

[0534] 1. Scanning film photos

[0535] First, the user places the film photograph into the scanner and performs the scan operation. The scanner used can be a general photo scanner (e.g., EPSON Perfection V600). The scanner converts the film photograph into a digital image.

[0536] 2. Local storage of digital images

[0537] The device temporarily stores digital images acquired from the scanner in local storage, which is implemented using the Windows or macOS file system.

[0538] 3. Sending digital images to the server

[0539] The device sends the stored digital images to a server using the HTTP protocol, for example using the Python requests library.

[0540] 4. Extracting Date Information

[0541] The server extracts date information from received digital images, using an image processing library (e.g., OpenCV) to detect date stamps in the images and the Pillow library to parse metadata, e.g., EXIF ​​information.

[0542] 5. Sorting photos with dates

[0543] The server sorts the digital images with date information in the order of the extracted dates, using Python's standard sort function.

[0544] 6. Dating undated photographs

[0545] The server uses generative AI models to estimate the age of undated digital images, for example, by using image classification libraries such as TensorFlow or PyTorch to evaluate features of the photo (such as facial expression, background, clothing, and buildings) to estimate the age.

[0546] 7. Photo Clustering

[0547] The server uses clustering algorithms to group digital images taken during the same event or scene, such as k-means clustering or DBSCAN (Density-Based Spatial Clustering of Applications with Noise).

[0548] 8. Selecting the best photo

[0549] The server evaluates each group's photos based on their resolution, focus, lighting, facial expression, composition, etc., and selects the highest quality photo. The evaluation module scores each item using an image evaluation library.

[0550] 9. Enter album conditions

[0551] Users can select themes and people they want to include in their albums, such as "family vacations" or "children growing up," through their device, either via a web interface or a mobile app.

[0552] 10. Search by people or topics

[0553] The server searches the database for relevant photos based on the specified criteria. The person recognition module performs face recognition processing using, for example, the dlib library, and the theme analysis module identifies photos that match a theme using an image captioning model.

[0554] 11. Album Creation and Display

[0555] The server then selects photos from the search results, lays them out in an album format, and displays them to the user. The layout process can be performed using an HTML template engine (e.g., Jinja2) to display them as a web page, or using a PDF generation library (e.g., ReportLab) to output them as PDF.

[0556] Specific examples

[0557] For example, if a user wants to create an album themed around a family trip in the 1980s, they first load the film photos stored at home into a scanner and scan all of the photos. The digital images obtained by the scanner are sent to a server, which sorts photos with date information by date and uses a generative AI model to identify photos without date information as being from around the 1980s. The server then uses a clustering algorithm to group photos taken at the same event or scene and selects the most suitable photo from each group. The user selects "family trip in the 1980s," and the server uses a person recognition module and theme analysis module to search for and select related photos, lay them out in an album format, and displays them to the user.

[0558] Prompt Sentence Examples

[0559] Example 1: "Create a family vacation album from the 1980s. Scan and digitize film photos, parse the date information, and arrange them chronologically. For undated photos, use a generative AI model to estimate the age and sort them appropriately."

[0560] Example 2: "Scan film photos from a family vacation. Send the resulting digital images to a server, group photos from the same event, and select and store the best ones."

[0561] By processing in this manner, users can efficiently digitize film photographs and easily organize and search them by era or theme.

[0562] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0563] Step 1:

[0564] The user places the film photograph in the scanner and performs the scanning operation.

[0565] Input: Film photo

[0566] Output: Digital image

[0567] Specific operation: Scan film photographs using a scanner (e.g., EPSON Perfection V600) and convert them into digital images.

[0568] Step 2:

[0569] The device stores the scanned digital images in local storage.

[0570] Input: Digital image

[0571] Output: Digital image saved to local storage

[0572] What it does: It uses the Windows or macOS file system to temporarily store scanned digital images.

[0573] Step 3:

[0574] The terminal transmits the stored digital images to a server.

[0575] Input: Digital image stored in local storage

[0576] Output: Digital image sent to server

[0577] Specific operation: Uploads a file to the server using the HTTP protocol and, for example, the Python requests library.

[0578] Step 4:

[0579] The server extracts date information from the received digital images.

[0580] Input: Digital image sent to the server

[0581] Output: Digital image with extracted date information

[0582] Specific operation: Detects date stamps in images using an image processing library (OpenCV), and analyzes metadata (EXIF information) using the Pillow library.

[0583] Step 5:

[0584] The server sorts the digital images containing date information in chronological order.

[0585] Input: Digital image with extracted date information

[0586] Output: Digital images sorted by date

[0587] What it does: Organizes digital images by date using Python's standard sorting functions.

[0588] Step 6:

[0589] The server uses generative AI models to estimate the age of undated digital images.

[0590] Input: undated digital image

[0591] Output: Dated digital image

[0592] Specific operation: Using image classification libraries such as TensorFlow and PyTorch, the features of the photo (facial expression, background, clothing, buildings, etc.) are evaluated to estimate the age.

[0593] Step 7:

[0594] The server uses a clustering algorithm to group digital images taken at the same event or scene.

[0595] Input: Dated digital image

[0596] Output: Clustered digital image groups

[0597] What it does: Group digital images using algorithms such as k-means clustering and DBSCAN.

[0598] Step 8:

[0599] The server evaluates the resolution, focus, lighting, facial expression, composition, etc. within each group and selects the highest quality photo.

[0600] Input: Clustered digital image groups

[0601] Output: Optimal digital images

[0602] Specific operation: Using an image evaluation library, each item (resolution, focus, lighting, facial expression, composition, etc.) is scored and the highest quality photo is selected.

[0603] Step 9:

[0604] The user specifies the themes and people they want to include in the album through the terminal.

[0605] Input: Album theme or person

[0606] Output: specified conditions

[0607] What it does: Enter a topic (e.g., "family vacation") and people using the web interface or mobile app.

[0608] Step 10:

[0609] The server searches the database for relevant photos based on the specified criteria.

[0610] Input: Specified conditions (theme or person)

[0611] Output: Related photos found

[0612] What it does: It uses a facial recognition library (dlib) to identify photos containing specified people, and a theme analysis module to identify photos that match specified themes.

[0613] Step 11:

[0614] The server lays out the photos selected from the search results in an album format and displays them to the user.

[0615] Input: Related photos found

[0616] Output: Layout album

[0617] Specific behavior: Display as a web page using an HTML template engine (Jinja2), or output as PDF using a PDF generation library (ReportLab).

[0618] (Application example 1)

[0619] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0620] It is difficult to efficiently digitize film photographs and automatically organize and display them, and it takes a great deal of effort, especially when digitizing and providing old film photographs in brick-and-mortar stores. Organizing photographs with missing date information and selecting the most suitable photograph from multiple similar photographs are also difficult challenges.

[0621] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0622] In this invention, the server includes means for capturing film images as digital data using a scanner, means for automatically sorting digital data with date information, means for predicting and sorting the age or period of digital data without date information, means for selecting the most suitable digital data from similar digital data, means for automatically displaying digital data based on a specified target person or theme, means for use with a smartphone in a physical store, means for estimating the age of digital data without date information using an AI model, means for identifying target people using a face recognition module, and means for identifying photos that match a specified theme using a theme recognition module.This makes it possible to efficiently digitize old film photos and easily organize and display them in a physical store using a smartphone.

[0623] A "scanner" is a device that captures film images as digital data.

[0624] "Digital data" refers to digital image files captured from film images.

[0625] "Date information" refers to data related to the date and time of shooting, such as date labels and meta information that appear on the photo.

[0626] "Sorting" is the process of organizing and rearranging digital data based on specific criteria.

[0627] "Decade" refers to a specific range that indicates when a photo was taken, such as the 1980s.

[0628] "Time" refers to the specific period of time when the photograph was taken, such as a particular season or year.

[0629] An "AI model" refers to a model that uses artificial intelligence to analyze multiple factors and infer specific information from the data.

[0630] "Facial Recognition Module" refers to technology for identifying and recognizing human faces contained in digital data.

[0631] "Theme recognition module" refers to technology that analyzes the image content and themes contained in digital data and identifies information that matches a specified theme.

[0632] "Brick and mortar store" refers to a commercial establishment that offers goods or services in a physical location, such as a photo studio or retail store.

[0633] A "smartphone" refers to a mobile device that not only has the functions of a mobile phone but also has advanced computing power and multifunctionality.

[0634] This invention builds a system that uses a scanner, a server, a smartphone terminal, and multiple software modules to efficiently digitize, organize, and display film photographs.

[0635] The user first places the film image on the scanner and operates the scanner to capture the film image as digital data. The captured digital data is saved in local storage and then sent to the server.

[0636] The server analyzes the digital data and automatically sorts it using the date information if it contains it, and if it does not, it uses a generative AI model to predict the age and time period of the photo and sorts it accordingly.

[0637] The server then uses a clustering algorithm to group digital images taken at the same event or scene. For each cluster, the server evaluates resolution, focus, lighting, facial expression, composition, and other factors to select the highest quality images. This selected data is then stored in a database.

[0638] Furthermore, users can use their smartphones to search and display digital data based on specified people or themes. The face recognition module identifies people, and the theme recognition module identifies photos that match the specified theme. Digital data selected from the search results is laid out and displayed to the user in an album format.

[0639] For example, let's say a user wants to create an album themed around a family trip in the 1980s. First, the user places all the film photos stored at home in a scanner and scans them all. The digital data obtained by the scanner is sent to a server. The server sorts photos that contain date information by date, and for photos without date information, it uses a generative AI model to identify photos from around the 1980s.

[0640] The server then uses a clustering algorithm to group digital data taken at the same event or scene and selects the most suitable data from each group.The user selects "1980s family vacation" on their smartphone, and the server uses face and theme recognition modules to search for and select related digital data, lays it out in an album format, and displays it to the user.

[0641] Example prompts for generative AI models:

[0642] To estimate the age of a photograph, pay attention to the following:

[0643] The clothing and hairstyle of the people in the photo

[0644] Background and buildings

[0645] Photo Tone

[0646] For example, characteristics of the 1980s include brightly colored clothing, big hairstyles, and a particular architectural style.

[0647] Photo: [Link to photo data]

[0648] What era do you estimate the photo was taken in?

[0649] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0650] Step 1:

[0651] A user places a film image on a scanner and operates the scanner to capture the film image as digital data.

[0652] Input: Film images, scanner

[0653] Processing: The scanner scans the film images and generates digital data, which is then stored in local storage.

[0654] Output: Digital data stored in local storage

[0655] Step 2:

[0656] The terminal transmits the digital data captured by the scanner to the server.

[0657] Input: Digital data stored in local storage, terminal, server

[0658] Processing: The device reads the digital data from local storage and sends it to the server over the network.

[0659] Output: Digital data received by the server

[0660] Step 3:

[0661] The server extracts date information from the received digital data and sorts the data having the date information.

[0662] Input: Digital data received by the server

[0663] Processing: The server uses an image analysis engine (e.g., pytesseract) to extract date information from the digital data. Based on the extracted date information, the digital data is sorted in chronological order.

[0664] Output: Digital data sorted by date

[0665] Step 4:

[0666] The server uses a generative AI model to predict and sort the age and time period of digital data that does not have date information.

[0667] Input: Digital data without date information

[0668] Processing: The server passes the input data to the generative AI model to obtain a predicted age and time period. The digital data is then sorted based on the obtained age and time period information.

[0669] Output: Digital data sorted and dated

[0670] Step 5:

[0671] The server uses a clustering algorithm to group digital data captured at the same event or scene.

[0672] Input: Sorted digital data stored on the server

[0673] Processing: The server applies a clustering algorithm to group the digital data based on the features.

[0674] Output: Digital data grouped into clusters

[0675] Step 6:

[0676] The server selects the most suitable digital data within each cluster.

[0677] Input: Digital data grouped into clusters

[0678] Processing: The server uses an evaluation module to evaluate resolution, focus, lighting, facial expression, composition, etc. within each cluster and select the most suitable digital data.

[0679] Output: Optimal digital data selected for each cluster

[0680] Step 7:

[0681] The server searches and displays digital data based on the target person or theme specified by the user.

[0682] Input: Specified target person or theme, selected and sorted digital data

[0683] Processing: The server uses a face recognition module to identify the target person, and a theme recognition module to search for digital data matching the specified theme. The search results are then laid out in an album format and displayed to the user on their smartphone.

[0684] Output: Digital data in album format displayed on a smartphone

[0685] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0686] This invention is a system that efficiently digitizes film photographs and automatically organizes and displays them, and also has the ability to recognize the user's emotions and select and display the most appropriate images.

[0687] Photo scanning process

[0688] The user first places a film photo in the scanner and performs the scan operation. The scanner scans the photo and generates a digital image. This digital image is saved in the device's local storage and then sent to the server.

[0689] Sorting photos with dates

[0690] The server extracts date information from the received digital images. If date information is included, it identifies the specific date of the photo by analyzing the date stamp and metadata in the photo. The digital images with this date information are sorted and stored in chronological order.

[0691] Dating and sorting undated photos

[0692] The server analyzes digital images that do not contain date information using a generative AI model. The generative AI model evaluates features such as facial expressions, backgrounds, clothing, and buildings to predict the age and time period in which the photo was taken. This allows digital images without date information to be sorted based on their estimated age.

[0693] Selecting the best photo

[0694] The server uses a clustering algorithm to group multiple digital images taken at the same event or scene. Within each group, an evaluation module evaluates resolution, focus, lighting, facial expression, composition, and other factors to select the best quality photos. The selected best photos are then stored in a database.

[0695] Implementing the Emotion Engine

[0696] The server includes an emotion engine that recognizes the user's emotions. The emotion engine includes means for analyzing emotions from the user's facial expressions, voice, and actions. The analyzed emotion data is reflected in the selection and display order of photos.

[0697] Display albums based on specified people or themes

[0698] Users specify themes and people they want to include in their albums, such as "family vacations" or "children growing up," or specific people. The server searches the database for relevant photos based on the specified criteria. The person recognition module identifies photos containing the specified people, and the theme analysis module identifies photos that match the specified theme.

[0699] Emotion-based selection and display of optimal photos

[0700] The album layout and display order are dynamically changed based on the user's emotions as recognized by the emotion engine. For example, if the user feels nostalgic for the past, photos that best match that emotion will be displayed preferentially. Furthermore, the system will display more photos of fun events that reflect the user's joy and happiness.

[0701] Specific examples

[0702] Let's say a user wants to create an album themed around a family trip in the 1980s. First, the user places film photos stored at home in a scanner and scans all of the photos. The digital images obtained by the scanner are sent to a server. The server sorts photos that contain date information by date, and for photos without date information, it uses a generative AI model to identify photos from around the 1980s.

[0703] The server then uses a clustering algorithm to group photos taken at the same event or scene and select the best photos from each group.The user selects "family vacations in the 1980s," and the server uses a person recognition module and a theme analysis module to search for and select related photos, lay them out in an album format, and display them to the user.

[0704] Furthermore, while a user is browsing an album, the emotion engine recognizes the user's emotions and dynamically adjusts the photos and album layout based on those emotions. For example, if the user smiles, photos with lots of happy family memories will be prioritized.

[0705] Through this process, users can efficiently digitize their old film photos without much effort and create albums organized chronologically or by theme. Furthermore, by displaying photos optimally according to their emotions, users can enjoy even greater satisfaction.

[0706] The processing flow will be explained below.

[0707] Step 1:

[0708] The user places the film photo in the scanner and presses the start scan button.

[0709] Step 2:

[0710] The device operates a scanner to scan film photographs and generate digital images.

[0711] Step 3:

[0712] The device stores the generated digital images in local storage.

[0713] Step 4:

[0714] The terminal transmits the stored digital images to a server.

[0715] Step 5:

[0716] The server analyzes the received digital images and extracts date information by analyzing the date stamps and metadata contained in the images.

[0717] Step 6:

[0718] The server classifies the digital images into those with date information and those without date information.

[0719] Step 7:

[0720] The server sorts the digital images with date information in chronological order.

[0721] Step 8:

[0722] The server uses a generative AI model to predict the age and time of a photo taken for digital images that do not have date information.

[0723] Step 9:

[0724] The server sorts undated digital images based on their predicted age.

[0725] Step 10:

[0726] The server uses a clustering algorithm to group digital images taken at the same event or scene.

[0727] Step 11:

[0728] For each group, the server selects the highest quality photo using an evaluation module that evaluates resolution, focus, lighting, facial expression, composition, etc.

[0729] Step 12:

[0730] The server stores the selected best photos in a database.

[0731] Step 13:

[0732] Users specify themes or people they want to include in their albums (e.g., "Family Vacations in the 1980s" or "Children Growing Up").

[0733] Step 14:

[0734] The server searches the database for relevant photos based on the specified criteria.

[0735] Step 15:

[0736] The server uses a people recognition module to identify photos that contain the specified people.

[0737] Step 16:

[0738] The server uses a theme analysis module to identify photos that match a specified theme.

[0739] Step 17:

[0740] The server lays out the photos selected from the search results in an album format.

[0741] Step 18:

[0742] The server generates the album and displays it to the user.

[0743] Step 19:

[0744] The server uses an emotion engine to recognize the user's emotions, which analyzes emotions from the user's facial expressions, voice, and behavior.

[0745] Step 20:

[0746] The server dynamically adjusts the display order and layout of the albums based on the analysis results of the emotion engine. For example, if a user is feeling nostalgic, photos that best match that emotion will be displayed first.

[0747] Step 21:

[0748] The server continuously monitors the user's emotional data and updates the displayed content in real time according to changes in the user's emotions. For example, if the user shows a happy expression, the server displays many photos of fun events that emphasize that emotion.

[0749] Specific examples

[0750] Consider a specific scenario where a user wants to create an album themed around a family trip in the 1980s.

[0751] 1. The user places the film photos stored at home into a scanner and scans all of the photos.

[0752] 2. The device performs the scan, generates and saves a digital image, and sends it to the server.

[0753] 3. The server analyzes the received images to extract date information, and if date information is included, sorts them by date.

[0754] 4. For photos without date information, a generative AI model identifies photos from around the 1980s.

[0755] 5. The server uses a clustering algorithm to group photos taken at the same event and selects and stores the best photos.

[0756] 6. The user specifies the theme "Family Vacations in the 1980s," and the server searches for related photos based on the specified criteria.

[0757] 7. The server lays out the photos in an album format and displays them to the user.

[0758] 8. While a user is browsing an album, the emotion engine recognizes the user's emotions and changes the layout of the photos and albums displayed based on those emotions.

[0759] In this way, the user can efficiently digitize past film photographs and realize optimal photo display according to his / her emotions.

[0760] Example 2

[0761] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0762] In today's digital society, there is a need to efficiently digitize, organize, and display old film photographs, but doing this manually requires a great deal of time and effort.In addition to automatically organizing extracted digital images by theme or era, there is also a need to provide a more satisfying user experience by selecting and displaying the most appropriate images based on the user's emotions.

[0763] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0764] In this invention, the server includes means for capturing film photographs as digital images using a scanner, means for automatically sorting digital images with date information, means for predicting and sorting digital images without date information by the year or time they were taken, means for selecting the most suitable image from similar digital images, means for automatically displaying digital images based on a specified person or theme, and means for recognizing a user's emotion and selecting and optimizing the display order of digital images based on that emotion. This allows for efficient digitization and organization of film photographs, and also enables optimal image display according to the user's emotion.

[0765] "Means for capturing film photographs as digital images using a scanner" refers to the device and its operating process for converting film photographs into high-resolution digital format.

[0766] "Means for automatically sorting date-identified digital images" refers to algorithms and software that analyze date labels and metadata contained in digital images and organize the images chronologically based on that information.

[0767] "Means for predicting and sorting the age and period of undated digital images" refers to a process and system that uses a generative AI model to analyze digital images that lack information about the date and time of their capture and estimate the age and period of their capture.

[0768] "Means for selecting optimal images from among similar digital images" refers to a process and system that uses a clustering algorithm to group multiple images taken of the same event or scene, and then evaluates the image quality and selects the optimal image from the group.

[0769] "Means for automatically displaying digital images based on a specified person or subject" means systems and algorithms that search, select, and automatically display relevant images from a database based on user-specified subject or person-specific input.

[0770] "Means for recognizing a user's emotions and selecting and optimizing the display order of digital images based on those emotions" refers to an emotion engine and related technologies that analyzes emotions from a user's facial expressions, voice, movements, etc., and dynamically adjusts the images to be displayed and their order based on the results of that analysis.

[0771] This invention provides a system that allows users to efficiently digitize film photographs and automatically organize and display them. It also has the ability to recognize the user's emotions and select and display the most suitable images.

[0772] Hardware and Software

[0773] The user places a film photograph into a scanner (e.g., a general-purpose film scanner), launches the scanner software, performs a scan, and generates a high-resolution digital image file (JPEG or TIFF format), which is temporarily stored in the device's local storage.

[0774] When a digital image is saved, the device detects it and transmits it over the Internet to a server using a secure channel (e.g., SSL / TLS).

[0775] The server processes the received digital images. For images with date information, it uses open source tools (e.g., OpenCV and ExifTool) to analyze the date labels and metadata to identify the date of capture. Based on the identified date, the images are sorted in chronological order and stored in a database.

[0776] For images that do not contain date information, the server uses a generative AI model (for example, a model built using TensorFlow or PyTorch) to analyze features in the image (for example, facial expressions, backgrounds, clothing, buildings, etc.) to estimate the age and time period in which the image was taken. Based on this information, the undated images are sorted chronologically and stored in a database.

[0777] The server then uses a clustering algorithm (e.g., K-means or DBSCAN) to group multiple images taken at the same event or scene. Within each group, an evaluation module evaluates resolution, focus, lighting, facial expression, composition, etc. to select the highest-quality image. The selected images are then stored in a database.

[0778] Furthermore, the server uses an emotion engine (e.g., Microsoft Azure Cognitive Services or Amazon Rekognition) to recognize the user's emotions. This emotion analysis engine analyzes the user's facial expressions, voice, and behavioral data in real time to estimate their current emotional state. The estimated emotion data is used to customize the image selection and display order.

[0779] The user logs into the system and specifies the theme and people for the album. For example, they can specify themes such as "family trips" or "children growing up" or a specific person. The server uses the person recognition module and theme analysis module to search and select photos from the database that match the specified criteria. The selected photos are laid out in an album format and displayed to the user.

[0780] As a concrete example, let's consider a case where a user creates a family travel album from the 1980s. First, the user places film photos from home in a scanner and scans all the photos. The digital images obtained by the scanner are sent to a server via the device. The server receives the images, analyzes photos with date information, and sorts them by date. For photos without date information, a generative AI model is used to identify photos from around the 1980s.

[0781] The server then uses a clustering algorithm to group photos taken at the same event or scene and select the most suitable photos.When a user selects a theme such as "1980s family vacation," the server uses a person recognition module and a theme analysis module to search for related photos, lays them out in an album format, and displays them to the user.

[0782] Furthermore, the emotion engine recognizes the user's emotions while browsing the album and dynamically adjusts the displayed photos and album layout based on those emotions. For example, when the user smiles, photos that evoke happy memories are displayed first. This system allows users to efficiently digitize film photos and effortlessly create albums organized chronologically or by theme. Displaying optimal photos based on emotions also enhances user satisfaction.

[0783] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0784] Step 1: Scan your film photos

[0785] The user places the film photograph in the scanner and performs the scan operation. Specifically, the user starts the scanner software and presses the scan button to capture the film as a digital image. The input is the film photograph, and the output is a high-resolution digital image file (JPEG or TIFF format).

[0786] Step 2: Save digital images locally and send them to the server

[0787] The device stores the digital image generated by the scanner in its local storage, then detects the image data and transmits it to a server over the Internet using a secure channel. The input is the digital image file, and the output is the digital image data transmitted to the server.

[0788] Step 3: Extract and sort date information

[0789] The server analyzes the received digital images. Specifically, it extracts date labels and metadata using analysis tools such as OpenCV and ExifTool. Photos containing date information are sorted chronologically based on the date they were taken and stored in a database. The input is the digital images and their metadata, and the output is the sorted digital images.

[0790] Step 4: Dating and Sorting Undated Photos

[0791] The server uses a generative AI model to estimate the age and time of a photo taken for digital images that do not contain date information. The model analyzes the image's features and assigns estimated age information. The images are then sorted based on the estimated age information and stored in a database. The input is digital images that do not contain date information, and the output is digital images sorted chronologically.

[0792] Step 5: Clustering and selecting the best photos

[0793] The server uses a clustering algorithm to group multiple images taken at the same event or scene. It forms clusters using K-means or DBSCAN, and within each group, it selects the best photo based on evaluation criteria such as resolution, focus, lighting, facial expression, and composition. The selected best photo is stored in a database. The input is multiple digital images, and the output is the selected best digital image.

[0794] Step 6: Sentiment analysis and database update

[0795] The server uses an emotion engine to analyze the user's emotions. It captures the user's facial expressions, voice, and actions in real time to estimate the user's emotional state. This emotion data is stored in a database for the selection and display order of digital images. The input is the user's real-time data, and the output is the analyzed emotion data.

[0796] Step 7: Album Settings and Search

[0797] The user inputs the album theme or a specific person into the system. The server uses the person recognition module and the theme analysis module to search and select relevant digital images from the database. The input is a prompt about the theme or person, and the output is the selected digital image.

[0798] Step 8: Dynamic optimization of album display based on user sentiment

[0799] The server uses an emotion engine to continuously monitor the user's emotions and dynamically optimize the displayed images and album layout based on those emotions. For example, when the user smiles, photos that reflect happy memories, such as family trips, are displayed preferentially. The input is real-time emotion data, and the output is a dynamically updated album layout.

[0800] (Application example 2)

[0801] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0802] While systems for efficiently digitizing film photographs already exist, these systems typically require a significant amount of effort to organize and select the digitized photographs. Furthermore, systems for suggesting entertainment and scenery based on passengers' emotions in autonomous vehicles are underdeveloped, necessitating a method for optimizing the riding experience. To address these challenges, a more advanced and efficient system is needed that automatically suggests and displays the most appropriate images and entertainment based on the user's emotions.

[0803] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for capturing film photographs as digital images using a scanner, means for automatically sorting digital images with date information, means for predicting and sorting digital images without date information by the year or time they were taken, means for selecting the most suitable image from similar digital images, means for automatically displaying digital images based on a specified person or theme, and means for analyzing images captured by a camera in an autonomous vehicle and suggesting the most suitable scenic image or entertainment based on the user's emotions. This enables efficient digitization and organization of film photographs and also makes it possible to provide an optimal entertainment experience tailored to the emotions of passengers in autonomous vehicles.

[0804] A "scanner" is a device that captures film photographs as digital images.

[0805] A "digital image" is image data that is acquired by a scanner, camera, or the like and can be stored and processed electronically.

[0806] "Sorting" is the process of rearranging digital images based on specific criteria.

[0807] "Date information" refers to the date and time of photography included in a photograph or time information embedded as metadata.

[0808] "Era and time period prediction" is the process of analyzing the characteristics of a digital image and estimating the age and time period when it was taken.

[0809] A "generative AI model" is an artificial intelligence that can analyze digital images and other data to recognize and predict various features and patterns.

[0810] "Best selection" means selecting high-quality images from similar digital images based on evaluation criteria such as resolution, focus, lighting, facial expression, and composition.

[0811] A "specified person or theme" refers to a particular person or theme that the user wants to include in the album.

[0812] "Display" means that the digital image is visually represented on a screen such as a display.

[0813] An "autonomous vehicle" is a vehicle that automatically performs driving operations under computer control.

[0814] A "camera" is a device for taking still or video images.

[0815] "Video analysis" refers to the analysis of video captured by cameras in self-driving vehicles using algorithms and AI.

[0816] "User's emotion" refers to the emotional state recognized from the user's facial expression, voice, behavior, etc.

[0817] A "landscape image" is a digital image of a natural or urban landscape.

[0818] "Entertainment" is content such as music, videos, podcasts, etc. that users can enjoy.

[0819] "Suggestion" means recommending certain content or images to the user.

[0820] This invention provides a system that efficiently digitizes film photographs and automatically organizes and displays them, as well as a system that analyzes images captured by cameras in autonomous vehicles and suggests optimal scenic images and entertainment based on the user's emotions.

[0821] First, the user places a film photo into a scanner to generate a digital image. The digital image captured by the scanner is saved to the device's local storage and then sent to the server. The server extracts date information from the digital image and automatically sorts photos with date information. For photos without date information, a generative AI model is used to analyze features such as facial expressions, backgrounds, clothing, and buildings to predict the era and time period and sort them.

[0822] The server then selects the best photos from among similar digital images, using criteria such as resolution, focus, lighting, facial expression, and composition to select the highest quality photos. Based on people or themes specified by the user, related digital images are searched and automatically displayed in an album format.

[0823] In autonomous vehicles, images captured by onboard cameras are analyzed in real time. The server uses an emotion engine to analyze the user's facial expressions and voice to recognize their emotions. Based on this emotion data, the server uses a generative AI model to select optimal scenery images and entertainment to display on the passenger's screen. For example, if the driver or passenger appears bored, the server will suggest relaxing music or scenery images.

[0824] The hardware required for the server to realize this system includes a scanner, an on-board camera, and an on-board computer with high-performance processing power. The software used also includes a generative AI model, an emotion recognition engine, and an entertainment recommendation engine. The specific technology stack includes OpenCV for image analysis, Amazon Rekognition and Google Cloud Vision for emotion recognition, and Spotify API and YouTube API for entertainment recommendations.

[0825] As a concrete example, a user can use a scanner to scan film photos of family trips from the 1980s stored at home and send them to the server. The server then analyzes the date information, sorts them, and selects the most suitable photos. If the user then selects the theme "Family trips in the 1980s," related photos are displayed in an album format. Furthermore, during long-distance drives in an autonomous vehicle, the system analyzes footage captured by the onboard camera and suggests the most suitable entertainment or scenic images depending on whether the user is relaxed or bored.

[0826] Example prompt sentence:

[0827] 1. "What is the best entertainment option for a bored passenger on a long drive?"

[0828] 2. "Based on the emotions you experience on a family trip, suggest a photo of the scenery seen from the car window."

[0829] This enables the efficient digitization, organization, and display of film photographs, providing an optimal entertainment experience for passengers in autonomous vehicles.

[0830] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0831] Step 1:

[0832] The user places the film photo in the scanner, which then captures the photo as a digital image, which is then saved in the device's local storage.

[0833] (Input) Film photo

[0834] (Output) Digital image

[0835] (Specific operation) The user operates the scanner software and starts scanning a film photograph. The scanner digitizes the photograph at high resolution and saves the resulting digital image on the device.

[0836] Step 2:

[0837] The terminal transmits the digital image stored in the local storage to the server.

[0838] (Input) Digital image

[0839] (Output) Digital image sent to server

[0840] (Specific operation) The terminal uploads the digital image to the server via the network, using a secure file transfer protocol (such as FTP).

[0841] Step 3:

[0842] The server extracts date information from the received digital images and automatically sorts the photos that have date information.

[0843] (Input) Digital image

[0844] (Output) A set of images sorted by date

[0845] (Specific operation) The server uses an algorithm (e.g., OpenCV) to analyze the metadata of image files and the date stamps on the photos to extract date information, and then sorts the digital images based on the extracted date information.

[0846] Step 4:

[0847] For digital images that do not have date information, the server uses a generative AI model to predict and sort the year and time of the photo.

[0848] (Input) Digital image with no date information

[0849] (Output) A set of images sorted based on age and time period

[0850] (Specific operation) The server uses a generative AI model (e.g., TensorFlow) to analyze features such as facial expressions, backgrounds, clothing, and buildings. Based on the analysis results, it predicts the age and time period in which the photos were taken and sorts these digital images.

[0851] Step 5:

[0852] The server groups digital images taken at the same event or scene and selects the highest quality photo based on criteria such as resolution, focus, lighting, facial expression, and composition.

[0853] (Input) Sorted digital images

[0854] (Output) Optimal photo set

[0855] (Specific operation) Uses a clustering algorithm (e.g., k-means) to group images by event or scene, then selects the best photo within each group based on quality criteria.

[0856] Step 6:

[0857] Based on people or themes specified by the user, the server searches for related digital images and automatically displays them in an album format.

[0858] (Input) Specify the person or theme

[0859] (Output) A set of images displayed in album format

[0860] (Specific operation) The server uses the person recognition module and the theme analysis module to search for images that match the specifications (e.g., face recognition algorithm and theme analysis algorithm). The found images are laid out in an album format and displayed to the user.

[0861] Step 7:

[0862] The server analyzes the video captured by the camera of the self-driving vehicle in real time and recognizes the user's emotions.

[0863] (Input) Video captured by an in-vehicle camera

[0864] (Output) Emotion data

[0865] (Specific operation) The server uses an emotion recognition engine (e.g., Amazon Rekognition) to analyze the user's facial expressions and voice in the video to recognize their emotions.

[0866] Step 8:

[0867] The server uses a generative AI model to suggest optimal landscape images and entertainment to the user based on the recognized user emotions.

[0868] (Input) Emotion data

[0869] (Output) Suggested landscape images and entertainment

[0870] (Specific operation) The server uses the generative AI model to select appropriate scenery images and entertainment based on emotional data and display them on the in-car display. For example, if it recognizes that the user is in a relaxing mood, it will suggest relaxing scenery images and music.

[0871] These steps will enable users to efficiently digitize and archive their film photos, while also optimizing the riding experience in autonomous vehicles.

[0872] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0873] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0874] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0875] [Third embodiment]

[0876] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0877] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0878] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0879] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0880] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0881] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0882] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0883] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0884] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0885] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0886] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0887] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0888] This invention relates to a system for efficiently digitizing film photographs and automatically organizing and displaying them.

[0889] Photo scanning process

[0890] The user first places a film photo in the scanner and performs the scan operation. The scanner scans the photo and generates a digital image. This digital image is saved in the device's local storage and then sent to the server.

[0891] Sorting photos with dates

[0892] The server extracts date information from the received digital images. If date information is included, it identifies the specific date of the photo by analyzing the date stamp and metadata in the photo. The digital images with this date information are sorted and stored in chronological order.

[0893] Dating and sorting undated photos

[0894] The server analyzes digital images that do not contain date information using a generative AI model. The generative AI model evaluates features such as facial expressions, backgrounds, clothing, and buildings to predict the age and time period in which the photo was taken. This allows digital images without date information to be sorted based on their estimated age.

[0895] Selecting the best photo

[0896] The server uses a clustering algorithm to group multiple digital images taken at the same event or scene. Within each group, an evaluation module evaluates resolution, focus, lighting, facial expression, composition, and other factors to select the best quality photos. The selected best photos are then stored in a database.

[0897] Display albums based on specified people or themes

[0898] Users specify themes and people they want to include in their albums, such as "family vacations" or "children growing up," or specific people. The server searches the database for relevant photos based on the specified criteria. The person recognition module identifies photos containing the specified people, and the theme analysis module identifies photos that match the specified theme.

[0899] Photos selected from the search results are laid out in an album format and displayed to the user, allowing users to easily digitize their old film photos, efficiently organize them, and immediately use them as albums tailored to their needs.

[0900] Specific examples

[0901] Let's say a user wants to create an album themed around a family trip in the 1980s. First, the user places film photos stored at home in a scanner and scans all of the photos. The digital images obtained by the scanner are sent to a server. The server sorts photos that contain date information by date, and for photos without date information, it uses a generative AI model to identify photos from around the 1980s.

[0902] The server then uses a clustering algorithm to group photos taken at the same event or scene and select the best photos from each group.The user selects "family vacations in the 1980s," and the server uses a person recognition module and a theme analysis module to search for and select related photos, lay them out in an album format, and display them to the user.

[0903] Through the above process, users can efficiently digitize old film photographs without much effort and easily create albums organized chronologically or by theme.

[0904] The processing flow will be explained below.

[0905] Step 1:

[0906] The user places the film photo in the scanner and presses the start scan button.

[0907] Step 2:

[0908] The device operates a scanner to scan film photographs and generate digital images.

[0909] Step 3:

[0910] The device stores the generated digital images in local storage.

[0911] Step 4:

[0912] The terminal transmits the stored digital images to a server.

[0913] Step 5:

[0914] The server analyzes the received digital images and extracts date information by analyzing the date stamps and metadata contained in the images.

[0915] Step 6:

[0916] The server classifies the digital images into those with date information and those without date information.

[0917] Step 7:

[0918] The server sorts the digital images with date information in chronological order.

[0919] Step 8:

[0920] The server uses a generative AI model to predict the age and time of a photo taken for digital images that do not have date information.

[0921] Step 9:

[0922] The server sorts undated digital images based on their predicted age.

[0923] Step 10:

[0924] The server uses a clustering algorithm to group digital images taken at the same event or scene.

[0925] Step 11:

[0926] For each group, the server selects the highest quality photo using an evaluation module that evaluates resolution, focus, lighting, facial expression, composition, etc.

[0927] Step 12:

[0928] The server stores the selected best photos in a database.

[0929] Step 13:

[0930] Users specify themes or people they want to include in their albums (e.g., "Family Vacations in the 1980s" or "Children Growing Up").

[0931] Step 14:

[0932] The server searches the database for relevant photos based on the specified criteria.

[0933] Step 15:

[0934] The server uses a people recognition module to identify photos that contain the specified people.

[0935] Step 16:

[0936] The server uses a theme analysis module to identify photos that match a specified theme.

[0937] Step 17:

[0938] The server lays out the photos selected from the search results in an album format.

[0939] Step 18:

[0940] The server generates the album and displays it to the user.

[0941] By following these steps, users can efficiently digitize their old film photos and easily create albums organized chronologically or thematically.

[0942] Example 1

[0943] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0944] Digitizing film photographs is time-consuming, and organizing and searching for specific photographs is complicated. Furthermore, it is difficult to identify the age or time period of photographs that do not have date information, and it is also extremely difficult to select the most suitable photographs from a large number of photographs. A system that can solve these problems is needed.

[0945] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0946] In this invention, the server includes means for capturing film photographs as digital images, means for saving the digital images in local storage, means for transmitting the saved digital images to the server, means for automatically sorting digital images with date information, means for predicting and sorting digital images without date information by the year or time they were taken, means for clustering digital images taken in similar scenes, means for selecting the most suitable digital image from the clustering results, and means for searching and displaying digital images based on a specified person or theme. This enables users to efficiently digitize film photographs and easily organize and search for them based on year or theme.

[0947] "Film photography" is an analogue form of photography taken using photographic film.

[0948] "Digital image" refers to an image file represented in an electronic data format.

[0949] "Local storage" refers to a storage medium, either internal or external to a computer, that is used to store data.

[0950] A "server" is a computer system that stores, processes, and manages data over a network.

[0951] "Date information" is data indicating the date and time a photo was taken, and is stored as metadata or text information included in the image.

[0952] "Clustering" is a method of grouping data based on similarity.

[0953] A "generative AI model" is an algorithm that uses artificial intelligence technology to analyze data and generate new information.

[0954] A "clustering algorithm" is a computational method for classifying data into groups based on similarity.

[0955] The "optimal digital image" is the digital image that is judged to be of the highest quality based on multiple evaluation criteria, including resolution, focus, lighting, facial expression, and composition.

[0956] A "person recognition module" is a software component for identifying specific people in an image.

[0957] A "theme analysis module" is a software component that analyzes the content of images and identifies matches to specific themes.

[0958] This invention is a system for efficiently digitizing film photographs and automatically organizing and displaying them. This system performs a series of steps, from scanning film photographs to generating, saving, and processing digital images. Specific embodiments are described below.

[0959] 1. Scanning film photos

[0960] First, the user places the film photograph into the scanner and performs the scan operation. The scanner used can be a general photo scanner (e.g., EPSON Perfection V600). The scanner converts the film photograph into a digital image.

[0961] 2. Local storage of digital images

[0962] The device temporarily stores digital images acquired from the scanner in local storage, which is implemented using the Windows or macOS file system.

[0963] 3. Sending digital images to the server

[0964] The device sends the stored digital images to a server using the HTTP protocol, for example using the Python requests library.

[0965] 4. Extracting Date Information

[0966] The server extracts date information from received digital images, using an image processing library (e.g., OpenCV) to detect date stamps in the images and the Pillow library to parse metadata, e.g., EXIF ​​information.

[0967] 5. Sorting photos with dates

[0968] The server sorts the digital images with date information in the order of the extracted dates, using Python's standard sort function.

[0969] 6. Dating undated photographs

[0970] The server uses generative AI models to estimate the age of undated digital images, for example, by using image classification libraries such as TensorFlow or PyTorch to evaluate features of the photo (such as facial expression, background, clothing, and buildings) to estimate the age.

[0971] 7. Photo Clustering

[0972] The server uses clustering algorithms to group digital images taken during the same event or scene, such as k-means clustering or DBSCAN (Density-Based Spatial Clustering of Applications with Noise).

[0973] 8. Selecting the best photo

[0974] The server evaluates each group's photos based on their resolution, focus, lighting, facial expression, composition, etc., and selects the highest quality photo. The evaluation module scores each item using an image evaluation library.

[0975] 9. Enter album conditions

[0976] Users can select themes and people they want to include in their albums, such as "family vacations" or "children growing up," through their device, either via a web interface or a mobile app.

[0977] 10. Search by people or topics

[0978] The server searches the database for relevant photos based on the specified criteria. The person recognition module performs face recognition processing using, for example, the dlib library, and the theme analysis module identifies photos that match a theme using an image captioning model.

[0979] 11. Album Creation and Display

[0980] The server then selects photos from the search results, lays them out in an album format, and displays them to the user. The layout process can be performed using an HTML template engine (e.g., Jinja2) to display them as a web page, or using a PDF generation library (e.g., ReportLab) to output them as PDF.

[0981] Specific examples

[0982] For example, if a user wants to create an album themed around a family trip in the 1980s, they first load the film photos stored at home into a scanner and scan all of the photos. The digital images obtained by the scanner are sent to a server, which sorts photos with date information by date and uses a generative AI model to identify photos without date information as being from around the 1980s. The server then uses a clustering algorithm to group photos taken at the same event or scene and selects the most suitable photo from each group. The user selects "family trip in the 1980s," and the server uses a person recognition module and theme analysis module to search for and select related photos, lay them out in an album format, and displays them to the user.

[0983] Prompt Sentence Examples

[0984] Example 1: "Create a family vacation album from the 1980s. Scan and digitize film photos, parse the date information, and arrange them chronologically. For undated photos, use a generative AI model to estimate the age and sort them appropriately."

[0985] Example 2: "Scan film photos from a family vacation. Send the resulting digital images to a server, group photos from the same event, and select and store the best ones."

[0986] By processing in this manner, users can efficiently digitize film photographs and easily organize and search them by era or theme.

[0987] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0988] Step 1:

[0989] The user places the film photograph in the scanner and performs the scanning operation.

[0990] Input: Film photo

[0991] Output: Digital image

[0992] Specific operation: Scan film photographs using a scanner (e.g., EPSON Perfection V600) and convert them into digital images.

[0993] Step 2:

[0994] The device stores the scanned digital images in local storage.

[0995] Input: Digital image

[0996] Output: Digital image saved to local storage

[0997] What it does: It uses the Windows or macOS file system to temporarily store scanned digital images.

[0998] Step 3:

[0999] The terminal transmits the stored digital images to a server.

[1000] Input: Digital image stored in local storage

[1001] Output: Digital image sent to server

[1002] Specific operation: Uploads a file to the server using the HTTP protocol and, for example, the Python requests library.

[1003] Step 4:

[1004] The server extracts date information from the received digital images.

[1005] Input: Digital image sent to the server

[1006] Output: Digital image with extracted date information

[1007] Specific operation: Detects date stamps in images using an image processing library (OpenCV), and analyzes metadata (EXIF information) using the Pillow library.

[1008] Step 5:

[1009] The server sorts the digital images containing date information in chronological order.

[1010] Input: Digital image with extracted date information

[1011] Output: Digital images sorted by date

[1012] What it does: Organizes digital images by date using Python's standard sorting functions.

[1013] Step 6:

[1014] The server uses generative AI models to estimate the age of undated digital images.

[1015] Input: undated digital image

[1016] Output: Dated digital image

[1017] Specific operation: Using image classification libraries such as TensorFlow and PyTorch, the features of the photo (facial expression, background, clothing, buildings, etc.) are evaluated to estimate the age.

[1018] Step 7:

[1019] The server uses a clustering algorithm to group digital images taken at the same event or scene.

[1020] Input: Dated digital image

[1021] Output: Clustered digital image groups

[1022] What it does: Group digital images using algorithms such as k-means clustering and DBSCAN.

[1023] Step 8:

[1024] The server evaluates the resolution, focus, lighting, facial expression, composition, etc. within each group and selects the highest quality photo.

[1025] Input: Clustered digital image groups

[1026] Output: Optimal digital images

[1027] Specific operation: Using an image evaluation library, each item (resolution, focus, lighting, facial expression, composition, etc.) is scored and the highest quality photo is selected.

[1028] Step 9:

[1029] The user specifies the themes and people they want to include in the album through the terminal.

[1030] Input: Album theme or person

[1031] Output: specified conditions

[1032] What it does: Enter a topic (e.g., "family vacation") and people using the web interface or mobile app.

[1033] Step 10:

[1034] The server searches the database for relevant photos based on the specified criteria.

[1035] Input: Specified conditions (theme or person)

[1036] Output: Related photos found

[1037] What it does: It uses a facial recognition library (dlib) to identify photos containing specified people, and a theme analysis module to identify photos that match specified themes.

[1038] Step 11:

[1039] The server lays out the photos selected from the search results in an album format and displays them to the user.

[1040] Input: Related photos found

[1041] Output: Layout album

[1042] Specific behavior: Display as a web page using an HTML template engine (Jinja2), or output as PDF using a PDF generation library (ReportLab).

[1043] (Application example 1)

[1044] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1045] It is difficult to efficiently digitize film photographs and automatically organize and display them, and it takes a great deal of effort, especially when digitizing and providing old film photographs in brick-and-mortar stores. Organizing photographs with missing date information and selecting the most suitable photograph from multiple similar photographs are also difficult challenges.

[1046] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1047] In this invention, the server includes means for capturing film images as digital data using a scanner, means for automatically sorting digital data with date information, means for predicting and sorting the age or period of digital data without date information, means for selecting the most suitable digital data from similar digital data, means for automatically displaying digital data based on a specified target person or theme, means for use with a smartphone in a physical store, means for estimating the age of digital data without date information using an AI model, means for identifying target people using a face recognition module, and means for identifying photos that match a specified theme using a theme recognition module.This makes it possible to efficiently digitize old film photos and easily organize and display them in a physical store using a smartphone.

[1048] A "scanner" is a device that captures film images as digital data.

[1049] "Digital data" refers to digital image files captured from film images.

[1050] "Date information" refers to data related to the date and time of shooting, such as date labels and meta information that appear on the photo.

[1051] "Sorting" is the process of organizing and rearranging digital data based on specific criteria.

[1052] "Decade" refers to a specific range that indicates when a photo was taken, such as the 1980s.

[1053] "Time" refers to the specific period of time when the photograph was taken, such as a particular season or year.

[1054] An "AI model" refers to a model that uses artificial intelligence to analyze multiple factors and infer specific information from the data.

[1055] "Facial Recognition Module" refers to technology for identifying and recognizing human faces contained in digital data.

[1056] "Theme recognition module" refers to technology that analyzes the image content and themes contained in digital data and identifies information that matches a specified theme.

[1057] "Brick and mortar store" refers to a commercial establishment that offers goods or services in a physical location, such as a photo studio or retail store.

[1058] A "smartphone" refers to a mobile device that not only has the functions of a mobile phone but also has advanced computing power and multifunctionality.

[1059] This invention builds a system that uses a scanner, a server, a smartphone terminal, and multiple software modules to efficiently digitize, organize, and display film photographs.

[1060] The user first places the film image on the scanner and operates the scanner to capture the film image as digital data. The captured digital data is saved in local storage and then sent to the server.

[1061] The server analyzes the digital data and automatically sorts it using the date information if it contains it, and if it does not, it uses a generative AI model to predict the age and time period of the photo and sorts it accordingly.

[1062] The server then uses a clustering algorithm to group digital images taken at the same event or scene. For each cluster, the server evaluates resolution, focus, lighting, facial expression, composition, and other factors to select the highest quality images. This selected data is then stored in a database.

[1063] Furthermore, users can use their smartphones to search and display digital data based on specified people or themes. The face recognition module identifies people, and the theme recognition module identifies photos that match the specified theme. Digital data selected from the search results is laid out and displayed to the user in an album format.

[1064] For example, let's say a user wants to create an album themed around a family trip in the 1980s. First, the user places all the film photos stored at home in a scanner and scans them all. The digital data obtained by the scanner is sent to a server. The server sorts photos that contain date information by date, and for photos without date information, it uses a generative AI model to identify photos from around the 1980s.

[1065] The server then uses a clustering algorithm to group digital data taken at the same event or scene and selects the most suitable data from each group.The user selects "1980s family vacation" on their smartphone, and the server uses face and theme recognition modules to search for and select related digital data, lays it out in an album format, and displays it to the user.

[1066] Example prompts for generative AI models:

[1067] To estimate the age of a photograph, pay attention to the following:

[1068] The clothing and hairstyle of the people in the photo

[1069] Background and buildings

[1070] Photo Tone

[1071] For example, characteristics of the 1980s include brightly colored clothing, big hairstyles, and a particular architectural style.

[1072] Photo: [Link to photo data]

[1073] What era do you estimate the photo was taken in?

[1074] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1075] Step 1:

[1076] A user places a film image on a scanner and operates the scanner to capture the film image as digital data.

[1077] Input: Film images, scanner

[1078] Processing: The scanner scans the film images and generates digital data, which is then stored in local storage.

[1079] Output: Digital data stored in local storage

[1080] Step 2:

[1081] The terminal transmits the digital data captured by the scanner to the server.

[1082] Input: Digital data stored in local storage, terminal, server

[1083] Processing: The device reads the digital data from local storage and sends it to the server over the network.

[1084] Output: Digital data received by the server

[1085] Step 3:

[1086] The server extracts date information from the received digital data and sorts the data having the date information.

[1087] Input: Digital data received by the server

[1088] Processing: The server uses an image analysis engine (e.g., pytesseract) to extract date information from the digital data. Based on the extracted date information, the digital data is sorted in chronological order.

[1089] Output: Digital data sorted by date

[1090] Step 4:

[1091] The server uses a generative AI model to predict and sort the age and time period of digital data that does not have date information.

[1092] Input: Digital data without date information

[1093] Processing: The server passes the input data to the generative AI model to obtain a predicted age and time period. The digital data is then sorted based on the obtained age and time period information.

[1094] Output: Digital data sorted and dated

[1095] Step 5:

[1096] The server uses a clustering algorithm to group digital data captured at the same event or scene.

[1097] Input: Sorted digital data stored on the server

[1098] Processing: The server applies a clustering algorithm to group the digital data based on the features.

[1099] Output: Digital data grouped into clusters

[1100] Step 6:

[1101] The server selects the most suitable digital data within each cluster.

[1102] Input: Digital data grouped into clusters

[1103] Processing: The server uses an evaluation module to evaluate resolution, focus, lighting, facial expression, composition, etc. within each cluster and select the most suitable digital data.

[1104] Output: Optimal digital data selected for each cluster

[1105] Step 7:

[1106] The server searches and displays digital data based on the target person or theme specified by the user.

[1107] Input: Specified target person or theme, selected and sorted digital data

[1108] Processing: The server uses a face recognition module to identify the target person, and a theme recognition module to search for digital data matching the specified theme. The search results are then laid out in an album format and displayed to the user on their smartphone.

[1109] Output: Digital data in album format displayed on a smartphone

[1110] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1111] This invention is a system that efficiently digitizes film photographs and automatically organizes and displays them, and also has the ability to recognize the user's emotions and select and display the most appropriate images.

[1112] Photo scanning process

[1113] The user first places a film photo in the scanner and performs the scan operation. The scanner scans the photo and generates a digital image. This digital image is saved in the device's local storage and then sent to the server.

[1114] Sorting photos with dates

[1115] The server extracts date information from the received digital images. If date information is included, it identifies the specific date of the photo by analyzing the date stamp and metadata in the photo. The digital images with this date information are sorted and stored in chronological order.

[1116] Dating and sorting undated photos

[1117] The server analyzes digital images that do not contain date information using a generative AI model. The generative AI model evaluates features such as facial expressions, backgrounds, clothing, and buildings to predict the age and time period in which the photo was taken. This allows digital images without date information to be sorted based on their estimated age.

[1118] Selecting the best photo

[1119] The server uses a clustering algorithm to group multiple digital images taken at the same event or scene. Within each group, an evaluation module evaluates resolution, focus, lighting, facial expression, composition, and other factors to select the best quality photos. The selected best photos are then stored in a database.

[1120] Implementing the Emotion Engine

[1121] The server includes an emotion engine that recognizes the user's emotions. The emotion engine includes means for analyzing emotions from the user's facial expressions, voice, and actions. The analyzed emotion data is reflected in the selection and display order of photos.

[1122] Display albums based on specified people or themes

[1123] Users specify themes and people they want to include in their albums, such as "family vacations" or "children growing up," or specific people. The server searches the database for relevant photos based on the specified criteria. The person recognition module identifies photos containing the specified people, and the theme analysis module identifies photos that match the specified theme.

[1124] Emotion-based selection and display of optimal photos

[1125] The album layout and display order are dynamically changed based on the user's emotions as recognized by the emotion engine. For example, if the user feels nostalgic for the past, photos that best match that emotion will be displayed preferentially. Furthermore, the system will display more photos of fun events that reflect the user's joy and happiness.

[1126] Specific examples

[1127] Let's say a user wants to create an album themed around a family trip in the 1980s. First, the user places film photos stored at home in a scanner and scans all of the photos. The digital images obtained by the scanner are sent to a server. The server sorts photos that contain date information by date, and for photos without date information, it uses a generative AI model to identify photos from around the 1980s.

[1128] The server then uses a clustering algorithm to group photos taken at the same event or scene and select the best photos from each group.The user selects "family vacations in the 1980s," and the server uses a person recognition module and a theme analysis module to search for and select related photos, lay them out in an album format, and display them to the user.

[1129] Furthermore, while a user is browsing an album, the emotion engine recognizes the user's emotions and dynamically adjusts the photos and album layout based on those emotions. For example, if the user smiles, photos with lots of happy family memories will be prioritized.

[1130] Through this process, users can efficiently digitize their old film photos without much effort and create albums organized chronologically or by theme. Furthermore, by displaying photos optimally according to their emotions, users can enjoy even greater satisfaction.

[1131] The processing flow will be explained below.

[1132] Step 1:

[1133] The user places the film photo in the scanner and presses the start scan button.

[1134] Step 2:

[1135] The device operates a scanner to scan film photographs and generate digital images.

[1136] Step 3:

[1137] The device stores the generated digital images in local storage.

[1138] Step 4:

[1139] The terminal transmits the stored digital images to a server.

[1140] Step 5:

[1141] The server analyzes the received digital images and extracts date information by analyzing the date stamps and metadata contained in the images.

[1142] Step 6:

[1143] The server classifies the digital images into those with date information and those without date information.

[1144] Step 7:

[1145] The server sorts the digital images with date information in chronological order.

[1146] Step 8:

[1147] The server uses a generative AI model to predict the age and time of a photo taken for digital images that do not have date information.

[1148] Step 9:

[1149] The server sorts undated digital images based on their predicted age.

[1150] Step 10:

[1151] The server uses a clustering algorithm to group digital images taken at the same event or scene.

[1152] Step 11:

[1153] For each group, the server selects the highest quality photo using an evaluation module that evaluates resolution, focus, lighting, facial expression, composition, etc.

[1154] Step 12:

[1155] The server stores the selected best photos in a database.

[1156] Step 13:

[1157] Users specify themes or people they want to include in their albums (e.g., "Family Vacations in the 1980s" or "Children Growing Up").

[1158] Step 14:

[1159] The server searches the database for relevant photos based on the specified criteria.

[1160] Step 15:

[1161] The server uses a people recognition module to identify photos that contain the specified people.

[1162] Step 16:

[1163] The server uses a theme analysis module to identify photos that match a specified theme.

[1164] Step 17:

[1165] The server lays out the photos selected from the search results in an album format.

[1166] Step 18:

[1167] The server generates the album and displays it to the user.

[1168] Step 19:

[1169] The server uses an emotion engine to recognize the user's emotions, which analyzes emotions from the user's facial expressions, voice, and behavior.

[1170] Step 20:

[1171] The server dynamically adjusts the display order and layout of the albums based on the analysis results of the emotion engine. For example, if a user is feeling nostalgic, photos that best match that emotion will be displayed first.

[1172] Step 21:

[1173] The server continuously monitors the user's emotional data and updates the displayed content in real time according to changes in the user's emotions. For example, if the user shows a happy expression, the server displays many photos of fun events that emphasize that emotion.

[1174] Specific examples

[1175] Consider a specific scenario where a user wants to create an album themed around a family trip in the 1980s.

[1176] 1. The user places the film photos stored at home into a scanner and scans all of the photos.

[1177] 2. The device performs the scan, generates and saves a digital image, and sends it to the server.

[1178] 3. The server analyzes the received images to extract date information, and if date information is included, sorts them by date.

[1179] 4. For photos without date information, a generative AI model identifies photos from around the 1980s.

[1180] 5. The server uses a clustering algorithm to group photos taken at the same event and selects and stores the best photos.

[1181] 6. The user specifies the theme "Family Vacations in the 1980s," and the server searches for related photos based on the specified criteria.

[1182] 7. The server lays out the photos in an album format and displays them to the user.

[1183] 8. While a user is browsing an album, the emotion engine recognizes the user's emotions and changes the layout of the photos and albums displayed based on those emotions.

[1184] In this way, the user can efficiently digitize past film photographs and realize optimal photo display according to his / her emotions.

[1185] Example 2

[1186] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1187] In today's digital society, there is a need to efficiently digitize, organize, and display old film photographs, but doing this manually requires a great deal of time and effort.In addition to automatically organizing extracted digital images by theme or era, there is also a need to provide a more satisfying user experience by selecting and displaying the most appropriate images based on the user's emotions.

[1188] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1189] In this invention, the server includes means for capturing film photographs as digital images using a scanner, means for automatically sorting digital images with date information, means for predicting and sorting digital images without date information by the year or time they were taken, means for selecting the most suitable image from similar digital images, means for automatically displaying digital images based on a specified person or theme, and means for recognizing a user's emotion and selecting and optimizing the display order of digital images based on that emotion. This allows for efficient digitization and organization of film photographs, and also enables optimal image display according to the user's emotion.

[1190] "Means for capturing film photographs as digital images using a scanner" refers to the device and its operating process for converting film photographs into high-resolution digital format.

[1191] "Means for automatically sorting date-identified digital images" refers to algorithms and software that analyze date labels and metadata contained in digital images and organize the images chronologically based on that information.

[1192] "Means for predicting and sorting the age and period of undated digital images" refers to a process and system that uses a generative AI model to analyze digital images that lack information about the date and time of their capture and estimate the age and period of their capture.

[1193] "Means for selecting optimal images from among similar digital images" refers to a process and system that uses a clustering algorithm to group multiple images taken of the same event or scene, and then evaluates the image quality and selects the optimal image from the group.

[1194] "Means for automatically displaying digital images based on a specified person or subject" means systems and algorithms that search, select, and automatically display relevant images from a database based on user-specified subject or person-specific input.

[1195] "Means for recognizing a user's emotions and selecting and optimizing the display order of digital images based on those emotions" refers to an emotion engine and related technologies that analyzes emotions from a user's facial expressions, voice, movements, etc., and dynamically adjusts the images to be displayed and their order based on the results of that analysis.

[1196] This invention provides a system that allows users to efficiently digitize film photographs and automatically organize and display them. It also has the ability to recognize the user's emotions and select and display the most suitable images.

[1197] Hardware and Software

[1198] The user places a film photograph into a scanner (e.g., a general-purpose film scanner), launches the scanner software, performs a scan, and generates a high-resolution digital image file (JPEG or TIFF format), which is temporarily stored in the device's local storage.

[1199] When a digital image is saved, the device detects it and transmits it over the Internet to a server using a secure channel (e.g., SSL / TLS).

[1200] The server processes the received digital images. For images with date information, it uses open source tools (e.g., OpenCV and ExifTool) to analyze the date labels and metadata to identify the date of capture. Based on the identified date, the images are sorted in chronological order and stored in a database.

[1201] For images that do not contain date information, the server uses a generative AI model (for example, a model built using TensorFlow or PyTorch) to analyze features in the image (for example, facial expressions, backgrounds, clothing, buildings, etc.) to estimate the age and time period in which the image was taken. Based on this information, the undated images are sorted chronologically and stored in a database.

[1202] The server then uses a clustering algorithm (e.g., K-means or DBSCAN) to group multiple images taken at the same event or scene. Within each group, an evaluation module evaluates resolution, focus, lighting, facial expression, composition, etc. to select the highest-quality image. The selected images are then stored in a database.

[1203] Furthermore, the server uses an emotion engine (e.g., Microsoft Azure Cognitive Services or Amazon Rekognition) to recognize the user's emotions. This emotion analysis engine analyzes the user's facial expressions, voice, and behavioral data in real time to estimate their current emotional state. The estimated emotion data is used to customize the image selection and display order.

[1204] The user logs into the system and specifies the theme and people for the album. For example, they can specify themes such as "family trips" or "children growing up" or a specific person. The server uses the person recognition module and theme analysis module to search and select photos from the database that match the specified criteria. The selected photos are laid out in an album format and displayed to the user.

[1205] As a concrete example, let's consider a case where a user creates a family travel album from the 1980s. First, the user places film photos from home in a scanner and scans all the photos. The digital images obtained by the scanner are sent to a server via the device. The server receives the images, analyzes photos with date information, and sorts them by date. For photos without date information, a generative AI model is used to identify photos from around the 1980s.

[1206] The server then uses a clustering algorithm to group photos taken at the same event or scene and select the most suitable photos.When a user selects a theme such as "1980s family vacation," the server uses a person recognition module and a theme analysis module to search for related photos, lays them out in an album format, and displays them to the user.

[1207] Furthermore, the emotion engine recognizes the user's emotions while browsing the album and dynamically adjusts the displayed photos and album layout based on those emotions. For example, when the user smiles, photos that evoke happy memories are displayed first. This system allows users to efficiently digitize film photos and effortlessly create albums organized chronologically or by theme. Displaying optimal photos based on emotions also enhances user satisfaction.

[1208] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1209] Step 1: Scan your film photos

[1210] The user places the film photograph in the scanner and performs the scan operation. Specifically, the user starts the scanner software and presses the scan button to capture the film as a digital image. The input is the film photograph, and the output is a high-resolution digital image file (JPEG or TIFF format).

[1211] Step 2: Save digital images locally and send them to the server

[1212] The device stores the digital image generated by the scanner in its local storage, then detects the image data and transmits it to a server over the Internet using a secure channel. The input is the digital image file, and the output is the digital image data transmitted to the server.

[1213] Step 3: Extract and sort date information

[1214] The server analyzes the received digital images. Specifically, it extracts date labels and metadata using analysis tools such as OpenCV and ExifTool. Photos containing date information are sorted chronologically based on the date they were taken and stored in a database. The input is the digital images and their metadata, and the output is the sorted digital images.

[1215] Step 4: Dating and Sorting Undated Photos

[1216] The server uses a generative AI model to estimate the age and time of a photo taken for digital images that do not contain date information. The model analyzes the image's features and assigns estimated age information. The images are then sorted based on the estimated age information and stored in a database. The input is digital images that do not contain date information, and the output is digital images sorted chronologically.

[1217] Step 5: Clustering and selecting the best photos

[1218] The server uses a clustering algorithm to group multiple images taken at the same event or scene. It forms clusters using K-means or DBSCAN, and within each group, it selects the best photo based on evaluation criteria such as resolution, focus, lighting, facial expression, and composition. The selected best photo is stored in a database. The input is multiple digital images, and the output is the selected best digital image.

[1219] Step 6: Sentiment analysis and database update

[1220] The server uses an emotion engine to analyze the user's emotions. It captures the user's facial expressions, voice, and actions in real time to estimate the user's emotional state. This emotion data is stored in a database for the selection and display order of digital images. The input is the user's real-time data, and the output is the analyzed emotion data.

[1221] Step 7: Album Settings and Search

[1222] The user inputs the album theme or a specific person into the system. The server uses the person recognition module and the theme analysis module to search and select relevant digital images from the database. The input is a prompt about the theme or person, and the output is the selected digital image.

[1223] Step 8: Dynamic optimization of album display based on user sentiment

[1224] The server uses an emotion engine to continuously monitor the user's emotions and dynamically optimize the displayed images and album layout based on those emotions. For example, when the user smiles, photos that reflect happy memories, such as family trips, are displayed preferentially. The input is real-time emotion data, and the output is a dynamically updated album layout.

[1225] (Application example 2)

[1226] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1227] While systems for efficiently digitizing film photographs already exist, these systems typically require a significant amount of effort to organize and select the digitized photographs. Furthermore, systems for suggesting entertainment and scenery based on passengers' emotions in autonomous vehicles are underdeveloped, necessitating a method for optimizing the riding experience. To address these challenges, a more advanced and efficient system is needed that automatically suggests and displays the most appropriate images and entertainment based on the user's emotions.

[1228] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for capturing film photographs as digital images using a scanner, means for automatically sorting digital images with date information, means for predicting and sorting digital images without date information by the year or time they were taken, means for selecting the most suitable image from similar digital images, means for automatically displaying digital images based on a specified person or theme, and means for analyzing images captured by a camera in an autonomous vehicle and suggesting the most suitable scenic image or entertainment based on the user's emotions. This enables efficient digitization and organization of film photographs and also makes it possible to provide an optimal entertainment experience tailored to the emotions of passengers in autonomous vehicles.

[1229] A "scanner" is a device that captures film photographs as digital images.

[1230] A "digital image" is image data that is acquired by a scanner, camera, or the like and can be stored and processed electronically.

[1231] "Sorting" is the process of rearranging digital images based on specific criteria.

[1232] "Date information" refers to the date and time of photography included in a photograph or time information embedded as metadata.

[1233] "Era and time period prediction" is the process of analyzing the characteristics of a digital image and estimating the age and time period when it was taken.

[1234] A "generative AI model" is an artificial intelligence that can analyze digital images and other data to recognize and predict various features and patterns.

[1235] "Best selection" means selecting high-quality images from similar digital images based on evaluation criteria such as resolution, focus, lighting, facial expression, and composition.

[1236] A "specified person or theme" refers to a particular person or theme that the user wants to include in the album.

[1237] "Display" means that the digital image is visually represented on a screen such as a display.

[1238] An "autonomous vehicle" is a vehicle that automatically performs driving operations under computer control.

[1239] A "camera" is a device for taking still or video images.

[1240] "Video analysis" refers to the analysis of video captured by cameras in self-driving vehicles using algorithms and AI.

[1241] "User's emotion" refers to the emotional state recognized from the user's facial expression, voice, behavior, etc.

[1242] A "landscape image" is a digital image of a natural or urban landscape.

[1243] "Entertainment" is content such as music, videos, podcasts, etc. that users can enjoy.

[1244] "Suggestion" means recommending certain content or images to the user.

[1245] This invention provides a system that efficiently digitizes film photographs and automatically organizes and displays them, as well as a system that analyzes images captured by cameras in autonomous vehicles and suggests optimal scenic images and entertainment based on the user's emotions.

[1246] First, the user places a film photo into a scanner to generate a digital image. The digital image captured by the scanner is saved to the device's local storage and then sent to the server. The server extracts date information from the digital image and automatically sorts photos with date information. For photos without date information, a generative AI model is used to analyze features such as facial expressions, backgrounds, clothing, and buildings to predict the era and time period and sort them.

[1247] The server then selects the best photos from among similar digital images, using criteria such as resolution, focus, lighting, facial expression, and composition to select the highest quality photos. Based on people or themes specified by the user, related digital images are searched and automatically displayed in an album format.

[1248] In autonomous vehicles, images captured by onboard cameras are analyzed in real time. The server uses an emotion engine to analyze the user's facial expressions and voice to recognize their emotions. Based on this emotion data, the server uses a generative AI model to select optimal scenery images and entertainment to display on the passenger's screen. For example, if the driver or passenger appears bored, the server will suggest relaxing music or scenery images.

[1249] The hardware required for the server to realize this system includes a scanner, an on-board camera, and an on-board computer with high-performance processing power. The software used also includes a generative AI model, an emotion recognition engine, and an entertainment recommendation engine. The specific technology stack includes OpenCV for image analysis, Amazon Rekognition and Google Cloud Vision for emotion recognition, and Spotify API and YouTube API for entertainment recommendations.

[1250] As a concrete example, a user can use a scanner to scan film photos of family trips from the 1980s stored at home and send them to the server. The server then analyzes the date information, sorts them, and selects the most suitable photos. If the user then selects the theme "Family trips in the 1980s," related photos are displayed in an album format. Furthermore, during long-distance drives in an autonomous vehicle, the system analyzes footage captured by the onboard camera and suggests the most suitable entertainment or scenic images depending on whether the user is relaxed or bored.

[1251] Example prompt sentence:

[1252] 1. "What is the best entertainment option for a bored passenger on a long drive?"

[1253] 2. "Based on the emotions you experience on a family trip, suggest a photo of the scenery seen from the car window."

[1254] This enables the efficient digitization, organization, and display of film photographs, providing an optimal entertainment experience for passengers in autonomous vehicles.

[1255] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1256] Step 1:

[1257] The user places the film photo in the scanner, which then captures the photo as a digital image, which is then saved in the device's local storage.

[1258] (Input) Film photo

[1259] (Output) Digital image

[1260] (Specific operation) The user operates the scanner software and starts scanning a film photograph. The scanner digitizes the photograph at high resolution and saves the resulting digital image on the device.

[1261] Step 2:

[1262] The terminal transmits the digital image stored in the local storage to the server.

[1263] (Input) Digital image

[1264] (Output) Digital image sent to server

[1265] (Specific operation) The terminal uploads the digital image to the server via the network, using a secure file transfer protocol (such as FTP).

[1266] Step 3:

[1267] The server extracts date information from the received digital images and automatically sorts the photos that have date information.

[1268] (Input) Digital image

[1269] (Output) A set of images sorted by date

[1270] (Specific operation) The server uses an algorithm (e.g., OpenCV) to analyze the metadata of image files and the date stamps on the photos to extract date information, and then sorts the digital images based on the extracted date information.

[1271] Step 4:

[1272] For digital images that do not have date information, the server uses a generative AI model to predict and sort the year and time of the photo.

[1273] (Input) Digital image with no date information

[1274] (Output) A set of images sorted based on age and time period

[1275] (Specific operation) The server uses a generative AI model (e.g., TensorFlow) to analyze features such as facial expressions, backgrounds, clothing, and buildings. Based on the analysis results, it predicts the age and time period in which the photos were taken and sorts these digital images.

[1276] Step 5:

[1277] The server groups digital images taken at the same event or scene and selects the highest quality photo based on criteria such as resolution, focus, lighting, facial expression, and composition.

[1278] (Input) Sorted digital images

[1279] (Output) Optimal photo set

[1280] (Specific operation) Uses a clustering algorithm (e.g., k-means) to group images by event or scene, then selects the best photo within each group based on quality criteria.

[1281] Step 6:

[1282] Based on people or themes specified by the user, the server searches for related digital images and automatically displays them in an album format.

[1283] (Input) Specify the person or theme

[1284] (Output) A set of images displayed in album format

[1285] (Specific operation) The server uses the person recognition module and the theme analysis module to search for images that match the specifications (e.g., face recognition algorithm and theme analysis algorithm). The found images are laid out in an album format and displayed to the user.

[1286] Step 7:

[1287] The server analyzes the video captured by the camera of the self-driving vehicle in real time and recognizes the user's emotions.

[1288] (Input) Video captured by an in-vehicle camera

[1289] (Output) Emotion data

[1290] (Specific operation) The server uses an emotion recognition engine (e.g., Amazon Rekognition) to analyze the user's facial expressions and voice in the video to recognize their emotions.

[1291] Step 8:

[1292] The server uses a generative AI model to suggest optimal landscape images and entertainment to the user based on the recognized user emotions.

[1293] (Input) Emotion data

[1294] (Output) Suggested landscape images and entertainment

[1295] (Specific operation) The server uses the generative AI model to select appropriate scenery images and entertainment based on emotional data and display them on the in-car display. For example, if it recognizes that the user is in a relaxing mood, it will suggest relaxing scenery images and music.

[1296] These steps will enable users to efficiently digitize and archive their film photos, while also optimizing the riding experience in autonomous vehicles.

[1297] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1298] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1299] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1300] [Fourth embodiment]

[1301] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1302] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1303] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1304] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1305] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1306] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1307] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1308] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1309] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1310] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1311] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1312] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1313] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1314] This invention relates to a system for efficiently digitizing film photographs and automatically organizing and displaying them.

[1315] Photo scanning process

[1316] The user first places a film photo in the scanner and performs the scan operation. The scanner scans the photo and generates a digital image. This digital image is saved in the device's local storage and then sent to the server.

[1317] Sorting photos with dates

[1318] The server extracts date information from the received digital images. If date information is included, it identifies the specific date of the photo by analyzing the date stamp and metadata in the photo. The digital images with this date information are sorted and stored in chronological order.

[1319] Dating and sorting undated photos

[1320] The server analyzes digital images that do not contain date information using a generative AI model. The generative AI model evaluates features such as facial expressions, backgrounds, clothing, and buildings to predict the age and time period in which the photo was taken. This allows digital images without date information to be sorted based on their estimated age.

[1321] Selecting the best photo

[1322] The server uses a clustering algorithm to group multiple digital images taken at the same event or scene. Within each group, an evaluation module evaluates resolution, focus, lighting, facial expression, composition, and other factors to select the best quality photos. The selected best photos are then stored in a database.

[1323] Display albums based on specified people or themes

[1324] Users specify themes and people they want to include in their albums, such as "family vacations" or "children growing up," or specific people. The server searches the database for relevant photos based on the specified criteria. The person recognition module identifies photos containing the specified people, and the theme analysis module identifies photos that match the specified theme.

[1325] Photos selected from the search results are laid out in an album format and displayed to the user, allowing users to easily digitize their old film photos, efficiently organize them, and immediately use them as albums tailored to their needs.

[1326] Specific examples

[1327] Let's say a user wants to create an album themed around a family trip in the 1980s. First, the user places film photos stored at home in a scanner and scans all of the photos. The digital images obtained by the scanner are sent to a server. The server sorts photos that contain date information by date, and for photos without date information, it uses a generative AI model to identify photos from around the 1980s.

[1328] The server then uses a clustering algorithm to group photos taken at the same event or scene and select the best photos from each group.The user selects "family vacations in the 1980s," and the server uses a person recognition module and a theme analysis module to search for and select related photos, lay them out in an album format, and display them to the user.

[1329] Through the above process, users can efficiently digitize old film photographs without much effort and easily create albums organized chronologically or by theme.

[1330] The processing flow will be explained below.

[1331] Step 1:

[1332] The user places the film photo in the scanner and presses the start scan button.

[1333] Step 2:

[1334] The device operates a scanner to scan film photographs and generate digital images.

[1335] Step 3:

[1336] The device stores the generated digital images in local storage.

[1337] Step 4:

[1338] The terminal transmits the stored digital images to a server.

[1339] Step 5:

[1340] The server analyzes the received digital images and extracts date information by analyzing the date stamps and metadata contained in the images.

[1341] Step 6:

[1342] The server classifies the digital images into those with date information and those without date information.

[1343] Step 7:

[1344] The server sorts the digital images with date information in chronological order.

[1345] Step 8:

[1346] The server uses a generative AI model to predict the age and time of a photo taken for digital images that do not have date information.

[1347] Step 9:

[1348] The server sorts undated digital images based on their predicted age.

[1349] Step 10:

[1350] The server uses a clustering algorithm to group digital images taken at the same event or scene.

[1351] Step 11:

[1352] For each group, the server selects the highest quality photo using an evaluation module that evaluates resolution, focus, lighting, facial expression, composition, etc.

[1353] Step 12:

[1354] The server stores the selected best photos in a database.

[1355] Step 13:

[1356] Users specify themes or people they want to include in their albums (e.g., "Family Vacations in the 1980s" or "Children Growing Up").

[1357] Step 14:

[1358] The server searches the database for relevant photos based on the specified criteria.

[1359] Step 15:

[1360] The server uses a people recognition module to identify photos that contain the specified people.

[1361] Step 16:

[1362] The server uses a theme analysis module to identify photos that match a specified theme.

[1363] Step 17:

[1364] The server lays out the photos selected from the search results in an album format.

[1365] Step 18:

[1366] The server generates the album and displays it to the user.

[1367] By following these steps, users can efficiently digitize their old film photos and easily create albums organized chronologically or thematically.

[1368] Example 1

[1369] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1370] Digitizing film photographs is time-consuming, and organizing and searching for specific photographs is complicated. Furthermore, it is difficult to identify the age or time period of photographs that do not have date information, and it is also extremely difficult to select the most suitable photographs from a large number of photographs. A system that can solve these problems is needed.

[1371] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1372] In this invention, the server includes means for capturing film photographs as digital images, means for saving the digital images in local storage, means for transmitting the saved digital images to the server, means for automatically sorting digital images with date information, means for predicting and sorting digital images without date information by the year or time they were taken, means for clustering digital images taken in similar scenes, means for selecting the most suitable digital image from the clustering results, and means for searching and displaying digital images based on a specified person or theme. This enables users to efficiently digitize film photographs and easily organize and search for them based on year or theme.

[1373] "Film photography" is an analogue form of photography taken using photographic film.

[1374] "Digital image" refers to an image file represented in an electronic data format.

[1375] "Local storage" refers to a storage medium, either internal or external to a computer, that is used to store data.

[1376] A "server" is a computer system that stores, processes, and manages data over a network.

[1377] "Date information" is data indicating the date and time a photo was taken, and is stored as metadata or text information included in the image.

[1378] "Clustering" is a method of grouping data based on similarity.

[1379] A "generative AI model" is an algorithm that uses artificial intelligence technology to analyze data and generate new information.

[1380] A "clustering algorithm" is a computational method for classifying data into groups based on similarity.

[1381] The "optimal digital image" is the digital image that is judged to be of the highest quality based on multiple evaluation criteria, including resolution, focus, lighting, facial expression, and composition.

[1382] A "person recognition module" is a software component for identifying specific people in an image.

[1383] A "theme analysis module" is a software component that analyzes the content of images and identifies matches to specific themes.

[1384] This invention is a system for efficiently digitizing film photographs and automatically organizing and displaying them. This system performs a series of steps, from scanning film photographs to generating, saving, and processing digital images. Specific embodiments are described below.

[1385] 1. Scanning film photos

[1386] First, the user places the film photograph into the scanner and performs the scan operation. The scanner used can be a general photo scanner (e.g., EPSON Perfection V600). The scanner converts the film photograph into a digital image.

[1387] 2. Local storage of digital images

[1388] The device temporarily stores digital images acquired from the scanner in local storage, which is implemented using the Windows or macOS file system.

[1389] 3. Sending digital images to the server

[1390] The device sends the stored digital images to a server using the HTTP protocol, for example using the Python requests library.

[1391] 4. Extracting Date Information

[1392] The server extracts date information from received digital images, using an image processing library (e.g., OpenCV) to detect date stamps in the images and the Pillow library to parse metadata, e.g., EXIF ​​information.

[1393] 5. Sorting photos with dates

[1394] The server sorts the digital images with date information in the order of the extracted dates, using Python's standard sort function.

[1395] 6. Dating undated photographs

[1396] The server uses generative AI models to estimate the age of undated digital images, for example, by using image classification libraries such as TensorFlow or PyTorch to evaluate features of the photo (such as facial expression, background, clothing, and buildings) to estimate the age.

[1397] 7. Photo Clustering

[1398] The server uses clustering algorithms to group digital images taken during the same event or scene, such as k-means clustering or DBSCAN (Density-Based Spatial Clustering of Applications with Noise).

[1399] 8. Selecting the best photo

[1400] The server evaluates each group's photos based on their resolution, focus, lighting, facial expression, composition, etc., and selects the highest quality photo. The evaluation module scores each item using an image evaluation library.

[1401] 9. Enter album conditions

[1402] Users can select themes and people they want to include in their albums, such as "family vacations" or "children growing up," through their device, either via a web interface or a mobile app.

[1403] 10. Search by people or topics

[1404] The server searches the database for relevant photos based on the specified criteria. The person recognition module performs face recognition processing using, for example, the dlib library, and the theme analysis module identifies photos that match a theme using an image captioning model.

[1405] 11. Album Creation and Display

[1406] The server then selects photos from the search results, lays them out in an album format, and displays them to the user. The layout process can be performed using an HTML template engine (e.g., Jinja2) to display them as a web page, or using a PDF generation library (e.g., ReportLab) to output them as PDF.

[1407] Specific examples

[1408] For example, if a user wants to create an album themed around a family trip in the 1980s, they first load the film photos stored at home into a scanner and scan all of the photos. The digital images obtained by the scanner are sent to a server, which sorts photos with date information by date and uses a generative AI model to identify photos without date information as being from around the 1980s. The server then uses a clustering algorithm to group photos taken at the same event or scene and selects the most suitable photo from each group. The user selects "family trip in the 1980s," and the server uses a person recognition module and theme analysis module to search for and select related photos, lay them out in an album format, and displays them to the user.

[1409] Prompt Sentence Examples

[1410] Example 1: "Create a family vacation album from the 1980s. Scan and digitize film photos, parse the date information, and arrange them chronologically. For undated photos, use a generative AI model to estimate the age and sort them appropriately."

[1411] Example 2: "Scan film photos from a family vacation. Send the resulting digital images to a server, group photos from the same event, and select and store the best ones."

[1412] By processing in this manner, users can efficiently digitize film photographs and easily organize and search them by era or theme.

[1413] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1414] Step 1:

[1415] The user places the film photograph in the scanner and performs the scanning operation.

[1416] Input: Film photo

[1417] Output: Digital image

[1418] Specific operation: Scan film photographs using a scanner (e.g., EPSON Perfection V600) and convert them into digital images.

[1419] Step 2:

[1420] The device stores the scanned digital images in local storage.

[1421] Input: Digital image

[1422] Output: Digital image saved to local storage

[1423] What it does: It uses the Windows or macOS file system to temporarily store scanned digital images.

[1424] Step 3:

[1425] The terminal transmits the stored digital images to a server.

[1426] Input: Digital image stored in local storage

[1427] Output: Digital image sent to server

[1428] Specific operation: Uploads a file to the server using the HTTP protocol and, for example, the Python requests library.

[1429] Step 4:

[1430] The server extracts date information from the received digital images.

[1431] Input: Digital image sent to the server

[1432] Output: Digital image with extracted date information

[1433] Specific operation: Detects date stamps in images using an image processing library (OpenCV), and analyzes metadata (EXIF information) using the Pillow library.

[1434] Step 5:

[1435] The server sorts the digital images containing date information in chronological order.

[1436] Input: Digital image with extracted date information

[1437] Output: Digital images sorted by date

[1438] What it does: Organizes digital images by date using Python's standard sorting functions.

[1439] Step 6:

[1440] The server uses generative AI models to estimate the age of undated digital images.

[1441] Input: undated digital image

[1442] Output: Dated digital image

[1443] Specific operation: Using image classification libraries such as TensorFlow and PyTorch, the features of the photo (facial expression, background, clothing, buildings, etc.) are evaluated to estimate the age.

[1444] Step 7:

[1445] The server uses a clustering algorithm to group digital images taken at the same event or scene.

[1446] Input: Dated digital image

[1447] Output: Clustered digital image groups

[1448] What it does: Group digital images using algorithms such as k-means clustering and DBSCAN.

[1449] Step 8:

[1450] The server evaluates the resolution, focus, lighting, facial expression, composition, etc. within each group and selects the highest quality photo.

[1451] Input: Clustered digital image groups

[1452] Output: Optimal digital images

[1453] Specific operation: Using an image evaluation library, each item (resolution, focus, lighting, facial expression, composition, etc.) is scored and the highest quality photo is selected.

[1454] Step 9:

[1455] The user specifies the themes and people they want to include in the album through the terminal.

[1456] Input: Album theme or person

[1457] Output: specified conditions

[1458] What it does: Enter a topic (e.g., "family vacation") and people using the web interface or mobile app.

[1459] Step 10:

[1460] The server searches the database for relevant photos based on the specified criteria.

[1461] Input: Specified conditions (theme or person)

[1462] Output: Related photos found

[1463] What it does: It uses a facial recognition library (dlib) to identify photos containing specified people, and a theme analysis module to identify photos that match specified themes.

[1464] Step 11:

[1465] The server lays out the photos selected from the search results in an album format and displays them to the user.

[1466] Input: Related photos found

[1467] Output: Layout album

[1468] Specific behavior: Display as a web page using an HTML template engine (Jinja2), or output as PDF using a PDF generation library (ReportLab).

[1469] (Application example 1)

[1470] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1471] It is difficult to efficiently digitize film photographs and automatically organize and display them, and it takes a great deal of effort, especially when digitizing and providing old film photographs in brick-and-mortar stores. Organizing photographs with missing date information and selecting the most suitable photograph from multiple similar photographs are also difficult challenges.

[1472] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1473] In this invention, the server includes means for capturing film images as digital data using a scanner, means for automatically sorting digital data with date information, means for predicting and sorting the age or period of digital data without date information, means for selecting the most suitable digital data from similar digital data, means for automatically displaying digital data based on a specified target person or theme, means for use with a smartphone in a physical store, means for estimating the age of digital data without date information using an AI model, means for identifying target people using a face recognition module, and means for identifying photos that match a specified theme using a theme recognition module.This makes it possible to efficiently digitize old film photos and easily organize and display them in a physical store using a smartphone.

[1474] A "scanner" is a device that captures film images as digital data.

[1475] "Digital data" refers to digital image files captured from film images.

[1476] "Date information" refers to data related to the date and time of shooting, such as date labels and meta information that appear on the photo.

[1477] "Sorting" is the process of organizing and rearranging digital data based on specific criteria.

[1478] "Decade" refers to a specific range that indicates when a photo was taken, such as the 1980s.

[1479] "Time" refers to the specific period of time when the photograph was taken, such as a particular season or year.

[1480] An "AI model" refers to a model that uses artificial intelligence to analyze multiple factors and infer specific information from the data.

[1481] "Facial Recognition Module" refers to technology for identifying and recognizing human faces contained in digital data.

[1482] "Theme recognition module" refers to technology that analyzes the image content and themes contained in digital data and identifies information that matches a specified theme.

[1483] "Brick and mortar store" refers to a commercial establishment that offers goods or services in a physical location, such as a photo studio or retail store.

[1484] A "smartphone" refers to a mobile device that not only has the functions of a mobile phone but also has advanced computing power and multifunctionality.

[1485] This invention builds a system that uses a scanner, a server, a smartphone terminal, and multiple software modules to efficiently digitize, organize, and display film photographs.

[1486] The user first places the film image on the scanner and operates the scanner to capture the film image as digital data. The captured digital data is saved in local storage and then sent to the server.

[1487] The server analyzes the digital data and automatically sorts it using the date information if it contains it, and if it does not, it uses a generative AI model to predict the age and time period of the photo and sorts it accordingly.

[1488] The server then uses a clustering algorithm to group digital images taken at the same event or scene. For each cluster, the server evaluates resolution, focus, lighting, facial expression, composition, and other factors to select the highest quality images. This selected data is then stored in a database.

[1489] Furthermore, users can use their smartphones to search and display digital data based on specified people or themes. The face recognition module identifies people, and the theme recognition module identifies photos that match the specified theme. Digital data selected from the search results is laid out and displayed to the user in an album format.

[1490] For example, let's say a user wants to create an album themed around a family trip in the 1980s. First, the user places all the film photos stored at home in a scanner and scans them all. The digital data obtained by the scanner is sent to a server. The server sorts photos that contain date information by date, and for photos without date information, it uses a generative AI model to identify photos from around the 1980s.

[1491] The server then uses a clustering algorithm to group digital data taken at the same event or scene and selects the most suitable data from each group.The user selects "1980s family vacation" on their smartphone, and the server uses face and theme recognition modules to search for and select related digital data, lays it out in an album format, and displays it to the user.

[1492] Example prompts for generative AI models:

[1493] To estimate the age of a photograph, pay attention to the following:

[1494] The clothing and hairstyle of the people in the photo

[1495] Background and buildings

[1496] Photo Tone

[1497] For example, characteristics of the 1980s include brightly colored clothing, big hairstyles, and a particular architectural style.

[1498] Photo: [Link to photo data]

[1499] What era do you estimate the photo was taken in?

[1500] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1501] Step 1:

[1502] A user places a film image on a scanner and operates the scanner to capture the film image as digital data.

[1503] Input: Film images, scanner

[1504] Processing: The scanner scans the film images and generates digital data, which is then stored in local storage.

[1505] Output: Digital data stored in local storage

[1506] Step 2:

[1507] The terminal transmits the digital data captured by the scanner to the server.

[1508] Input: Digital data stored in local storage, terminal, server

[1509] Processing: The device reads the digital data from local storage and sends it to the server over the network.

[1510] Output: Digital data received by the server

[1511] Step 3:

[1512] The server extracts date information from the received digital data and sorts the data having the date information.

[1513] Input: Digital data received by the server

[1514] Processing: The server uses an image analysis engine (e.g., pytesseract) to extract date information from the digital data. Based on the extracted date information, the digital data is sorted in chronological order.

[1515] Output: Digital data sorted by date

[1516] Step 4:

[1517] The server uses a generative AI model to predict and sort the age and time period of digital data that does not have date information.

[1518] Input: Digital data without date information

[1519] Processing: The server passes the input data to the generative AI model to obtain a predicted age and time period. The digital data is then sorted based on the obtained age and time period information.

[1520] Output: Digital data sorted and dated

[1521] Step 5:

[1522] The server uses a clustering algorithm to group digital data captured at the same event or scene.

[1523] Input: Sorted digital data stored on the server

[1524] Processing: The server applies a clustering algorithm to group the digital data based on the features.

[1525] Output: Digital data grouped into clusters

[1526] Step 6:

[1527] The server selects the most suitable digital data within each cluster.

[1528] Input: Digital data grouped into clusters

[1529] Processing: The server uses an evaluation module to evaluate resolution, focus, lighting, facial expression, composition, etc. within each cluster and select the most suitable digital data.

[1530] Output: Optimal digital data selected for each cluster

[1531] Step 7:

[1532] The server searches and displays digital data based on the target person or theme specified by the user.

[1533] Input: Specified target person or theme, selected and sorted digital data

[1534] Processing: The server uses a face recognition module to identify the target person, and a theme recognition module to search for digital data matching the specified theme. The search results are then laid out in an album format and displayed to the user on their smartphone.

[1535] Output: Digital data in album format displayed on a smartphone

[1536] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1537] This invention is a system that efficiently digitizes film photographs and automatically organizes and displays them, and also has the ability to recognize the user's emotions and select and display the most appropriate images.

[1538] Photo scanning process

[1539] The user first places a film photo in the scanner and performs the scan operation. The scanner scans the photo and generates a digital image. This digital image is saved in the device's local storage and then sent to the server.

[1540] Sorting photos with dates

[1541] The server extracts date information from the received digital images. If date information is included, it identifies the specific date of the photo by analyzing the date stamp and metadata in the photo. The digital images with this date information are sorted and stored in chronological order.

[1542] Dating and sorting undated photos

[1543] The server analyzes digital images that do not contain date information using a generative AI model. The generative AI model evaluates features such as facial expressions, backgrounds, clothing, and buildings to predict the age and time period in which the photo was taken. This allows digital images without date information to be sorted based on their estimated age.

[1544] Selecting the best photo

[1545] The server uses a clustering algorithm to group multiple digital images taken at the same event or scene. Within each group, an evaluation module evaluates resolution, focus, lighting, facial expression, composition, and other factors to select the best quality photos. The selected best photos are then stored in a database.

[1546] Implementing the Emotion Engine

[1547] The server includes an emotion engine that recognizes the user's emotions. The emotion engine includes means for analyzing emotions from the user's facial expressions, voice, and actions. The analyzed emotion data is reflected in the selection and display order of photos.

[1548] Display albums based on specified people or themes

[1549] Users specify themes and people they want to include in their albums, such as "family vacations" or "children growing up," or specific people. The server searches the database for relevant photos based on the specified criteria. The person recognition module identifies photos containing the specified people, and the theme analysis module identifies photos that match the specified theme.

[1550] Emotion-based selection and display of optimal photos

[1551] The album layout and display order are dynamically changed based on the user's emotions as recognized by the emotion engine. For example, if the user feels nostalgic for the past, photos that best match that emotion will be displayed preferentially. Furthermore, the system will display more photos of fun events that reflect the user's joy and happiness.

[1552] Specific examples

[1553] Let's say a user wants to create an album themed around a family trip in the 1980s. First, the user places film photos stored at home in a scanner and scans all of the photos. The digital images obtained by the scanner are sent to a server. The server sorts photos that contain date information by date, and for photos without date information, it uses a generative AI model to identify photos from around the 1980s.

[1554] The server then uses a clustering algorithm to group photos taken at the same event or scene and select the best photos from each group.The user selects "family vacations in the 1980s," and the server uses a person recognition module and a theme analysis module to search for and select related photos, lay them out in an album format, and display them to the user.

[1555] Furthermore, while a user is browsing an album, the emotion engine recognizes the user's emotions and dynamically adjusts the photos and album layout based on those emotions. For example, if the user smiles, photos with lots of happy family memories will be prioritized.

[1556] Through this process, users can efficiently digitize their old film photos without much effort and create albums organized chronologically or by theme. Furthermore, by displaying photos optimally according to their emotions, users can enjoy even greater satisfaction.

[1557] The processing flow will be explained below.

[1558] Step 1:

[1559] The user places the film photo in the scanner and presses the start scan button.

[1560] Step 2:

[1561] The device operates a scanner to scan film photographs and generate digital images.

[1562] Step 3:

[1563] The device stores the generated digital images in local storage.

[1564] Step 4:

[1565] The terminal transmits the stored digital images to a server.

[1566] Step 5:

[1567] The server analyzes the received digital images and extracts date information by analyzing the date stamps and metadata contained in the images.

[1568] Step 6:

[1569] The server classifies the digital images into those with date information and those without date information.

[1570] Step 7:

[1571] The server sorts the digital images with date information in chronological order.

[1572] Step 8:

[1573] The server uses a generative AI model to predict the age and time of a photo taken for digital images that do not have date information.

[1574] Step 9:

[1575] The server sorts undated digital images based on their predicted age.

[1576] Step 10:

[1577] The server uses a clustering algorithm to group digital images taken at the same event or scene.

[1578] Step 11:

[1579] For each group, the server selects the highest quality photo using an evaluation module that evaluates resolution, focus, lighting, facial expression, composition, etc.

[1580] Step 12:

[1581] The server stores the selected best photos in a database.

[1582] Step 13:

[1583] Users specify themes or people they want to include in their albums (e.g., "Family Vacations in the 1980s" or "Children Growing Up").

[1584] Step 14:

[1585] The server searches the database for relevant photos based on the specified criteria.

[1586] Step 15:

[1587] The server uses a people recognition module to identify photos that contain the specified people.

[1588] Step 16:

[1589] The server uses a theme analysis module to identify photos that match a specified theme.

[1590] Step 17:

[1591] The server lays out the photos selected from the search results in an album format.

[1592] Step 18:

[1593] The server generates the album and displays it to the user.

[1594] Step 19:

[1595] The server uses an emotion engine to recognize the user's emotions, which analyzes emotions from the user's facial expressions, voice, and behavior.

[1596] Step 20:

[1597] The server dynamically adjusts the display order and layout of the albums based on the analysis results of the emotion engine. For example, if a user is feeling nostalgic, photos that best match that emotion will be displayed first.

[1598] Step 21:

[1599] The server continuously monitors the user's emotional data and updates the displayed content in real time according to changes in the user's emotions. For example, if the user shows a happy expression, the server displays many photos of fun events that emphasize that emotion.

[1600] Specific examples

[1601] Consider a specific scenario where a user wants to create an album themed around a family trip in the 1980s.

[1602] 1. The user places the film photos stored at home into a scanner and scans all of the photos.

[1603] 2. The device performs the scan, generates and saves a digital image, and sends it to the server.

[1604] 3. The server analyzes the received images to extract date information, and if date information is included, sorts them by date.

[1605] 4. For photos without date information, a generative AI model identifies photos from around the 1980s.

[1606] 5. The server uses a clustering algorithm to group photos taken at the same event and selects and stores the best photos.

[1607] 6. The user specifies the theme "Family Vacations in the 1980s," and the server searches for related photos based on the specified criteria.

[1608] 7. The server lays out the photos in an album format and displays them to the user.

[1609] 8. While a user is browsing an album, the emotion engine recognizes the user's emotions and changes the layout of the photos and albums displayed based on those emotions.

[1610] In this way, the user can efficiently digitize past film photographs and realize optimal photo display according to his / her emotions.

[1611] Example 2

[1612] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1613] In today's digital society, there is a need to efficiently digitize, organize, and display old film photographs, but doing this manually requires a great deal of time and effort.In addition to automatically organizing extracted digital images by theme or era, there is also a need to provide a more satisfying user experience by selecting and displaying the most appropriate images based on the user's emotions.

[1614] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1615] In this invention, the server includes means for capturing film photographs as digital images using a scanner, means for automatically sorting digital images with date information, means for predicting and sorting digital images without date information by the year or time they were taken, means for selecting the most suitable image from similar digital images, means for automatically displaying digital images based on a specified person or theme, and means for recognizing a user's emotion and selecting and optimizing the display order of digital images based on that emotion. This allows for efficient digitization and organization of film photographs, and also enables optimal image display according to the user's emotion.

[1616] "Means for capturing film photographs as digital images using a scanner" refers to the device and its operating process for converting film photographs into high-resolution digital format.

[1617] "Means for automatically sorting date-identified digital images" refers to algorithms and software that analyze date labels and metadata contained in digital images and organize the images chronologically based on that information.

[1618] "Means for predicting and sorting the age and period of undated digital images" refers to a process and system that uses a generative AI model to analyze digital images that lack information about the date and time of their capture and estimate the age and period of their capture.

[1619] "Means for selecting optimal images from among similar digital images" refers to a process and system that uses a clustering algorithm to group multiple images taken of the same event or scene, and then evaluates the image quality and selects the optimal image from the group.

[1620] "Means for automatically displaying digital images based on a specified person or subject" means systems and algorithms that search, select, and automatically display relevant images from a database based on user-specified subject or person-specific input.

[1621] "Means for recognizing a user's emotions and selecting and optimizing the display order of digital images based on those emotions" refers to an emotion engine and related technologies that analyzes emotions from a user's facial expressions, voice, movements, etc., and dynamically adjusts the images to be displayed and their order based on the results of that analysis.

[1622] This invention provides a system that allows users to efficiently digitize film photographs and automatically organize and display them. It also has the ability to recognize the user's emotions and select and display the most suitable images.

[1623] Hardware and Software

[1624] The user places a film photograph into a scanner (e.g., a general-purpose film scanner), launches the scanner software, performs a scan, and generates a high-resolution digital image file (JPEG or TIFF format), which is temporarily stored in the device's local storage.

[1625] When a digital image is saved, the device detects it and transmits it over the Internet to a server using a secure channel (e.g., SSL / TLS).

[1626] The server processes the received digital images. For images with date information, it uses open source tools (e.g., OpenCV and ExifTool) to analyze the date labels and metadata to identify the date of capture. Based on the identified date, the images are sorted in chronological order and stored in a database.

[1627] For images that do not contain date information, the server uses a generative AI model (for example, a model built using TensorFlow or PyTorch) to analyze features in the image (for example, facial expressions, backgrounds, clothing, buildings, etc.) to estimate the age and time period in which the image was taken. Based on this information, the undated images are sorted chronologically and stored in a database.

[1628] The server then uses a clustering algorithm (e.g., K-means or DBSCAN) to group multiple images taken at the same event or scene. Within each group, an evaluation module evaluates resolution, focus, lighting, facial expression, composition, etc. to select the highest-quality image. The selected images are then stored in a database.

[1629] Furthermore, the server uses an emotion engine (e.g., Microsoft Azure Cognitive Services or Amazon Rekognition) to recognize the user's emotions. This emotion analysis engine analyzes the user's facial expressions, voice, and behavioral data in real time to estimate their current emotional state. The estimated emotion data is used to customize the image selection and display order.

[1630] The user logs into the system and specifies the theme and people for the album. For example, they can specify themes such as "family trips" or "children growing up" or a specific person. The server uses the person recognition module and theme analysis module to search and select photos from the database that match the specified criteria. The selected photos are laid out in an album format and displayed to the user.

[1631] As a concrete example, let's consider a case where a user creates a family travel album from the 1980s. First, the user places film photos from home in a scanner and scans all the photos. The digital images obtained by the scanner are sent to a server via the device. The server receives the images, analyzes photos with date information, and sorts them by date. For photos without date information, a generative AI model is used to identify photos from around the 1980s.

[1632] The server then uses a clustering algorithm to group photos taken at the same event or scene and select the most suitable photos.When a user selects a theme such as "1980s family vacation," the server uses a person recognition module and a theme analysis module to search for related photos, lays them out in an album format, and displays them to the user.

[1633] Furthermore, the emotion engine recognizes the user's emotions while browsing the album and dynamically adjusts the displayed photos and album layout based on those emotions. For example, when the user smiles, photos that evoke happy memories are displayed first. This system allows users to efficiently digitize film photos and effortlessly create albums organized chronologically or by theme. Displaying optimal photos based on emotions also enhances user satisfaction.

[1634] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1635] Step 1: Scan your film photos

[1636] The user places the film photograph in the scanner and performs the scan operation. Specifically, the user starts the scanner software and presses the scan button to capture the film as a digital image. The input is the film photograph, and the output is a high-resolution digital image file (JPEG or TIFF format).

[1637] Step 2: Save digital images locally and send them to the server

[1638] The device stores the digital image generated by the scanner in its local storage, then detects the image data and transmits it to a server over the Internet using a secure channel. The input is the digital image file, and the output is the digital image data transmitted to the server.

[1639] Step 3: Extract and sort date information

[1640] The server analyzes the received digital images. Specifically, it extracts date labels and metadata using analysis tools such as OpenCV and ExifTool. Photos containing date information are sorted chronologically based on the date they were taken and stored in a database. The input is the digital images and their metadata, and the output is the sorted digital images.

[1641] Step 4: Dating and Sorting Undated Photos

[1642] The server uses a generative AI model to estimate the age and time of a photo taken for digital images that do not contain date information. The model analyzes the image's features and assigns estimated age information. The images are then sorted based on the estimated age information and stored in a database. The input is digital images that do not contain date information, and the output is digital images sorted chronologically.

[1643] Step 5: Clustering and selecting the best photos

[1644] The server uses a clustering algorithm to group multiple images taken at the same event or scene. It forms clusters using K-means or DBSCAN, and within each group, it selects the best photo based on evaluation criteria such as resolution, focus, lighting, facial expression, and composition. The selected best photo is stored in a database. The input is multiple digital images, and the output is the selected best digital image.

[1645] Step 6: Sentiment analysis and database update

[1646] The server uses an emotion engine to analyze the user's emotions. It captures the user's facial expressions, voice, and actions in real time to estimate the user's emotional state. This emotion data is stored in a database for the selection and display order of digital images. The input is the user's real-time data, and the output is the analyzed emotion data.

[1647] Step 7: Album Settings and Search

[1648] The user inputs the album theme or a specific person into the system. The server uses the person recognition module and the theme analysis module to search and select relevant digital images from the database. The input is a prompt about the theme or person, and the output is the selected digital image.

[1649] Step 8: Dynamic optimization of album display based on user sentiment

[1650] The server uses an emotion engine to continuously monitor the user's emotions and dynamically optimize the displayed images and album layout based on those emotions. For example, when the user smiles, photos that reflect happy memories, such as family trips, are displayed preferentially. The input is real-time emotion data, and the output is a dynamically updated album layout.

[1651] (Application example 2)

[1652] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1653] While systems for efficiently digitizing film photographs already exist, these systems typically require a significant amount of effort to organize and select the digitized photographs. Furthermore, systems for suggesting entertainment and scenery based on passengers' emotions in autonomous vehicles are underdeveloped, necessitating a method for optimizing the riding experience. To address these challenges, a more advanced and efficient system is needed that automatically suggests and displays the most appropriate images and entertainment based on the user's emotions.

[1654] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for capturing film photographs as digital images using a scanner, means for automatically sorting digital images with date information, means for predicting and sorting digital images without date information by the year or time they were taken, means for selecting the most suitable image from similar digital images, means for automatically displaying digital images based on a specified person or theme, and means for analyzing images captured by a camera in an autonomous vehicle and suggesting the most suitable scenic image or entertainment based on the user's emotions. This enables efficient digitization and organization of film photographs and also makes it possible to provide an optimal entertainment experience tailored to the emotions of passengers in autonomous vehicles.

[1655] A "scanner" is a device that captures film photographs as digital images.

[1656] A "digital image" is image data that is acquired by a scanner, camera, or the like and can be stored and processed electronically.

[1657] "Sorting" is the process of rearranging digital images based on specific criteria.

[1658] "Date information" refers to the date and time of photography included in a photograph or time information embedded as metadata.

[1659] "Era and time period prediction" is the process of analyzing the characteristics of a digital image and estimating the age and time period when it was taken.

[1660] A "generative AI model" is an artificial intelligence that can analyze digital images and other data to recognize and predict various features and patterns.

[1661] "Best selection" means selecting high-quality images from similar digital images based on evaluation criteria such as resolution, focus, lighting, facial expression, and composition.

[1662] A "specified person or theme" refers to a particular person or theme that the user wants to include in the album.

[1663] "Display" means that the digital image is visually represented on a screen such as a display.

[1664] An "autonomous vehicle" is a vehicle that automatically performs driving operations under computer control.

[1665] A "camera" is a device for taking still or video images.

[1666] "Video analysis" refers to the analysis of video captured by cameras in self-driving vehicles using algorithms and AI.

[1667] "User's emotion" refers to the emotional state recognized from the user's facial expression, voice, behavior, etc.

[1668] A "landscape image" is a digital image of a natural or urban landscape.

[1669] "Entertainment" is content such as music, videos, podcasts, etc. that users can enjoy.

[1670] "Suggestion" means recommending certain content or images to the user.

[1671] This invention provides a system that efficiently digitizes film photographs and automatically organizes and displays them, as well as a system that analyzes images captured by cameras in autonomous vehicles and suggests optimal scenic images and entertainment based on the user's emotions.

[1672] First, the user places a film photo into a scanner to generate a digital image. The digital image captured by the scanner is saved to the device's local storage and then sent to the server. The server extracts date information from the digital image and automatically sorts photos with date information. For photos without date information, a generative AI model is used to analyze features such as facial expressions, backgrounds, clothing, and buildings to predict the era and time period and sort them.

[1673] The server then selects the best photos from among similar digital images, using criteria such as resolution, focus, lighting, facial expression, and composition to select the highest quality photos. Based on people or themes specified by the user, related digital images are searched and automatically displayed in an album format.

[1674] In autonomous vehicles, images captured by onboard cameras are analyzed in real time. The server uses an emotion engine to analyze the user's facial expressions and voice to recognize their emotions. Based on this emotion data, the server uses a generative AI model to select optimal scenery images and entertainment to display on the passenger's screen. For example, if the driver or passenger appears bored, the server will suggest relaxing music or scenery images.

[1675] The hardware required for the server to realize this system includes a scanner, an on-board camera, and an on-board computer with high-performance processing power. The software used also includes a generative AI model, an emotion recognition engine, and an entertainment recommendation engine. The specific technology stack includes OpenCV for image analysis, Amazon Rekognition and Google Cloud Vision for emotion recognition, and Spotify API and YouTube API for entertainment recommendations.

[1676] As a concrete example, a user can use a scanner to scan film photos of family trips from the 1980s stored at home and send them to the server. The server then analyzes the date information, sorts them, and selects the most suitable photos. If the user then selects the theme "Family trips in the 1980s," related photos are displayed in an album format. Furthermore, during long-distance drives in an autonomous vehicle, the system analyzes footage captured by the onboard camera and suggests the most suitable entertainment or scenic images depending on whether the user is relaxed or bored.

[1677] Example prompt sentence:

[1678] 1. "What is the best entertainment option for a bored passenger on a long drive?"

[1679] 2. "Based on the emotions you experience on a family trip, suggest a photo of the scenery seen from the car window."

[1680] This enables the efficient digitization, organization, and display of film photographs, providing an optimal entertainment experience for passengers in autonomous vehicles.

[1681] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1682] Step 1:

[1683] The user places the film photo in the scanner, which then captures the photo as a digital image, which is then saved in the device's local storage.

[1684] (Input) Film photo

[1685] (Output) Digital image

[1686] (Specific operation) The user operates the scanner software and starts scanning a film photograph. The scanner digitizes the photograph at high resolution and saves the resulting digital image on the device.

[1687] Step 2:

[1688] The terminal transmits the digital image stored in the local storage to the server.

[1689] (Input) Digital image

[1690] (Output) Digital image sent to server

[1691] (Specific operation) The terminal uploads the digital image to the server via the network, using a secure file transfer protocol (such as FTP).

[1692] Step 3:

[1693] The server extracts date information from the received digital images and automatically sorts the photos that have date information.

[1694] (Input) Digital image

[1695] (Output) A set of images sorted by date

[1696] (Specific operation) The server uses an algorithm (e.g., OpenCV) to analyze the metadata of image files and the date stamps on the photos to extract date information, and then sorts the digital images based on the extracted date information.

[1697] Step 4:

[1698] For digital images that do not have date information, the server uses a generative AI model to predict and sort the year and time of the photo.

[1699] (Input) Digital image with no date information

[1700] (Output) A set of images sorted based on age and time period

[1701] (Specific operation) The server uses a generative AI model (e.g., TensorFlow) to analyze features such as facial expressions, backgrounds, clothing, and buildings. Based on the analysis results, it predicts the age and time period in which the photos were taken and sorts these digital images.

[1702] Step 5:

[1703] The server groups digital images taken at the same event or scene and selects the highest quality photo based on criteria such as resolution, focus, lighting, facial expression, and composition.

[1704] (Input) Sorted digital images

[1705] (Output) Optimal photo set

[1706] (Specific operation) Uses a clustering algorithm (e.g., k-means) to group images by event or scene, then selects the best photo within each group based on quality criteria.

[1707] Step 6:

[1708] Based on people or themes specified by the user, the server searches for related digital images and automatically displays them in an album format.

[1709] (Input) Specify the person or theme

[1710] (Output) A set of images displayed in album format

[1711] (Specific operation) The server uses the person recognition module and the theme analysis module to search for images that match the specifications (e.g., face recognition algorithm and theme analysis algorithm). The found images are laid out in an album format and displayed to the user.

[1712] Step 7:

[1713] The server analyzes the video captured by the camera of the self-driving vehicle in real time and recognizes the user's emotions.

[1714] (Input) Video captured by an in-vehicle camera

[1715] (Output) Emotion data

[1716] (Specific operation) The server uses an emotion recognition engine (e.g., Amazon Rekognition) to analyze the user's facial expressions and voice in the video to recognize their emotions.

[1717] Step 8:

[1718] The server uses a generative AI model to suggest optimal landscape images and entertainment to the user based on the recognized user emotions.

[1719] (Input) Emotion data

[1720] (Output) Suggested landscape images and entertainment

[1721] (Specific operation) The server uses the generative AI model to select appropriate scenery images and entertainment based on emotional data and display them on the in-car display. For example, if it recognizes that the user is in a relaxing mood, it will suggest relaxing scenery images and music.

[1722] These steps will enable users to efficiently digitize and archive their film photos, while also optimizing the riding experience in autonomous vehicles.

[1723] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1724] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1725] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1726] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1727] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1728] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1729] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1730] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1731] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1732] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1733] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1734] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1735] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1736] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1737] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1738] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1739] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1740] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1741] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1742] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1743] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1744] The following is further disclosed regarding the above embodiment.

[1745] (Claim 1)

[1746] A method to capture film photos as digital images using a scanner,

[1747] means for automatically sorting date-bearing digital images;

[1748] A method for predicting and sorting the age and time of digital images that do not have date information,

[1749] A means of selecting the best among similar digital images;

[1750] a means for automatically displaying digital images based on specified people or subjects;

[1751] A system including:

[1752] (Claim 2)

[1753] 10. The system of claim 1, wherein sorting the date-bearing digital images is performed by analyzing date labels and metadata present on the photographs.

[1754] (Claim 3)

[1755] 10. The system of claim 1, wherein the prediction of the age or time period is performed using a generative AI model that analyzes features such as facial expressions, background, clothing, and buildings.

[1756] (Claim 4)

[1757] 10. The system of claim 1, wherein the selection of the optimal digital image is performed using an evaluation module that evaluates factors such as resolution, focus, lighting, facial expression, and composition.

[1758] (Claim 5)

[1759] 10. The system of claim 1, wherein the automated display of digital images based on specified people and themes is accomplished using a people recognition module and a theme analysis module.

[1760] "Example 1"

[1761] (Claim 1)

[1762] A means of capturing film photographs as digital images;

[1763] a means for storing the digital image in local storage;

[1764] means for transmitting the stored digital images to a server;

[1765] means for automatically sorting date-bearing digital images;

[1766] A method for predicting and sorting the age and time of digital images that do not have date information,

[1767] A means for clustering digital images taken of similar scenes;

[1768] A means for selecting an optimal digital image from the clustering results;

[1769] A means for searching and displaying digital images based on specified people or themes;

[1770] A system including:

[1771] (Claim 2)

[1772] 10. The system of claim 1, wherein sorting the date-bearing digital images is performed by analyzing date labels and metadata present on the photographs.

[1773] (Claim 3)

[1774] 10. The system of claim 1, wherein the prediction of the age or time period is performed using a generative AI model that analyzes features such as facial expressions, background, clothing, and buildings.

[1775] "Application Example 1"

[1776] (Claim 1)

[1777] A method to capture film images as digital data using a scanner,

[1778] A means for automatically sorting digital data having date information;

[1779] A method for predicting and sorting the age and time of digital data that does not have date information,

[1780] A means for selecting the most suitable digital data from among similar digital data;

[1781] A means for automatically displaying digital data based on designated subjects or themes;

[1782] A method used in a physical store using a smartphone;

[1783] A method for estimating the age of undated digital data using AI models; and

[1784] means for identifying a person of interest using a facial recognition module;

[1785] means for identifying photos matching a specified theme using a theme recognition module;

[1786] A system including:

[1787] (Claim 2)

[1788] 10. The system of claim 1, wherein sorting of digital data having date information is performed by analyzing date labels and meta information appearing on film images.

[1789] (Claim 3)

[1790] 10. The system of claim 1, wherein the prediction of era or time period is performed using a generative AI model that analyzes features such as facial expressions, background, clothing, and buildings.

[1791] "Example 2: Combining Emotion Engines"

[1792] (Claim 1)

[1793] A method to capture film photos as digital images using a scanner,

[1794] means for automatically sorting date-bearing digital images;

[1795] A method for predicting and sorting the age and time of digital images that do not have date information,

[1796] A means of selecting the best among similar digital images;

[1797] a means for automatically displaying digital images based on specified people or subjects;

[1798] means for recognizing a user's emotion and selecting and optimizing the display order of digital images based on the emotion;

[1799] A system including:

[1800] (Claim 2)

[1801] 10. The system of claim 1, wherein sorting the date-bearing digital images is performed by analyzing date labels and metadata present on the photographs.

[1802] (Claim 3)

[1803] 10. The system of claim 1, wherein the prediction of the age or time period is performed using a generative AI model that analyzes features such as facial expressions, background, clothing, and buildings.

[1804] *It is okay to leave claim 3 as it is.

[1805] "Application example 2 when combining emotion engines"

[1806] (Claim 1)

[1807] A method to capture film photos as digital images using a scanner,

[1808] means for automatically sorting date-bearing digital images;

[1809] A method for predicting and sorting the age and time of digital images that do not have date information,

[1810] A means of selecting the best among similar digital images;

[1811] a means for automatically displaying digital images based on specified people or subjects;

[1812] A method to analyze images captured by cameras in autonomous vehicles and suggest optimal landscape images and entertainment based on the user's emotions.

[1813] A system including:

[1814] (Claim 2)

[1815] 10. The system of claim 1, wherein sorting the date-bearing digital images is performed by analyzing date labels and metadata present on the photographs.

[1816] (Claim 3)

[1817] 10. The system of claim 1, wherein the prediction of the age or time period is performed using a generative AI model that analyzes features such as facial expressions, background, clothing, and buildings. [Explanation of symbols]

[1818] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A method to capture film photos as digital images using a scanner, means for automatically sorting date-bearing digital images; A method for predicting and sorting the age and time of digital images that do not have date information, A means of selecting the best among similar digital images; a means for automatically displaying digital images based on specified people or subjects; A system including:

2. 10. The system of claim 1, wherein sorting the date-bearing digital images is performed by analyzing date labels and metadata present on the photographs.

3. 10. The system of claim 1, wherein the prediction of the age and time of the photo is performed using a generative AI model that analyzes features such as facial expressions, background, clothing, and buildings.

4. 10. The system of claim 1, wherein the selection of the optimal digital image is performed using an evaluation module that evaluates factors such as resolution, focus, lighting, facial expression, and composition.

5. 10. The system of claim 1, wherein the automatic display of digital images based on specified people and themes is accomplished using a people recognition module and a theme analysis module.

Images