System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving image data from a user; artificial intelligence module means for analyzing the received image data to generate an evaluation score; means for feeding back the evaluation score to the user; means for storing the image data and the evaluation score in a database; and means for generating ranking information based on the stored data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Image sharing applications have become widespread in recent years, but many applications are limited to simply sharing images and provide little feedback on the images posted by users, resulting in insufficient user satisfaction and engagement. Furthermore, the lack of functionality for users to compare their posts with those of other users makes it difficult to improve their skills and maintain motivation. Furthermore, existing systems do not efficiently collect and analyze image data and its evaluation results, making it difficult to improve services based on data. [Means for solving the problem]

[0005] In order to solve the above-mentioned problems, the present invention provides the following means. Specifically, a system is provided that includes means for receiving image data from users, artificial intelligence module means for analyzing the received image data and generating an evaluation score, means for providing feedback on the evaluation score to the user, means for storing the image data and the evaluation score in a database, and means for generating ranking information based on the stored data. This allows users to receive scores and feedback on images they post, and further improve engagement by utilizing a ranking function that allows comparisons with other users. Furthermore, efficient analysis of collected data contributes to improving the quality of the overall service.

[0006] "Image data" refers to digital files of still or moving images that users upload from their devices.

[0007] "User" means a person or entity that uses an image sharing application to post image data and receive feedback.

[0008] The "receiving means" refers to a mechanism or method for importing image data sent from a user into the server.

[0009] An "artificial intelligence module" is an algorithm or software that analyzes uploaded image data and generates an evaluation score based on its content.

[0010] An "evaluation score" is a numerical evaluation result calculated by an artificial intelligence module for image data, and is a numerical representation of the image quality and aesthetic sense.

[0011] "Feedback means" refers to the mechanisms and methods for notifying users of their evaluation scores and communicating the evaluation results.

[0012] A "database" is a collection of digital information that stores received image data and evaluation scores for later reference and analysis.

[0013] "Ranking information" is data that aggregates and organizes the evaluation scores of multiple users and presents them in a form that allows for comparison.

[0014] "Analysis" refers to the series of processes and calculations that the artificial intelligence module performs to convert image data into an evaluation score.

[0015] An "image sharing application" is software and its platform that allows users to post image data and share it with other users. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This system allows users to post image data via an image sharing application, and an artificial intelligence module evaluates the image data to calculate an evaluation score and provide feedback to the user. It also generates and publishes ranking information based on the collected data.

[0038] User uploads an image

[0039] Users launch an image sharing application from their smartphone, PC, or other device and tap the "Post Image" button.

[0040] The device provides users with a gallery and camera function, allowing them to select the images they want to post.

[0041] The user selects an image through the device's interface and taps the "Upload" button.

[0042] The device compresses the image data and generates and transmits an API request to the server.

[0043] Examples:

[0044] Users can upload landscape photos they have taken during their trip using the app's "upload" button, at which point the device will send the selected images to the server.

[0045] Images are received on the server side and evaluated by AI

[0046] The server temporarily stores the image data sent from the terminal and passes it to the artificial intelligence module.

[0047] The artificial intelligence module analyzes the image data and generates an evaluation score based on evaluation criteria (e.g., color, brightness, composition, and match with tags).

[0048] The server stores the generated evaluation scores and image data in a database.

[0049] Examples:

[0050] An artificial intelligence module analyzes the color and composition of the landscape photo and generates an evaluation score of 85 points, which is then stored in a database by the server.

[0051] Provide feedback to users on the evaluation results

[0052] The server generates a feedback message based on the rating score, including an example comment such as "This image scores 85 points. Great composition and colors! A little more sharpness would make it even better."

[0053] The server sends the generated feedback message to the user's terminal.

[0054] The device receives and displays the feedback message and provides the user with the evaluation results.

[0055] Examples:

[0056] A feedback message such as "This image scores 85 points. Great composition and colors! A little more sharpness would make it even better" will be displayed on the user's device.

[0057] Image data collection and analysis

[0058] The server keeps a database of all submitted image data and evaluation scores.

[0059] The server uses the stored data to analyze trends and understand evaluation trends.

[0060] The server uses the obtained analysis results to improve the evaluation algorithm of the artificial intelligence module.

[0061] Examples:

[0062] The server analyzes landscape photos posted over a certain period of time, analyzes current trends, and analyzes the use of popular filters and tags, and uses this data to improve the rating algorithm.

[0063] Ranking function and comparison with other users

[0064] The server aggregates the evaluation scores of all users and generates ranking information.

[0065] The server makes the generated ranking information available to users.

[0066] Users can access the ranking page from their device and check their position.

[0067] Examples:

[0068] The server aggregates the evaluation scores of all users and displays the top 10 users every week. When user "Taro Sato" accesses the ranking page, he can see that he is in 5th place.

[0069] The present invention, configured as described above, allows users to receive instant feedback on images they post and easily compare their images with those of other users. It also makes it possible to improve the service based on the collected data. This increases user engagement and improves the quality of image sharing applications.

[0070] The processing flow will be explained below.

[0071] Step 1:

[0072] The user launches an image sharing application on a device such as a smartphone or PC and taps the "Post Image" button.

[0073] Step 2:

[0074] The device presents the user with a file selection screen, and the user selects an image from the gallery or camera.

[0075] Step 3:

[0076] The user selects the image they want to post and taps the "Upload" button.

[0077] Step 4:

[0078] The terminal compresses the selected image data and generates and transmits an API request to the server.

[0079] Step 5:

[0080] The server receives the image data sent from the terminal and temporarily stores it.

[0081] Step 6:

[0082] The server passes the temporarily stored image data to the artificial intelligence module.

[0083] Step 7:

[0084] An artificial intelligence module analyzes the image data and generates an evaluation score based on evaluation criteria (e.g., color, brightness, composition, and match with tags).

[0085] Step 8:

[0086] The server receives the evaluation score from the artificial intelligence module and stores it in a database together with the image data.

[0087] Step 9:

[0088] The server generates a feedback message based on the rating score, including a comment like "This image scores 85 points. Great composition and colors! A little more sharpness would make it even better."

[0089] Step 10:

[0090] The server sends the generated feedback message to the user's terminal.

[0091] Step 11:

[0092] The device receives the feedback message and displays it to the user.

[0093] Step 12:

[0094] The server continues to store the posted image data and evaluation scores in a database.

[0095] Step 13:

[0096] The server uses the stored data to analyze trends and improve evaluation algorithms.

[0097] Step 14:

[0098] The server aggregates the evaluation scores of all users and generates ranking information.

[0099] Step 15:

[0100] The server publishes the generated ranking information to the user.

[0101] Step 16:

[0102] Users access the ranking page on their device and check their ranking position.

[0103] Example 1

[0104] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0105] In conventional image sharing applications, image ratings and feedback were done manually, making it difficult for users to obtain detailed ratings immediately. Furthermore, there were problems with inconsistent rating criteria, making it difficult to generate fair rankings. Furthermore, trend analysis and algorithm improvements were done manually, resulting in low efficiency of the overall system.

[0106] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0107] In this invention, the server includes means for receiving image data from users, means for compressing and transmitting the received image data, means for temporarily storing the received image data, an artificial intelligence module for analyzing the received image data and generating an evaluation score, means for providing feedback on the evaluation score to the user, means for storing the image data and the evaluation score in a database, means for performing trend analysis and algorithm improvement based on the stored data, and means for generating ranking information based on the stored data. This allows users to receive detailed evaluations in real time, enabling the generation of fair rankings. Furthermore, because trend analysis and algorithm improvement are performed automatically, the efficiency of the entire system is improved.

[0108] "Image data" refers to visual information represented in digital form.

[0109] "User" means a person who uses an image sharing application to post images and receive ratings and feedback.

[0110] "Means for receiving" refers to a mechanism for capturing image data sent by a user.

[0111] "Means for compressing and transmitting" refers to a method for reducing the size of image data and transmitting it to the server.

[0112] "Temporary storage means" refers to a mechanism for temporarily storing received image data.

[0113] "Artificial Intelligence Module" refers to a software or hardware component for analyzing image data based on evaluation criteria and generating an evaluation score.

[0114] The "means for providing feedback" refers to a method for forming a feedback message based on the generated evaluation score and providing it to the user.

[0115] "Database" refers to a system that systematically stores and manages image data and evaluation scores.

[0116] "Trend analysis" refers to the act of extracting and analyzing current trends and patterns based on collected data.

[0117] "Means for improving algorithms" refers to methods for using the results of trend analysis to improve the evaluation process of artificial intelligence modules and other system components.

[0118] "Ranking information" refers to ranking information generated based on the evaluation scores of all users.

[0119] This system allows users to post image data via an image sharing application, and an artificial intelligence module evaluates the image data to calculate an evaluation score and provide feedback to the user. It also generates and publishes ranking information based on the collected data.

[0120] Overall system configuration

[0121] The system consists of various components, including an image sharing application, a server, a database, an artificial intelligence module, a feedback generation system, and a ranking generation system.

[0122] Hardware and software used

[0123] Device: The smartphone or PC used by the user.

[0124] Server: A remote computer that receives, stores, and processes data.

[0125] Database: A system for systematically managing data, such as MySQL (registered trademark) or PostgreSQL.

[0126] Artificial Intelligence Module: An evaluation system built using machine learning libraries such as TENSORFLOW® and PyTorch.

[0127] Specific implementation flow

[0128] 1. User uploads an image

[0129] Users launch an image sharing application from their smartphone, PC, or other device and tap the "Post Image" button.

[0130] The device provides users with a gallery and camera function, allowing them to select the images they want to post.

[0131] The user selects an image through the device's interface and taps the "Upload" button.

[0132] The device compresses the image data and generates and transmits an API request to the server.

[0133] Examples:

[0134] Users can upload landscape photos they have taken during their trip using the app's "upload" button, at which point the device will send the selected images to the server.

[0135] 2. The image is received on the server side and evaluated by AI

[0136] The server temporarily stores the image data sent from the terminal and passes it to the artificial intelligence module.

[0137] The artificial intelligence module analyzes the image data and generates an evaluation score based on evaluation criteria (e.g., color, brightness, composition, and match with tags).

[0138] The server stores the generated evaluation scores and image data in a database.

[0139] Examples:

[0140] An artificial intelligence module analyzes the color and composition of the landscape photo and generates an evaluation score of 85 points, which is then stored in a database by the server.

[0141] 3. Provide feedback to users on the evaluation results

[0142] The server generates a feedback message based on the rating score, including an example comment such as "This image scores 85 points. Great composition and colors! A little more sharpness would make it even better."

[0143] The server sends the generated feedback message to the user's terminal.

[0144] The device receives and displays the feedback message and provides the user with the evaluation results.

[0145] Examples:

[0146] A feedback message such as "This image scores 85 points. Great composition and colors! A little more sharpness would make it even better" will be displayed on the user's device.

[0147] 4. Image data collection and analysis

[0148] The server keeps a database of all submitted image data and evaluation scores.

[0149] The server uses the stored data to analyze trends and understand evaluation trends.

[0150] The server uses the obtained analysis results to improve the evaluation algorithm of the artificial intelligence module.

[0151] Examples:

[0152] The server analyzes landscape photos posted over a certain period of time, analyzes current trends, and analyzes the use of popular filters and tags, and uses this data to improve the rating algorithm.

[0153] 5. Ranking function and comparison with other users

[0154] The server aggregates the evaluation scores of all users and generates ranking information.

[0155] The server makes the generated ranking information available to users.

[0156] Users can access the ranking page from their device and check their position.

[0157] Examples:

[0158] The server aggregates the evaluation scores of all users and displays the top 10 users every week. When a user visits the ranking page, they can see that they are in 5th place.

[0159] Example prompts for generative AI models

[0160] "Rate the color and composition of landscape photos posted to an image sharing application. Also, generate a feedback message based on your rating. For example, this image gets a score of 85. Great composition and color! A little more sharpness would make it even better."

[0161] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0162] Specific flow of program processing

[0163] Step 1: Launching the Application

[0164] Users launch an image sharing application from their smartphone or PC.

[0165] Input: terminal

[0166] Output: The interface that the application launches.

[0167] What happens: A user taps an app icon on their device to launch an application.

[0168] Step 2: Tap the image posting button

[0169] The user taps the "Post Image" button on the application.

[0170] Input: Application interface

[0171] Output: Access to gallery and camera functions

[0172] Specific operation: The user taps the "Post Image" button in the app and moves to the next operation screen.

[0173] Step 3: Select an image

[0174] The device provides users with a gallery and camera function to select images to post.

[0175] The user selects an image using the gallery or camera.

[0176] Input: Device gallery or camera

[0177] Output: Selected image data

[0178] What happens: The user selects a specific image from the device's image gallery or takes a new one using the device's camera.

[0179] Step 4: Upload an image

[0180] The user selects an image through the device's interface and taps the "Upload" button.

[0181] Input: Selected image data

[0182] Output: Compressed image data and API request

[0183] The device compresses the image data and generates and transmits an API request to the server.

[0184] Specific operation: The user taps the "Upload" button in the app, and the device compresses the image data and sends it to the server.

[0185] Step 5: Receiving image data

[0186] The server receives the image data sent from the terminal.

[0187] Input: Compressed image data and API request

[0188] Output: Temporarily saved image data

[0189] Specific operation: The server receives the API request and temporarily stores the image data.

[0190] Step 6: Temporarily save image data

[0191] The server temporarily stores the received image data.

[0192] Input: Received image data

[0193] Output: Temporarily saved image data

[0194] Specific operation: Saves image data in a specified folder or memory area on the server.

[0195] Step 7: Transferring data to the AI ​​module

[0196] The server passes the image data to an artificial intelligence module.

[0197] Input: Temporarily saved image data

[0198] Output: Image data to be analyzed

[0199] Specific operation: The server reads the temporarily stored image data and sends it to the AI ​​platform.

[0200] Step 8: Image analysis using AI

[0201] The artificial intelligence module analyzes the image data and generates an evaluation score based on evaluation criteria (e.g., color, brightness, composition, and match with tags).

[0202] Input: Image data to be analyzed

[0203] Output: Evaluation score

[0204] How it works: The AI ​​analyzes elements such as color, brightness, composition, and tags, and generates an evaluation score based on a scoring model.

[0205] Step 9: Saving to the Database

[0206] The server stores the generated evaluation scores and image data in a database.

[0207] Input: Evaluation scores and image data

[0208] Output: Evaluation scores and image data stored in a database

[0209] Specific operation: The server stores the evaluation score and image data as a database entry.

[0210] Step 10: Generate feedback messages

[0211] The server generates a feedback message based on the rating score.

[0212] Input: Rating score

[0213] Output: Feedback message

[0214] Specific behavior: The server selects a feedback template from the rating score and generates a message.

[0215] Step 11: Sending a feedback message

[0216] The server sends the generated feedback message to the user's terminal.

[0217] Input: Feedback message

[0218] Output: Feedback message displayed on the user's terminal

[0219] Specific operation: The server sends a message to the user's device via push notification or API.

[0220] Step 12: Displaying Messages

[0221] The device receives and displays the feedback message and provides the user with the evaluation results.

[0222] Input: Feedback message

[0223] Output: Feedback that is displayed to the user

[0224] What happens: Your device receives a notification and displays a message in an in-app pop-up or in the notification bar.

[0225] Step 13: Continually save your data

[0226] The server keeps a database of all submitted image data and evaluation scores.

[0227] Input: Newly submitted image data and rating score

[0228] Output: A continuously updated database

[0229] What it does: Every time new data is submitted, the database entry is updated.

[0230] Step 14: Trend analysis

[0231] The server uses the stored data to analyze trends and understand evaluation trends.

[0232] Input: Historical data

[0233] Output: Trend reports and analysis results

[0234] What it does: The server analyzes historical data and extracts trends and popularity.

[0235] Step 15: Improve the algorithm

[0236] The server uses the obtained analysis results to improve the evaluation algorithm of the artificial intelligence module.

[0237] Input: Trend analysis results

[0238] Output: Improved evaluation algorithm

[0239] What it does: The server learns from the data and improves the rating algorithm.

[0240] Step 16: Generate ranking information

[0241] The server aggregates the evaluation scores of all users and generates ranking information.

[0242] Input: User rating score

[0243] Output: Ranking information

[0244] What happens: The server aggregates all the rating scores and creates a ranking list.

[0245] Step 17: Publish ranking information

[0246] The server makes the generated ranking information available to users.

[0247] Input: Ranking information

[0248] Output: Published ranking information

[0249] Specific operation: Publish ranking information on a web page or within an application.

[0250] Step 18: Check your rankings

[0251] Users can access the ranking page from their device and check their position.

[0252] Input: Ranking page URL or link

[0253] Output: User ranking information

[0254] What happens: A user visits the rankings page and checks their rating score and ranking.

[0255] (Application example 1)

[0256] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0257] Conventional image sharing applications lack sufficient ratings and feedback on the images posted by users, resulting in low user engagement. Furthermore, the mechanisms for comparing photos with other users and analyzing trends were inadequate, making it difficult to motivate users to post. In particular, specific advice on how to improve photography skills was not provided.

[0258] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0259] In this invention, the server includes means for receiving image data from users, artificial intelligence module means for analyzing the received image data and generating an evaluation score, means for providing the evaluation score as feedback to the user in real time, means for generating a feedback message based on the evaluation score, means for storing the image data and the evaluation score in a database, means for generating ranking information based on the stored data, and means for publishing the ranking information to users so that users can check their own ranking position. This allows users to receive immediate feedback on the images they post, compare their images with those of other users, and check their own ranking position. Comments containing specific suggestions for improvement are also provided, contributing to the improvement of users' photography skills.

[0260] "Image data" refers to digital image files that a user takes or selects from their device and posts through the application.

[0261] An "artificial intelligence module" is a machine learning model or algorithm for analyzing received image data and generating an evaluation score.

[0262] The "evaluation score" is an evaluation score for the image data generated by the artificial intelligence module.

[0263] "Feedback" refers to messages about evaluation results and areas for improvement provided to users based on their evaluation scores.

[0264] "Database" means a digital storage system for storing and managing image data and evaluation scores.

[0265] "Ranking information" is information that aggregates the evaluation scores of all users and indicates their relative position among other users.

[0266] "Real-time" refers to the time interval during which a user can receive immediate feedback after posting image data.

[0267] A "feedback message" is a comment containing specific improvements that is generated based on the evaluation score.

[0268] This invention is a system in which users post image data through an image sharing application, an AI module evaluates the image data, calculates an evaluation score, and provides feedback to the user. It also has a configuration in which ranking information is generated and published based on the collected data.

[0269] Hardware / Software used

[0270] Hardware:

[0271] Server environment: For example, AWS (registered trademark) EC2 instance

[0272] User device: Smartphone or PC

[0273] software:

[0274] Framework:Flask

[0275] AI model: ResNet50 model provided by PyTorch

[0276] Database: MySQL or PostgreSQL

[0277] Specific processing flow

[0278] 1. Receiving image data:

[0279] Users launch an image sharing application on their smartphone or PC and tap the "Post Image" button. In response to this operation, the device provides a gallery or camera function, allowing the user to select the image they want to post. After selection, the device compresses the image data and sends it to the server.

[0280] 2. AI evaluation:

[0281] The server temporarily stores the received image data and passes it to an artificial intelligence module. The AI ​​model used is PyTorch's ResNet50. The model analyzes the image data and generates an evaluation score based on color, brightness, composition, and degree of match with tags.

[0282] 3. Feedback Processing:

[0283] The server generates a feedback message based on the generated evaluation score. For example, if the score is 85, it will provide a comment with specific suggestions for improvement, such as "This image scores 85. Great composition and colors! A little more sharpness would make it even better."

[0284] 4. Save to database:

[0285] The server stores the evaluation scores and image data in a database, which enables trend analysis and evaluation trends to be identified, and can also be used to improve the AI ​​evaluation algorithm.

[0286] 5. Generating and publishing ranking information:

[0287] The server aggregates the evaluation scores of all users and generates ranking information, which is then made public to users so that they can check their own ranking position.

[0288] Specific examples

[0289] Specifically, users submit landscape photos taken during their travels using the app's "Upload" button. The device compresses the image data and sends it to the server. The server receives the image data, temporarily stores it, and then passes it to a PyTorch ResNet50 model. The model analyzes color, lighting, composition, and tag information to generate an evaluation score. For example, if an evaluation score of 85 points is generated, the server returns a feedback message to the user saying, "This image scores 85 points. Excellent composition and color! A little more sharpness would make it even better." The evaluation scores and image data are also stored in a database, and the server aggregates all user data to generate and publish weekly ranking information.

[0290] Prompt Sentence Examples

[0291] Based on the adjusted settings, the system uses PyTorch's ResNet50 model to evaluate images uploaded by users. ResNet50 is used as a pre-trained model for image classification. The model analyzes the color and composition of the image and generates an evaluation score. Based on the score, the system provides specific feedback to the user in the form of comments suggesting areas for improvement.

[0292] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0293] Step 1:

[0294] The user uploads an image.

[0295] Specific operation: The user launches an image sharing application on a smartphone or PC and taps the "Post Image" button. The device responds to this action by providing gallery and camera functions. The user selects the image they want to post, selects the image through the device's interface, and taps the "Upload" button. At this time, the input is the image data selected by the user. The device compresses this image data and generates and sends an API request to the server. The output is the compressed image data sent to the server.

[0296] Step 2:

[0297] The server receives and stores the image data.

[0298] Specific operation: The server receives image data sent from the terminal. The input is compressed image data sent from the terminal. The server temporarily stores this image data. The output is the temporarily stored image data.

[0299] Step 3:

[0300] The server analyzes the image data and generates an evaluation score.

[0301] Specific operation: The server passes the temporarily stored image data to the artificial intelligence module. The input is the temporarily stored image data. The artificial intelligence module used is the PyTorch ResNet50 model. The model analyzes the image data and generates an evaluation score based on color, brightness, composition, and degree of match with tags. The output is the generated evaluation score. For example, the analysis result of a landscape photo will be a score of 85 points.

[0302] Step 4:

[0303] The server generates a feedback message based on the rating score.

[0304] Specific behavior: The server generates a feedback message based on the generated evaluation score. The input is the evaluation score. The feedback message includes specific improvements. For example, a message like "This image scores 85 points. Great composition and color! A little more sharpness would make it even better." is generated. The output is the feedback message.

[0305] Step 5:

[0306] The server sends a feedback message to the user's device.

[0307] Specific operation: The server sends the generated feedback message to the user's terminal. The input is the generated feedback message. The terminal receives this message and displays it to the user. The output is the feedback message displayed on the terminal.

[0308] Step 6:

[0309] The server stores the evaluation scores and image data in a database.

[0310] Specific operation: The server saves the generated evaluation score and corresponding image data in a database. The input is the evaluation score and image data. The server stores this in the database. The output is the evaluation score and image data saved in the database.

[0311] Step 7:

[0312] The server generates and publishes ranking information.

[0313] Specific operation: The server aggregates the evaluation scores of all users and generates ranking information. The input is all the evaluation score data. The server generates ranking information based on this and publishes it to users. The output is the published ranking information. Users can access the ranking page from their devices and check their position.

[0314] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0315] This invention is a system in which a user posts image data through an image sharing application, and an artificial intelligence module evaluates the image data to calculate an evaluation score and provide feedback to the user. In addition, by combining this system with an emotion engine that recognizes the user's emotions, it is possible to adjust the evaluation score and improve the user experience.

[0316] User uploads an image

[0317] Users launch an image sharing application on their smartphone, PC, or other device and tap the "Post Image" button.

[0318] The device displays a file selection screen to the user, and the user selects an image from the gallery or camera.

[0319] Users select the image they want to post and tap the "Upload" button.

[0320] The device compresses the image data and generates and transmits an API request to the server.

[0321] Examples:

[0322] Users can upload landscape photos they have taken during their trip using the app's "upload" button, at which point the device will send the selected images to the server.

[0323] Images are received on the server side and evaluated by AI

[0324] The server receives the image data sent from the terminal and temporarily stores it.

[0325] The server passes the temporarily stored image data to the artificial intelligence module.

[0326] The artificial intelligence module analyzes the image data and generates an evaluation score based on evaluation criteria (e.g., color, brightness, composition, and match with tags).

[0327] The server stores the generated evaluation scores and image data in a database.

[0328] Examples:

[0329] An artificial intelligence module analyzes the color and composition of the landscape photo and generates an evaluation score of 85 points, which is then stored in a database by the server.

[0330] User sentiment analysis and feedback adjustment

[0331] When uploading an image, the server uses the camera and microphone of the device the user is using to analyze the user's facial expressions and voice, collecting data to recognize emotions.

[0332] The emotion engine analyzes the collected data and identifies the user's emotional state (e.g., happy, sad, excited, etc.).

[0333] The server adjusts the rating score to include the perceived emotional state, for example, if the user is very happy, the rating score may be set somewhat higher.

[0334] Examples:

[0335] When a user uploads an image, the camera analyzes his facial expression and the emotion engine recognizes "happiness," adjusting the rating score from 85 to 88.

[0336] Providing Feedback

[0337] The server generates a feedback message based on the rating score and the user's sentiment analysis, including an example comment such as "This image scores 88 points. Great composition and colors! A little more sharpness would make it even better. I can see that you are pleased with this photo."

[0338] The server sends the generated feedback message to the user's terminal.

[0339] The terminal receives the feedback message and displays it to the user.

[0340] Examples:

[0341] A feedback message such as "This image scores 88 points. Great composition and colors! A little more sharpness would make it even better. I can see that you are happy with this photo" will be displayed on the user's device.

[0342] Image data collection and analysis

[0343] The server keeps a database of all submitted image data and evaluation scores.

[0344] The server analyzes trends and improves the evaluation algorithm based on the stored data and user sentiment data.

[0345] Examples:

[0346] The server analyzes landscape photos posted within a certain period of time and the associated user emotional data, and analyzes the relationship between current trends, popular filters, tag usage, and emotional states. This data is used to improve the evaluation algorithm.

[0347] Ranking function and comparison with other users

[0348] The server aggregates the evaluation scores and emotional data of all users and generates ranking information.

[0349] The server makes the generated ranking information available to users.

[0350] Users can access the ranking page from their device and check their position.

[0351] Examples:

[0352] The server aggregates the evaluation scores and emotional data of all users and displays the top 10 users each week. When user "Sato Taro" accesses the ranking page, he sees that he is in 5th place and the reason for this is also displayed.

[0353] The present invention, configured as described above, allows users to receive instant feedback on images they post and easily compare their views with those of other users. Furthermore, by incorporating user emotional data, it is possible to provide more personalized feedback and experiences, further increasing user engagement. Furthermore, it is possible to improve services based on the collected data.

[0354] The processing flow will be explained below.

[0355] Step 1:

[0356] The user launches an image sharing application on a device such as a smartphone or PC and taps the "Post Image" button.

[0357] Step 2:

[0358] The device presents the user with a file selection screen, and the user selects an image from the gallery or camera.

[0359] Step 3:

[0360] The user selects the image they want to post and taps the "Upload" button.

[0361] Step 4:

[0362] The device compresses the image data and generates and sends an API request to the server.

[0363] Step 5:

[0364] The server receives the image data sent from the terminal and temporarily stores it.

[0365] Step 6:

[0366] The server passes the temporarily stored image data to the artificial intelligence module.

[0367] Step 7:

[0368] An artificial intelligence module analyzes the image data and generates an evaluation score based on evaluation criteria (e.g., color, brightness, composition, and match with tags).

[0369] Step 8:

[0370] The server receives the evaluation score from the artificial intelligence module and stores it in a database together with the image data.

[0371] Step 9:

[0372] The server analyzes the user's facial expressions and voice using the device's camera and microphone to collect emotional data when uploading images.

[0373] Step 10:

[0374] The emotion engine analyzes the collected data and identifies the user's emotional state (e.g., happy, sad, excited, etc.).

[0375] Step 11:

[0376] The server adjusts the rating score to include the perceived emotional state and determines the final rating score, for example, slightly increasing the score if the user is very happy.

[0377] Step 12:

[0378] The server generates a feedback message based on the rating score and the user's sentiment analysis, including a comment like "This image scores 88 points. Great composition and colors! A little more sharpness would make it even better. I can see you're happy with this photo."

[0379] Step 13:

[0380] The server sends the generated feedback message to the user's terminal.

[0381] Step 14:

[0382] The device receives the feedback message and displays it to the user.

[0383] Step 15:

[0384] The server continuously stores the posted image data, evaluation scores, and related user emotional data in a database.

[0385] Step 16:

[0386] The server uses the stored data to analyze trends and improve evaluation algorithms.

[0387] Step 17:

[0388] The server aggregates the evaluation scores and emotional data of all users and generates ranking information.

[0389] Step 18:

[0390] The server publishes the generated ranking information to the user.

[0391] Step 19:

[0392] Users access the ranking page on their device and check their ranking position.

[0393] Example 2

[0394] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0395] In conventional image sharing systems, evaluations of images posted by users are subjective, making it difficult to obtain consistent feedback. Furthermore, they do not take into account the user's emotional state, making it impossible to provide a personalized experience. This has led to challenges in improving user experience and reducing engagement.

[0396] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0397] In this invention, the server includes means for receiving image data from a user, artificial intelligence module means for analyzing the received image data and generating an evaluation score, means for feeding back the evaluation score to the user, means for storing the image data and the evaluation score in a database, means for generating ranking information based on the stored data, emotion engine means for collecting and recognizing the emotional state of the user, and means for adjusting the evaluation score based on the emotional state. This allows users to receive consistent evaluations and feedback, and further enables engagement to be improved through personalized feedback based on the emotional state.

[0398] "Image data" refers to digital image files photographed or acquired by a user.

[0399] "Means for receiving" refers to a process or device for transferring image data from a user to a server over the Internet.

[0400] "Artificial Intelligence Module" refers to the computational algorithms and software used to analyze image data and automatically generate an evaluation score.

[0401] An "evaluation score" is a numerical indicator of the quality and characteristics of an image calculated by an artificial intelligence module.

[0402] "Feedback means" refers to the process or system for notifying users of the evaluation score and providing the results as feedback.

[0403] A "database" is a digital information storage system for efficiently storing and managing image data and evaluation scores.

[0404] "Ranking information" refers to information that ranks image data posted by multiple users based on their evaluation scores.

[0405] An "emotion engine" is software and algorithms used to analyze and recognize a user's emotional state.

[0406] "Emotional state" refers to the psychological state of the user, such as joy, sadness, or excitement, analyzed from their facial expressions and voice.

[0407] "Adjustment" refers to a process or system for appropriately modifying or changing the rating score based on the user's emotional state.

[0408] This invention is a system in which a user posts image data through an image sharing application, and an artificial intelligence module evaluates the image data to calculate an evaluation score and provide feedback to the user. In addition, by combining this system with an emotion engine that recognizes the user's emotions, it is possible to adjust the evaluation score and improve the user experience.

[0409] First, the user launches an image sharing application on a device such as a smartphone or PC and taps the "Post Image" button. Next, the device displays a file selection screen for the user, allowing the user to select an image from the gallery or camera. Once the user selects the image they want to post and taps the "Upload" button, the device compresses the image data and generates and sends an API request to the server.

[0410] For example, when a user posts a landscape photo taken during a trip using the app's "upload" button, the device will compress the selected image and send it to the server.

[0411] The server receives the image data sent from the device and temporarily stores it. The server then passes the temporarily stored image data to an artificial intelligence module. The artificial intelligence module analyzes the image data and generates an evaluation score based on evaluation criteria such as color, brightness, composition, and degree of match with tags. The server then stores the generated evaluation score and image data in a database.

[0412] For example, an artificial intelligence module analyzes the color and composition of a landscape photo and generates an evaluation score of 85 points, which is then stored in a database by the server.

[0413] Next, the server uses the user's device's camera and microphone to analyze the user's facial expressions and voice when uploading an image, collecting data for emotion recognition. The emotion engine analyzes the collected data and identifies the user's emotional state (happiness, sadness, excitement, etc.). The server adjusts the evaluation score taking into account the recognized emotional state. For example, if the user is very happy, the evaluation score can be set higher.

[0414] Specifically, when a user uploads an image, the camera analyzes his facial expression, and the emotion engine recognizes "joy," adjusting the rating score from 85 to 88.

[0415] The server then generates a feedback message based on the evaluation score and the user's sentiment analysis results. For example, it might include a comment like, "This image scores 88 points. Excellent composition and color. A little more sharpness would make it even better. I can see that you are pleased with this photo." The server then sends the generated feedback message to the user's device. The device receives the feedback message and displays it to the user.

[0416] For example, a feedback message such as "This image scores 88 points. Great composition and color. A little more sharpness would make it even better. I can see that you are happy with this photo" will be displayed on the user's device.

[0417] In addition, the server stores all posted image data and evaluation scores in a database. Trend analysis and improvements to the evaluation algorithm are performed based on the stored data and user emotional data. For example, the server analyzes landscape photos posted within a certain period of time and the associated emotional data of users, and analyzes the relationship between current trends, popular filters, tag usage, and emotional states. This data is used to improve the evaluation algorithm.

[0418] Finally, the server aggregates the evaluation scores and sentiment data of all users to generate ranking information, which is then made public to users, who can access the ranking page from their devices to check their ranking.

[0419] For example, the server aggregates the evaluation scores and emotional data of all users and displays the top 10 users every week. When user "Taro Sato" accesses the ranking page, he sees that he is in 5th place and the reason for his ranking is also displayed.

[0420] This invention allows users to receive instant feedback on the images they post and easily compare their images with those of other users. Furthermore, by incorporating user emotional data, it is possible to provide more personalized feedback and experiences, further increasing user engagement. Furthermore, the collected data can be used to improve services.

[0421] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0422] Step 1:

[0423] Users launch an image sharing application on their smartphone, PC, or other device and tap the "Post Image" button. Next, the device displays a file selection screen for the user, allowing them to select an image from their gallery or camera. Finally, the user selects the image they want to post and taps the "Upload" button.

[0424] Input: User-selected image data

[0425] Output: Compressed image data sent to the server

[0426] Specific operation: When a user posts a landscape photo taken during a trip through the app's "Upload" button, the device compresses the selected image, generates an API request, and sends it to the server.

[0427] Step 2:

[0428] The server receives the image data sent from the device and temporarily stores it. The server then passes the temporarily stored image data to an artificial intelligence module. The artificial intelligence module analyzes the image data and generates an evaluation score based on evaluation criteria (color, brightness, composition, and degree of match with tags). The server then stores the generated evaluation score and the image data in a database.

[0429] Input: Received image data

[0430] Output: Evaluation scores and image data stored in a database

[0431] How it works: The AI ​​module analyzes the color and composition of a landscape photo and generates an evaluation score of 85 points. This score is then stored in a database by the server.

[0432] Step 3:

[0433] When uploading an image, the server uses the user's device's camera and microphone to analyze the user's facial expressions and voice, collecting data for emotion recognition. The emotion engine then analyzes the collected data to identify the user's emotional state (happiness, sadness, excitement, etc.). The server then adjusts the evaluation score taking into account the recognized emotional state.

[0434] Input: facial expression and voice data when uploading images

[0435] Output: Adjusted rating score

[0436] How it works: When a user uploads an image, the camera analyzes his facial expression, and the emotion engine recognizes "joy," adjusting the rating score from 85 to 88.

[0437] Step 4:

[0438] The server generates a feedback message based on the evaluation score and the user's sentiment analysis results. The feedback message includes details of the evaluation score, areas for improvement, and comments based on the user's sentiment. The server then sends the generated feedback message to the user's device. The device receives the feedback message and displays it to the user.

[0439] Input: Adjusted rating scores and user sentiment analysis results

[0440] Output: The feedback message that is displayed to the user

[0441] What it does: The device displays a feedback message like, "This image scores 88 points. Great composition and colors. A little more sharpness would make it even better. I can see that you are happy with this photo."

[0442] Step 5:

[0443] The server continuously stores all posted image data and evaluation scores in a database. Based on the stored data and user sentiment data, it analyzes trends and improves the evaluation algorithm.

[0444] Input: Stored image data, rating scores, and emotion data

[0445] Output: Trend analysis and evaluation algorithm improvements

[0446] How it works: The server analyzes landscape photos posted within a certain period of time and the associated user emotional data, and analyzes the relationship between current trends, popular filters, tag usage, and emotional state. This data is used to improve the evaluation algorithm.

[0447] Step 6:

[0448] The server aggregates the evaluation scores and emotion data of all users and generates ranking information, which is then made public to users, who can access the ranking page from their devices to check their ranking.

[0449] Input: Rating scores and sentiment data for all users

[0450] Output: Public ranking information

[0451] How it works: The server aggregates the evaluation scores and sentiment data of all users and displays the top 10 users every week. When a user accesses the ranking page, they can see their ranking and the reason for their ranking.

[0452] (Application example 2)

[0453] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0454] Conventional methods for measuring the effectiveness of advertising have the problem of making it difficult to grasp the viewer's emotional state in real time and reflect it in feedback. Furthermore, to accurately evaluate the effectiveness of advertising, it is necessary to utilize user emotional data, but there has been a lack of means to achieve this. Furthermore, a system that can instantly evaluate and improve advertising performance is needed.

[0455] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0456] In this invention, the server includes means for receiving image data from a user, artificial intelligence module means for analyzing the received image data and generating an evaluation score, emotion engine means for recognizing the user's emotions, means for adjusting the evaluation score based on the user's emotional state, means for providing feedback on the evaluation score to the user, means for storing the image data and the evaluation score in a database, means for generating ranking information based on the stored data, and means for disclosing the ranking information to users so that they can check their ranking position. This enables emotions to be analyzed in real time when viewing an advertisement and provides effective feedback. Furthermore, it becomes possible to generate advertisement evaluations and ranking information based on emotion data, thereby enabling more accurate improvement of advertising performance.

[0457] "Image data" is a collection of visual information posted by users from their devices.

[0458] "User" means an individual or group that uses the system or application to post image data and receive evaluation results and feedback.

[0459] The "artificial intelligence module" is a software component for analyzing received image data and generating an evaluation score.

[0460] The "emotion engine" is a software component that analyzes the user's facial expressions and voice data to recognize their emotional state.

[0461] The "evaluation score" is a numerical evaluation value generated by the artificial intelligence module based on the analyzed image data.

[0462] "Feedback" is information provided to users, such as comments and advice based on rating scores and sentiment analysis.

[0463] A "database" is a storage system for storing image data, evaluation scores, and related information.

[0464] "Ranking information" is information obtained by aggregating and ranking evaluation scores from multiple users.

[0465] The "server" is a computer system that controls the entire process, including receiving, analyzing, and storing image data, and sending feedback.

[0466] The present invention relates to a system for evaluating the effectiveness of advertisements in real time. The system analyzes the emotional state of a user when viewing an advertisement using smart glasses and generates an evaluation score. Specific embodiments are described below.

[0467] System Configuration

[0468] The system consists of the following major components:

[0469] 1. User device: Smart glasses

[0470] 2. Server

[0471] 3. Artificial Intelligence Module

[0472] 4. Emotion Engine

[0473] 5. Database

[0474] 6. Feedback Generation Module

[0475] 7. Ranking Generation Module

[0476] Hardware and Software Use

[0477] Hardware

[0478] Smart glasses: Use built-in cameras and microphones to collect facial expression and voice data from the user.

[0479] Server: Processes and stores data using high-performance processors and large-capacity storage.

[0480] software

[0481] Python: The main implementation language for programs

[0482] OpenCV: A library for video capture and image processing

[0483] EmotionEngine: A software component (virtual module) for analyzing user emotions

[0484] AdEvaluator: A software component (virtual module) for generating ad evaluation scores.

[0485] FeedbackGenerator: A software component (virtual module) for generating feedback messages.

[0486] Cloud services (e.g., AWS, Google® Cloud, Azure®): Distributed computing environments for data storage and analysis

[0487] System Operation

[0488] User's device

[0489] When a user wears the smart glasses and watches an advertisement, the built-in camera and microphone collect facial expression and voice data, which is then transmitted to a server in real time.

[0490] server

[0491] The server receives and temporarily stores the image and audio data sent from the user's device. The received data is processed as follows:

[0492] 1. Emotion analysis: The server uses the Emotion Engine to analyze the user's emotions from the image data. Specifically, it recognizes emotions such as "happiness," "surprise," and "sadness" from facial expressions and voice tone.

[0493] 2. Generating an ad rating score: The server uses AdEvaluator to generate a rating score for the received ad data. This rating score is calculated based on the visual elements of the ad, its content, and the user's viewing time.

[0494] 3. Rating Score Adjustment: The server adjusts the generated rating score based on the user's recognized emotional state. For example, if the user has the emotion "joy," the server may set the rating score slightly higher.

[0495] 4. Feedback Generation: The server uses the FeedbackGenerator to generate a feedback message based on the adjusted rating score. The feedback message includes specific advice and rating results to show to the user.

[0496] Data storage and ranking information generation

[0497] The server stores the generated evaluation scores and related data in a database. Based on the stored data, a ranking generation module generates ranking information, allowing users to check their own evaluation scores and ranking positions.

[0498] Specific examples

[0499] As a specific example, data processing when viewing an advertisement will be shown.

[0500] 1. A user puts on the smart glasses and watches an advertisement.

[0501] 2. The camera and microphone in the smart glasses collect data and send it to the server.

[0502] 3. The server uses EmotionEngine to perform emotion analysis and recognize the emotion of "joy."

[0503] 4. The server uses the AdEvaluator to generate a rating score for the ad (e.g., 85 points).

[0504] 5. Adjust the rating score slightly (e.g., 88 points) based on the perceived emotional state.

[0505] 6. Use FeedbackGenerator to generate a feedback message such as "This ad scores 88 points."

[0506] 7. The server sends the evaluation score and feedback message to the smart glasses and displays it to the user.

[0507] Prompt Sentence Examples

[0508] "Design a system that uses the camera and microphone on smart glasses to analyze emotions in real time when a user watches an advertisement, and evaluate the effectiveness of the advertisement. Adjust the evaluation score based on the emotion analysis data and generate a feedback message. Design this as a Python application that runs on the smart glasses, and clearly state the necessary modules and cloud services."

[0509] The system of the present invention configured as described above contributes to improving the quality of advertising and engagement by evaluating the effectiveness of advertising in real time and providing feedback based on the user's emotional state.

[0510] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0511] Step 1:

[0512] Users wear the smart glasses and watch advertisements, while the built-in camera and microphone collect facial expression and voice data.

[0513] Specifically, the smart glasses' camera captures the user's face frame by frame and simultaneously acquires voice data. These data are sent from the device to the server in real time. The input is facial expression and voice data, and the output is the data sent to the server.

[0514] Step 2:

[0515] The server receives the image data and audio data sent from the terminal and temporarily stores them.

[0516] Specifically, the server temporarily stores the received data in a specific folder or memory. The input is the user's facial expression and voice data, and the output is the temporarily stored raw data.

[0517] Step 3:

[0518] The server uses EmotionEngine to analyze the user's emotions from the temporarily stored image data.

[0519] Specifically, the Emotion Engine analyzes facial features and voice tones to recognize emotional states such as "happiness," "surprise," and "sadness." The input is temporarily saved image and voice data, and the output is recognized emotional data.

[0520] Step 4:

[0521] The server uses the AdEvaluator to generate an evaluation score for the advertising data.

[0522] Specifically, AdEvaluator calculates a rating score based on the visual elements of the ad, its content, the user's viewing time, etc. The input is the ad data, and the output is a rating score (e.g., 85 points).

[0523] Step 5:

[0524] The server adjusts the generated evaluation score based on the recognized emotion data.

[0525] Specifically, AdEvaluator adjusts the final rating score by taking into account the emotional data obtained from EmotionEngine. For example, if the emotion is "joy," the rating score is adjusted from 85 to 88. The input is the rating score and emotional data, and the output is the adjusted rating score.

[0526] Step 6:

[0527] The server uses a FeedbackGenerator to generate feedback messages based on the adjusted reputation scores.

[0528] Specifically, the FeedbackGenerator generates a feedback message containing the evaluation result and advice to be shown to the user based on the adjusted evaluation score. The input is the adjusted evaluation score, and the output is the feedback message.

[0529] Step 7:

[0530] The server sends the generated feedback message to the user's terminal and displays it on the smart glasses.

[0531] Specifically, the server sends the generated feedback message to the user's smart glasses in real time and visually displays it. The input is the feedback message, and the output is the display on the smart glasses.

[0532] Step 8:

[0533] The server stores the generated evaluation scores and related data in a database.

[0534] Specifically, the server stores the evaluation scores, sentiment analysis data, and feedback messages in a database. The input is the evaluation scores and related data, and the output is the saved data.

[0535] Step 9:

[0536] The server generates ranking information based on the stored data and allows users to check their ranking position.

[0537] Specifically, the ranking generation module analyzes the data of all users and generates a ranking of their evaluation scores. Users can view the ranking information through smart glasses or other devices. The input is the stored data, and the output is the generated ranking information.

[0538] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0539] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0540] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0541] [Second embodiment]

[0542] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0543] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0544] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0545] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0546] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0547] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0548] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0549] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0550] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0551] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0552] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0553] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0554] This system allows users to post image data via an image sharing application, and an artificial intelligence module evaluates the image data to calculate an evaluation score and provide feedback to the user. It also generates and publishes ranking information based on the collected data.

[0555] User uploads an image

[0556] Users launch an image sharing application from their smartphone, PC, or other device and tap the "Post Image" button.

[0557] The device provides users with a gallery and camera function, allowing them to select the images they want to post.

[0558] The user selects an image through the device's interface and taps the "Upload" button.

[0559] The device compresses the image data and generates and transmits an API request to the server.

[0560] Examples:

[0561] Users can upload landscape photos they have taken during their trip using the app's "upload" button, at which point the device will send the selected images to the server.

[0562] Images are received on the server side and evaluated by AI

[0563] The server temporarily stores the image data sent from the terminal and passes it to the artificial intelligence module.

[0564] The artificial intelligence module analyzes the image data and generates an evaluation score based on evaluation criteria (e.g., color, brightness, composition, and match with tags).

[0565] The server stores the generated evaluation scores and image data in a database.

[0566] Examples:

[0567] An artificial intelligence module analyzes the color and composition of the landscape photo and generates an evaluation score of 85 points, which is then stored in a database by the server.

[0568] Provide feedback to users on the evaluation results

[0569] The server generates a feedback message based on the rating score, including an example comment such as "This image scores 85 points. Great composition and colors! A little more sharpness would make it even better."

[0570] The server sends the generated feedback message to the user's terminal.

[0571] The device receives and displays the feedback message and provides the user with the evaluation results.

[0572] Examples:

[0573] A feedback message such as "This image scores 85 points. Great composition and colors! A little more sharpness would make it even better" will be displayed on the user's device.

[0574] Image data collection and analysis

[0575] The server keeps a database of all submitted image data and evaluation scores.

[0576] The server uses the stored data to analyze trends and understand evaluation trends.

[0577] The server uses the obtained analysis results to improve the evaluation algorithm of the artificial intelligence module.

[0578] Examples:

[0579] The server analyzes landscape photos posted over a certain period of time, analyzes current trends, and analyzes the use of popular filters and tags, and uses this data to improve the rating algorithm.

[0580] Ranking function and comparison with other users

[0581] The server aggregates the evaluation scores of all users and generates ranking information.

[0582] The server makes the generated ranking information available to users.

[0583] Users can access the ranking page from their device and check their position.

[0584] Examples:

[0585] The server aggregates the evaluation scores of all users and displays the top 10 users every week. When user "Taro Sato" accesses the ranking page, he can see that he is in 5th place.

[0586] The present invention, configured as described above, allows users to receive instant feedback on images they post and easily compare their images with those of other users. It also makes it possible to improve the service based on the collected data. This increases user engagement and improves the quality of image sharing applications.

[0587] The processing flow will be explained below.

[0588] Step 1:

[0589] The user launches an image sharing application on a device such as a smartphone or PC and taps the "Post Image" button.

[0590] Step 2:

[0591] The device presents the user with a file selection screen, and the user selects an image from the gallery or camera.

[0592] Step 3:

[0593] The user selects the image they want to post and taps the "Upload" button.

[0594] Step 4:

[0595] The terminal compresses the selected image data and generates and transmits an API request to the server.

[0596] Step 5:

[0597] The server receives the image data sent from the terminal and temporarily stores it.

[0598] Step 6:

[0599] The server passes the temporarily stored image data to the artificial intelligence module.

[0600] Step 7:

[0601] An artificial intelligence module analyzes the image data and generates an evaluation score based on evaluation criteria (e.g., color, brightness, composition, and match with tags).

[0602] Step 8:

[0603] The server receives the evaluation score from the artificial intelligence module and stores it in a database together with the image data.

[0604] Step 9:

[0605] The server generates a feedback message based on the rating score, including a comment like "This image scores 85 points. Great composition and colors! A little more sharpness would make it even better."

[0606] Step 10:

[0607] The server sends the generated feedback message to the user's terminal.

[0608] Step 11:

[0609] The device receives the feedback message and displays it to the user.

[0610] Step 12:

[0611] The server continues to store the posted image data and evaluation scores in a database.

[0612] Step 13:

[0613] The server uses the stored data to analyze trends and improve evaluation algorithms.

[0614] Step 14:

[0615] The server aggregates the evaluation scores of all users and generates ranking information.

[0616] Step 15:

[0617] The server publishes the generated ranking information to the user.

[0618] Step 16:

[0619] Users access the ranking page on their device and check their ranking position.

[0620] Example 1

[0621] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0622] In conventional image sharing applications, image ratings and feedback were done manually, making it difficult for users to obtain detailed ratings immediately. Furthermore, there were problems with inconsistent rating criteria, making it difficult to generate fair rankings. Furthermore, trend analysis and algorithm improvements were done manually, resulting in low efficiency of the overall system.

[0623] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0624] In this invention, the server includes means for receiving image data from users, means for compressing and transmitting the received image data, means for temporarily storing the received image data, an artificial intelligence module for analyzing the received image data and generating an evaluation score, means for providing feedback on the evaluation score to the user, means for storing the image data and the evaluation score in a database, means for performing trend analysis and algorithm improvement based on the stored data, and means for generating ranking information based on the stored data. This allows users to receive detailed evaluations in real time, enabling the generation of fair rankings. Furthermore, because trend analysis and algorithm improvement are performed automatically, the efficiency of the entire system is improved.

[0625] "Image data" refers to visual information represented in digital form.

[0626] "User" means a person who uses an image sharing application to post images and receive ratings and feedback.

[0627] "Means for receiving" refers to a mechanism for capturing image data sent by a user.

[0628] "Means for compressing and transmitting" refers to a method for reducing the size of image data and transmitting it to the server.

[0629] "Temporary storage means" refers to a mechanism for temporarily storing received image data.

[0630] "Artificial Intelligence Module" refers to a software or hardware component for analyzing image data based on evaluation criteria and generating an evaluation score.

[0631] The "means for providing feedback" refers to a method for forming a feedback message based on the generated evaluation score and providing it to the user.

[0632] "Database" refers to a system that systematically stores and manages image data and evaluation scores.

[0633] "Trend analysis" refers to the act of extracting and analyzing current trends and patterns based on collected data.

[0634] "Means for improving algorithms" refers to methods for using the results of trend analysis to improve the evaluation process of artificial intelligence modules and other system components.

[0635] "Ranking information" refers to ranking information generated based on the evaluation scores of all users.

[0636] This system allows users to post image data via an image sharing application, and an artificial intelligence module evaluates the image data to calculate an evaluation score and provide feedback to the user. It also generates and publishes ranking information based on the collected data.

[0637] Overall system configuration

[0638] The system consists of various components, including an image sharing application, a server, a database, an artificial intelligence module, a feedback generation system, and a ranking generation system.

[0639] Hardware and software used

[0640] Device: The smartphone or PC used by the user.

[0641] Server: A remote computer that receives, stores, and processes data.

[0642] Database: A system for managing data in an organized manner, such as MySQL or PostgreSQL.

[0643] Artificial Intelligence Module: A rating system built using machine learning libraries such as TensorFlow and PyTorch.

[0644] Specific implementation flow

[0645] 1. User uploads an image

[0646] Users launch an image sharing application from their smartphone, PC, or other device and tap the "Post Image" button.

[0647] The device provides users with a gallery and camera function, allowing them to select the images they want to post.

[0648] The user selects an image through the device's interface and taps the "Upload" button.

[0649] The device compresses the image data and generates and transmits an API request to the server.

[0650] Examples:

[0651] Users can upload landscape photos they have taken during their trip using the app's "upload" button, at which point the device will send the selected images to the server.

[0652] 2. The image is received on the server side and evaluated by AI

[0653] The server temporarily stores the image data sent from the terminal and passes it to the artificial intelligence module.

[0654] The artificial intelligence module analyzes the image data and generates an evaluation score based on evaluation criteria (e.g., color, brightness, composition, and match with tags).

[0655] The server stores the generated evaluation scores and image data in a database.

[0656] Examples:

[0657] An artificial intelligence module analyzes the color and composition of the landscape photo and generates an evaluation score of 85 points, which is then stored in a database by the server.

[0658] 3. Provide feedback to users on the evaluation results

[0659] The server generates a feedback message based on the rating score, including an example comment such as "This image scores 85 points. Great composition and colors! A little more sharpness would make it even better."

[0660] The server sends the generated feedback message to the user's terminal.

[0661] The device receives and displays the feedback message and provides the user with the evaluation results.

[0662] Examples:

[0663] A feedback message such as "This image scores 85 points. Great composition and colors! A little more sharpness would make it even better" will be displayed on the user's device.

[0664] 4. Image data collection and analysis

[0665] The server keeps a database of all submitted image data and evaluation scores.

[0666] The server uses the stored data to analyze trends and understand evaluation trends.

[0667] The server uses the obtained analysis results to improve the evaluation algorithm of the artificial intelligence module.

[0668] Examples:

[0669] The server analyzes landscape photos posted over a certain period of time, analyzes current trends, and analyzes the use of popular filters and tags, and uses this data to improve the rating algorithm.

[0670] 5. Ranking function and comparison with other users

[0671] The server aggregates the evaluation scores of all users and generates ranking information.

[0672] The server makes the generated ranking information available to users.

[0673] Users can access the ranking page from their device and check their position.

[0674] Examples:

[0675] The server aggregates the evaluation scores of all users and displays the top 10 users every week. When a user visits the ranking page, they can see that they are in 5th place.

[0676] Example prompts for generative AI models

[0677] "Rate the color and composition of landscape photos posted to an image sharing application. Also, generate a feedback message based on your rating. For example, this image gets a score of 85. Great composition and color! A little more sharpness would make it even better."

[0678] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0679] Specific flow of program processing

[0680] Step 1: Launching the Application

[0681] Users launch an image sharing application from their smartphone or PC.

[0682] Input: terminal

[0683] Output: The interface that the application launches.

[0684] What happens: A user taps an app icon on their device to launch an application.

[0685] Step 2: Tap the image posting button

[0686] The user taps the "Post Image" button on the application.

[0687] Input: Application interface

[0688] Output: Access to gallery and camera functions

[0689] Specific operation: The user taps the "Post Image" button in the app and moves to the next operation screen.

[0690] Step 3: Select an image

[0691] The device provides users with a gallery and camera function to select images to post.

[0692] The user selects an image using the gallery or camera.

[0693] Input: Device gallery or camera

[0694] Output: Selected image data

[0695] What happens: The user selects a specific image from the device's image gallery or takes a new one using the device's camera.

[0696] Step 4: Upload an image

[0697] The user selects an image through the device's interface and taps the "Upload" button.

[0698] Input: Selected image data

[0699] Output: Compressed image data and API request

[0700] The device compresses the image data and generates and transmits an API request to the server.

[0701] Specific operation: The user taps the "Upload" button in the app, and the device compresses the image data and sends it to the server.

[0702] Step 5: Receiving image data

[0703] The server receives the image data sent from the terminal.

[0704] Input: Compressed image data and API request

[0705] Output: Temporarily saved image data

[0706] Specific operation: The server receives the API request and temporarily stores the image data.

[0707] Step 6: Temporarily save image data

[0708] The server temporarily stores the received image data.

[0709] Input: Received image data

[0710] Output: Temporarily saved image data

[0711] Specific operation: Saves image data in a specified folder or memory area on the server.

[0712] Step 7: Transferring data to the AI ​​module

[0713] The server passes the image data to an artificial intelligence module.

[0714] Input: Temporarily saved image data

[0715] Output: Image data to be analyzed

[0716] Specific operation: The server reads the temporarily stored image data and sends it to the AI ​​platform.

[0717] Step 8: Image analysis using AI

[0718] The artificial intelligence module analyzes the image data and generates an evaluation score based on evaluation criteria (e.g., color, brightness, composition, and match with tags).

[0719] Input: Image data to be analyzed

[0720] Output: Evaluation score

[0721] How it works: The AI ​​analyzes elements such as color, brightness, composition, and tags, and generates an evaluation score based on a scoring model.

[0722] Step 9: Saving to the Database

[0723] The server stores the generated evaluation scores and image data in a database.

[0724] Input: Evaluation scores and image data

[0725] Output: Evaluation scores and image data stored in a database

[0726] Specific operation: The server stores the evaluation score and image data as a database entry.

[0727] Step 10: Generate feedback messages

[0728] The server generates a feedback message based on the rating score.

[0729] Input: Rating score

[0730] Output: Feedback message

[0731] Specific behavior: The server selects a feedback template from the rating score and generates a message.

[0732] Step 11: Sending a feedback message

[0733] The server sends the generated feedback message to the user's terminal.

[0734] Input: Feedback message

[0735] Output: Feedback message displayed on the user's terminal

[0736] Specific operation: The server sends a message to the user's device via push notification or API.

[0737] Step 12: Displaying Messages

[0738] The device receives and displays the feedback message and provides the user with the evaluation results.

[0739] Input: Feedback message

[0740] Output: Feedback that is displayed to the user

[0741] What happens: Your device receives a notification and displays a message in an in-app pop-up or in the notification bar.

[0742] Step 13: Continually save your data

[0743] The server keeps a database of all submitted image data and evaluation scores.

[0744] Input: Newly submitted image data and rating score

[0745] Output: A continuously updated database

[0746] What it does: Every time new data is submitted, the database entry is updated.

[0747] Step 14: Trend analysis

[0748] The server uses the stored data to analyze trends and understand evaluation trends.

[0749] Input: Historical data

[0750] Output: Trend reports and analysis results

[0751] What it does: The server analyzes historical data and extracts trends and popularity.

[0752] Step 15: Improve the algorithm

[0753] The server uses the obtained analysis results to improve the evaluation algorithm of the artificial intelligence module.

[0754] Input: Trend analysis results

[0755] Output: Improved evaluation algorithm

[0756] What it does: The server learns from the data and improves the rating algorithm.

[0757] Step 16: Generate ranking information

[0758] The server aggregates the evaluation scores of all users and generates ranking information.

[0759] Input: User rating score

[0760] Output: Ranking information

[0761] What happens: The server aggregates all the rating scores and creates a ranking list.

[0762] Step 17: Publish ranking information

[0763] The server makes the generated ranking information available to users.

[0764] Input: Ranking information

[0765] Output: Published ranking information

[0766] Specific operation: Publish ranking information on a web page or within an application.

[0767] Step 18: Check your rankings

[0768] Users can access the ranking page from their device and check their position.

[0769] Input: Ranking page URL or link

[0770] Output: User ranking information

[0771] What happens: A user visits the rankings page and checks their rating score and ranking.

[0772] (Application example 1)

[0773] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0774] Conventional image sharing applications lack sufficient ratings and feedback on the images posted by users, resulting in low user engagement. Furthermore, the mechanisms for comparing photos with other users and analyzing trends were inadequate, making it difficult to motivate users to post. In particular, specific advice on how to improve photography skills was not provided.

[0775] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0776] In this invention, the server includes means for receiving image data from users, artificial intelligence module means for analyzing the received image data and generating an evaluation score, means for providing the evaluation score as feedback to the user in real time, means for generating a feedback message based on the evaluation score, means for storing the image data and the evaluation score in a database, means for generating ranking information based on the stored data, and means for publishing the ranking information to users so that users can check their own ranking position. This allows users to receive immediate feedback on the images they post, compare their images with those of other users, and check their own ranking position. Comments containing specific suggestions for improvement are also provided, contributing to the improvement of users' photography skills.

[0777] "Image data" refers to digital image files that a user takes or selects from their device and posts through the application.

[0778] An "artificial intelligence module" is a machine learning model or algorithm for analyzing received image data and generating an evaluation score.

[0779] The "evaluation score" is an evaluation score for the image data generated by the artificial intelligence module.

[0780] "Feedback" refers to messages about evaluation results and areas for improvement provided to users based on their evaluation scores.

[0781] "Database" means a digital storage system for storing and managing image data and evaluation scores.

[0782] "Ranking information" is information that aggregates the evaluation scores of all users and indicates their relative position among other users.

[0783] "Real-time" refers to the time interval during which a user can receive immediate feedback after posting image data.

[0784] A "feedback message" is a comment containing specific improvements that is generated based on the evaluation score.

[0785] This invention is a system in which users post image data through an image sharing application, an AI module evaluates the image data, calculates an evaluation score, and provides feedback to the user. It also has a configuration in which ranking information is generated and published based on the collected data.

[0786] Hardware / Software used

[0787] Hardware:

[0788] Server environment: AWS EC2 instance as an example

[0789] User device: Smartphone or PC

[0790] software:

[0791] Framework:Flask

[0792] AI model: ResNet50 model provided by PyTorch

[0793] Database: MySQL or PostgreSQL

[0794] Specific processing flow

[0795] 1. Receiving image data:

[0796] Users launch an image sharing application on their smartphone or PC and tap the "Post Image" button. In response to this operation, the device provides a gallery or camera function, allowing the user to select the image they want to post. After selection, the device compresses the image data and sends it to the server.

[0797] 2. AI evaluation:

[0798] The server temporarily stores the received image data and passes it to an artificial intelligence module. The AI ​​model used is PyTorch's ResNet50. The model analyzes the image data and generates an evaluation score based on color, brightness, composition, and degree of match with tags.

[0799] 3. Feedback Processing:

[0800] The server generates a feedback message based on the generated evaluation score. For example, if the score is 85, it will provide a comment with specific suggestions for improvement, such as "This image scores 85. Great composition and colors! A little more sharpness would make it even better."

[0801] 4. Save to database:

[0802] The server stores the evaluation scores and image data in a database, which enables trend analysis and evaluation trends to be identified, and can also be used to improve the AI ​​evaluation algorithm.

[0803] 5. Generating and publishing ranking information:

[0804] The server aggregates the evaluation scores of all users and generates ranking information, which is then made public to users so that they can check their own ranking position.

[0805] Specific examples

[0806] Specifically, users submit landscape photos taken during their travels using the app's "Upload" button. The device compresses the image data and sends it to the server. The server receives the image data, temporarily stores it, and then passes it to a PyTorch ResNet50 model. The model analyzes color, lighting, composition, and tag information to generate an evaluation score. For example, if an evaluation score of 85 points is generated, the server returns a feedback message to the user saying, "This image scores 85 points. Excellent composition and color! A little more sharpness would make it even better." The evaluation scores and image data are also stored in a database, and the server aggregates all user data to generate and publish weekly ranking information.

[0807] Prompt Sentence Examples

[0808] Based on the adjusted settings, the system uses PyTorch's ResNet50 model to evaluate images uploaded by users. ResNet50 is used as a pre-trained model for image classification. The model analyzes the color and composition of the image and generates an evaluation score. Based on the score, the system provides specific feedback to the user in the form of comments suggesting areas for improvement.

[0809] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0810] Step 1:

[0811] The user uploads an image.

[0812] Specific operation: The user launches an image sharing application on a smartphone or PC and taps the "Post Image" button. The device responds to this action by providing gallery and camera functions. The user selects the image they want to post, selects the image through the device's interface, and taps the "Upload" button. At this time, the input is the image data selected by the user. The device compresses this image data and generates and sends an API request to the server. The output is the compressed image data sent to the server.

[0813] Step 2:

[0814] The server receives and stores the image data.

[0815] Specific operation: The server receives image data sent from the terminal. The input is compressed image data sent from the terminal. The server temporarily stores this image data. The output is the temporarily stored image data.

[0816] Step 3:

[0817] The server analyzes the image data and generates an evaluation score.

[0818] Specific operation: The server passes the temporarily stored image data to the artificial intelligence module. The input is the temporarily stored image data. The artificial intelligence module used is the PyTorch ResNet50 model. The model analyzes the image data and generates an evaluation score based on color, brightness, composition, and degree of match with tags. The output is the generated evaluation score. For example, the analysis result of a landscape photo will be a score of 85 points.

[0819] Step 4:

[0820] The server generates a feedback message based on the rating score.

[0821] Specific behavior: The server generates a feedback message based on the generated evaluation score. The input is the evaluation score. The feedback message includes specific improvements. For example, a message like "This image scores 85 points. Great composition and color! A little more sharpness would make it even better." is generated. The output is the feedback message.

[0822] Step 5:

[0823] The server sends a feedback message to the user's device.

[0824] Specific operation: The server sends the generated feedback message to the user's terminal. The input is the generated feedback message. The terminal receives this message and displays it to the user. The output is the feedback message displayed on the terminal.

[0825] Step 6:

[0826] The server stores the evaluation scores and image data in a database.

[0827] Specific operation: The server saves the generated evaluation score and corresponding image data in a database. The input is the evaluation score and image data. The server stores this in the database. The output is the evaluation score and image data saved in the database.

[0828] Step 7:

[0829] The server generates and publishes ranking information.

[0830] Specific operation: The server aggregates the evaluation scores of all users and generates ranking information. The input is all the evaluation score data. The server generates ranking information based on this and publishes it to users. The output is the published ranking information. Users can access the ranking page from their devices and check their position.

[0831] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0832] This invention is a system in which a user posts image data through an image sharing application, and an artificial intelligence module evaluates the image data to calculate an evaluation score and provide feedback to the user. In addition, by combining this system with an emotion engine that recognizes the user's emotions, it is possible to adjust the evaluation score and improve the user experience.

[0833] User uploads an image

[0834] Users launch an image sharing application on their smartphone, PC, or other device and tap the "Post Image" button.

[0835] The device displays a file selection screen to the user, and the user selects an image from the gallery or camera.

[0836] Users select the image they want to post and tap the "Upload" button.

[0837] The device compresses the image data and generates and transmits an API request to the server.

[0838] Examples:

[0839] Users can upload landscape photos they have taken during their trip using the app's "upload" button, at which point the device will send the selected images to the server.

[0840] Images are received on the server side and evaluated by AI

[0841] The server receives the image data sent from the terminal and temporarily stores it.

[0842] The server passes the temporarily stored image data to the artificial intelligence module.

[0843] The artificial intelligence module analyzes the image data and generates an evaluation score based on evaluation criteria (e.g., color, brightness, composition, and match with tags).

[0844] The server stores the generated evaluation scores and image data in a database.

[0845] Examples:

[0846] An artificial intelligence module analyzes the color and composition of the landscape photo and generates an evaluation score of 85 points, which is then stored in a database by the server.

[0847] User sentiment analysis and feedback adjustment

[0848] When uploading an image, the server uses the camera and microphone of the device the user is using to analyze the user's facial expressions and voice, collecting data to recognize emotions.

[0849] The emotion engine analyzes the collected data and identifies the user's emotional state (e.g., happy, sad, excited, etc.).

[0850] The server adjusts the rating score to include the perceived emotional state, for example, if the user is very happy, the rating score may be set somewhat higher.

[0851] Examples:

[0852] When a user uploads an image, the camera analyzes his facial expression and the emotion engine recognizes "happiness," adjusting the rating score from 85 to 88.

[0853] Providing Feedback

[0854] The server generates a feedback message based on the rating score and the user's sentiment analysis, including an example comment such as "This image scores 88 points. Great composition and colors! A little more sharpness would make it even better. I can see that you are pleased with this photo."

[0855] The server sends the generated feedback message to the user's terminal.

[0856] The terminal receives the feedback message and displays it to the user.

[0857] Examples:

[0858] A feedback message such as "This image scores 88 points. Great composition and colors! A little more sharpness would make it even better. I can see that you are happy with this photo" will be displayed on the user's device.

[0859] Image data collection and analysis

[0860] The server keeps a database of all submitted image data and evaluation scores.

[0861] The server analyzes trends and improves the evaluation algorithm based on the stored data and user sentiment data.

[0862] Examples:

[0863] The server analyzes landscape photos posted within a certain period of time and the associated user emotional data, and analyzes the relationship between current trends, popular filters, tag usage, and emotional states. This data is used to improve the evaluation algorithm.

[0864] Ranking function and comparison with other users

[0865] The server aggregates the evaluation scores and emotional data of all users and generates ranking information.

[0866] The server makes the generated ranking information available to users.

[0867] Users can access the ranking page from their device and check their position.

[0868] Examples:

[0869] The server aggregates the evaluation scores and emotional data of all users and displays the top 10 users each week. When user "Sato Taro" accesses the ranking page, he sees that he is in 5th place and the reason for this is also displayed.

[0870] The present invention, configured as described above, allows users to receive instant feedback on images they post and easily compare their views with those of other users. Furthermore, by incorporating user emotional data, it is possible to provide more personalized feedback and experiences, further increasing user engagement. Furthermore, it is possible to improve services based on the collected data.

[0871] The processing flow will be explained below.

[0872] Step 1:

[0873] The user launches an image sharing application on a device such as a smartphone or PC and taps the "Post Image" button.

[0874] Step 2:

[0875] The device presents the user with a file selection screen, and the user selects an image from the gallery or camera.

[0876] Step 3:

[0877] The user selects the image they want to post and taps the "Upload" button.

[0878] Step 4:

[0879] The device compresses the image data and generates and sends an API request to the server.

[0880] Step 5:

[0881] The server receives the image data sent from the terminal and temporarily stores it.

[0882] Step 6:

[0883] The server passes the temporarily stored image data to the artificial intelligence module.

[0884] Step 7:

[0885] An artificial intelligence module analyzes the image data and generates an evaluation score based on evaluation criteria (e.g., color, brightness, composition, and match with tags).

[0886] Step 8:

[0887] The server receives the evaluation score from the artificial intelligence module and stores it in a database together with the image data.

[0888] Step 9:

[0889] The server analyzes the user's facial expressions and voice using the device's camera and microphone to collect emotional data when uploading images.

[0890] Step 10:

[0891] The emotion engine analyzes the collected data and identifies the user's emotional state (e.g., happy, sad, excited, etc.).

[0892] Step 11:

[0893] The server adjusts the rating score to include the perceived emotional state and determines the final rating score, for example, slightly increasing the score if the user is very happy.

[0894] Step 12:

[0895] The server generates a feedback message based on the rating score and the user's sentiment analysis, including a comment like "This image scores 88 points. Great composition and colors! A little more sharpness would make it even better. I can see you're happy with this photo."

[0896] Step 13:

[0897] The server sends the generated feedback message to the user's terminal.

[0898] Step 14:

[0899] The device receives the feedback message and displays it to the user.

[0900] Step 15:

[0901] The server continuously stores the posted image data, evaluation scores, and related user emotional data in a database.

[0902] Step 16:

[0903] The server uses the stored data to analyze trends and improve evaluation algorithms.

[0904] Step 17:

[0905] The server aggregates the evaluation scores and emotional data of all users and generates ranking information.

[0906] Step 18:

[0907] The server publishes the generated ranking information to the user.

[0908] Step 19:

[0909] Users access the ranking page on their device and check their ranking position.

[0910] Example 2

[0911] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0912] In conventional image sharing systems, evaluations of images posted by users are subjective, making it difficult to obtain consistent feedback. Furthermore, they do not take into account the user's emotional state, making it impossible to provide a personalized experience. This has led to challenges in improving user experience and reducing engagement.

[0913] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0914] In this invention, the server includes means for receiving image data from a user, artificial intelligence module means for analyzing the received image data and generating an evaluation score, means for feeding back the evaluation score to the user, means for storing the image data and the evaluation score in a database, means for generating ranking information based on the stored data, emotion engine means for collecting and recognizing the emotional state of the user, and means for adjusting the evaluation score based on the emotional state. This allows users to receive consistent evaluations and feedback, and further enables engagement to be improved through personalized feedback based on the emotional state.

[0915] "Image data" refers to digital image files photographed or acquired by a user.

[0916] "Means for receiving" refers to a process or device for transferring image data from a user to a server over the Internet.

[0917] "Artificial Intelligence Module" refers to the computational algorithms and software used to analyze image data and automatically generate an evaluation score.

[0918] An "evaluation score" is a numerical indicator of the quality and characteristics of an image calculated by an artificial intelligence module.

[0919] "Feedback means" refers to the process or system for notifying users of the evaluation score and providing the results as feedback.

[0920] A "database" is a digital information storage system for efficiently storing and managing image data and evaluation scores.

[0921] "Ranking information" refers to information that ranks image data posted by multiple users based on their evaluation scores.

[0922] An "emotion engine" is software and algorithms used to analyze and recognize a user's emotional state.

[0923] "Emotional state" refers to the psychological state of the user, such as joy, sadness, or excitement, analyzed from their facial expressions and voice.

[0924] "Adjustment" refers to a process or system for appropriately modifying or changing the rating score based on the user's emotional state.

[0925] This invention is a system in which a user posts image data through an image sharing application, and an artificial intelligence module evaluates the image data to calculate an evaluation score and provide feedback to the user. In addition, by combining this system with an emotion engine that recognizes the user's emotions, it is possible to adjust the evaluation score and improve the user experience.

[0926] First, the user launches an image sharing application on a device such as a smartphone or PC and taps the "Post Image" button. Next, the device displays a file selection screen for the user, allowing the user to select an image from the gallery or camera. Once the user selects the image they want to post and taps the "Upload" button, the device compresses the image data and generates and sends an API request to the server.

[0927] For example, when a user posts a landscape photo taken during a trip using the app's "upload" button, the device will compress the selected image and send it to the server.

[0928] The server receives the image data sent from the device and temporarily stores it. The server then passes the temporarily stored image data to an artificial intelligence module. The artificial intelligence module analyzes the image data and generates an evaluation score based on evaluation criteria such as color, brightness, composition, and degree of match with tags. The server then stores the generated evaluation score and image data in a database.

[0929] For example, an artificial intelligence module analyzes the color and composition of a landscape photo and generates an evaluation score of 85 points, which is then stored in a database by the server.

[0930] Next, the server uses the user's device's camera and microphone to analyze the user's facial expressions and voice when uploading an image, collecting data for emotion recognition. The emotion engine analyzes the collected data and identifies the user's emotional state (happiness, sadness, excitement, etc.). The server adjusts the evaluation score taking into account the recognized emotional state. For example, if the user is very happy, the evaluation score can be set higher.

[0931] Specifically, when a user uploads an image, the camera analyzes his facial expression, and the emotion engine recognizes "joy," adjusting the rating score from 85 to 88.

[0932] The server then generates a feedback message based on the evaluation score and the user's sentiment analysis results. For example, it might include a comment like, "This image scores 88 points. Excellent composition and color. A little more sharpness would make it even better. I can see that you are pleased with this photo." The server then sends the generated feedback message to the user's device. The device receives the feedback message and displays it to the user.

[0933] For example, a feedback message such as "This image scores 88 points. Great composition and color. A little more sharpness would make it even better. I can see that you are happy with this photo" will be displayed on the user's device.

[0934] In addition, the server stores all posted image data and evaluation scores in a database. Trend analysis and improvements to the evaluation algorithm are performed based on the stored data and user emotional data. For example, the server analyzes landscape photos posted within a certain period of time and the associated emotional data of users, and analyzes the relationship between current trends, popular filters, tag usage, and emotional states. This data is used to improve the evaluation algorithm.

[0935] Finally, the server aggregates the evaluation scores and sentiment data of all users to generate ranking information, which is then made public to users, who can access the ranking page from their devices to check their ranking.

[0936] For example, the server aggregates the evaluation scores and emotional data of all users and displays the top 10 users every week. When user "Taro Sato" accesses the ranking page, he sees that he is in 5th place and the reason for his ranking is also displayed.

[0937] This invention allows users to receive instant feedback on the images they post and easily compare their images with those of other users. Furthermore, by incorporating user emotional data, it is possible to provide more personalized feedback and experiences, further increasing user engagement. Furthermore, the collected data can be used to improve services.

[0938] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0939] Step 1:

[0940] Users launch an image sharing application on their smartphone, PC, or other device and tap the "Post Image" button. Next, the device displays a file selection screen for the user, allowing them to select an image from their gallery or camera. Finally, the user selects the image they want to post and taps the "Upload" button.

[0941] Input: User-selected image data

[0942] Output: Compressed image data sent to the server

[0943] Specific operation: When a user posts a landscape photo taken during a trip through the app's "Upload" button, the device compresses the selected image, generates an API request, and sends it to the server.

[0944] Step 2:

[0945] The server receives the image data sent from the device and temporarily stores it. The server then passes the temporarily stored image data to an artificial intelligence module. The artificial intelligence module analyzes the image data and generates an evaluation score based on evaluation criteria (color, brightness, composition, and degree of match with tags). The server then stores the generated evaluation score and the image data in a database.

[0946] Input: Received image data

[0947] Output: Evaluation scores and image data stored in a database

[0948] How it works: The AI ​​module analyzes the color and composition of a landscape photo and generates an evaluation score of 85 points. This score is then stored in a database by the server.

[0949] Step 3:

[0950] When uploading an image, the server uses the user's device's camera and microphone to analyze the user's facial expressions and voice, collecting data for emotion recognition. The emotion engine then analyzes the collected data to identify the user's emotional state (happiness, sadness, excitement, etc.). The server then adjusts the evaluation score taking into account the recognized emotional state.

[0951] Input: facial expression and voice data when uploading images

[0952] Output: Adjusted rating score

[0953] How it works: When a user uploads an image, the camera analyzes his facial expression, and the emotion engine recognizes "joy," adjusting the rating score from 85 to 88.

[0954] Step 4:

[0955] The server generates a feedback message based on the evaluation score and the user's sentiment analysis results. The feedback message includes details of the evaluation score, areas for improvement, and comments based on the user's sentiment. The server then sends the generated feedback message to the user's device. The device receives the feedback message and displays it to the user.

[0956] Input: Adjusted rating scores and user sentiment analysis results

[0957] Output: The feedback message that is displayed to the user

[0958] What it does: The device displays a feedback message like, "This image scores 88 points. Great composition and colors. A little more sharpness would make it even better. I can see that you are happy with this photo."

[0959] Step 5:

[0960] The server continuously stores all posted image data and evaluation scores in a database. Based on the stored data and user sentiment data, it analyzes trends and improves the evaluation algorithm.

[0961] Input: Stored image data, rating scores, and emotion data

[0962] Output: Trend analysis and evaluation algorithm improvements

[0963] How it works: The server analyzes landscape photos posted within a certain period of time and the associated user emotional data, and analyzes the relationship between current trends, popular filters, tag usage, and emotional state. This data is used to improve the evaluation algorithm.

[0964] Step 6:

[0965] The server aggregates the evaluation scores and emotion data of all users and generates ranking information, which is then made public to users, who can access the ranking page from their devices to check their ranking.

[0966] Input: Rating scores and sentiment data for all users

[0967] Output: Public ranking information

[0968] How it works: The server aggregates the evaluation scores and sentiment data of all users and displays the top 10 users every week. When a user accesses the ranking page, they can see their ranking and the reason for their ranking.

[0969] (Application example 2)

[0970] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0971] Conventional methods for measuring the effectiveness of advertising have the problem of making it difficult to grasp the viewer's emotional state in real time and reflect it in feedback. Furthermore, to accurately evaluate the effectiveness of advertising, it is necessary to utilize user emotional data, but there has been a lack of means to achieve this. Furthermore, a system that can instantly evaluate and improve advertising performance is needed.

[0972] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0973] In this invention, the server includes means for receiving image data from a user, artificial intelligence module means for analyzing the received image data and generating an evaluation score, emotion engine means for recognizing the user's emotions, means for adjusting the evaluation score based on the user's emotional state, means for providing feedback on the evaluation score to the user, means for storing the image data and the evaluation score in a database, means for generating ranking information based on the stored data, and means for disclosing the ranking information to users so that they can check their ranking position. This enables emotions to be analyzed in real time when viewing an advertisement and provides effective feedback. Furthermore, it becomes possible to generate advertisement evaluations and ranking information based on emotion data, thereby enabling more accurate improvement of advertising performance.

[0974] "Image data" is a collection of visual information posted by users from their devices.

[0975] "User" means an individual or group that uses the system or application to post image data and receive evaluation results and feedback.

[0976] The "artificial intelligence module" is a software component for analyzing received image data and generating an evaluation score.

[0977] The "emotion engine" is a software component that analyzes the user's facial expressions and voice data to recognize their emotional state.

[0978] The "evaluation score" is a numerical evaluation value generated by the artificial intelligence module based on the analyzed image data.

[0979] "Feedback" is information provided to users, such as comments and advice based on rating scores and sentiment analysis.

[0980] A "database" is a storage system for storing image data, evaluation scores, and related information.

[0981] "Ranking information" is information obtained by aggregating and ranking evaluation scores from multiple users.

[0982] The "server" is a computer system that controls the entire process, including receiving, analyzing, and storing image data, and sending feedback.

[0983] The present invention relates to a system for evaluating the effectiveness of advertisements in real time. The system analyzes the emotional state of a user when viewing an advertisement using smart glasses and generates an evaluation score. Specific embodiments are described below.

[0984] System Configuration

[0985] The system consists of the following major components:

[0986] 1. User device: Smart glasses

[0987] 2. Server

[0988] 3. Artificial Intelligence Module

[0989] 4. Emotion Engine

[0990] 5. Database

[0991] 6. Feedback Generation Module

[0992] 7. Ranking Generation Module

[0993] Hardware and Software Use

[0994] Hardware

[0995] Smart glasses: Use built-in cameras and microphones to collect facial expression and voice data from the user.

[0996] Server: Processes and stores data using high-performance processors and large-capacity storage.

[0997] software

[0998] Python: The main implementation language for programs

[0999] OpenCV: A library for video capture and image processing

[1000] EmotionEngine: A software component (virtual module) for analyzing user emotions

[1001] AdEvaluator: A software component (virtual module) for generating ad evaluation scores.

[1002] FeedbackGenerator: A software component (virtual module) for generating feedback messages.

[1003] Cloud services (e.g., AWS, Google Cloud, Azure): Distributed computing environments for data storage and analysis

[1004] System Operation

[1005] User's device

[1006] When a user wears the smart glasses and watches an advertisement, the built-in camera and microphone collect facial expression and voice data, which is then transmitted to a server in real time.

[1007] server

[1008] The server receives and temporarily stores the image and audio data sent from the user's device. The received data is processed as follows:

[1009] 1. Emotion analysis: The server uses the Emotion Engine to analyze the user's emotions from the image data. Specifically, it recognizes emotions such as "happiness," "surprise," and "sadness" from facial expressions and voice tone.

[1010] 2. Generating an ad rating score: The server uses AdEvaluator to generate a rating score for the received ad data. This rating score is calculated based on the visual elements of the ad, its content, and the user's viewing time.

[1011] 3. Rating Score Adjustment: The server adjusts the generated rating score based on the user's recognized emotional state. For example, if the user has the emotion "joy," the server may set the rating score slightly higher.

[1012] 4. Feedback Generation: The server uses the FeedbackGenerator to generate a feedback message based on the adjusted rating score. The feedback message includes specific advice and rating results to show to the user.

[1013] Data storage and ranking information generation

[1014] The server stores the generated evaluation scores and related data in a database. Based on the stored data, a ranking generation module generates ranking information, allowing users to check their own evaluation scores and ranking positions.

[1015] Specific examples

[1016] As a specific example, data processing when viewing an advertisement will be shown.

[1017] 1. A user puts on the smart glasses and watches an advertisement.

[1018] 2. The camera and microphone in the smart glasses collect data and send it to the server.

[1019] 3. The server uses EmotionEngine to perform emotion analysis and recognize the emotion of "joy."

[1020] 4. The server uses the AdEvaluator to generate a rating score for the ad (e.g., 85 points).

[1021] 5. Adjust the rating score slightly (e.g., 88 points) based on the perceived emotional state.

[1022] 6. Use FeedbackGenerator to generate a feedback message such as "This ad scores 88 points."

[1023] 7. The server sends the evaluation score and feedback message to the smart glasses and displays it to the user.

[1024] Prompt Sentence Examples

[1025] "Design a system that uses the camera and microphone on smart glasses to analyze emotions in real time when a user watches an advertisement, and evaluate the effectiveness of the advertisement. Adjust the evaluation score based on the emotion analysis data and generate a feedback message. Design this as a Python application that runs on the smart glasses, and clearly state the necessary modules and cloud services."

[1026] The system of the present invention configured as described above contributes to improving the quality of advertising and engagement by evaluating the effectiveness of advertising in real time and providing feedback based on the user's emotional state.

[1027] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1028] Step 1:

[1029] Users wear the smart glasses and watch advertisements, while the built-in camera and microphone collect facial expression and voice data.

[1030] Specifically, the smart glasses' camera captures the user's face frame by frame and simultaneously acquires voice data. These data are sent from the device to the server in real time. The input is facial expression and voice data, and the output is the data sent to the server.

[1031] Step 2:

[1032] The server receives the image data and audio data sent from the terminal and temporarily stores them.

[1033] Specifically, the server temporarily stores the received data in a specific folder or memory. The input is the user's facial expression and voice data, and the output is the temporarily stored raw data.

[1034] Step 3:

[1035] The server uses EmotionEngine to analyze the user's emotions from the temporarily stored image data.

[1036] Specifically, the Emotion Engine analyzes facial features and voice tones to recognize emotional states such as "happiness," "surprise," and "sadness." The input is temporarily saved image and voice data, and the output is recognized emotional data.

[1037] Step 4:

[1038] The server uses the AdEvaluator to generate an evaluation score for the advertising data.

[1039] Specifically, AdEvaluator calculates a rating score based on the visual elements of the ad, its content, the user's viewing time, etc. The input is the ad data, and the output is a rating score (e.g., 85 points).

[1040] Step 5:

[1041] The server adjusts the generated evaluation score based on the recognized emotion data.

[1042] Specifically, AdEvaluator adjusts the final rating score by taking into account the emotional data obtained from EmotionEngine. For example, if the emotion is "joy," the rating score is adjusted from 85 to 88. The input is the rating score and emotional data, and the output is the adjusted rating score.

[1043] Step 6:

[1044] The server uses a FeedbackGenerator to generate feedback messages based on the adjusted reputation scores.

[1045] Specifically, the FeedbackGenerator generates a feedback message containing the evaluation result and advice to be shown to the user based on the adjusted evaluation score. The input is the adjusted evaluation score, and the output is the feedback message.

[1046] Step 7:

[1047] The server sends the generated feedback message to the user's terminal and displays it on the smart glasses.

[1048] Specifically, the server sends the generated feedback message to the user's smart glasses in real time and visually displays it. The input is the feedback message, and the output is the display on the smart glasses.

[1049] Step 8:

[1050] The server stores the generated evaluation scores and related data in a database.

[1051] Specifically, the server stores the evaluation scores, sentiment analysis data, and feedback messages in a database. The input is the evaluation scores and related data, and the output is the saved data.

[1052] Step 9:

[1053] The server generates ranking information based on the stored data and allows users to check their ranking position.

[1054] Specifically, the ranking generation module analyzes the data of all users and generates a ranking of their evaluation scores. Users can view the ranking information through smart glasses or other devices. The input is the stored data, and the output is the generated ranking information.

[1055] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1056] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1057] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1058] [Third embodiment]

[1059] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1060] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1061] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1062] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1063] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1064] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1065] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1066] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1067] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1068] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1069] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1070] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1071] This system allows users to post image data via an image sharing application, and an artificial intelligence module evaluates the image data to calculate an evaluation score and provide feedback to the user. It also generates and publishes ranking information based on the collected data.

[1072] User uploads an image

[1073] Users launch an image sharing application from their smartphone, PC, or other device and tap the "Post Image" button.

[1074] The device provides users with a gallery and camera function, allowing them to select the images they want to post.

[1075] The user selects an image through the device's interface and taps the "Upload" button.

[1076] The device compresses the image data and generates and transmits an API request to the server.

[1077] Examples:

[1078] Users can upload landscape photos they have taken during their trip using the app's "upload" button, at which point the device will send the selected images to the server.

[1079] Images are received on the server side and evaluated by AI

[1080] The server temporarily stores the image data sent from the terminal and passes it to the artificial intelligence module.

[1081] The artificial intelligence module analyzes the image data and generates an evaluation score based on evaluation criteria (e.g., color, brightness, composition, and match with tags).

[1082] The server stores the generated evaluation scores and image data in a database.

[1083] Examples:

[1084] An artificial intelligence module analyzes the color and composition of the landscape photo and generates an evaluation score of 85 points, which is then stored in a database by the server.

[1085] Provide feedback to users on the evaluation results

[1086] The server generates a feedback message based on the rating score, including an example comment such as "This image scores 85 points. Great composition and colors! A little more sharpness would make it even better."

[1087] The server sends the generated feedback message to the user's terminal.

[1088] The device receives and displays the feedback message and provides the user with the evaluation results.

[1089] Examples:

[1090] A feedback message such as "This image scores 85 points. Great composition and colors! A little more sharpness would make it even better" will be displayed on the user's device.

[1091] Image data collection and analysis

[1092] The server keeps a database of all submitted image data and evaluation scores.

[1093] The server uses the stored data to analyze trends and understand evaluation trends.

[1094] The server uses the obtained analysis results to improve the evaluation algorithm of the artificial intelligence module.

[1095] Examples:

[1096] The server analyzes landscape photos posted over a certain period of time, analyzes current trends, and analyzes the use of popular filters and tags, and uses this data to improve the rating algorithm.

[1097] Ranking function and comparison with other users

[1098] The server aggregates the evaluation scores of all users and generates ranking information.

[1099] The server makes the generated ranking information available to users.

[1100] Users can access the ranking page from their device and check their position.

[1101] Examples:

[1102] The server aggregates the evaluation scores of all users and displays the top 10 users every week. When user "Taro Sato" accesses the ranking page, he can see that he is in 5th place.

[1103] The present invention, configured as described above, allows users to receive instant feedback on images they post and easily compare their images with those of other users. It also makes it possible to improve the service based on the collected data. This increases user engagement and improves the quality of image sharing applications.

[1104] The processing flow will be explained below.

[1105] Step 1:

[1106] The user launches an image sharing application on a device such as a smartphone or PC and taps the "Post Image" button.

[1107] Step 2:

[1108] The device presents the user with a file selection screen, and the user selects an image from the gallery or camera.

[1109] Step 3:

[1110] The user selects the image they want to post and taps the "Upload" button.

[1111] Step 4:

[1112] The terminal compresses the selected image data and generates and transmits an API request to the server.

[1113] Step 5:

[1114] The server receives the image data sent from the terminal and temporarily stores it.

[1115] Step 6:

[1116] The server passes the temporarily stored image data to the artificial intelligence module.

[1117] Step 7:

[1118] An artificial intelligence module analyzes the image data and generates an evaluation score based on evaluation criteria (e.g., color, brightness, composition, and match with tags).

[1119] Step 8:

[1120] The server receives the evaluation score from the artificial intelligence module and stores it in a database together with the image data.

[1121] Step 9:

[1122] The server generates a feedback message based on the rating score, including a comment like "This image scores 85 points. Great composition and colors! A little more sharpness would make it even better."

[1123] Step 10:

[1124] The server sends the generated feedback message to the user's terminal.

[1125] Step 11:

[1126] The device receives the feedback message and displays it to the user.

[1127] Step 12:

[1128] The server continues to store the posted image data and evaluation scores in a database.

[1129] Step 13:

[1130] The server uses the stored data to analyze trends and improve evaluation algorithms.

[1131] Step 14:

[1132] The server aggregates the evaluation scores of all users and generates ranking information.

[1133] Step 15:

[1134] The server publishes the generated ranking information to the user.

[1135] Step 16:

[1136] Users access the ranking page on their device and check their ranking position.

[1137] Example 1

[1138] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1139] In conventional image sharing applications, image ratings and feedback were done manually, making it difficult for users to obtain detailed ratings immediately. Furthermore, there were problems with inconsistent rating criteria, making it difficult to generate fair rankings. Furthermore, trend analysis and algorithm improvements were done manually, resulting in low efficiency of the overall system.

[1140] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1141] In this invention, the server includes means for receiving image data from users, means for compressing and transmitting the received image data, means for temporarily storing the received image data, an artificial intelligence module for analyzing the received image data and generating an evaluation score, means for providing feedback on the evaluation score to the user, means for storing the image data and the evaluation score in a database, means for performing trend analysis and algorithm improvement based on the stored data, and means for generating ranking information based on the stored data. This allows users to receive detailed evaluations in real time, enabling the generation of fair rankings. Furthermore, because trend analysis and algorithm improvement are performed automatically, the efficiency of the entire system is improved.

[1142] "Image data" refers to visual information represented in digital form.

[1143] "User" means a person who uses an image sharing application to post images and receive ratings and feedback.

[1144] "Means for receiving" refers to a mechanism for capturing image data sent by a user.

[1145] "Means for compressing and transmitting" refers to a method for reducing the size of image data and transmitting it to the server.

[1146] "Temporary storage means" refers to a mechanism for temporarily storing received image data.

[1147] "Artificial Intelligence Module" refers to a software or hardware component for analyzing image data based on evaluation criteria and generating an evaluation score.

[1148] The "means for providing feedback" refers to a method for forming a feedback message based on the generated evaluation score and providing it to the user.

[1149] "Database" refers to a system that systematically stores and manages image data and evaluation scores.

[1150] "Trend analysis" refers to the act of extracting and analyzing current trends and patterns based on collected data.

[1151] "Means for improving algorithms" refers to methods for using the results of trend analysis to improve the evaluation process of artificial intelligence modules and other system components.

[1152] "Ranking information" refers to ranking information generated based on the evaluation scores of all users.

[1153] This system allows users to post image data via an image sharing application, and an artificial intelligence module evaluates the image data to calculate an evaluation score and provide feedback to the user. It also generates and publishes ranking information based on the collected data.

[1154] Overall system configuration

[1155] The system consists of various components, including an image sharing application, a server, a database, an artificial intelligence module, a feedback generation system, and a ranking generation system.

[1156] Hardware and software used

[1157] Device: The smartphone or PC used by the user.

[1158] Server: A remote computer that receives, stores, and processes data.

[1159] Database: A system for managing data in an organized manner, such as MySQL or PostgreSQL.

[1160] Artificial Intelligence Module: A rating system built using machine learning libraries such as TensorFlow and PyTorch.

[1161] Specific implementation flow

[1162] 1. User uploads an image

[1163] Users launch an image sharing application from their smartphone, PC, or other device and tap the "Post Image" button.

[1164] The device provides users with a gallery and camera function, allowing them to select the images they want to post.

[1165] The user selects an image through the device's interface and taps the "Upload" button.

[1166] The device compresses the image data and generates and transmits an API request to the server.

[1167] Examples:

[1168] Users can upload landscape photos they have taken during their trip using the app's "upload" button, at which point the device will send the selected images to the server.

[1169] 2. The image is received on the server side and evaluated by AI

[1170] The server temporarily stores the image data sent from the terminal and passes it to the artificial intelligence module.

[1171] The artificial intelligence module analyzes the image data and generates an evaluation score based on evaluation criteria (e.g., color, brightness, composition, and match with tags).

[1172] The server stores the generated evaluation scores and image data in a database.

[1173] Examples:

[1174] An artificial intelligence module analyzes the color and composition of the landscape photo and generates an evaluation score of 85 points, which is then stored in a database by the server.

[1175] 3. Provide feedback to users on the evaluation results

[1176] The server generates a feedback message based on the rating score, including an example comment such as "This image scores 85 points. Great composition and colors! A little more sharpness would make it even better."

[1177] The server sends the generated feedback message to the user's terminal.

[1178] The device receives and displays the feedback message and provides the user with the evaluation results.

[1179] Examples:

[1180] A feedback message such as "This image scores 85 points. Great composition and colors! A little more sharpness would make it even better" will be displayed on the user's device.

[1181] 4. Image data collection and analysis

[1182] The server keeps a database of all submitted image data and evaluation scores.

[1183] The server uses the stored data to analyze trends and understand evaluation trends.

[1184] The server uses the obtained analysis results to improve the evaluation algorithm of the artificial intelligence module.

[1185] Examples:

[1186] The server analyzes landscape photos posted over a certain period of time, analyzes current trends, and analyzes the use of popular filters and tags, and uses this data to improve the rating algorithm.

[1187] 5. Ranking function and comparison with other users

[1188] The server aggregates the evaluation scores of all users and generates ranking information.

[1189] The server makes the generated ranking information available to users.

[1190] Users can access the ranking page from their device and check their position.

[1191] Examples:

[1192] The server aggregates the evaluation scores of all users and displays the top 10 users every week. When a user visits the ranking page, they can see that they are in 5th place.

[1193] Example prompts for generative AI models

[1194] "Rate the color and composition of landscape photos posted to an image sharing application. Also, generate a feedback message based on your rating. For example, this image gets a score of 85. Great composition and color! A little more sharpness would make it even better."

[1195] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1196] Specific flow of program processing

[1197] Step 1: Launching the Application

[1198] Users launch an image sharing application from their smartphone or PC.

[1199] Input: terminal

[1200] Output: The interface that the application launches.

[1201] What happens: A user taps an app icon on their device to launch an application.

[1202] Step 2: Tap the image posting button

[1203] The user taps the "Post Image" button on the application.

[1204] Input: Application interface

[1205] Output: Access to gallery and camera functions

[1206] Specific operation: The user taps the "Post Image" button in the app and moves to the next operation screen.

[1207] Step 3: Select an image

[1208] The device provides users with a gallery and camera function to select images to post.

[1209] The user selects an image using the gallery or camera.

[1210] Input: Device gallery or camera

[1211] Output: Selected image data

[1212] What happens: The user selects a specific image from the device's image gallery or takes a new one using the device's camera.

[1213] Step 4: Upload an image

[1214] The user selects an image through the device's interface and taps the "Upload" button.

[1215] Input: Selected image data

[1216] Output: Compressed image data and API request

[1217] The device compresses the image data and generates and transmits an API request to the server.

[1218] Specific operation: The user taps the "Upload" button in the app, and the device compresses the image data and sends it to the server.

[1219] Step 5: Receiving image data

[1220] The server receives the image data sent from the terminal.

[1221] Input: Compressed image data and API request

[1222] Output: Temporarily saved image data

[1223] Specific operation: The server receives the API request and temporarily stores the image data.

[1224] Step 6: Temporarily save image data

[1225] The server temporarily stores the received image data.

[1226] Input: Received image data

[1227] Output: Temporarily saved image data

[1228] Specific operation: Saves image data in a specified folder or memory area on the server.

[1229] Step 7: Transferring data to the AI ​​module

[1230] The server passes the image data to an artificial intelligence module.

[1231] Input: Temporarily saved image data

[1232] Output: Image data to be analyzed

[1233] Specific operation: The server reads the temporarily stored image data and sends it to the AI ​​platform.

[1234] Step 8: Image analysis using AI

[1235] The artificial intelligence module analyzes the image data and generates an evaluation score based on evaluation criteria (e.g., color, brightness, composition, and match with tags).

[1236] Input: Image data to be analyzed

[1237] Output: Evaluation score

[1238] How it works: The AI ​​analyzes elements such as color, brightness, composition, and tags, and generates an evaluation score based on a scoring model.

[1239] Step 9: Saving to the Database

[1240] The server stores the generated evaluation scores and image data in a database.

[1241] Input: Evaluation scores and image data

[1242] Output: Evaluation scores and image data stored in a database

[1243] Specific operation: The server stores the evaluation score and image data as a database entry.

[1244] Step 10: Generate feedback messages

[1245] The server generates a feedback message based on the rating score.

[1246] Input: Rating score

[1247] Output: Feedback message

[1248] Specific behavior: The server selects a feedback template from the rating score and generates a message.

[1249] Step 11: Sending a feedback message

[1250] The server sends the generated feedback message to the user's terminal.

[1251] Input: Feedback message

[1252] Output: Feedback message displayed on the user's terminal

[1253] Specific operation: The server sends a message to the user's device via push notification or API.

[1254] Step 12: Displaying Messages

[1255] The device receives and displays the feedback message and provides the user with the evaluation results.

[1256] Input: Feedback message

[1257] Output: Feedback that is displayed to the user

[1258] What happens: Your device receives a notification and displays a message in an in-app pop-up or in the notification bar.

[1259] Step 13: Continually save your data

[1260] The server keeps a database of all submitted image data and evaluation scores.

[1261] Input: Newly submitted image data and rating score

[1262] Output: A continuously updated database

[1263] What it does: Every time new data is submitted, the database entry is updated.

[1264] Step 14: Trend analysis

[1265] The server uses the stored data to analyze trends and understand evaluation trends.

[1266] Input: Historical data

[1267] Output: Trend reports and analysis results

[1268] What it does: The server analyzes historical data and extracts trends and popularity.

[1269] Step 15: Improve the algorithm

[1270] The server uses the obtained analysis results to improve the evaluation algorithm of the artificial intelligence module.

[1271] Input: Trend analysis results

[1272] Output: Improved evaluation algorithm

[1273] What it does: The server learns from the data and improves the rating algorithm.

[1274] Step 16: Generate ranking information

[1275] The server aggregates the evaluation scores of all users and generates ranking information.

[1276] Input: User rating score

[1277] Output: Ranking information

[1278] What happens: The server aggregates all the rating scores and creates a ranking list.

[1279] Step 17: Publish ranking information

[1280] The server makes the generated ranking information available to users.

[1281] Input: Ranking information

[1282] Output: Published ranking information

[1283] Specific operation: Publish ranking information on a web page or within an application.

[1284] Step 18: Check your rankings

[1285] Users can access the ranking page from their device and check their position.

[1286] Input: Ranking page URL or link

[1287] Output: User ranking information

[1288] What happens: A user visits the rankings page and checks their rating score and ranking.

[1289] (Application example 1)

[1290] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1291] Conventional image sharing applications lack sufficient ratings and feedback on the images posted by users, resulting in low user engagement. Furthermore, the mechanisms for comparing photos with other users and analyzing trends were inadequate, making it difficult to motivate users to post. In particular, specific advice on how to improve photography skills was not provided.

[1292] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1293] In this invention, the server includes means for receiving image data from users, artificial intelligence module means for analyzing the received image data and generating an evaluation score, means for providing the evaluation score as feedback to the user in real time, means for generating a feedback message based on the evaluation score, means for storing the image data and the evaluation score in a database, means for generating ranking information based on the stored data, and means for publishing the ranking information to users so that users can check their own ranking position. This allows users to receive immediate feedback on the images they post, compare their images with those of other users, and check their own ranking position. Comments containing specific suggestions for improvement are also provided, contributing to the improvement of users' photography skills.

[1294] "Image data" refers to digital image files that a user takes or selects from their device and posts through the application.

[1295] An "artificial intelligence module" is a machine learning model or algorithm for analyzing received image data and generating an evaluation score.

[1296] The "evaluation score" is an evaluation score for the image data generated by the artificial intelligence module.

[1297] "Feedback" refers to messages about evaluation results and areas for improvement provided to users based on their evaluation scores.

[1298] "Database" means a digital storage system for storing and managing image data and evaluation scores.

[1299] "Ranking information" is information that aggregates the evaluation scores of all users and indicates their relative position among other users.

[1300] "Real-time" refers to the time interval during which a user can receive immediate feedback after posting image data.

[1301] A "feedback message" is a comment containing specific improvements that is generated based on the evaluation score.

[1302] This invention is a system in which users post image data through an image sharing application, an AI module evaluates the image data, calculates an evaluation score, and provides feedback to the user. It also has a configuration in which ranking information is generated and published based on the collected data.

[1303] Hardware / Software used

[1304] Hardware:

[1305] Server environment: AWS EC2 instance as an example

[1306] User device: Smartphone or PC

[1307] software:

[1308] Framework:Flask

[1309] AI model: ResNet50 model provided by PyTorch

[1310] Database: MySQL or PostgreSQL

[1311] Specific processing flow

[1312] 1. Receiving image data:

[1313] Users launch an image sharing application on their smartphone or PC and tap the "Post Image" button. In response to this operation, the device provides a gallery or camera function, allowing the user to select the image they want to post. After selection, the device compresses the image data and sends it to the server.

[1314] 2. AI evaluation:

[1315] The server temporarily stores the received image data and passes it to an artificial intelligence module. The AI ​​model used is PyTorch's ResNet50. The model analyzes the image data and generates an evaluation score based on color, brightness, composition, and degree of match with tags.

[1316] 3. Feedback Processing:

[1317] The server generates a feedback message based on the generated evaluation score. For example, if the score is 85, it will provide a comment with specific suggestions for improvement, such as "This image scores 85. Great composition and colors! A little more sharpness would make it even better."

[1318] 4. Save to database:

[1319] The server stores the evaluation scores and image data in a database, which enables trend analysis and evaluation trends to be identified, and can also be used to improve the AI ​​evaluation algorithm.

[1320] 5. Generating and publishing ranking information:

[1321] The server aggregates the evaluation scores of all users and generates ranking information, which is then made public to users so that they can check their own ranking position.

[1322] Specific examples

[1323] Specifically, users submit landscape photos taken during their travels using the app's "Upload" button. The device compresses the image data and sends it to the server. The server receives the image data, temporarily stores it, and then passes it to a PyTorch ResNet50 model. The model analyzes color, lighting, composition, and tag information to generate an evaluation score. For example, if an evaluation score of 85 points is generated, the server returns a feedback message to the user saying, "This image scores 85 points. Excellent composition and color! A little more sharpness would make it even better." The evaluation scores and image data are also stored in a database, and the server aggregates all user data to generate and publish weekly ranking information.

[1324] Prompt Sentence Examples

[1325] Based on the adjusted settings, the system uses PyTorch's ResNet50 model to evaluate images uploaded by users. ResNet50 is used as a pre-trained model for image classification. The model analyzes the color and composition of the image and generates an evaluation score. Based on the score, the system provides specific feedback to the user in the form of comments suggesting areas for improvement.

[1326] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1327] Step 1:

[1328] The user uploads an image.

[1329] Specific operation: The user launches an image sharing application on a smartphone or PC and taps the "Post Image" button. The device responds to this action by providing gallery and camera functions. The user selects the image they want to post, selects the image through the device's interface, and taps the "Upload" button. At this time, the input is the image data selected by the user. The device compresses this image data and generates and sends an API request to the server. The output is the compressed image data sent to the server.

[1330] Step 2:

[1331] The server receives and stores the image data.

[1332] Specific operation: The server receives image data sent from the terminal. The input is compressed image data sent from the terminal. The server temporarily stores this image data. The output is the temporarily stored image data.

[1333] Step 3:

[1334] The server analyzes the image data and generates an evaluation score.

[1335] Specific operation: The server passes the temporarily stored image data to the artificial intelligence module. The input is the temporarily stored image data. The artificial intelligence module used is the PyTorch ResNet50 model. The model analyzes the image data and generates an evaluation score based on color, brightness, composition, and degree of match with tags. The output is the generated evaluation score. For example, the analysis result of a landscape photo will be a score of 85 points.

[1336] Step 4:

[1337] The server generates a feedback message based on the rating score.

[1338] Specific behavior: The server generates a feedback message based on the generated evaluation score. The input is the evaluation score. The feedback message includes specific improvements. For example, a message like "This image scores 85 points. Great composition and color! A little more sharpness would make it even better." is generated. The output is the feedback message.

[1339] Step 5:

[1340] The server sends a feedback message to the user's device.

[1341] Specific operation: The server sends the generated feedback message to the user's terminal. The input is the generated feedback message. The terminal receives this message and displays it to the user. The output is the feedback message displayed on the terminal.

[1342] Step 6:

[1343] The server stores the evaluation scores and image data in a database.

[1344] Specific operation: The server saves the generated evaluation score and corresponding image data in a database. The input is the evaluation score and image data. The server stores this in the database. The output is the evaluation score and image data saved in the database.

[1345] Step 7:

[1346] The server generates and publishes ranking information.

[1347] Specific operation: The server aggregates the evaluation scores of all users and generates ranking information. The input is all the evaluation score data. The server generates ranking information based on this and publishes it to users. The output is the published ranking information. Users can access the ranking page from their devices and check their position.

[1348] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1349] This invention is a system in which a user posts image data through an image sharing application, and an artificial intelligence module evaluates the image data to calculate an evaluation score and provide feedback to the user. In addition, by combining this system with an emotion engine that recognizes the user's emotions, it is possible to adjust the evaluation score and improve the user experience.

[1350] User uploads an image

[1351] Users launch an image sharing application on their smartphone, PC, or other device and tap the "Post Image" button.

[1352] The device displays a file selection screen to the user, and the user selects an image from the gallery or camera.

[1353] Users select the image they want to post and tap the "Upload" button.

[1354] The device compresses the image data and generates and transmits an API request to the server.

[1355] Examples:

[1356] Users can upload landscape photos they have taken during their trip using the app's "upload" button, at which point the device will send the selected images to the server.

[1357] Images are received on the server side and evaluated by AI

[1358] The server receives the image data sent from the terminal and temporarily stores it.

[1359] The server passes the temporarily stored image data to the artificial intelligence module.

[1360] The artificial intelligence module analyzes the image data and generates an evaluation score based on evaluation criteria (e.g., color, brightness, composition, and match with tags).

[1361] The server stores the generated evaluation scores and image data in a database.

[1362] Examples:

[1363] An artificial intelligence module analyzes the color and composition of the landscape photo and generates an evaluation score of 85 points, which is then stored in a database by the server.

[1364] User sentiment analysis and feedback adjustment

[1365] When uploading an image, the server uses the camera and microphone of the device the user is using to analyze the user's facial expressions and voice, collecting data to recognize emotions.

[1366] The emotion engine analyzes the collected data and identifies the user's emotional state (e.g., happy, sad, excited, etc.).

[1367] The server adjusts the rating score to include the perceived emotional state, for example, if the user is very happy, the rating score may be set somewhat higher.

[1368] Examples:

[1369] When a user uploads an image, the camera analyzes his facial expression and the emotion engine recognizes "happiness," adjusting the rating score from 85 to 88.

[1370] Providing Feedback

[1371] The server generates a feedback message based on the rating score and the user's sentiment analysis, including an example comment such as "This image scores 88 points. Great composition and colors! A little more sharpness would make it even better. I can see that you are pleased with this photo."

[1372] The server sends the generated feedback message to the user's terminal.

[1373] The terminal receives the feedback message and displays it to the user.

[1374] Examples:

[1375] A feedback message such as "This image scores 88 points. Great composition and colors! A little more sharpness would make it even better. I can see that you are happy with this photo" will be displayed on the user's device.

[1376] Image data collection and analysis

[1377] The server keeps a database of all submitted image data and evaluation scores.

[1378] The server analyzes trends and improves the evaluation algorithm based on the stored data and user sentiment data.

[1379] Examples:

[1380] The server analyzes landscape photos posted within a certain period of time and the associated user emotional data, and analyzes the relationship between current trends, popular filters, tag usage, and emotional states. This data is used to improve the evaluation algorithm.

[1381] Ranking function and comparison with other users

[1382] The server aggregates the evaluation scores and emotional data of all users and generates ranking information.

[1383] The server makes the generated ranking information available to users.

[1384] Users can access the ranking page from their device and check their position.

[1385] Examples:

[1386] The server aggregates the evaluation scores and emotional data of all users and displays the top 10 users each week. When user "Sato Taro" accesses the ranking page, he sees that he is in 5th place and the reason for this is also displayed.

[1387] The present invention, configured as described above, allows users to receive instant feedback on images they post and easily compare their views with those of other users. Furthermore, by incorporating user emotional data, it is possible to provide more personalized feedback and experiences, further increasing user engagement. Furthermore, it is possible to improve services based on the collected data.

[1388] The processing flow will be explained below.

[1389] Step 1:

[1390] The user launches an image sharing application on a device such as a smartphone or PC and taps the "Post Image" button.

[1391] Step 2:

[1392] The device presents the user with a file selection screen, and the user selects an image from the gallery or camera.

[1393] Step 3:

[1394] The user selects the image they want to post and taps the "Upload" button.

[1395] Step 4:

[1396] The device compresses the image data and generates and sends an API request to the server.

[1397] Step 5:

[1398] The server receives the image data sent from the terminal and temporarily stores it.

[1399] Step 6:

[1400] The server passes the temporarily stored image data to the artificial intelligence module.

[1401] Step 7:

[1402] An artificial intelligence module analyzes the image data and generates an evaluation score based on evaluation criteria (e.g., color, brightness, composition, and match with tags).

[1403] Step 8:

[1404] The server receives the evaluation score from the artificial intelligence module and stores it in a database together with the image data.

[1405] Step 9:

[1406] The server analyzes the user's facial expressions and voice using the device's camera and microphone to collect emotional data when uploading images.

[1407] Step 10:

[1408] The emotion engine analyzes the collected data and identifies the user's emotional state (e.g., happy, sad, excited, etc.).

[1409] Step 11:

[1410] The server adjusts the rating score to include the perceived emotional state and determines the final rating score, for example, slightly increasing the score if the user is very happy.

[1411] Step 12:

[1412] The server generates a feedback message based on the rating score and the user's sentiment analysis, including a comment like "This image scores 88 points. Great composition and colors! A little more sharpness would make it even better. I can see you're happy with this photo."

[1413] Step 13:

[1414] The server sends the generated feedback message to the user's terminal.

[1415] Step 14:

[1416] The device receives the feedback message and displays it to the user.

[1417] Step 15:

[1418] The server continuously stores the posted image data, evaluation scores, and related user emotional data in a database.

[1419] Step 16:

[1420] The server uses the stored data to analyze trends and improve evaluation algorithms.

[1421] Step 17:

[1422] The server aggregates the evaluation scores and emotional data of all users and generates ranking information.

[1423] Step 18:

[1424] The server publishes the generated ranking information to the user.

[1425] Step 19:

[1426] Users access the ranking page on their device and check their ranking position.

[1427] Example 2

[1428] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1429] In conventional image sharing systems, evaluations of images posted by users are subjective, making it difficult to obtain consistent feedback. Furthermore, they do not take into account the user's emotional state, making it impossible to provide a personalized experience. This has led to challenges in improving user experience and reducing engagement.

[1430] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1431] In this invention, the server includes means for receiving image data from a user, artificial intelligence module means for analyzing the received image data and generating an evaluation score, means for feeding back the evaluation score to the user, means for storing the image data and the evaluation score in a database, means for generating ranking information based on the stored data, emotion engine means for collecting and recognizing the emotional state of the user, and means for adjusting the evaluation score based on the emotional state. This allows users to receive consistent evaluations and feedback, and further enables engagement to be improved through personalized feedback based on the emotional state.

[1432] "Image data" refers to digital image files photographed or acquired by a user.

[1433] "Means for receiving" refers to a process or device for transferring image data from a user to a server over the Internet.

[1434] "Artificial Intelligence Module" refers to the computational algorithms and software used to analyze image data and automatically generate an evaluation score.

[1435] An "evaluation score" is a numerical indicator of the quality and characteristics of an image calculated by an artificial intelligence module.

[1436] "Feedback means" refers to the process or system for notifying users of the evaluation score and providing the results as feedback.

[1437] A "database" is a digital information storage system for efficiently storing and managing image data and evaluation scores.

[1438] "Ranking information" refers to information that ranks image data posted by multiple users based on their evaluation scores.

[1439] An "emotion engine" is software and algorithms used to analyze and recognize a user's emotional state.

[1440] "Emotional state" refers to the psychological state of the user, such as joy, sadness, or excitement, analyzed from their facial expressions and voice.

[1441] "Adjustment" refers to a process or system for appropriately modifying or changing the rating score based on the user's emotional state.

[1442] This invention is a system in which a user posts image data through an image sharing application, and an artificial intelligence module evaluates the image data to calculate an evaluation score and provide feedback to the user. In addition, by combining this system with an emotion engine that recognizes the user's emotions, it is possible to adjust the evaluation score and improve the user experience.

[1443] First, the user launches an image sharing application on a device such as a smartphone or PC and taps the "Post Image" button. Next, the device displays a file selection screen for the user, allowing the user to select an image from the gallery or camera. Once the user selects the image they want to post and taps the "Upload" button, the device compresses the image data and generates and sends an API request to the server.

[1444] For example, when a user posts a landscape photo taken during a trip using the app's "upload" button, the device will compress the selected image and send it to the server.

[1445] The server receives the image data sent from the device and temporarily stores it. The server then passes the temporarily stored image data to an artificial intelligence module. The artificial intelligence module analyzes the image data and generates an evaluation score based on evaluation criteria such as color, brightness, composition, and degree of match with tags. The server then stores the generated evaluation score and image data in a database.

[1446] For example, an artificial intelligence module analyzes the color and composition of a landscape photo and generates an evaluation score of 85 points, which is then stored in a database by the server.

[1447] Next, the server uses the user's device's camera and microphone to analyze the user's facial expressions and voice when uploading an image, collecting data for emotion recognition. The emotion engine analyzes the collected data and identifies the user's emotional state (happiness, sadness, excitement, etc.). The server adjusts the evaluation score taking into account the recognized emotional state. For example, if the user is very happy, the evaluation score can be set higher.

[1448] Specifically, when a user uploads an image, the camera analyzes his facial expression, and the emotion engine recognizes "joy," adjusting the rating score from 85 to 88.

[1449] The server then generates a feedback message based on the evaluation score and the user's sentiment analysis results. For example, it might include a comment like, "This image scores 88 points. Excellent composition and color. A little more sharpness would make it even better. I can see that you are pleased with this photo." The server then sends the generated feedback message to the user's device. The device receives the feedback message and displays it to the user.

[1450] For example, a feedback message such as "This image scores 88 points. Great composition and color. A little more sharpness would make it even better. I can see that you are happy with this photo" will be displayed on the user's device.

[1451] In addition, the server stores all posted image data and evaluation scores in a database. Trend analysis and improvements to the evaluation algorithm are performed based on the stored data and user emotional data. For example, the server analyzes landscape photos posted within a certain period of time and the associated emotional data of users, and analyzes the relationship between current trends, popular filters, tag usage, and emotional states. This data is used to improve the evaluation algorithm.

[1452] Finally, the server aggregates the evaluation scores and sentiment data of all users to generate ranking information, which is then made public to users, who can access the ranking page from their devices to check their ranking.

[1453] For example, the server aggregates the evaluation scores and emotional data of all users and displays the top 10 users every week. When user "Taro Sato" accesses the ranking page, he sees that he is in 5th place and the reason for his ranking is also displayed.

[1454] This invention allows users to receive instant feedback on the images they post and easily compare their images with those of other users. Furthermore, by incorporating user emotional data, it is possible to provide more personalized feedback and experiences, further increasing user engagement. Furthermore, the collected data can be used to improve services.

[1455] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1456] Step 1:

[1457] Users launch an image sharing application on their smartphone, PC, or other device and tap the "Post Image" button. Next, the device displays a file selection screen for the user, allowing them to select an image from their gallery or camera. Finally, the user selects the image they want to post and taps the "Upload" button.

[1458] Input: User-selected image data

[1459] Output: Compressed image data sent to the server

[1460] Specific operation: When a user posts a landscape photo taken during a trip through the app's "Upload" button, the device compresses the selected image, generates an API request, and sends it to the server.

[1461] Step 2:

[1462] The server receives the image data sent from the device and temporarily stores it. The server then passes the temporarily stored image data to an artificial intelligence module. The artificial intelligence module analyzes the image data and generates an evaluation score based on evaluation criteria (color, brightness, composition, and degree of match with tags). The server then stores the generated evaluation score and the image data in a database.

[1463] Input: Received image data

[1464] Output: Evaluation scores and image data stored in a database

[1465] How it works: The AI ​​module analyzes the color and composition of a landscape photo and generates an evaluation score of 85 points. This score is then stored in a database by the server.

[1466] Step 3:

[1467] When uploading an image, the server uses the user's device's camera and microphone to analyze the user's facial expressions and voice, collecting data for emotion recognition. The emotion engine then analyzes the collected data to identify the user's emotional state (happiness, sadness, excitement, etc.). The server then adjusts the evaluation score taking into account the recognized emotional state.

[1468] Input: facial expression and voice data when uploading images

[1469] Output: Adjusted rating score

[1470] How it works: When a user uploads an image, the camera analyzes his facial expression, and the emotion engine recognizes "joy," adjusting the rating score from 85 to 88.

[1471] Step 4:

[1472] The server generates a feedback message based on the evaluation score and the user's sentiment analysis results. The feedback message includes details of the evaluation score, areas for improvement, and comments based on the user's sentiment. The server then sends the generated feedback message to the user's device. The device receives the feedback message and displays it to the user.

[1473] Input: Adjusted rating scores and user sentiment analysis results

[1474] Output: The feedback message that is displayed to the user

[1475] What it does: The device displays a feedback message like, "This image scores 88 points. Great composition and colors. A little more sharpness would make it even better. I can see that you are happy with this photo."

[1476] Step 5:

[1477] The server continuously stores all posted image data and evaluation scores in a database. Based on the stored data and user sentiment data, it analyzes trends and improves the evaluation algorithm.

[1478] Input: Stored image data, rating scores, and emotion data

[1479] Output: Trend analysis and evaluation algorithm improvements

[1480] How it works: The server analyzes landscape photos posted within a certain period of time and the associated user emotional data, and analyzes the relationship between current trends, popular filters, tag usage, and emotional state. This data is used to improve the evaluation algorithm.

[1481] Step 6:

[1482] The server aggregates the evaluation scores and emotion data of all users and generates ranking information, which is then made public to users, who can access the ranking page from their devices to check their ranking.

[1483] Input: Rating scores and sentiment data for all users

[1484] Output: Public ranking information

[1485] How it works: The server aggregates the evaluation scores and sentiment data of all users and displays the top 10 users every week. When a user accesses the ranking page, they can see their ranking and the reason for their ranking.

[1486] (Application example 2)

[1487] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1488] Conventional methods for measuring the effectiveness of advertising have the problem of making it difficult to grasp the viewer's emotional state in real time and reflect it in feedback. Furthermore, to accurately evaluate the effectiveness of advertising, it is necessary to utilize user emotional data, but there has been a lack of means to achieve this. Furthermore, a system that can instantly evaluate and improve advertising performance is needed.

[1489] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1490] In this invention, the server includes means for receiving image data from a user, artificial intelligence module means for analyzing the received image data and generating an evaluation score, emotion engine means for recognizing the user's emotions, means for adjusting the evaluation score based on the user's emotional state, means for providing feedback on the evaluation score to the user, means for storing the image data and the evaluation score in a database, means for generating ranking information based on the stored data, and means for disclosing the ranking information to users so that they can check their ranking position. This enables emotions to be analyzed in real time when viewing an advertisement and provides effective feedback. Furthermore, it becomes possible to generate advertisement evaluations and ranking information based on emotion data, thereby enabling more accurate improvement of advertising performance.

[1491] "Image data" is a collection of visual information posted by users from their devices.

[1492] "User" means an individual or group that uses the system or application to post image data and receive evaluation results and feedback.

[1493] The "artificial intelligence module" is a software component for analyzing received image data and generating an evaluation score.

[1494] The "emotion engine" is a software component that analyzes the user's facial expressions and voice data to recognize their emotional state.

[1495] The "evaluation score" is a numerical evaluation value generated by the artificial intelligence module based on the analyzed image data.

[1496] "Feedback" is information provided to users, such as comments and advice based on rating scores and sentiment analysis.

[1497] A "database" is a storage system for storing image data, evaluation scores, and related information.

[1498] "Ranking information" is information obtained by aggregating and ranking evaluation scores from multiple users.

[1499] The "server" is a computer system that controls the entire process, including receiving, analyzing, and storing image data, and sending feedback.

[1500] The present invention relates to a system for evaluating the effectiveness of advertisements in real time. The system analyzes the emotional state of a user when viewing an advertisement using smart glasses and generates an evaluation score. Specific embodiments are described below.

[1501] System Configuration

[1502] The system consists of the following major components:

[1503] 1. User device: Smart glasses

[1504] 2. Server

[1505] 3. Artificial Intelligence Module

[1506] 4. Emotion Engine

[1507] 5. Database

[1508] 6. Feedback Generation Module

[1509] 7. Ranking Generation Module

[1510] Hardware and Software Use

[1511] Hardware

[1512] Smart glasses: Use built-in cameras and microphones to collect facial expression and voice data from the user.

[1513] Server: Processes and stores data using high-performance processors and large-capacity storage.

[1514] software

[1515] Python: The main implementation language for programs

[1516] OpenCV: A library for video capture and image processing

[1517] EmotionEngine: A software component (virtual module) for analyzing user emotions

[1518] AdEvaluator: A software component (virtual module) for generating ad evaluation scores.

[1519] FeedbackGenerator: A software component (virtual module) for generating feedback messages.

[1520] Cloud services (e.g., AWS, Google Cloud, Azure): Distributed computing environments for data storage and analysis

[1521] System Operation

[1522] User's device

[1523] When a user wears the smart glasses and watches an advertisement, the built-in camera and microphone collect facial expression and voice data, which is then transmitted to a server in real time.

[1524] server

[1525] The server receives and temporarily stores the image and audio data sent from the user's device. The received data is processed as follows:

[1526] 1. Emotion analysis: The server uses the Emotion Engine to analyze the user's emotions from the image data. Specifically, it recognizes emotions such as "happiness," "surprise," and "sadness" from facial expressions and voice tone.

[1527] 2. Generating an ad rating score: The server uses AdEvaluator to generate a rating score for the received ad data. This rating score is calculated based on the visual elements of the ad, its content, and the user's viewing time.

[1528] 3. Rating Score Adjustment: The server adjusts the generated rating score based on the user's recognized emotional state. For example, if the user has the emotion "joy," the server may set the rating score slightly higher.

[1529] 4. Feedback Generation: The server uses the FeedbackGenerator to generate a feedback message based on the adjusted rating score. The feedback message includes specific advice and rating results to show to the user.

[1530] Data storage and ranking information generation

[1531] The server stores the generated evaluation scores and related data in a database. Based on the stored data, a ranking generation module generates ranking information, allowing users to check their own evaluation scores and ranking positions.

[1532] Specific examples

[1533] As a specific example, data processing when viewing an advertisement will be shown.

[1534] 1. A user puts on the smart glasses and watches an advertisement.

[1535] 2. The camera and microphone in the smart glasses collect data and send it to the server.

[1536] 3. The server uses EmotionEngine to perform emotion analysis and recognize the emotion of "joy."

[1537] 4. The server uses the AdEvaluator to generate a rating score for the ad (e.g., 85 points).

[1538] 5. Adjust the rating score slightly (e.g., 88 points) based on the perceived emotional state.

[1539] 6. Use FeedbackGenerator to generate a feedback message such as "This ad scores 88 points."

[1540] 7. The server sends the evaluation score and feedback message to the smart glasses and displays it to the user.

[1541] Prompt Sentence Examples

[1542] "Design a system that uses the camera and microphone on smart glasses to analyze emotions in real time when a user watches an advertisement, and evaluate the effectiveness of the advertisement. Adjust the evaluation score based on the emotion analysis data and generate a feedback message. Design this as a Python application that runs on the smart glasses, and clearly state the necessary modules and cloud services."

[1543] The system of the present invention configured as described above contributes to improving the quality of advertising and engagement by evaluating the effectiveness of advertising in real time and providing feedback based on the user's emotional state.

[1544] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1545] Step 1:

[1546] Users wear the smart glasses and watch advertisements, while the built-in camera and microphone collect facial expression and voice data.

[1547] Specifically, the smart glasses' camera captures the user's face frame by frame and simultaneously acquires voice data. These data are sent from the device to the server in real time. The input is facial expression and voice data, and the output is the data sent to the server.

[1548] Step 2:

[1549] The server receives the image data and audio data sent from the terminal and temporarily stores them.

[1550] Specifically, the server temporarily stores the received data in a specific folder or memory. The input is the user's facial expression and voice data, and the output is the temporarily stored raw data.

[1551] Step 3:

[1552] The server uses EmotionEngine to analyze the user's emotions from the temporarily stored image data.

[1553] Specifically, the Emotion Engine analyzes facial features and voice tones to recognize emotional states such as "happiness," "surprise," and "sadness." The input is temporarily saved image and voice data, and the output is recognized emotional data.

[1554] Step 4:

[1555] The server uses the AdEvaluator to generate an evaluation score for the advertising data.

[1556] Specifically, AdEvaluator calculates a rating score based on the visual elements of the ad, its content, the user's viewing time, etc. The input is the ad data, and the output is a rating score (e.g., 85 points).

[1557] Step 5:

[1558] The server adjusts the generated evaluation score based on the recognized emotion data.

[1559] Specifically, AdEvaluator adjusts the final rating score by taking into account the emotional data obtained from EmotionEngine. For example, if the emotion is "joy," the rating score is adjusted from 85 to 88. The input is the rating score and emotional data, and the output is the adjusted rating score.

[1560] Step 6:

[1561] The server uses a FeedbackGenerator to generate feedback messages based on the adjusted reputation scores.

[1562] Specifically, the FeedbackGenerator generates a feedback message containing the evaluation result and advice to be shown to the user based on the adjusted evaluation score. The input is the adjusted evaluation score, and the output is the feedback message.

[1563] Step 7:

[1564] The server sends the generated feedback message to the user's terminal and displays it on the smart glasses.

[1565] Specifically, the server sends the generated feedback message to the user's smart glasses in real time and visually displays it. The input is the feedback message, and the output is the display on the smart glasses.

[1566] Step 8:

[1567] The server stores the generated evaluation scores and related data in a database.

[1568] Specifically, the server stores the evaluation scores, sentiment analysis data, and feedback messages in a database. The input is the evaluation scores and related data, and the output is the saved data.

[1569] Step 9:

[1570] The server generates ranking information based on the stored data and allows users to check their ranking position.

[1571] Specifically, the ranking generation module analyzes the data of all users and generates a ranking of their evaluation scores. Users can view the ranking information through smart glasses or other devices. The input is the stored data, and the output is the generated ranking information.

[1572] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1573] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1574] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1575] [Fourth embodiment]

[1576] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1577] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1578] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1579] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1580] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1581] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1582] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1583] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1584] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1585] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1586] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1587] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1588] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1589] This system allows users to post image data via an image sharing application, and an artificial intelligence module evaluates the image data to calculate an evaluation score and provide feedback to the user. It also generates and publishes ranking information based on the collected data.

[1590] User uploads an image

[1591] Users launch an image sharing application from their smartphone, PC, or other device and tap the "Post Image" button.

[1592] The device provides users with a gallery and camera function, allowing them to select the images they want to post.

[1593] The user selects an image through the device's interface and taps the "Upload" button.

[1594] The device compresses the image data and generates and transmits an API request to the server.

[1595] Examples:

[1596] Users can upload landscape photos they have taken during their trip using the app's "upload" button, at which point the device will send the selected images to the server.

[1597] Images are received on the server side and evaluated by AI

[1598] The server temporarily stores the image data sent from the terminal and passes it to the artificial intelligence module.

[1599] The artificial intelligence module analyzes the image data and generates an evaluation score based on evaluation criteria (e.g., color, brightness, composition, and match with tags).

[1600] The server stores the generated evaluation scores and image data in a database.

[1601] Examples:

[1602] An artificial intelligence module analyzes the color and composition of the landscape photo and generates an evaluation score of 85 points, which is then stored in a database by the server.

[1603] Provide feedback to users on the evaluation results

[1604] The server generates a feedback message based on the rating score, including an example comment such as "This image scores 85 points. Great composition and colors! A little more sharpness would make it even better."

[1605] The server sends the generated feedback message to the user's terminal.

[1606] The device receives and displays the feedback message and provides the user with the evaluation results.

[1607] Examples:

[1608] A feedback message such as "This image scores 85 points. Great composition and colors! A little more sharpness would make it even better" will be displayed on the user's device.

[1609] Image data collection and analysis

[1610] The server keeps a database of all submitted image data and evaluation scores.

[1611] The server uses the stored data to analyze trends and understand evaluation trends.

[1612] The server uses the obtained analysis results to improve the evaluation algorithm of the artificial intelligence module.

[1613] Examples:

[1614] The server analyzes landscape photos posted over a certain period of time, analyzes current trends, and analyzes the use of popular filters and tags, and uses this data to improve the rating algorithm.

[1615] Ranking function and comparison with other users

[1616] The server aggregates the evaluation scores of all users and generates ranking information.

[1617] The server makes the generated ranking information available to users.

[1618] Users can access the ranking page from their device and check their position.

[1619] Examples:

[1620] The server aggregates the evaluation scores of all users and displays the top 10 users every week. When user "Taro Sato" accesses the ranking page, he can see that he is in 5th place.

[1621] The present invention, configured as described above, allows users to receive instant feedback on images they post and easily compare their images with those of other users. It also makes it possible to improve the service based on the collected data. This increases user engagement and improves the quality of image sharing applications.

[1622] The processing flow will be explained below.

[1623] Step 1:

[1624] The user launches an image sharing application on a device such as a smartphone or PC and taps the "Post Image" button.

[1625] Step 2:

[1626] The device presents the user with a file selection screen, and the user selects an image from the gallery or camera.

[1627] Step 3:

[1628] The user selects the image they want to post and taps the "Upload" button.

[1629] Step 4:

[1630] The terminal compresses the selected image data and generates and transmits an API request to the server.

[1631] Step 5:

[1632] The server receives the image data sent from the terminal and temporarily stores it.

[1633] Step 6:

[1634] The server passes the temporarily stored image data to the artificial intelligence module.

[1635] Step 7:

[1636] An artificial intelligence module analyzes the image data and generates an evaluation score based on evaluation criteria (e.g., color, brightness, composition, and match with tags).

[1637] Step 8:

[1638] The server receives the evaluation score from the artificial intelligence module and stores it in a database together with the image data.

[1639] Step 9:

[1640] The server generates a feedback message based on the rating score, including a comment like "This image scores 85 points. Great composition and colors! A little more sharpness would make it even better."

[1641] Step 10:

[1642] The server sends the generated feedback message to the user's terminal.

[1643] Step 11:

[1644] The device receives the feedback message and displays it to the user.

[1645] Step 12:

[1646] The server continues to store the posted image data and evaluation scores in a database.

[1647] Step 13:

[1648] The server uses the stored data to analyze trends and improve evaluation algorithms.

[1649] Step 14:

[1650] The server aggregates the evaluation scores of all users and generates ranking information.

[1651] Step 15:

[1652] The server publishes the generated ranking information to the user.

[1653] Step 16:

[1654] Users access the ranking page on their device and check their ranking position.

[1655] Example 1

[1656] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1657] In conventional image sharing applications, image ratings and feedback were done manually, making it difficult for users to obtain detailed ratings immediately. Furthermore, there were problems with inconsistent rating criteria, making it difficult to generate fair rankings. Furthermore, trend analysis and algorithm improvements were done manually, resulting in low efficiency of the overall system.

[1658] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1659] In this invention, the server includes means for receiving image data from users, means for compressing and transmitting the received image data, means for temporarily storing the received image data, an artificial intelligence module for analyzing the received image data and generating an evaluation score, means for providing feedback on the evaluation score to the user, means for storing the image data and the evaluation score in a database, means for performing trend analysis and algorithm improvement based on the stored data, and means for generating ranking information based on the stored data. This allows users to receive detailed evaluations in real time, enabling the generation of fair rankings. Furthermore, because trend analysis and algorithm improvement are performed automatically, the efficiency of the entire system is improved.

[1660] "Image data" refers to visual information represented in digital form.

[1661] "User" means a person who uses an image sharing application to post images and receive ratings and feedback.

[1662] "Means for receiving" refers to a mechanism for capturing image data sent by a user.

[1663] "Means for compressing and transmitting" refers to a method for reducing the size of image data and transmitting it to the server.

[1664] "Temporary storage means" refers to a mechanism for temporarily storing received image data.

[1665] "Artificial Intelligence Module" refers to a software or hardware component for analyzing image data based on evaluation criteria and generating an evaluation score.

[1666] The "means for providing feedback" refers to a method for forming a feedback message based on the generated evaluation score and providing it to the user.

[1667] "Database" refers to a system that systematically stores and manages image data and evaluation scores.

[1668] "Trend analysis" refers to the act of extracting and analyzing current trends and patterns based on collected data.

[1669] "Means for improving algorithms" refers to methods for using the results of trend analysis to improve the evaluation process of artificial intelligence modules and other system components.

[1670] "Ranking information" refers to ranking information generated based on the evaluation scores of all users.

[1671] This system allows users to post image data via an image sharing application, and an artificial intelligence module evaluates the image data to calculate an evaluation score and provide feedback to the user. It also generates and publishes ranking information based on the collected data.

[1672] Overall system configuration

[1673] The system consists of various components, including an image sharing application, a server, a database, an artificial intelligence module, a feedback generation system, and a ranking generation system.

[1674] Hardware and software used

[1675] Device: The smartphone or PC used by the user.

[1676] Server: A remote computer that receives, stores, and processes data.

[1677] Database: A system for managing data in an organized manner, such as MySQL or PostgreSQL.

[1678] Artificial Intelligence Module: A rating system built using machine learning libraries such as TensorFlow and PyTorch.

[1679] Specific implementation flow

[1680] 1. User uploads an image

[1681] Users launch an image sharing application from their smartphone, PC, or other device and tap the "Post Image" button.

[1682] The device provides users with a gallery and camera function, allowing them to select the images they want to post.

[1683] The user selects an image through the device's interface and taps the "Upload" button.

[1684] The device compresses the image data and generates and transmits an API request to the server.

[1685] Examples:

[1686] Users can upload landscape photos they have taken during their trip using the app's "upload" button, at which point the device will send the selected images to the server.

[1687] 2. The image is received on the server side and evaluated by AI

[1688] The server temporarily stores the image data sent from the terminal and passes it to the artificial intelligence module.

[1689] The artificial intelligence module analyzes the image data and generates an evaluation score based on evaluation criteria (e.g., color, brightness, composition, and match with tags).

[1690] The server stores the generated evaluation scores and image data in a database.

[1691] Examples:

[1692] An artificial intelligence module analyzes the color and composition of the landscape photo and generates an evaluation score of 85 points, which is then stored in a database by the server.

[1693] 3. Provide feedback to users on the evaluation results

[1694] The server generates a feedback message based on the rating score, including an example comment such as "This image scores 85 points. Great composition and colors! A little more sharpness would make it even better."

[1695] The server sends the generated feedback message to the user's terminal.

[1696] The device receives and displays the feedback message and provides the user with the evaluation results.

[1697] Examples:

[1698] A feedback message such as "This image scores 85 points. Great composition and colors! A little more sharpness would make it even better" will be displayed on the user's device.

[1699] 4. Image data collection and analysis

[1700] The server keeps a database of all submitted image data and evaluation scores.

[1701] The server uses the stored data to analyze trends and understand evaluation trends.

[1702] The server uses the obtained analysis results to improve the evaluation algorithm of the artificial intelligence module.

[1703] Examples:

[1704] The server analyzes landscape photos posted over a certain period of time, analyzes current trends, and analyzes the use of popular filters and tags, and uses this data to improve the rating algorithm.

[1705] 5. Ranking function and comparison with other users

[1706] The server aggregates the evaluation scores of all users and generates ranking information.

[1707] The server makes the generated ranking information available to users.

[1708] Users can access the ranking page from their device and check their position.

[1709] Examples:

[1710] The server aggregates the evaluation scores of all users and displays the top 10 users every week. When a user visits the ranking page, they can see that they are in 5th place.

[1711] Example prompts for generative AI models

[1712] "Rate the color and composition of landscape photos posted to an image sharing application. Also, generate a feedback message based on your rating. For example, this image gets a score of 85. Great composition and color! A little more sharpness would make it even better."

[1713] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1714] Specific flow of program processing

[1715] Step 1: Launching the Application

[1716] Users launch an image sharing application from their smartphone or PC.

[1717] Input: terminal

[1718] Output: The interface that the application launches.

[1719] What happens: A user taps an app icon on their device to launch an application.

[1720] Step 2: Tap the image posting button

[1721] The user taps the "Post Image" button on the application.

[1722] Input: Application interface

[1723] Output: Access to gallery and camera functions

[1724] Specific operation: The user taps the "Post Image" button in the app and moves to the next operation screen.

[1725] Step 3: Select an image

[1726] The device provides users with a gallery and camera function to select images to post.

[1727] The user selects an image using the gallery or camera.

[1728] Input: Device gallery or camera

[1729] Output: Selected image data

[1730] What happens: The user selects a specific image from the device's image gallery or takes a new one using the device's camera.

[1731] Step 4: Upload an image

[1732] The user selects an image through the device's interface and taps the "Upload" button.

[1733] Input: Selected image data

[1734] Output: Compressed image data and API request

[1735] The device compresses the image data and generates and transmits an API request to the server.

[1736] Specific operation: The user taps the "Upload" button in the app, and the device compresses the image data and sends it to the server.

[1737] Step 5: Receiving image data

[1738] The server receives the image data sent from the terminal.

[1739] Input: Compressed image data and API request

[1740] Output: Temporarily saved image data

[1741] Specific operation: The server receives the API request and temporarily stores the image data.

[1742] Step 6: Temporarily save image data

[1743] The server temporarily stores the received image data.

[1744] Input: Received image data

[1745] Output: Temporarily saved image data

[1746] Specific operation: Saves image data in a specified folder or memory area on the server.

[1747] Step 7: Transferring data to the AI ​​module

[1748] The server passes the image data to an artificial intelligence module.

[1749] Input: Temporarily saved image data

[1750] Output: Image data to be analyzed

[1751] Specific operation: The server reads the temporarily stored image data and sends it to the AI ​​platform.

[1752] Step 8: Image analysis using AI

[1753] The artificial intelligence module analyzes the image data and generates an evaluation score based on evaluation criteria (e.g., color, brightness, composition, and match with tags).

[1754] Input: Image data to be analyzed

[1755] Output: Evaluation score

[1756] How it works: The AI ​​analyzes elements such as color, brightness, composition, and tags, and generates an evaluation score based on a scoring model.

[1757] Step 9: Saving to the Database

[1758] The server stores the generated evaluation scores and image data in a database.

[1759] Input: Evaluation scores and image data

[1760] Output: Evaluation scores and image data stored in a database

[1761] Specific operation: The server stores the evaluation score and image data as a database entry.

[1762] Step 10: Generate feedback messages

[1763] The server generates a feedback message based on the rating score.

[1764] Input: Rating score

[1765] Output: Feedback message

[1766] Specific behavior: The server selects a feedback template from the rating score and generates a message.

[1767] Step 11: Sending a feedback message

[1768] The server sends the generated feedback message to the user's terminal.

[1769] Input: Feedback message

[1770] Output: Feedback message displayed on the user's terminal

[1771] Specific operation: The server sends a message to the user's device via push notification or API.

[1772] Step 12: Displaying Messages

[1773] The device receives and displays the feedback message and provides the user with the evaluation results.

[1774] Input: Feedback message

[1775] Output: Feedback that is displayed to the user

[1776] What happens: Your device receives a notification and displays a message in an in-app pop-up or in the notification bar.

[1777] Step 13: Continually save your data

[1778] The server keeps a database of all submitted image data and evaluation scores.

[1779] Input: Newly submitted image data and rating score

[1780] Output: A continuously updated database

[1781] What it does: Every time new data is submitted, the database entry is updated.

[1782] Step 14: Trend analysis

[1783] The server uses the stored data to analyze trends and understand evaluation trends.

[1784] Input: Historical data

[1785] Output: Trend reports and analysis results

[1786] What it does: The server analyzes historical data and extracts trends and popularity.

[1787] Step 15: Improve the algorithm

[1788] The server uses the obtained analysis results to improve the evaluation algorithm of the artificial intelligence module.

[1789] Input: Trend analysis results

[1790] Output: Improved evaluation algorithm

[1791] What it does: The server learns from the data and improves the rating algorithm.

[1792] Step 16: Generate ranking information

[1793] The server aggregates the evaluation scores of all users and generates ranking information.

[1794] Input: User rating score

[1795] Output: Ranking information

[1796] What happens: The server aggregates all the rating scores and creates a ranking list.

[1797] Step 17: Publish ranking information

[1798] The server makes the generated ranking information available to users.

[1799] Input: Ranking information

[1800] Output: Published ranking information

[1801] Specific operation: Publish ranking information on a web page or within an application.

[1802] Step 18: Check your rankings

[1803] Users can access the ranking page from their device and check their position.

[1804] Input: Ranking page URL or link

[1805] Output: User ranking information

[1806] What happens: A user visits the rankings page and checks their rating score and ranking.

[1807] (Application example 1)

[1808] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1809] Conventional image sharing applications lack sufficient ratings and feedback on the images posted by users, resulting in low user engagement. Furthermore, the mechanisms for comparing photos with other users and analyzing trends were inadequate, making it difficult to motivate users to post. In particular, specific advice on how to improve photography skills was not provided.

[1810] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1811] In this invention, the server includes means for receiving image data from users, artificial intelligence module means for analyzing the received image data and generating an evaluation score, means for providing the evaluation score as feedback to the user in real time, means for generating a feedback message based on the evaluation score, means for storing the image data and the evaluation score in a database, means for generating ranking information based on the stored data, and means for publishing the ranking information to users so that users can check their own ranking position. This allows users to receive immediate feedback on the images they post, compare their images with those of other users, and check their own ranking position. Comments containing specific suggestions for improvement are also provided, contributing to the improvement of users' photography skills.

[1812] "Image data" refers to digital image files that a user takes or selects from their device and posts through the application.

[1813] An "artificial intelligence module" is a machine learning model or algorithm for analyzing received image data and generating an evaluation score.

[1814] The "evaluation score" is an evaluation score for the image data generated by the artificial intelligence module.

[1815] "Feedback" refers to messages about evaluation results and areas for improvement provided to users based on their evaluation scores.

[1816] "Database" means a digital storage system for storing and managing image data and evaluation scores.

[1817] "Ranking information" is information that aggregates the evaluation scores of all users and indicates their relative position among other users.

[1818] "Real-time" refers to the time interval during which a user can receive immediate feedback after posting image data.

[1819] A "feedback message" is a comment containing specific improvements that is generated based on the evaluation score.

[1820] This invention is a system in which users post image data through an image sharing application, an AI module evaluates the image data, calculates an evaluation score, and provides feedback to the user. It also has a configuration in which ranking information is generated and published based on the collected data.

[1821] Hardware / Software used

[1822] Hardware:

[1823] Server environment: AWS EC2 instance as an example

[1824] User device: Smartphone or PC

[1825] software:

[1826] Framework:Flask

[1827] AI model: ResNet50 model provided by PyTorch

[1828] Database: MySQL or PostgreSQL

[1829] Specific processing flow

[1830] 1. Receiving image data:

[1831] Users launch an image sharing application on their smartphone or PC and tap the "Post Image" button. In response to this operation, the device provides a gallery or camera function, allowing the user to select the image they want to post. After selection, the device compresses the image data and sends it to the server.

[1832] 2. AI evaluation:

[1833] The server temporarily stores the received image data and passes it to an artificial intelligence module. The AI ​​model used is PyTorch's ResNet50. The model analyzes the image data and generates an evaluation score based on color, brightness, composition, and degree of match with tags.

[1834] 3. Feedback Processing:

[1835] The server generates a feedback message based on the generated evaluation score. For example, if the score is 85, it will provide a comment with specific suggestions for improvement, such as "This image scores 85. Great composition and colors! A little more sharpness would make it even better."

[1836] 4. Save to database:

[1837] The server stores the evaluation scores and image data in a database, which enables trend analysis and evaluation trends to be identified, and can also be used to improve the AI ​​evaluation algorithm.

[1838] 5. Generating and publishing ranking information:

[1839] The server aggregates the evaluation scores of all users and generates ranking information, which is then made public to users so that they can check their own ranking position.

[1840] Specific examples

[1841] Specifically, users submit landscape photos taken during their travels using the app's "Upload" button. The device compresses the image data and sends it to the server. The server receives the image data, temporarily stores it, and then passes it to a PyTorch ResNet50 model. The model analyzes color, lighting, composition, and tag information to generate an evaluation score. For example, if an evaluation score of 85 points is generated, the server returns a feedback message to the user saying, "This image scores 85 points. Excellent composition and color! A little more sharpness would make it even better." The evaluation scores and image data are also stored in a database, and the server aggregates all user data to generate and publish weekly ranking information.

[1842] Prompt Sentence Examples

[1843] Based on the adjusted settings, the system uses PyTorch's ResNet50 model to evaluate images uploaded by users. ResNet50 is used as a pre-trained model for image classification. The model analyzes the color and composition of the image and generates an evaluation score. Based on the score, the system provides specific feedback to the user in the form of comments suggesting areas for improvement.

[1844] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1845] Step 1:

[1846] The user uploads an image.

[1847] Specific operation: The user launches an image sharing application on a smartphone or PC and taps the "Post Image" button. The device responds to this action by providing gallery and camera functions. The user selects the image they want to post, selects the image through the device's interface, and taps the "Upload" button. At this time, the input is the image data selected by the user. The device compresses this image data and generates and sends an API request to the server. The output is the compressed image data sent to the server.

[1848] Step 2:

[1849] The server receives and stores the image data.

[1850] Specific operation: The server receives image data sent from the terminal. The input is compressed image data sent from the terminal. The server temporarily stores this image data. The output is the temporarily stored image data.

[1851] Step 3:

[1852] The server analyzes the image data and generates an evaluation score.

[1853] Specific operation: The server passes the temporarily stored image data to the artificial intelligence module. The input is the temporarily stored image data. The artificial intelligence module used is the PyTorch ResNet50 model. The model analyzes the image data and generates an evaluation score based on color, brightness, composition, and degree of match with tags. The output is the generated evaluation score. For example, the analysis result of a landscape photo will be a score of 85 points.

[1854] Step 4:

[1855] The server generates a feedback message based on the rating score.

[1856] Specific behavior: The server generates a feedback message based on the generated evaluation score. The input is the evaluation score. The feedback message includes specific improvements. For example, a message like "This image scores 85 points. Great composition and color! A little more sharpness would make it even better." is generated. The output is the feedback message.

[1857] Step 5:

[1858] The server sends a feedback message to the user's device.

[1859] Specific operation: The server sends the generated feedback message to the user's terminal. The input is the generated feedback message. The terminal receives this message and displays it to the user. The output is the feedback message displayed on the terminal.

[1860] Step 6:

[1861] The server stores the evaluation scores and image data in a database.

[1862] Specific operation: The server saves the generated evaluation score and corresponding image data in a database. The input is the evaluation score and image data. The server stores this in the database. The output is the evaluation score and image data saved in the database.

[1863] Step 7:

[1864] The server generates and publishes ranking information.

[1865] Specific operation: The server aggregates the evaluation scores of all users and generates ranking information. The input is all the evaluation score data. The server generates ranking information based on this and publishes it to users. The output is the published ranking information. Users can access the ranking page from their devices and check their position.

[1866] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1867] This invention is a system in which a user posts image data through an image sharing application, and an artificial intelligence module evaluates the image data to calculate an evaluation score and provide feedback to the user. In addition, by combining this system with an emotion engine that recognizes the user's emotions, it is possible to adjust the evaluation score and improve the user experience.

[1868] User uploads an image

[1869] Users launch an image sharing application on their smartphone, PC, or other device and tap the "Post Image" button.

[1870] The device displays a file selection screen to the user, and the user selects an image from the gallery or camera.

[1871] Users select the image they want to post and tap the "Upload" button.

[1872] The device compresses the image data and generates and transmits an API request to the server.

[1873] Examples:

[1874] Users can upload landscape photos they have taken during their trip using the app's "upload" button, at which point the device will send the selected images to the server.

[1875] Images are received on the server side and evaluated by AI

[1876] The server receives the image data sent from the terminal and temporarily stores it.

[1877] The server passes the temporarily stored image data to the artificial intelligence module.

[1878] The artificial intelligence module analyzes the image data and generates an evaluation score based on evaluation criteria (e.g., color, brightness, composition, and match with tags).

[1879] The server stores the generated evaluation scores and image data in a database.

[1880] Examples:

[1881] An artificial intelligence module analyzes the color and composition of the landscape photo and generates an evaluation score of 85 points, which is then stored in a database by the server.

[1882] User sentiment analysis and feedback adjustment

[1883] When uploading an image, the server uses the camera and microphone of the device the user is using to analyze the user's facial expressions and voice, collecting data to recognize emotions.

[1884] The emotion engine analyzes the collected data and identifies the user's emotional state (e.g., happy, sad, excited, etc.).

[1885] The server adjusts the rating score to include the perceived emotional state, for example, if the user is very happy, the rating score may be set somewhat higher.

[1886] Examples:

[1887] When a user uploads an image, the camera analyzes his facial expression and the emotion engine recognizes "happiness," adjusting the rating score from 85 to 88.

[1888] Providing Feedback

[1889] The server generates a feedback message based on the rating score and the user's sentiment analysis, including an example comment such as "This image scores 88 points. Great composition and colors! A little more sharpness would make it even better. I can see that you are pleased with this photo."

[1890] The server sends the generated feedback message to the user's terminal.

[1891] The terminal receives the feedback message and displays it to the user.

[1892] Examples:

[1893] A feedback message such as "This image scores 88 points. Great composition and colors! A little more sharpness would make it even better. I can see that you are happy with this photo" will be displayed on the user's device.

[1894] Image data collection and analysis

[1895] The server keeps a database of all submitted image data and evaluation scores.

[1896] The server analyzes trends and improves the evaluation algorithm based on the stored data and user sentiment data.

[1897] Examples:

[1898] The server analyzes landscape photos posted within a certain period of time and the associated user emotional data, and analyzes the relationship between current trends, popular filters, tag usage, and emotional states. This data is used to improve the evaluation algorithm.

[1899] Ranking function and comparison with other users

[1900] The server aggregates the evaluation scores and emotional data of all users and generates ranking information.

[1901] The server makes the generated ranking information available to users.

[1902] Users can access the ranking page from their device and check their position.

[1903] Examples:

[1904] The server aggregates the evaluation scores and emotional data of all users and displays the top 10 users each week. When user "Sato Taro" accesses the ranking page, he sees that he is in 5th place and the reason for this is also displayed.

[1905] The present invention, configured as described above, allows users to receive instant feedback on images they post and easily compare their views with those of other users. Furthermore, by incorporating user emotional data, it is possible to provide more personalized feedback and experiences, further increasing user engagement. Furthermore, it is possible to improve services based on the collected data.

[1906] The processing flow will be explained below.

[1907] Step 1:

[1908] The user launches an image sharing application on a device such as a smartphone or PC and taps the "Post Image" button.

[1909] Step 2:

[1910] The device presents the user with a file selection screen, and the user selects an image from the gallery or camera.

[1911] Step 3:

[1912] The user selects the image they want to post and taps the "Upload" button.

[1913] Step 4:

[1914] The device compresses the image data and generates and sends an API request to the server.

[1915] Step 5:

[1916] The server receives the image data sent from the terminal and temporarily stores it.

[1917] Step 6:

[1918] The server passes the temporarily stored image data to the artificial intelligence module.

[1919] Step 7:

[1920] An artificial intelligence module analyzes the image data and generates an evaluation score based on evaluation criteria (e.g., color, brightness, composition, and match with tags).

[1921] Step 8:

[1922] The server receives the evaluation score from the artificial intelligence module and stores it in a database together with the image data.

[1923] Step 9:

[1924] The server analyzes the user's facial expressions and voice using the device's camera and microphone to collect emotional data when uploading images.

[1925] Step 10:

[1926] The emotion engine analyzes the collected data and identifies the user's emotional state (e.g., happy, sad, excited, etc.).

[1927] Step 11:

[1928] The server adjusts the rating score to include the perceived emotional state and determines the final rating score, for example, slightly increasing the score if the user is very happy.

[1929] Step 12:

[1930] The server generates a feedback message based on the rating score and the user's sentiment analysis, including a comment like "This image scores 88 points. Great composition and colors! A little more sharpness would make it even better. I can see you're happy with this photo."

[1931] Step 13:

[1932] The server sends the generated feedback message to the user's terminal.

[1933] Step 14:

[1934] The device receives the feedback message and displays it to the user.

[1935] Step 15:

[1936] The server continuously stores the posted image data, evaluation scores, and related user emotional data in a database.

[1937] Step 16:

[1938] The server uses the stored data to analyze trends and improve evaluation algorithms.

[1939] Step 17:

[1940] The server aggregates the evaluation scores and emotional data of all users and generates ranking information.

[1941] Step 18:

[1942] The server publishes the generated ranking information to the user.

[1943] Step 19:

[1944] Users access the ranking page on their device and check their ranking position.

[1945] Example 2

[1946] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1947] In conventional image sharing systems, evaluations of images posted by users are subjective, making it difficult to obtain consistent feedback. Furthermore, they do not take into account the user's emotional state, making it impossible to provide a personalized experience. This has led to challenges in improving user experience and reducing engagement.

[1948] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1949] In this invention, the server includes means for receiving image data from a user, artificial intelligence module means for analyzing the received image data and generating an evaluation score, means for feeding back the evaluation score to the user, means for storing the image data and the evaluation score in a database, means for generating ranking information based on the stored data, emotion engine means for collecting and recognizing the emotional state of the user, and means for adjusting the evaluation score based on the emotional state. This allows users to receive consistent evaluations and feedback, and further enables engagement to be improved through personalized feedback based on the emotional state.

[1950] "Image data" refers to digital image files photographed or acquired by a user.

[1951] "Means for receiving" refers to a process or device for transferring image data from a user to a server over the Internet.

[1952] "Artificial Intelligence Module" refers to the computational algorithms and software used to analyze image data and automatically generate an evaluation score.

[1953] An "evaluation score" is a numerical indicator of the quality and characteristics of an image calculated by an artificial intelligence module.

[1954] "Feedback means" refers to the process or system for notifying users of the evaluation score and providing the results as feedback.

[1955] A "database" is a digital information storage system for efficiently storing and managing image data and evaluation scores.

[1956] "Ranking information" refers to information that ranks image data posted by multiple users based on their evaluation scores.

[1957] An "emotion engine" is software and algorithms used to analyze and recognize a user's emotional state.

[1958] "Emotional state" refers to the psychological state of the user, such as joy, sadness, or excitement, analyzed from their facial expressions and voice.

[1959] "Adjustment" refers to a process or system for appropriately modifying or changing the rating score based on the user's emotional state.

[1960] This invention is a system in which a user posts image data through an image sharing application, and an artificial intelligence module evaluates the image data to calculate an evaluation score and provide feedback to the user. In addition, by combining this system with an emotion engine that recognizes the user's emotions, it is possible to adjust the evaluation score and improve the user experience.

[1961] First, the user launches an image sharing application on a device such as a smartphone or PC and taps the "Post Image" button. Next, the device displays a file selection screen for the user, allowing the user to select an image from the gallery or camera. Once the user selects the image they want to post and taps the "Upload" button, the device compresses the image data and generates and sends an API request to the server.

[1962] For example, when a user posts a landscape photo taken during a trip using the app's "upload" button, the device will compress the selected image and send it to the server.

[1963] The server receives the image data sent from the device and temporarily stores it. The server then passes the temporarily stored image data to an artificial intelligence module. The artificial intelligence module analyzes the image data and generates an evaluation score based on evaluation criteria such as color, brightness, composition, and degree of match with tags. The server then stores the generated evaluation score and image data in a database.

[1964] For example, an artificial intelligence module analyzes the color and composition of a landscape photo and generates an evaluation score of 85 points, which is then stored in a database by the server.

[1965] Next, the server uses the user's device's camera and microphone to analyze the user's facial expressions and voice when uploading an image, collecting data for emotion recognition. The emotion engine analyzes the collected data and identifies the user's emotional state (happiness, sadness, excitement, etc.). The server adjusts the evaluation score taking into account the recognized emotional state. For example, if the user is very happy, the evaluation score can be set higher.

[1966] Specifically, when a user uploads an image, the camera analyzes his facial expression, and the emotion engine recognizes "joy," adjusting the rating score from 85 to 88.

[1967] The server then generates a feedback message based on the evaluation score and the user's sentiment analysis results. For example, it might include a comment like, "This image scores 88 points. Excellent composition and color. A little more sharpness would make it even better. I can see that you are pleased with this photo." The server then sends the generated feedback message to the user's device. The device receives the feedback message and displays it to the user.

[1968] For example, a feedback message such as "This image scores 88 points. Great composition and color. A little more sharpness would make it even better. I can see that you are happy with this photo" will be displayed on the user's device.

[1969] In addition, the server stores all posted image data and evaluation scores in a database. Trend analysis and improvements to the evaluation algorithm are performed based on the stored data and user emotional data. For example, the server analyzes landscape photos posted within a certain period of time and the associated emotional data of users, and analyzes the relationship between current trends, popular filters, tag usage, and emotional states. This data is used to improve the evaluation algorithm.

[1970] Finally, the server aggregates the evaluation scores and sentiment data of all users to generate ranking information, which is then made public to users, who can access the ranking page from their devices to check their ranking.

[1971] For example, the server aggregates the evaluation scores and emotional data of all users and displays the top 10 users every week. When user "Taro Sato" accesses the ranking page, he sees that he is in 5th place and the reason for his ranking is also displayed.

[1972] This invention allows users to receive instant feedback on the images they post and easily compare their images with those of other users. Furthermore, by incorporating user emotional data, it is possible to provide more personalized feedback and experiences, further increasing user engagement. Furthermore, the collected data can be used to improve services.

[1973] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1974] Step 1:

[1975] Users launch an image sharing application on their smartphone, PC, or other device and tap the "Post Image" button. Next, the device displays a file selection screen for the user, allowing them to select an image from their gallery or camera. Finally, the user selects the image they want to post and taps the "Upload" button.

[1976] Input: User-selected image data

[1977] Output: Compressed image data sent to the server

[1978] Specific operation: When a user posts a landscape photo taken during a trip through the app's "Upload" button, the device compresses the selected image, generates an API request, and sends it to the server.

[1979] Step 2:

[1980] The server receives the image data sent from the device and temporarily stores it. The server then passes the temporarily stored image data to an artificial intelligence module. The artificial intelligence module analyzes the image data and generates an evaluation score based on evaluation criteria (color, brightness, composition, and degree of match with tags). The server then stores the generated evaluation score and the image data in a database.

[1981] Input: Received image data

[1982] Output: Evaluation scores and image data stored in a database

[1983] How it works: The AI ​​module analyzes the color and composition of a landscape photo and generates an evaluation score of 85 points. This score is then stored in a database by the server.

[1984] Step 3:

[1985] When uploading an image, the server uses the user's device's camera and microphone to analyze the user's facial expressions and voice, collecting data for emotion recognition. The emotion engine then analyzes the collected data to identify the user's emotional state (happiness, sadness, excitement, etc.). The server then adjusts the evaluation score taking into account the recognized emotional state.

[1986] Input: facial expression and voice data when uploading images

[1987] Output: Adjusted rating score

[1988] How it works: When a user uploads an image, the camera analyzes his facial expression, and the emotion engine recognizes "joy," adjusting the rating score from 85 to 88.

[1989] Step 4:

[1990] The server generates a feedback message based on the evaluation score and the user's sentiment analysis results. The feedback message includes details of the evaluation score, areas for improvement, and comments based on the user's sentiment. The server then sends the generated feedback message to the user's device. The device receives the feedback message and displays it to the user.

[1991] Input: Adjusted rating scores and user sentiment analysis results

[1992] Output: The feedback message that is displayed to the user

[1993] What it does: The device displays a feedback message like, "This image scores 88 points. Great composition and colors. A little more sharpness would make it even better. I can see that you are happy with this photo."

[1994] Step 5:

[1995] The server continuously stores all posted image data and evaluation scores in a database. Based on the stored data and user sentiment data, it analyzes trends and improves the evaluation algorithm.

[1996] Input: Stored image data, rating scores, and emotion data

[1997] Output: Trend analysis and evaluation algorithm improvements

[1998] How it works: The server analyzes landscape photos posted within a certain period of time and the associated user emotional data, and analyzes the relationship between current trends, popular filters, tag usage, and emotional state. This data is used to improve the evaluation algorithm.

[1999] Step 6:

[2000] The server aggregates the evaluation scores and emotion data of all users and generates ranking information, which is then made public to users, who can access the ranking page from their devices to check their ranking.

[2001] Input: Rating scores and sentiment data for all users

[2002] Output: Public ranking information

[2003] How it works: The server aggregates the evaluation scores and sentiment data of all users and displays the top 10 users every week. When a user accesses the ranking page, they can see their ranking and the reason for their ranking.

[2004] (Application example 2)

[2005] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2006] Conventional methods for measuring the effectiveness of advertising have the problem of making it difficult to grasp the viewer's emotional state in real time and reflect it in feedback. Furthermore, to accurately evaluate the effectiveness of advertising, it is necessary to utilize user emotional data, but there has been a lack of means to achieve this. Furthermore, a system that can instantly evaluate and improve advertising performance is needed.

[2007] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2008] In this invention, the server includes means for receiving image data from a user, artificial intelligence module means for analyzing the received image data and generating an evaluation score, emotion engine means for recognizing the user's emotions, means for adjusting the evaluation score based on the user's emotional state, means for providing feedback on the evaluation score to the user, means for storing the image data and the evaluation score in a database, means for generating ranking information based on the stored data, and means for disclosing the ranking information to users so that they can check their ranking position. This enables emotions to be analyzed in real time when viewing an advertisement and provides effective feedback. Furthermore, it becomes possible to generate advertisement evaluations and ranking information based on emotion data, thereby enabling more accurate improvement of advertising performance.

[2009] "Image data" is a collection of visual information posted by users from their devices.

[2010] "User" means an individual or group that uses the system or application to post image data and receive evaluation results and feedback.

[2011] The "artificial intelligence module" is a software component for analyzing received image data and generating an evaluation score.

[2012] The "emotion engine" is a software component that analyzes the user's facial expressions and voice data to recognize their emotional state.

[2013] The "evaluation score" is a numerical evaluation value generated by the artificial intelligence module based on the analyzed image data.

[2014] "Feedback" is information provided to users, such as comments and advice based on rating scores and sentiment analysis.

[2015] A "database" is a storage system for storing image data, evaluation scores, and related information.

[2016] "Ranking information" is information obtained by aggregating and ranking evaluation scores from multiple users.

[2017] The "server" is a computer system that controls the entire process, including receiving, analyzing, and storing image data, and sending feedback.

[2018] The present invention relates to a system for evaluating the effectiveness of advertisements in real time. The system analyzes the emotional state of a user when viewing an advertisement using smart glasses and generates an evaluation score. Specific embodiments are described below.

[2019] System Configuration

[2020] The system consists of the following major components:

[2021] 1. User device: Smart glasses

[2022] 2. Server

[2023] 3. Artificial Intelligence Module

[2024] 4. Emotion Engine

[2025] 5. Database

[2026] 6. Feedback Generation Module

[2027] 7. Ranking Generation Module

[2028] Hardware and Software Use

[2029] Hardware

[2030] Smart glasses: Use built-in cameras and microphones to collect facial expression and voice data from the user.

[2031] Server: Processes and stores data using high-performance processors and large-capacity storage.

[2032] software

[2033] Python: The main implementation language for programs

[2034] OpenCV: A library for video capture and image processing

[2035] EmotionEngine: A software component (virtual module) for analyzing user emotions

[2036] AdEvaluator: A software component (virtual module) for generating ad evaluation scores.

[2037] FeedbackGenerator: A software component (virtual module) for generating feedback messages.

[2038] Cloud services (e.g., AWS, Google Cloud, Azure): Distributed computing environments for data storage and analysis

[2039] System Operation

[2040] User's device

[2041] When a user wears the smart glasses and watches an advertisement, the built-in camera and microphone collect facial expression and voice data, which is then transmitted to a server in real time.

[2042] server

[2043] The server receives and temporarily stores the image and audio data sent from the user's device. The received data is processed as follows:

[2044] 1. Emotion analysis: The server uses the Emotion Engine to analyze the user's emotions from the image data. Specifically, it recognizes emotions such as "happiness," "surprise," and "sadness" from facial expressions and voice tone.

[2045] 2. Generating an ad rating score: The server uses AdEvaluator to generate a rating score for the received ad data. This rating score is calculated based on the visual elements of the ad, its content, and the user's viewing time.

[2046] 3. Rating Score Adjustment: The server adjusts the generated rating score based on the user's recognized emotional state. For example, if the user has the emotion "joy," the server may set the rating score slightly higher.

[2047] 4. Feedback Generation: The server uses the FeedbackGenerator to generate a feedback message based on the adjusted rating score. The feedback message includes specific advice and rating results to show to the user.

[2048] Data storage and ranking information generation

[2049] The server stores the generated evaluation scores and related data in a database. Based on the stored data, a ranking generation module generates ranking information, allowing users to check their own evaluation scores and ranking positions.

[2050] Specific examples

[2051] As a specific example, data processing when viewing an advertisement will be shown.

[2052] 1. A user puts on the smart glasses and watches an advertisement.

[2053] 2. The camera and microphone in the smart glasses collect data and send it to the server.

[2054] 3. The server uses EmotionEngine to perform emotion analysis and recognize the emotion of "joy."

[2055] 4. The server uses the AdEvaluator to generate a rating score for the ad (e.g., 85 points).

[2056] 5. Adjust the rating score slightly (e.g., 88 points) based on the perceived emotional state.

[2057] 6. Use FeedbackGenerator to generate a feedback message such as "This ad scores 88 points."

[2058] 7. The server sends the evaluation score and feedback message to the smart glasses and displays it to the user.

[2059] Prompt Sentence Examples

[2060] "Design a system that uses the camera and microphone on smart glasses to analyze emotions in real time when a user watches an advertisement, and evaluate the effectiveness of the advertisement. Adjust the evaluation score based on the emotion analysis data and generate a feedback message. Design this as a Python application that runs on the smart glasses, and clearly state the necessary modules and cloud services."

[2061] The system of the present invention configured as described above contributes to improving the quality of advertising and engagement by evaluating the effectiveness of advertising in real time and providing feedback based on the user's emotional state.

[2062] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2063] Step 1:

[2064] Users wear the smart glasses and watch advertisements, while the built-in camera and microphone collect facial expression and voice data.

[2065] Specifically, the smart glasses' camera captures the user's face frame by frame and simultaneously acquires voice data. These data are sent from the device to the server in real time. The input is facial expression and voice data, and the output is the data sent to the server.

[2066] Step 2:

[2067] The server receives the image data and audio data sent from the terminal and temporarily stores them.

[2068] Specifically, the server temporarily stores the received data in a specific folder or memory. The input is the user's facial expression and voice data, and the output is the temporarily stored raw data.

[2069] Step 3:

[2070] The server uses EmotionEngine to analyze the user's emotions from the temporarily stored image data.

[2071] Specifically, the Emotion Engine analyzes facial features and voice tones to recognize emotional states such as "happiness," "surprise," and "sadness." The input is temporarily saved image and voice data, and the output is recognized emotional data.

[2072] Step 4:

[2073] The server uses the AdEvaluator to generate an evaluation score for the advertising data.

[2074] Specifically, AdEvaluator calculates a rating score based on the visual elements of the ad, its content, the user's viewing time, etc. The input is the ad data, and the output is a rating score (e.g., 85 points).

[2075] Step 5:

[2076] The server adjusts the generated evaluation score based on the recognized emotion data.

[2077] Specifically, AdEvaluator adjusts the final rating score by taking into account the emotional data obtained from EmotionEngine. For example, if the emotion is "joy," the rating score is adjusted from 85 to 88. The input is the rating score and emotional data, and the output is the adjusted rating score.

[2078] Step 6:

[2079] The server uses a FeedbackGenerator to generate feedback messages based on the adjusted reputation scores.

[2080] Specifically, the FeedbackGenerator generates a feedback message containing the evaluation result and advice to be shown to the user based on the adjusted evaluation score. The input is the adjusted evaluation score, and the output is the feedback message.

[2081] Step 7:

[2082] The server sends the generated feedback message to the user's terminal and displays it on the smart glasses.

[2083] Specifically, the server sends the generated feedback message to the user's smart glasses in real time and visually displays it. The input is the feedback message, and the output is the display on the smart glasses.

[2084] Step 8:

[2085] The server stores the generated evaluation scores and related data in a database.

[2086] Specifically, the server stores the evaluation scores, sentiment analysis data, and feedback messages in a database. The input is the evaluation scores and related data, and the output is the saved data.

[2087] Step 9:

[2088] The server generates ranking information based on the stored data and allows users to check their ranking position.

[2089] Specifically, the ranking generation module analyzes the data of all users and generates a ranking of their evaluation scores. Users can view the ranking information through smart glasses or other devices. The input is the stored data, and the output is the generated ranking information.

[2090] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2091] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2092] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2093] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2094] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2095] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2096] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2097] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2098] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2099] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2100] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2101] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2102] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2103] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2104] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2105] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2106] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2107] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2108] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2109] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2110] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2111] The following is further disclosed regarding the above embodiment.

[2112] (Claim 1)

[2113] means for receiving image data from a user;

[2114] an artificial intelligence module means for analyzing the received image data and generating an evaluation score;

[2115] a means for feeding back the evaluation score to the user;

[2116] means for storing the image data and evaluation scores in a database;

[2117] means for generating ranking information based on the stored data;

[2118] A system including:

[2119] (Claim 2)

[2120] The system according to claim 1, further comprising means for using the color, brightness, composition, and degree of match with tags of the image as evaluation criteria when generating the evaluation score.

[2121] (Claim 3)

[2122] The system according to claim 1, further comprising means for disclosing said ranking information to users, allowing users to confirm their own ranking positions.

[2123] "Example 1"

[2124] (Claim 1)

[2125] means for receiving image data from a user;

[2126] means for compressing the received image data and transmitting it to a server;

[2127] means for temporarily storing the received image data;

[2128] an artificial intelligence module means for analyzing the received image data and generating an evaluation score;

[2129] a means for feeding back the evaluation score to the user;

[2130] means for storing the image data and evaluation scores in a database;

[2131] means for trend analysis and algorithm refinement based on said stored data;

[2132] means for generating ranking information based on the stored data;

[2133] A system including:

[2134] (Claim 2)

[2135] The system according to claim 1, further comprising means for using the color, brightness, composition, and degree of match with tags of the image as evaluation criteria when generating the evaluation score.

[2136] (Claim 3)

[2137] The system according to claim 1, further comprising means for disclosing said ranking information to users, allowing users to confirm their own ranking positions.

[2138] "Application Example 1"

[2139] (Claim 1)

[2140] means for receiving image data from a user;

[2141] an artificial intelligence module means for analyzing the received image data and generating an evaluation score;

[2142] a means for providing feedback of the evaluation score to the user in real time;

[2143] means for generating a feedback message based on the evaluation score;

[2144] means for storing the image data and evaluation scores in a database;

[2145] means for generating ranking information based on the stored data;

[2146] a means for disclosing the ranking information to users, allowing users to check their own ranking positions;

[2147] A system including:

[2148] (Claim 2)

[2149] The system according to claim 1, further comprising means for using the color, brightness, composition, and degree of match with tags of the image as evaluation criteria when generating the evaluation score.

[2150] (Claim 3)

[2151] 10. The system of claim 1, further comprising: means for providing a message including specific improvements based on evaluation criteria when generating the feedback message.

[2152] "Example 2: Combining Emotion Engines"

[2153] (Claim 1)

[2154] means for receiving image data from a user;

[2155] an artificial intelligence module means for analyzing the received image data and generating an evaluation score;

[2156] a means for feeding back the evaluation score to the user;

[2157] means for storing the image data and evaluation scores in a database;

[2158] means for generating ranking information based on the stored data;

[2159] an emotion engine means for collecting and recognizing an emotional state of a user;

[2160] means for adjusting an evaluation score based on said emotional state;

[2161] A system including:

[2162] (Claim 2)

[2163] The system according to claim 1, further comprising means for using the color, brightness, composition, and degree of match with tags of the image as evaluation criteria when generating the evaluation score.

[2164] (Claim 3)

[2165] The system according to claim 1, further comprising means for disclosing said ranking information to users, allowing users to confirm their own ranking positions.

[2166] "Applicatio...

Claims

1. means for receiving image data from a user; an artificial intelligence module means for analyzing the received image data and generating an evaluation score; a means for feeding back the evaluation score to the user; means for storing the image data and evaluation scores in a database; means for generating ranking information based on the stored data; A system including:

2. The system according to claim 1 , further comprising means for using the color, brightness, composition, and degree of match with tags of the image as evaluation criteria when generating the evaluation score.

3. The system according to claim 1 , further comprising means for disclosing said ranking information to users, allowing users to confirm their own ranking positions.

Images