System

Abstract

A system according to an embodiment aims to improve the viewing experience by providing a detailed explanation of a work of art.SOLUTION: A system according to an embodiment includes an imaging unit, an identification unit, and an explanation generation unit. The photographing unit photographs an artwork. The specification part specifies the artwork from the image photographed by the photographing part. The explanation generation unit generates a detailed explanation about the artwork specified by the specification unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] With conventional technology, specialized knowledge is required to obtain detailed explanations of artworks, making them difficult for the average viewer to understand.

[0005] The system according to the embodiment aims to enhance the viewing experience by providing detailed descriptions of artworks. [Means for solving the problem]

[0006] The system according to the embodiment includes a photographing unit, an identifying unit, and an explanation generating unit. The photographing unit photographs a work of art. The identifying unit identifies the work of art from the image photographed by the photographing unit. The explanation generating unit generates a detailed explanation about the work of art identified by the identifying unit. [Effects of the Invention]

[0007] The system according to the embodiment can enhance the viewing experience by providing detailed descriptions of artworks. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE INVENTION

[0009] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), and Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (for example, a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., audio and / or text). Display 40A displays visible information such as text and images in accordance with instructions from processor 46. Speaker 40B outputs audio in accordance with instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device owned by a user (e.g., a mobile phone, a robot, a home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) The art appreciation system according to the embodiment of the present invention allows a user to photograph a work of art and a generating AI provides a detailed commentary about the work, thereby enabling the user to gain a deeper understanding of the work of art and improving the appreciation experience.

[0029] An art appreciation system according to an embodiment includes a photographing unit, an identification unit, and a commentary generation unit. The photographing unit photographs artworks. For example, the photographing unit photographs artworks using a smartphone camera. The photographing unit can also capture high-resolution images using a digital camera. The photographing unit can also correct the captured images using image processing technology. For example, the smartphone camera is used when a user photographs artworks at an art museum. The digital camera is used to capture high-resolution images. Image processing technology is used to adjust the brightness and contrast of the captured images. The identification unit identifies artworks from images photographed by the photographing unit. For example, the identification unit identifies artworks using an image recognition algorithm. The identification unit can also identify artworks by comparing them with a database on the cloud. The identification unit can also identify information such as the title, artist, and year of production of the artwork. For example, the image recognition algorithm analyzes the photographed image to identify the artwork. The database on the cloud stores information about artworks, and the identification unit identifies the artwork by comparing it with the database. The identification unit obtains information such as the title, artist, and year of production of the artwork from the database. The commentary generation unit generates a detailed commentary about the artwork identified by the identification unit. For example, the commentary generation unit generates the commentary using a generation AI (e.g., GPT-3). The commentary generation unit can also generate a commentary that includes the artwork's historical background, technique, and artist's intention. The commentary generation unit can also generate a commentary based on a user's prompt. For example, the generation AI has learned a large amount of text data and has advanced natural language processing capabilities. The commentary generation unit generates a detailed commentary that includes the artwork's historical background, technique, and artist's intention. The user's prompt instructs the generation AI on the content of the commentary, and is input in the form of, for example, "Please tell me the historical background of this painting." This allows the art appreciation system according to the embodiment to provide a deeper understanding of the artwork and improve the appreciation experience. For example, a user can take a photo of an artwork using a smartphone and obtain a detailed commentary about the artwork.Users can also learn about the historical background, techniques, and artist's intentions of the artwork through the explanations provided by the generative AI. Furthermore, by introducing this system, museums and galleries can provide visitors with a more fulfilling viewing experience.

[0030] The photographing unit can simultaneously collect information about the surrounding environment using the smartphone's sensors, which can be used to help the identification unit identify the artwork. For example, the photographing unit can use the smartphone's optical sensor to measure the light intensity at the time of photographing and use the data to identify the artwork. For example, it can compare artworks photographed under specific lighting conditions with a database. The photographing unit can also use the smartphone's microphone to collect ambient sounds at the time of photographing and use the sound data to help identify the artwork. For example, it can identify artworks based on a specific acoustic environment within a museum. The photographing unit can also use the smartphone's acceleration sensor to detect camera shake and movement during photographing and use the data to identify the artwork. For example, it can identify artworks based on specific movement patterns. This allows the use of surrounding environmental information to improve the accuracy of artwork identification.

[0031] The identification unit can use the user's location information to identify artworks in cooperation with a database of a specific museum or gallery. The identification unit, for example, uses the GPS function of a smartphone to acquire the user's location information and, based on that location information, identifies artworks in cooperation with a database of a specific museum or gallery. For example, the identification unit compares artworks photographed within a specific museum with the database. The identification unit also identifies a specific exhibition area of ​​a museum or gallery based on the user's location information and identifies artworks in cooperation with a database of artworks within that area. For example, the identification unit compares artworks photographed within a specific exhibition area with the database. The identification unit also uses the location information to record a history of museums or galleries visited by the user and identifies artworks based on that history. For example, the identification unit identifies artworks in cooperation with a database of museums visited in the past. In this way, the user's location information can be used to identify artworks in cooperation with a database of a specific museum or gallery.

[0032] The identification unit can not only identify artworks, but also automatically evaluate the artwork's state of preservation and degree of deterioration. For example, the identification unit analyzes captured images and automatically evaluates the degree of deterioration, such as cracks and discoloration on the surface of the artwork. For example, it uses image processing technology to identify deteriorated areas and evaluate their condition. The identification unit also analyzes the light reflection and shadow at the time of capture to evaluate the preservation state of the artwork. For example, it evaluates the surface condition of the artwork based on the degree of light reflection. The identification unit also compares captured images with past images to evaluate the progress of deterioration. For example, it compares past image data with current image data to identify the progress of deterioration. This makes it possible to automatically evaluate the preservation state and degree of deterioration of the artwork.

[0033] The specifying unit can convert captured images into 3D models, allowing users to view artworks from various angles. For example, the specifying unit can generate 3D models based on captured images, allowing users to view artworks from various angles on their smartphones. For example, the specifying unit can provide a function for rotating and viewing the 3D model. The specifying unit can also capture multiple images and integrate them to generate a highly accurate 3D model. For example, the specifying unit can create a 3D model based on images captured from different angles. The specifying unit can also store the generated 3D model on the cloud, allowing it to be shared with other users. For example, the specifying unit can provide a function for publishing the 3D model as an online gallery, allowing users to view artworks from various angles.

[0034] The commentary generation unit uses a generation AI to automatically generate an audio guide and background music for a work, providing a more immersive viewing experience. For example, the commentary generation unit uses a generation AI to automatically generate an audio guide to match the commentary for the work, allowing the user to listen to the commentary aloud. For example, the history and techniques of the work are explained aloud. The commentary generation unit also uses a generation AI to automatically generate background music to match the work, making the viewing experience more immersive. For example, music that matches the theme of the work is played. The commentary generation unit also combines the audio guide and background music to provide a more integrated experience when the user views the work. For example, the audio guide and music are adjusted to synchronize. This allows the automatic generation of an audio guide and background music for the work, providing a more immersive viewing experience.

[0035] The commentary generation unit can add information about other works and artists related to the work as links to the commentary, allowing users to easily search for related information. For example, the commentary generation unit adds information about other works related to the work as links to the commentary provided by the generation AI. For example, it provides links to other works by the same artist. The commentary generation unit also adds information about artists related to the work as links to the commentary, allowing users to easily search for the artist's background and other works. For example, it provides links to the artist's official website. The commentary generation unit also adds information about exhibitions and events related to the work as links, allowing users to easily search for related information. For example, it provides links to detailed information about related exhibitions. This allows users to easily search for related information.

[0036] The explanation generation unit automatically generates explanations in multiple languages, making it possible to accommodate international users. The explanation generation unit, for example, automatically generates explanations provided by the generation AI in multiple languages, making it possible to accommodate international users. For example, explanations are provided in languages ​​such as English, French, and Chinese. The explanation generation unit also provides a function to automatically translate the explanations and display them in a language selected by the user. For example, the explanations are displayed in the language set by the user on their smartphone. The explanation generation unit also provides a function that allows the user to compare explanations in different languages ​​based on the explanations provided in multiple languages. For example, the same explanation is displayed in different languages. This makes it possible to accommodate international users.

[0037] The commentary generation unit can add an interactive function that allows users to add their own impressions and thoughts to the commentary. The commentary generation unit provides, for example, an interactive function that allows users to add their own impressions and thoughts to the commentary provided by the generation AI. For example, it provides a function that allows users to add comments to the commentary. The commentary generation unit also provides a function that allows users to post questions and opinions about the commentary. For example, it collects feedback on the commentary in real time. The commentary generation unit also provides a function that allows users to discuss the commentary with other users. For example, it provides a discussion forum for the commentary. This makes it possible to provide an interactive function that allows users to add their own impressions and thoughts.

[0038] The explanation generation unit can incorporate interactive quizzes and assignments into the explanations provided by the generation AI to check the student's level of understanding. For example, the explanation generation unit can incorporate interactive quizzes into the explanations provided by the generation AI, providing a function that allows students to check their level of understanding after reading the explanation. For example, it can provide a multiple-choice quiz based on the explanation. The explanation generation unit can also incorporate assignments into the explanation, providing a function that allows students to check their level of understanding by solving the assignments after reading the explanation. For example, it can provide an essay assignment based on the explanation. The explanation generation unit can also analyze the results of quizzes and assignments in real time and build a system to evaluate the student's level of understanding. For example, it can evaluate the student's level of understanding based on the rate of correct answers to the quiz. This makes it possible to provide interactive quizzes and assignments to check the student's level of understanding.

[0039] When used as teaching materials, the explanation generation unit allows the generation AI to provide customized explanations according to the student's learning progress. For example, the explanation generation unit allows the generation AI to analyze the student's learning progress in real time and provide customized explanations according to that progress. For example, separate explanations for beginners and advanced learners are provided. The explanation generation unit also provides individually customized explanations based on the student's past learning history. For example, it provides explanations that review content that has been learned in the past. The explanation generation unit also builds a system in which the generation AI adjusts the difficulty of the explanations according to the student's learning progress. For example, it provides more detailed explanations for students with a high level of understanding. This makes it possible to provide customized explanations according to the student's learning progress.

[0040] The commentary generation unit can expand the commentary function of the generative AI so that it can be applied not only to art education but also to other subjects (history, literature, etc.). For example, the commentary generation unit expands the commentary function of the generative AI so that it can be applied not only to art education but also to other subjects such as history and literature. For example, it provides commentary on historical events and literary works. The commentary generation unit also builds a system that customizes the content of the commentary provided by the generative AI for application to other subjects. For example, it generates commentary to be used in history classes. The commentary generation unit also expands the commentary function of the generative AI and develops a system that automatically generates teaching materials for multiple subjects. For example, it provides teaching materials that combine art and history. This makes it possible to expand the commentary function so that it can be applied not only to art education but also to other subjects.

[0041] The commentary generation unit can add a function that enables students to create original teaching materials using works they have photographed themselves. The commentary generation unit provides a function in which a generative AI creates original teaching materials based on, for example, artworks that students have photographed themselves. For example, it automatically generates commentaries about the photographed works. The commentary generation unit also builds a system in which works photographed by students are uploaded to the cloud and the generative AI creates teaching materials based on that data. For example, it generates quizzes and assignments based on the uploaded works. The commentary generation unit also provides a function in which a generative AI creates interactive teaching materials using works photographed by students. For example, it provides a discussion forum for the works. This makes it possible to provide a function in which students can create original teaching materials using works photographed by themselves.

[0042] The commentary generation unit can automatically add information about museum and gallery events and special exhibitions to the commentary provided by the generation AI. The commentary generation unit, for example, provides a function to automatically add information about museum and gallery events to the commentary provided by the generation AI. For example, it incorporates information about specific exhibitions or workshops into the commentary. The commentary generation unit also provides a function to automatically add information about special exhibitions to the commentary to serve as a reference for users when they visit. For example, it displays the period and location of special exhibitions in the commentary. The commentary generation unit also builds a system that updates event information for museums and galleries in real time and allows the generation AI to reflect that information in the commentary. For example, it incorporates newly added event information into the commentary. This makes it possible to automatically add information about museum and gallery events and special exhibitions.

[0043] The commentary generation unit can propose a personalized visit plan to the user based on the visit history of the museum or gallery. The commentary generation unit, for example, analyzes the user's visit history to museums or galleries, and builds a system that proposes a personalized visit plan based on that data. For example, it proposes a new exhibition based on previously visited exhibitions. The commentary generation unit also generates a visit plan tailored to the user's interests and concerns based on the visit history. For example, it proposes exhibitions related to a specific artist or theme. The commentary generation unit also updates the user's visit history in real time, and develops a system that provides a personalized visit plan based on that data. For example, it proposes the next visit plan based on a newly visited exhibition. This makes it possible to propose a personalized visit plan to the user.

[0044] The commentary generation unit uses a generation AI to automatically generate online tours of museums and galleries, allowing users in remote locations to enjoy a viewing experience. The commentary generation unit builds a system in which a generation AI automatically generates online tours of museums and galleries, allowing users in remote locations to enjoy a viewing experience. For example, it provides virtual tours. The commentary generation unit also automatically generates online tours, allowing users to experience visiting museums and galleries on their smartphones or computers. For example, it provides 360-degree virtual tours. The commentary generation unit also develops a system in which a generation AI updates the online tour in real time to reflect new exhibit and event information. For example, it incorporates newly added exhibits into the online tour. This allows users in remote locations to enjoy a viewing experience.

[0045] The commentary generation unit incorporates product information from cafes and shops at museums and galleries into the commentary, thereby increasing purchasing motivation. The commentary generation unit, for example, provides a function that automatically incorporates product information from cafes and shops at museums and galleries into the commentary provided by the generation AI. For example, product information related to a specific exhibition is displayed in the commentary. The commentary generation unit also provides a function that incorporates product information from cafes and shops into the commentary to serve as reference for users when they visit. For example, information about books and goods related to the exhibition is displayed in the commentary. The commentary generation unit also builds a system that updates product information from cafes and shops at museums and galleries in real time, allowing the generation AI to reflect that information in the commentary. For example, newly added product information is incorporated into the commentary. This allows product information from cafes and shops at museums and galleries to be incorporated into the commentary, increasing purchasing motivation.

[0046] The commentary generation unit can add personalized recommended works based on the user's past viewing history. For example, the commentary generation unit builds a system in which a generation AI analyzes the user's past viewing history and adds personalized recommended works to the commentary based on that data. For example, it provides recommended works related to works previously viewed. The commentary generation unit also generates recommended works based on the user's viewing history, tailored to the user's interests and concerns. For example, it recommends works related to a specific artist or theme. The commentary generation unit also updates the user's viewing history in real time, and develops a system that provides personalized recommended works based on that data. For example, it suggests the next recommended work based on a newly viewed work. This makes it possible to provide personalized recommended works based on the user's past viewing history.

[0047] The commentary generation unit can incorporate the work's production process and interview videos of the artist into the commentary to promote deeper understanding. For example, the commentary generation unit provides a function to incorporate videos showing the work's production process into the commentary provided by the generation AI. For example, it adds footage of the artist creating the work to the commentary. The commentary generation unit also incorporates interview videos of the artist into the commentary to allow users to understand the artist's intentions and background. For example, it adds videos of the artist talking about the work to the commentary. The commentary generation unit also builds a system that updates the work's production process and interview videos of the artist in real time, and the generation AI reflects that information in the commentary. For example, it incorporates newly added interview videos into the commentary. This allows the work's production process and interview videos of the artist to be incorporated to promote deeper understanding.

[0048] The commentary generation unit can use AR technology to overlay the commentary provided by the generation AI on top of the artwork. The commentary generation unit, for example, builds a system that uses AR technology to overlay the commentary provided by the generation AI on top of the artwork. For example, the commentary is overlaid on the artwork using a smartphone camera. The commentary generation unit also provides a function that uses AR technology to overlay the commentary on specific parts of the artwork. For example, the commentary is displayed on important parts of the artwork. The commentary generation unit also develops a system that updates the commentary provided by the generation AI in real time using AR technology, and displays the latest commentary as the user views the artwork. For example, newly added commentary is displayed in real time. This makes it possible to overlay the commentary on top of the artwork using AR technology.

[0049] The commentary generation unit can add a social function that allows users to share their impressions with other viewers in real time. For example, the commentary generation unit adds a social function to the commentary provided by the generation AI that allows users to share their impressions with other viewers in real time. For example, it provides a function to add comments to the commentary. The commentary generation unit also provides a function that allows users to post their impressions on the commentary in real time. For example, it collects feedback on the commentary in real time. The commentary generation unit also provides a function that allows users to discuss the commentary with other viewers. For example, it provides a discussion forum for the commentary. This makes it possible to provide a social function that allows users to share their impressions with other viewers in real time.

[0050] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0051] The art appreciation system can also be equipped with a user's eye-tracking function. The eye-tracking function detects which parts of the artwork the user is focusing on and customizes the commentary based on that data. For example, if the user focuses on a particular part for a long time, a detailed commentary on that part can be provided. The eye-tracking data can also be used to identify parts of the artwork that the user is interested in and provide other artworks or information related to that part. Furthermore, the eye-tracking data can be analyzed to understand the user's viewing patterns and provide a more personalized viewing experience.

[0052] The art appreciation system can also be equipped with a function that recognizes user voice commands. The voice command function allows the user to request explanations or search for specific information by voice. For example, the user can say, "Tell me about the techniques used in this work," and the system will provide an explanation based on the request. The voice command function also allows the user to search for information about a specific artist or theme. The voice command function also allows the user to play, stop, skip, and perform other operations on the explanation.

[0053] The art appreciation system can also be equipped with a function to detect the user's walking patterns. The walking pattern detection function understands how the user moves around the museum and optimizes the viewing route based on that data. For example, if the user spends a long time in a particular area, other works related to that area can be recommended. The walking pattern data can also be used to notify the user of works that they may have missed. Furthermore, by analyzing walking patterns, the system can suggest a personalized viewing route based on the user's interests.

[0054] The processing flow of the first embodiment will be briefly explained below.

[0055] Step 1: The camera unit takes a photograph of the artwork. For example, the artwork can be photographed using a smartphone camera or a digital camera. The camera unit can also enhance the captured image using image processing technology. For example, a smartphone camera is used when a user photographs artwork in an art museum, and a digital camera is used to capture high-resolution images. Image processing technology is used to adjust the brightness and contrast of the captured image. Step 2: The identification unit identifies the artwork from the image captured by the photography unit. For example, the identification unit can use an image recognition algorithm to identify the artwork and compare it with a database on the cloud to identify the artwork. The identification unit can also identify information such as the title, artist, and year of production of the artwork. The image recognition algorithm analyzes the captured image to identify the artwork, and the database on the cloud stores information about the artwork, and the identification unit compares it with that database to identify the artwork. The identification unit obtains information such as the title, artist, and year of production of the artwork from the database. Step 3: The explanation generation unit generates a detailed explanation about the artwork identified by the identification unit. For example, the explanation generation unit can generate an explanation using a generation AI (e.g., GPT-3) and generate an explanation that includes the work's historical background, techniques, and the artist's intentions. The explanation generation unit can also generate an explanation based on a user prompt. The generation AI has learned from a large amount of text data and has advanced natural language processing capabilities. The explanation generation unit generates a detailed explanation that includes the work's historical background, techniques, and the artist's intentions. The user's prompt instructs the generation AI on the content of the explanation, and is input in the form of, for example, "Please tell me the historical background of this painting."

[0056] (Example 2) The art appreciation system according to the embodiment of the present invention allows a user to photograph a work of art and a generating AI provides a detailed commentary about the work, thereby enabling the user to gain a deeper understanding of the work of art and improving the appreciation experience.

[0057] An art appreciation system according to an embodiment includes a photographing unit, an identification unit, and a commentary generation unit. The photographing unit photographs artworks. For example, the photographing unit photographs artworks using a smartphone camera. The photographing unit can also capture high-resolution images using a digital camera. The photographing unit can also correct the captured images using image processing technology. For example, the smartphone camera is used when a user photographs artworks at an art museum. The digital camera is used to capture high-resolution images. Image processing technology is used to adjust the brightness and contrast of the captured images. The identification unit identifies artworks from images photographed by the photographing unit. For example, the identification unit identifies artworks using an image recognition algorithm. The identification unit can also identify artworks by comparing them with a database on the cloud. The identification unit can also identify information such as the title, artist, and year of production of the artwork. For example, the image recognition algorithm analyzes the photographed image to identify the artwork. The database on the cloud stores information about artworks, and the identification unit identifies the artwork by comparing it with the database. The identification unit obtains information such as the title, artist, and year of production of the artwork from the database. The commentary generation unit generates a detailed commentary about the artwork identified by the identification unit. For example, the commentary generation unit generates the commentary using a generation AI (e.g., GPT-3). The commentary generation unit can also generate a commentary that includes the artwork's historical background, technique, and artist's intention. The commentary generation unit can also generate a commentary based on a user's prompt. For example, the generation AI has learned a large amount of text data and has advanced natural language processing capabilities. The commentary generation unit generates a detailed commentary that includes the artwork's historical background, technique, and artist's intention. The user's prompt instructs the generation AI on the content of the commentary, and is input in the form of, for example, "Please tell me the historical background of this painting." This allows the art appreciation system according to the embodiment to provide a deeper understanding of the artwork and improve the appreciation experience. For example, a user can take a photo of an artwork using a smartphone and obtain a detailed commentary about the artwork.Users can also learn about the historical background, techniques, and artist's intentions of the artwork through the explanations provided by the generative AI. Furthermore, by introducing this system, museums and galleries can provide visitors with a more fulfilling viewing experience.

[0058] The photographing unit can simultaneously collect information about the surrounding environment using the smartphone's sensors, which can be used to help the identification unit identify the artwork. For example, the photographing unit can use the smartphone's optical sensor to measure the light intensity at the time of photographing and use the data to identify the artwork. For example, it can compare artworks photographed under specific lighting conditions with a database. The photographing unit can also use the smartphone's microphone to collect ambient sounds at the time of photographing and use the sound data to help identify the artwork. For example, it can identify artworks based on a specific acoustic environment within a museum. The photographing unit can also use the smartphone's acceleration sensor to detect camera shake and movement during photographing and use the data to identify the artwork. For example, it can identify artworks based on specific movement patterns. This allows the use of surrounding environmental information to improve the accuracy of artwork identification.

[0059] The identification unit can use the user's location information to identify artworks in cooperation with a database of a specific museum or gallery. The identification unit, for example, uses the GPS function of a smartphone to acquire the user's location information and, based on that location information, identifies artworks in cooperation with a database of a specific museum or gallery. For example, the identification unit compares artworks photographed within a specific museum with the database. The identification unit also identifies a specific exhibition area of ​​a museum or gallery based on the user's location information and identifies artworks in cooperation with a database of artworks within that area. For example, the identification unit compares artworks photographed within a specific exhibition area with the database. The identification unit also uses the location information to record a history of museums or galleries visited by the user and identifies artworks based on that history. For example, the identification unit identifies artworks in cooperation with a database of museums visited in the past. In this way, the user's location information can be used to identify artworks in cooperation with a database of a specific museum or gallery.

[0060] The identification unit uses the emotion estimation function to analyze the emotion the user felt when taking a photo and can improve the accuracy of identifying a work based on that emotion. The identification unit, for example, analyzes the user's facial expression when taking a photo and improves the accuracy of identifying a work based on that emotion data. For example, it prioritizes identifying works that the user felt surprised or moved by. The identification unit also analyzes audio data taken at the time of taking a photo and analyzes the user's tone of voice and emotion to help identify the work. For example, it prioritizes identifying specific works based on an excited tone of voice. The identification unit also measures the user's heart rate and electrodermal response when taking a photo and estimates the emotion based on that data to improve the accuracy of identifying a work. For example, it prioritizes identifying works that were taken when the heart rate was elevated. This makes it possible to analyze the user's emotions and improve the accuracy of identifying a work.

[0061] The identification unit can not only identify artworks, but also automatically evaluate the artwork's state of preservation and degree of deterioration. For example, the identification unit analyzes captured images and automatically evaluates the degree of deterioration, such as cracks and discoloration on the surface of the artwork. For example, it uses image processing technology to identify deteriorated areas and evaluate their condition. The identification unit also analyzes the light reflection and shadow at the time of capture to evaluate the preservation state of the artwork. For example, it evaluates the surface condition of the artwork based on the degree of light reflection. The identification unit also compares captured images with past images to evaluate the progress of deterioration. For example, it compares past image data with current image data to identify the progress of deterioration. This makes it possible to automatically evaluate the preservation state and degree of deterioration of the artwork.

[0062] The specifying unit can convert captured images into 3D models, allowing users to view artworks from various angles. For example, the specifying unit can generate 3D models based on captured images, allowing users to view artworks from various angles on their smartphones. For example, the specifying unit can provide a function for rotating and viewing the 3D model. The specifying unit can also capture multiple images and integrate them to generate a highly accurate 3D model. For example, the specifying unit can create a 3D model based on images captured from different angles. The specifying unit can also store the generated 3D model on the cloud, allowing it to be shared with other users. For example, the specifying unit can provide a function for publishing the 3D model as an online gallery, allowing users to view artworks from various angles.

[0063] The identification unit uses the emotion estimation function to display the user's emotions toward a photographed work in real time, allowing the user to share the emotions with other users. The identification unit, for example, adds a function to analyze the user's emotions in real time when photographing and display the emotion data. For example, it displays an emotion score for a work that moved the user. The identification unit also provides a function to share the emotion data toward the photographed work with other users. For example, it adds a function to share the emotion score on a social networking site. The identification unit also provides a function to compare and display the emotions that other users felt toward the same work based on the emotion data. For example, it displays the emotion scores of multiple users in a graph. This allows the user to display the emotions toward the photographed work in real time and share the emotions with other users.

[0064] The commentary generation unit uses a generation AI to automatically generate an audio guide and background music for a work, providing a more immersive viewing experience. For example, the commentary generation unit uses a generation AI to automatically generate an audio guide to match the commentary for the work, allowing the user to listen to the commentary aloud. For example, the history and techniques of the work are explained aloud. The commentary generation unit also uses a generation AI to automatically generate background music to match the work, making the viewing experience more immersive. For example, music that matches the theme of the work is played. The commentary generation unit also combines the audio guide and background music to provide a more integrated experience when the user views the work. For example, the audio guide and music are adjusted to synchronize. This allows the automatic generation of an audio guide and background music for the work, providing a more immersive viewing experience.

[0065] The commentary generation unit can add information about other works and artists related to the work as links to the commentary, allowing users to easily search for related information. For example, the commentary generation unit adds information about other works related to the work as links to the commentary provided by the generation AI. For example, it provides links to other works by the same artist. The commentary generation unit also adds information about artists related to the work as links to the commentary, allowing users to easily search for the artist's background and other works. For example, it provides links to the artist's official website. The commentary generation unit also adds information about exhibitions and events related to the work as links, allowing users to easily search for related information. For example, it provides links to detailed information about related exhibitions. This allows users to easily search for related information.

[0066] The commentary generation unit can customize the tone and content of the commentary according to the user's emotions using the emotion estimation function. The commentary generation unit, for example, uses the emotion estimation function to customize the tone of the commentary according to the user's emotions. For example, if the user is excited, the commentary is provided in a more energetic tone. The commentary generation unit also customizes the content of the commentary based on the user's emotions. For example, if the user is moved, the commentary generation unit emphasizes an inspiring episode in the work. The commentary generation unit also provides commentary in real time that matches the user's emotions based on the emotion estimation data. For example, the tone and content of the commentary are adjusted each time the user's emotions change. This makes it possible to customize the tone and content of the commentary according to the user's emotions.

[0067] The explanation generation unit automatically generates explanations in multiple languages, making it possible to accommodate international users. The explanation generation unit, for example, automatically generates explanations provided by the generation AI in multiple languages, making it possible to accommodate international users. For example, explanations are provided in languages ​​such as English, French, and Chinese. The explanation generation unit also provides a function to automatically translate the explanations and display them in a language selected by the user. For example, the explanations are displayed in the language set by the user on their smartphone. The explanation generation unit also provides a function that allows the user to compare explanations in different languages ​​based on the explanations provided in multiple languages. For example, the same explanation is displayed in different languages. This makes it possible to accommodate international users.

[0068] The commentary generation unit can add an interactive function that allows users to add their own impressions and thoughts to the commentary. The commentary generation unit provides, for example, an interactive function that allows users to add their own impressions and thoughts to the commentary provided by the generation AI. For example, it provides a function that allows users to add comments to the commentary. The commentary generation unit also provides a function that allows users to post questions and opinions about the commentary. For example, it collects feedback on the commentary in real time. The commentary generation unit also provides a function that allows users to discuss the commentary with other users. For example, it provides a discussion forum for the commentary. This makes it possible to provide an interactive function that allows users to add their own impressions and thoughts.

[0069] The commentary generation unit uses the emotion estimation function to analyze the emotions a user feels after reading a commentary and can improve the commentary content based on that feedback. For example, the commentary generation unit uses the emotion estimation function to analyze the emotions a user feels after reading a commentary in real time and improve the commentary content based on that data. For example, if the user is moved, an inspiring episode is added. The commentary generation unit also identifies areas for improvement in the commentary based on the user's emotional response data, and the generation AI automatically corrects the commentary content. For example, it corrects parts that receive a lot of negative responses. The commentary generation unit also builds a system that provides commentary tailored to the user's emotions based on the emotion estimation data. For example, it adjusts the commentary content each time the user's emotions change. This makes it possible to analyze the emotions a user feels after reading a commentary and improve the commentary content based on that feedback.

[0070] The explanation generation unit can incorporate interactive quizzes and assignments into the explanations provided by the generation AI to check the student's level of understanding. For example, the explanation generation unit can incorporate interactive quizzes into the explanations provided by the generation AI, providing a function that allows students to check their level of understanding after reading the explanation. For example, it can provide a multiple-choice quiz based on the explanation. The explanation generation unit can also incorporate assignments into the explanation, providing a function that allows students to check their level of understanding by solving the assignments after reading the explanation. For example, it can provide an essay assignment based on the explanation. The explanation generation unit can also analyze the results of quizzes and assignments in real time and build a system to evaluate the student's level of understanding. For example, it can evaluate the student's level of understanding based on the rate of correct answers to the quiz. This makes it possible to provide interactive quizzes and assignments to check the student's level of understanding.

[0071] When used as teaching materials, the explanation generation unit allows the generation AI to provide customized explanations according to the student's learning progress. For example, the explanation generation unit allows the generation AI to analyze the student's learning progress in real time and provide customized explanations according to that progress. For example, separate explanations for beginners and advanced learners are provided. The explanation generation unit also provides individually customized explanations based on the student's past learning history. For example, it provides explanations that review content that has been learned in the past. The explanation generation unit also builds a system in which the generation AI adjusts the difficulty of the explanations according to the student's learning progress. For example, it provides more detailed explanations for students with a high level of understanding. This makes it possible to provide customized explanations according to the student's learning progress.

[0072] The commentary generation unit can use the emotion estimation function to analyze students' emotions while studying and provide feedback to increase their motivation. For example, the commentary generation unit can use the emotion estimation function to analyze students' emotions while studying in real time and provide feedback to increase their motivation based on the data. For example, if a student is tired, it can display an encouraging message. The commentary generation unit can also build a system to adjust learning content and increase motivation based on students' emotional response data. For example, it can provide commentary based on a topic that interests the student. The commentary generation unit can also analyze students' emotions while studying based on the emotion estimation data and provide specific advice to increase their motivation based on the results. For example, it can display words of praise according to their learning progress. In this way, it is possible to analyze students' emotions while studying and provide feedback to increase their motivation.

[0073] The commentary generation unit can expand the commentary function of the generative AI so that it can be applied not only to art education but also to other subjects (history, literature, etc.). For example, the commentary generation unit expands the commentary function of the generative AI so that it can be applied not only to art education but also to other subjects such as history and literature. For example, it provides commentary on historical events and literary works. The commentary generation unit also builds a system that customizes the content of the commentary provided by the generative AI for application to other subjects. For example, it generates commentary to be used in history classes. The commentary generation unit also expands the commentary function of the generative AI and develops a system that automatically generates teaching materials for multiple subjects. For example, it provides teaching materials that combine art and history. This makes it possible to expand the commentary function so that it can be applied not only to art education but also to other subjects.

[0074] The commentary generation unit can add a function that enables students to create original teaching materials using works they have photographed themselves. The commentary generation unit provides a function in which a generative AI creates original teaching materials based on, for example, artworks that students have photographed themselves. For example, it automatically generates commentaries about the photographed works. The commentary generation unit also builds a system in which works photographed by students are uploaded to the cloud and the generative AI creates teaching materials based on that data. For example, it generates quizzes and assignments based on the uploaded works. The commentary generation unit also provides a function in which a generative AI creates interactive teaching materials using works photographed by students. For example, it provides a discussion forum for the works. This makes it possible to provide a function in which students can create original teaching materials using works photographed by themselves.

[0075] The commentary generation unit can use the emotion estimation function to identify the theme in which the student is most interested and provide customized teaching materials based on that theme. The commentary generation unit, for example, uses the emotion estimation function to identify the theme in which the student is most interested and builds a system to provide customized teaching materials based on that theme. For example, it provides commentary on artists in which the student is interested. The commentary generation unit also identifies the theme of interest based on the student's emotional response data and generates teaching materials based on that theme. For example, it provides detailed commentary on works that have moved the student. The commentary generation unit also identifies the theme in which the student is most interested based on the emotion estimation data and provides interactive teaching materials based on that theme. For example, it provides quizzes and assignments related to the theme of interest. In this way, it is possible to identify the theme in which the student is most interested and provide customized teaching materials based on that theme.

[0076] The commentary generation unit can automatically add information about museum and gallery events and special exhibitions to the commentary provided by the generation AI. The commentary generation unit, for example, provides a function to automatically add information about museum and gallery events to the commentary provided by the generation AI. For example, it incorporates information about specific exhibitions or workshops into the commentary. The commentary generation unit also provides a function to automatically add information about special exhibitions to the commentary to serve as a reference for users when they visit. For example, it displays the period and location of special exhibitions in the commentary. The commentary generation unit also builds a system that updates event information for museums and galleries in real time and allows the generation AI to reflect that information in the commentary. For example, it incorporates newly added event information into the commentary. This makes it possible to automatically add information about museum and gallery events and special exhibitions.

[0077] The commentary generation unit can propose a personalized visit plan to the user based on the visit history of the museum or gallery. The commentary generation unit, for example, analyzes the user's visit history to museums or galleries, and builds a system that proposes a personalized visit plan based on that data. For example, it proposes a new exhibition based on previously visited exhibitions. The commentary generation unit also generates a visit plan tailored to the user's interests and concerns based on the visit history. For example, it proposes exhibitions related to a specific artist or theme. The commentary generation unit also updates the user's visit history in real time, and develops a system that provides a personalized visit plan based on that data. For example, it proposes the next visit plan based on a newly visited exhibition. This makes it possible to propose a personalized visit plan to the user.

[0078] The commentary generation unit can use the emotion estimation function to provide special offers and discounts based on the user's emotions. The commentary generation unit, for example, uses the emotion estimation function to build a system that provides special offers and discounts based on the user's emotions. For example, a special discount is provided for a work that moves the user. The commentary generation unit also customizes special offers and discounts based on the user's emotional response data. For example, if the user is excited, a discount is provided for the next visit. The commentary generation unit also develops a system that provides special offers and discounts tailored to the user's emotions in real time based on the emotion estimation data. For example, the offer is adjusted every time the user's emotions change. This makes it possible to provide special offers and discounts based on the user's emotions.

[0079] The commentary generation unit uses a generation AI to automatically generate online tours of museums and galleries, allowing users in remote locations to enjoy a viewing experience. The commentary generation unit builds a system in which a generation AI automatically generates online tours of museums and galleries, allowing users in remote locations to enjoy a viewing experience. For example, it provides virtual tours. The commentary generation unit also automatically generates online tours, allowing users to experience visiting museums and galleries on their smartphones or computers. For example, it provides 360-degree virtual tours. The commentary generation unit also develops a system in which a generation AI updates the online tour in real time to reflect new exhibit and event information. For example, it incorporates newly added exhibits into the online tour. This allows users in remote locations to enjoy a viewing experience.

[0080] The commentary generation unit incorporates product information from cafes and shops at museums and galleries into the commentary, thereby increasing purchasing motivation. The commentary generation unit, for example, provides a function that automatically incorporates product information from cafes and shops at museums and galleries into the commentary provided by the generation AI. For example, product information related to a specific exhibition is displayed in the commentary. The commentary generation unit also provides a function that incorporates product information from cafes and shops into the commentary to serve as reference for users when they visit. For example, information about books and goods related to the exhibition is displayed in the commentary. The commentary generation unit also builds a system that updates product information from cafes and shops at museums and galleries in real time, allowing the generation AI to reflect that information in the commentary. For example, newly added product information is incorporated into the commentary. This allows product information from cafes and shops at museums and galleries to be incorporated into the commentary, increasing purchasing motivation.

[0081] The commentary generation unit can use the emotion estimation function to identify exhibits and events that interest the user most and provide that information via push notification. The commentary generation unit, for example, uses the emotion estimation function to build a system that identifies exhibits and events that interest the user most and provides that information via push notification. For example, the commentary generation unit notifies the user of information about exhibits that impressed the user. The commentary generation unit also identifies exhibits and events that interest the user based on the user's emotional response data and notifies the user of that information in real time. For example, if the user is excited, the commentary generation unit notifies the user of related event information. The commentary generation unit also develops a system that identifies exhibits and events that interest the user most based on the emotion estimation data and provides the user with that information via push notification. For example, the notification content is adjusted each time the user's emotion changes. This makes it possible to identify exhibits and events that interest the user most and provide the user with that information via push notification.

[0082] The commentary generation unit can add personalized recommended works based on the user's past viewing history. For example, the commentary generation unit builds a system in which a generation AI analyzes the user's past viewing history and adds personalized recommended works to the commentary based on that data. For example, it provides recommended works related to works previously viewed. The commentary generation unit also generates recommended works based on the user's viewing history, tailored to the user's interests and concerns. For example, it recommends works related to a specific artist or theme. The commentary generation unit also updates the user's viewing history in real time, and develops a system that provides personalized recommended works based on that data. For example, it suggests the next recommended work based on a newly viewed work. This makes it possible to provide personalized recommended works based on the user's past viewing history.

[0083] The commentary generation unit can incorporate the work's production process and interview videos of the artist into the commentary to promote deeper understanding. For example, the commentary generation unit provides a function to incorporate videos showing the work's production process into the commentary provided by the generation AI. For example, it adds footage of the artist creating the work to the commentary. The commentary generation unit also incorporates interview videos of the artist into the commentary to allow users to understand the artist's intentions and background. For example, it adds videos of the artist talking about the work to the commentary. The commentary generation unit also builds a system that updates the work's production process and interview videos of the artist in real time, and the generation AI reflects that information in the commentary. For example, it incorporates newly added interview videos into the commentary. This allows the work's production process and interview videos of the artist to be incorporated to promote deeper understanding.

[0084] The commentary generation unit uses the emotion estimation function to suggest a viewing route that corresponds to the user's emotions, thereby providing an optimal viewing experience. The commentary generation unit, for example, uses the emotion estimation function to build a system that suggests a viewing route that corresponds to the user's emotions. For example, a viewing route is set that focuses on works that have moved the user. The commentary generation unit also generates an optimal viewing route based on the user's emotional response data and suggests it to the user. For example, if the user is excited, a viewing route is set that focuses on energetic works. The commentary generation unit also develops a system that provides a viewing route that matches the user's emotions in real time based on the emotion estimation data. For example, the viewing route is adjusted each time the user's emotions change. This makes it possible to suggest a viewing route that corresponds to the user's emotions, thereby providing an optimal viewing experience.

[0085] The commentary generation unit can use AR technology to overlay the commentary provided by the generation AI on top of the artwork. The commentary generation unit, for example, builds a system that uses AR technology to overlay the commentary provided by the generation AI on top of the artwork. For example, the commentary is overlaid on the artwork using a smartphone camera. The commentary generation unit also provides a function that uses AR technology to overlay the commentary on specific parts of the artwork. For example, the commentary is displayed on important parts of the artwork. The commentary generation unit also develops a system that updates the commentary provided by the generation AI in real time using AR technology, and displays the latest commentary as the user views the artwork. For example, newly added commentary is displayed in real time. This makes it possible to overlay the commentary on top of the artwork using AR technology.

[0086] The commentary generation unit can add a social function that allows users to share their impressions with other viewers in real time. For example, the commentary generation unit adds a social function to the commentary provided by the generation AI that allows users to share their impressions with other viewers in real time. For example, it provides a function to add comments to the commentary. The commentary generation unit also provides a function that allows users to post their impressions on the commentary in real time. For example, it collects feedback on the commentary in real time. The commentary generation unit also provides a function that allows users to discuss the commentary with other viewers. For example, it provides a discussion forum for the commentary. This makes it possible to provide a social function that allows users to share their impressions with other viewers in real time.

[0087] The commentary generation unit can use the emotion estimation function to identify the work that moved the user the most and recommend other works related to that work. For example, the commentary generation unit uses the emotion estimation function to build a system that identifies the work that moved the user the most and recommends other works related to that work. For example, it provides other works by an artist related to the moving work. The commentary generation unit also identifies the moving work based on the user's emotional response data and generates recommended works related to that work. For example, it recommends works with the same theme or style. The commentary generation unit also identifies the work that moved the user the most based on the emotion estimation data and provides an interactive commentary related to that work. For example, it provides a detailed commentary on the moving work. This allows the user to identify the work that moved the user the most and recommend other works related to that work.

[0088] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0089] The art appreciation system can also be equipped with a user's eye-tracking function. The eye-tracking function detects which parts of the artwork the user is focusing on and customizes the commentary based on that data. For example, if the user focuses on a particular part for a long time, a detailed commentary on that part can be provided. The eye-tracking data can also be used to identify parts of the artwork that the user is interested in and provide other artworks or information related to that part. Furthermore, the eye-tracking data can be analyzed to understand the user's viewing patterns and provide a more personalized viewing experience.

[0090] The art appreciation system can also be equipped with a function that recognizes user voice commands. The voice command function allows the user to request explanations or search for specific information by voice. For example, the user can say, "Tell me about the techniques used in this work," and the system will provide an explanation based on the request. The voice command function also allows the user to search for information about a specific artist or theme. The voice command function also allows the user to play, stop, skip, and perform other operations on the explanation.

[0091] The art appreciation system can also be equipped with a function to detect the user's walking patterns. The walking pattern detection function understands how the user moves around the museum and optimizes the viewing route based on that data. For example, if the user spends a long time in a particular area, other works related to that area can be recommended. The walking pattern data can also be used to notify the user of works that they may have missed. Furthermore, by analyzing walking patterns, the system can suggest a personalized viewing route based on the user's interests.

[0092] The art appreciation system can further estimate the user's emotions and customize the tone and content of the commentary based on those emotions. For example, if the user is surprised or moved, the commentary tone can be made more energetic to match that emotion. On the other hand, if the user is relaxed, the commentary can be provided in a calmer tone. Furthermore, the content of the commentary can be adjusted based on the user's emotions, emphasizing moving episodes and interesting facts. This makes it possible to provide a personalized commentary that matches the user's emotions.

[0093] The art appreciation system can further estimate the user's emotions and recommend specific artworks based on those emotions. For example, if the user is moved, related inspiring artworks can be recommended to further deepen that emotion. If the user is excited, energetic artworks can be recommended. Furthermore, artworks related to a specific theme or style can be recommended based on the user's emotions. This makes it possible to provide personalized artwork recommendations according to the user's emotions.

[0094] The art appreciation system can further estimate the user's emotions and provide special offers or discounts based on the emotions. For example, if the user is moved, a special discount can be offered to commemorate the emotion. If the user is excited, a discount can be offered on the user's next visit. Furthermore, based on the user's emotions, an invitation to a specific event or workshop can be offered. This makes it possible to provide special offers and discounts according to the user's emotions.

[0095] The art appreciation system can further estimate the user's emotions and suggest an appreciation route based on those emotions. For example, if the user is moved, the system can set an appreciation route centered around related, inspiring works to further deepen that emotion. Alternatively, if the user is excited, the system can set an appreciation route centered around energetic works. Furthermore, based on the user's emotions, the system can suggest an appreciation route centered around works related to a specific theme or style. This makes it possible to provide an optimal appreciation route that matches the user's emotions.

[0096] The art appreciation system can further estimate the user's emotions and customize the tone and content of the commentary based on those emotions. For example, if the user is surprised or moved, the commentary tone can be made more energetic to match that emotion. On the other hand, if the user is relaxed, the commentary can be provided in a calmer tone. Furthermore, the content of the commentary can be adjusted based on the user's emotions, emphasizing moving episodes and interesting facts. This makes it possible to provide a personalized commentary that matches the user's emotions.

[0097] The art appreciation system can further estimate the user's emotions and recommend specific artworks based on those emotions. For example, if the user is moved, related inspiring artworks can be recommended to further deepen that emotion. If the user is excited, energetic artworks can be recommended. Furthermore, artworks related to a specific theme or style can be recommended based on the user's emotions. This makes it possible to provide personalized artwork recommendations according to the user's emotions.

[0098] The art appreciation system can further estimate the user's emotions and provide special offers or discounts based on the emotions. For example, if the user is moved, a special discount can be offered to commemorate the emotion. If the user is excited, a discount can be offered on the user's next visit. Furthermore, based on the user's emotions, an invitation to a specific event or workshop can be offered. This makes it possible to provide special offers and discounts according to the user's emotions.

[0099] The processing flow of the second embodiment will be briefly explained below.

[0100] Step 1: The camera unit takes a photograph of the artwork. For example, the artwork can be photographed using a smartphone camera or a digital camera. The camera unit can also enhance the captured image using image processing technology. For example, a smartphone camera is used when a user photographs artwork in an art museum, and a digital camera is used to capture high-resolution images. Image processing technology is used to adjust the brightness and contrast of the captured image. Step 2: The identification unit identifies the artwork from the image captured by the photography unit. For example, the identification unit can use an image recognition algorithm to identify the artwork and compare it with a database on the cloud to identify the artwork. The identification unit can also identify information such as the title, artist, and year of production of the artwork. The image recognition algorithm analyzes the captured image to identify the artwork, and the database on the cloud stores information about the artwork, and the identification unit compares it with that database to identify the artwork. The identification unit obtains information such as the title, artist, and year of production of the artwork from the database. Step 3: The explanation generation unit generates a detailed explanation about the artwork identified by the identification unit. For example, the explanation generation unit can generate an explanation using a generation AI (e.g., GPT-3) and generate an explanation that includes the work's historical background, techniques, and the artist's intentions. The explanation generation unit can also generate an explanation based on a user prompt. The generation AI has learned from a large amount of text data and has advanced natural language processing capabilities. The explanation generation unit generates a detailed explanation that includes the work's historical background, techniques, and the artist's intentions. The user's prompt instructs the generation AI on the content of the explanation, and is input in the form of, for example, "Please tell me the historical background of this painting."

[0101] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0102] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> Examples of generative AIs include the data generation model 58, such as a neural network model (e.g., a neural network model), and a neural network model (e.g., a neural network model). The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating speech, text data indicating text, and image data indicating an image is also input to the data generation model 58. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0103] Furthermore, the processing by the data processing system 10 described above is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0104] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0105] 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0106] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0107] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0108] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0109] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0110] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0111] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0112] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0113] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0114] In the smart glasses 214, the specific processing is performed by the processor 46. A specific processing program 60 is stored in the storage 50. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0115] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0116] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0117] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0118] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, etc., and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0119] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0120] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0121] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0122] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0123] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0124] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0125] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0126] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0127] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0128] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0129] In the headset type terminal 314, the specific processing is performed by the processor 46. A specific processing program 60 is stored in the storage 50. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0130] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0131] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0132] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0133] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, etc., and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0134] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0135] 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0136] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0137] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0138] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0139] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0140] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0141] The control object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0142] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0143] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0144] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0145] In the robot 414, the specific processing is performed by the processor 46. A specific processing program 60 is stored in the storage 50. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. The robot 414 also has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0146] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0147] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0148] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0149] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0150] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0151] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion encompasses both emotions and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0152] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0153] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0154] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. Emotions can also be created for robots, cars, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on speech emotion recognition and brain physiological signal analysis systems for emotions, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the area called "reaction," where sensation is dominant. The right half of the emotion map lists emotions belonging to the area called "situation," where situational awareness is dominant.

[0155] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[0156] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[0157] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers including computer 22.

[0158] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0159] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0160] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0161] The hardware resource for executing a specific process can be any of the following types of processors: A processor, for example, is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. A processor also includes a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[0162] The hardware resource that executes the specific process may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0163] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0164] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[0165] In the above example, the first to fourth embodiments have been described separately, but some or all of these embodiments may be combined. The smart device 14, smart glasses 214, headset terminal 314, and robot 414 are merely examples, and they may be combined, or other devices may be used. In the above example, the first and second embodiments have been described separately, but they may be combined.

[0166] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0167] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference. [Explanation of symbols]

[0168] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. A photography department that photographs artworks, an identification unit that identifies a work of art from the image captured by the imaging unit; a commentary generating unit that generates a detailed commentary on the artwork identified by the identifying unit. A system characterized by:

2. The imaging unit is The smartphone's sensors are used to simultaneously collect information about the surrounding environment, which is then used by the identification unit to identify the artwork.

2. The system of claim 1.

3. The identification unit Uses user location information to identify artworks by linking with specific museum or gallery databases 2. The system of claim 1.

4. The identification unit Analyze the emotions felt by the user when taking a photo and improve the accuracy of identifying the work based on those emotions.

2. The system of claim 1.

5. The identification unit Not only can artworks be identified, but their condition and deterioration can also be automatically assessed.

2. The system of claim 1.

Images