System

Abstract

The system of the embodiment aims at enabling audiences to easily obtain related information of items in video content. A system according to an embodiment includes an acquisition unit, a recognition unit, a reference unit, and a provision unit. The acquisition part is used for acquiring click information. The recognition unit recognizes an item based on the click information acquired by the acquisition unit. The reference unit acquires related information based on the item information recognized by the recognition unit. The providing unit provides the information acquired by the reference unit.

Description

Technical Field

[0001] The technology disclosed herein relates to a system. Background Technology

[0002] Patent Document 1 discloses a personalized chatbot control method executed by at least one processor, the method comprising: receiving user speech; adding the user speech to a prompt containing instructions related to a chatbot role; encoding the prompt; and inputting the encoded prompt into a language model to generate chatbot speech in response to the user speech.

[0003] Patent Document 1: Japanese Patent Application Publication No. 2022-180282 Summary of the Invention

[0004] In existing technologies, there is a problem that viewers cannot easily obtain project-related information from video content.

[0005] The purpose of this technical solution is to enable viewers to easily obtain relevant information about items in video content.

[0006] The system of this technical solution comprises an acquisition unit, an identification unit, a reference unit, and a provision unit. The acquisition unit acquires click information. The identification unit identifies items based on the click information acquired by the acquisition unit. The reference unit acquires relevant information based on the item information identified by the identification unit. The provision unit provides the information acquired by the reference unit.

[0007] The system of this technical solution enables viewers to easily obtain relevant information about the items in the video content. Attached Figure Description

[0008] Figure 1 This is a conceptual diagram illustrating an example of the configuration of a data processing system according to the first embodiment.

[0009] Figure 2 This is a conceptual diagram illustrating an example of the main functions of the data processing apparatus and smart device according to the first embodiment.

[0010] Figure 3 This is a conceptual diagram illustrating an example of the data processing system configuration in the second embodiment.

[0011] Figure 4 This is a conceptual diagram illustrating an example of the main functions of the data processing device and smart glasses according to the second embodiment.

[0012] Figure 5 This is a conceptual diagram illustrating an example of the data processing system configuration in the third embodiment.

[0013] Figure 6This is a conceptual diagram illustrating an example of the main functions of the data processing device and head-mounted terminal according to the third embodiment.

[0014] Figure 7 This is a conceptual diagram illustrating an example of the data processing system configuration in the fourth embodiment.

[0015] Figure 8 This is a conceptual diagram illustrating an example of the functions of the main parts of the data processing device and robot according to the fourth embodiment.

[0016] Figure 9 It represents an emotion graph that maps multiple emotions.

[0017] Figure 10 It represents an emotion graph that maps multiple emotions.

[0018] Explanation of reference numerals in the attached figures

[0019] Data processing systems 10, 210, 310, and 410

[0020] 12 Data processing device

[0021] 14 Smart devices

[0022] 214 Smart Glasses

[0023] 314 Head-mounted terminal

[0024] 414 Robot Detailed Implementation

[0025] Hereinafter, an example of an implementation of the system involved in this disclosure will be described with reference to the accompanying drawings.

[0026] First, let's explain the terms used in the following description.

[0027] In the following embodiments, the processor (hereinafter referred to as "processor"), as indicated by the reference numerals, can be a single computing device or a combination of multiple computing devices. Furthermore, a processor can be a single computing device or a combination of multiple computing devices. Examples of computing devices include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose Computing on Graphics Processing Units), APU (Accelerated Processing Unit), or TPU (Tensor Processing Unit), etc.

[0028] In the following embodiments, RAM (Random Access Memory), as indicated in the figures, is a memory for temporary storage of information, which is used by the processor as working memory.

[0029] In the following embodiments, the memory, as indicated by the reference numerals, is one or more non-volatile storage devices used to store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), disk (e.g., hard disk), or magnetic tape, etc.

[0030] In the following embodiments, the communication I / F (Interface) with reference numerals is an interface including a communication processor and an antenna, etc. The communication I / F is responsible for communication between multiple computers. Examples of communication standards applicable to the communication I / F include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0031] In the following implementation, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it can be only A, only B, or a combination of A and B. Furthermore, in this specification, when "and / or" connects more than three items, the same approach as "A and / or B" applies.

[0032] First Implementation Method

[0033] Figure 1 An example of the configuration of the data processing system 10 according to the first embodiment is shown.

[0034] like Figure 1 As shown, the data processing system 10 includes a data processing device 12 and an intelligent device 14. An example of the data processing device 12 is a server.

[0035] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. An example of the network 54 includes a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0036] The smart device 14 includes a computer 36, a receiver 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. In addition, the receiver 38, output device 40, and camera 42 are also connected to the bus 52.

[0037] The receiving device 38 includes a touchscreen 38A and a microphone 38B, etc., for receiving user input. The touchscreen 38A receives user input generated by contact with an indicator (e.g., a pen or finger). The microphone 38B receives user input generated by sound by detecting the user's voice. The control unit 46A sends data representing user input received via the touchscreen 38A and microphone 38B to the data processing unit 12. In the data processing unit 12, a specific processing unit 290 (see reference...) Figure 2 Get the data that represents the user input.

[0038] The output device 40 includes a display 40A and a speaker 40B, which present data to the user by outputting data in a user-perceptible format (e.g., sound and / or text). The display 40A displays visual information such as text and images according to instructions from the processor 46. The speaker 40B outputs sound according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0039] Communication I / F 44 is connected to network 54. Communication I / F 44 and 26 are responsible for sending and receiving various information between processor 46 and processor 28 via network 54.

[0040] Figure 2 An example of the main functions of the data processing device 12 and the smart device 14 is shown.

[0041] like Figure 2 As shown, in the data processing apparatus 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the memory 32. The specific processing program 56 is an example of a "program" as understood in this disclosure. The processor 28 reads the specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. Specific processing is implemented by the processor 28 operating as a specific processing unit 290 based on the specific processing program 56 executed on the RAM 30.

[0042] The memory 32 stores a data generation model 58 and an emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290. The specific processing unit 290 can use the emotion-specific model 59 to infer the user's emotions and perform specific processing based on the user's emotions. The emotion inference function using the emotion-specific model 59 (emotion-specific function) includes inferring and predicting the user's emotions, performing various inferences and predictions related to the user's emotions, but is not limited to this example. Furthermore, emotion inference and prediction also include, for example, emotion analysis (analysis).

[0043] In the smart device 14, specific processing is performed by the processor 46. A specific processing program 60 is stored in the memory 50. The specific processing program 60 is used in conjunction with the data processing system 10. The processor 46 reads the specific processing program 60 from the memory 50 and executes the read specific processing program 60 on the RAM 48. Specific processing is implemented by the processor 46 operating as a control unit 46A based on the specific processing program 60 executed on the RAM 48. Furthermore, the smart device 14 may also have the same data generation model and emotion-specific model as the data generation model 58 and the emotion-specific model 59, and use these models to perform the same processing as the specific processing unit 290.

[0044] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains the processing results (prediction results, etc.) using the data generation model 58 by communicating with the server device that has the data generation model 58. Furthermore, the data processing device 12 may be a server device or a user-owned terminal device (e.g., a mobile phone, robot, home appliance, etc.). Next, an example of the processing performed by the data processing system 10 of the first embodiment will be described.

[0045] Implementation Method 1

[0046] The information provision system of this invention is an information provision system in which viewers click on items appearing in video content on their devices, and AI obtains and provides this information to the viewers. The information provision system obtains information by viewers watching video content and clicking on items of interest. For example, clicking on an actress displays her profile, and clicking on clothing provides information about the sales website. Even with low image resolution, the AI ​​can provide accurate information by referring to EC data. Thus, viewers can easily obtain information about items of interest while watching videos. Through this information provision system, viewers can easily obtain information about items of interest while enjoying video content, increasing their desire to purchase. Furthermore, businesses can also use video content to provide product information and guide viewers to EC websites, thereby expecting to achieve marketing results.

[0047] The information providing system of this embodiment includes an acquisition unit, an identification unit, a reference unit, and a providing unit. The acquisition unit acquires click information by having viewers watch video content on a device and click on items of interest. For example, the acquisition unit can acquire information such as the location, time, and intensity of the click. Furthermore, the acquisition unit can infer the user's emotions and adjust the timing of acquiring click information based on the inferred user emotions. The identification unit identifies the clicked items based on the click information acquired by the acquisition unit. For example, the identification unit can use image recognition technology to identify the clicked items. Furthermore, the identification unit can infer the user's emotions and adjust the accuracy of item recognition based on the inferred user emotions. The reference unit acquires relevant information based on the item information identified by the identification unit. For example, the reference unit can refer to EC databases and other relevant databases to acquire detailed product information, reviews, prices, etc. Furthermore, the reference unit can infer the user's emotions and select a reference database based on the inferred user emotions. The providing unit provides the information acquired by the reference unit to the viewer. For example, the providing unit can provide information through pop-up displays, notifications, emails, etc. Furthermore, the providing unit can infer the user's emotions and adjust the way information is provided based on the inferred user emotions. Therefore, the information providing system of this embodiment enables viewers to easily obtain information about items of interest while watching video content. For example, clicking on an actress will display her profile, and clicking on clothing will provide information about the sales website. Even with low image resolution, AI can provide accurate information by referring to EC data. Thus, viewers can easily obtain information about items of interest while watching videos.

[0048] The reference unit can refer to EC databases. EC databases include, but are not limited to, databases such as Yahoo! Shopping (registered trademark) and Amazon (registered trademark). The reference unit can, for example, refer to EC databases to obtain detailed product information, reviews, prices, etc. Thus, by referring to EC databases, accurate product information can be obtained. Some or all of the above processing in the reference unit can be performed by AI, or it can be performed without AI. For example, the reference unit can input EC databases into AI, which can then retrieve detailed product information, reviews, prices, etc.

[0049] The reference unit can reference other relevant databases. Relevant databases include, but are not limited to, industry-specific databases and open databases. For example, the reference unit can reference relevant databases to obtain a wide range of information. By referencing relevant databases, a wide range of information can be obtained. Some or all of the above processing in the reference unit can be performed by AI, or it can be performed without AI. For example, the reference unit can input relevant databases into AI, which will then perform the acquisition of a wide range of information.

[0050] The recognition unit is capable of identifying clicked items. For example, the recognition unit can utilize image recognition technology to identify clicked items. For instance, the recognition unit can analyze the image of the clicked item, extract its features, and perform recognition. Furthermore, the recognition unit can also utilize location information to identify clicked items. For instance, the recognition unit can determine the location of the item and obtain that information based on the location information of the click. In addition, the recognition unit can infer the user's sentiment and adjust the accuracy of item recognition based on the inferred user sentiment. Thus, the clicked item can be accurately identified. Some or all of the above processing in the recognition unit can be performed by AI, or AI can be used without it. For example, the recognition unit can input the image data of the clicked item into AI, and AI can perform item recognition.

[0051] The information delivery department can provide information to the audience. This can be done through methods such as pop-ups, notifications, and emails. For example, the department can provide information about the clicked item to the audience via a pop-up. Additionally, it can provide this information via notifications or emails. This allows for rapid information delivery to the audience. Some or all of the above processing in the delivery department can be performed using AI, or it can be done without AI. For example, the department can input the clicked item information into AI, which will then execute the information delivery method.

[0052] The acquisition department can analyze a user's past click history and select appropriate acquisition methods. For example, it can analyze frequently clicked item patterns and prioritize acquiring similar items. Alternatively, it can analyze click trends over specific time periods from a user's click history and select the optimal acquisition method for that period. Furthermore, the acquisition department can customize acquisition methods for specific content types based on a user's click history. Thus, it can select the optimal acquisition method based on past click history. Some or all of the above processing in the acquisition department can be performed using AI, or it can be done without AI. For example, the acquisition department can input the user's click history data into AI, which will then select the optimal acquisition method.

[0053] The acquisition department can filter click information based on the user's current viewing content and areas of interest. For example, it can prioritize acquiring click information for items related to the type of content the user is currently viewing. Alternatively, it can filter and acquire click information for highly relevant items based on the user's areas of interest. Furthermore, the acquisition department can acquire click information at appropriate times based on the context of the user's viewing. Thus, it can filter click information based on viewing content and areas of interest. Some or all of the above processing in the acquisition department can be performed using AI, or it can be done without AI. For example, the acquisition department can input the user's viewing content and areas of interest data into AI, which can then perform the filtering of click information.

[0054] The acquisition unit can select the appropriate acquisition method based on the user's input method when acquiring click information. For example, when the user uses voice input, the acquisition unit acquires click information based on voice commands. Similarly, when the user uses gesture input, the acquisition unit acquires click information based on gesture actions. Furthermore, when the user uses text input, the acquisition unit can also acquire click information based on the input text. Thus, the optimal acquisition method can be selected based on the user's input method. Some or all of the above processing in the acquisition unit can be performed by AI, or it can be performed without AI. For example, the acquisition unit can input the user's input method data into AI, which can then select the optimal acquisition method.

[0055] The acquisition unit can consider the user's geographical location information when acquiring click information, prioritizing the acquisition of highly relevant information. For example, when the user is located in a specific region, the acquisition unit prioritizes acquiring click information for items related to that region. Similarly, when the user is traveling, the acquisition unit prioritizes acquiring click information for items related to the travel destination. Furthermore, when the user is at home, the acquisition unit can also prioritize acquiring click information for items related to nearby stores or services. Thus, it is possible to consider geographical location information to acquire highly relevant information. Some or all of the above processing in the acquisition unit can be performed by AI, or it can be performed without AI. For example, the acquisition unit can input the user's geographical location information into AI, which can then perform the acquisition of highly relevant information.

[0056] The acquisition department can analyze users' social media activities to obtain relevant information when acquiring click information. For example, it can acquire click information related to users' check-in locations on social media. For instance, it can analyze users' social media posts to acquire click information related to those locations. Furthermore, it can refer to the activity of users' friends on social media to acquire click information related to those friends. Thus, it can acquire relevant information based on social media activity. Some or all of the above processing in the acquisition department can be performed by AI, or it can be done without AI. For example, the acquisition department can input users' social media activity data into AI, which will then perform the acquisition of relevant information.

[0057] The acquisition department can incorporate past user feedback when acquiring click information, allowing for customized acquisition methods. For example, it can optimize acquisition methods based on past user feedback. Specifically, it can customize acquisition methods for specific projects based on past user feedback. Furthermore, the acquisition department can analyze user feedback to identify areas for improvement in the acquisition methods. Thus, it can customize acquisition methods based on past feedback. Some or all of the above processing in the acquisition department can be performed using AI, or it can be done without AI. For example, the acquisition department can input past user feedback data into AI, which can then customize the acquisition methods.

[0058] The recognition unit can adjust the recognition accuracy based on the importance of the items during recognition. For example, it may perform detailed recognition of highly important items, or simplified recognition of less important items. Furthermore, the recognition unit can dynamically adjust the recognition accuracy according to the importance of the items. Thus, the recognition accuracy can be adjusted based on the importance of the items. Some or all of the above processing in the recognition unit can be performed by AI, or it can be performed without AI. For example, the recognition unit can input the importance data of the items into the AI, which will then adjust the recognition accuracy.

[0059] The recognition unit can apply different recognition algorithms based on the category of the item during recognition. For example, it can apply a dedicated fashion recognition algorithm for clothing items. It can also apply a dedicated technology recognition algorithm for electronic devices. Furthermore, it can apply a dedicated food recognition algorithm for food items. Thus, it is possible to apply recognition algorithms based on the category of the item. Some or all of the above processing in the recognition unit can be performed by AI, or it can be performed without AI. For example, the recognition unit can input the category data of the item into the AI, which will then execute the application of the recognition algorithm.

[0060] The recognition unit can improve recognition accuracy by referencing the user's past recognition results. For example, it can improve accuracy based on the user's past recognition data. Specifically, it can adjust the recognition accuracy for specific items based on the user's past results. Furthermore, the recognition unit can analyze the user's recognition history to optimize the recognition algorithm. Thus, it can improve recognition accuracy based on past results. Some or all of the above processing in the recognition unit can be performed by AI, or it can be done without AI. For example, the recognition unit can input the user's past recognition result data into the AI, which will then perform the accuracy improvement.

[0061] The identification unit can determine the identification priority based on the submission time of the items. For example, it may prioritize identifying items that have been recently clicked. Alternatively, it may postpone identifying items that haven't been clicked for a long time. Furthermore, the identification unit can dynamically adjust the identification priority based on the submission time. Thus, it can determine the identification priority based on the submission time. Some or all of the above processing in the identification unit can be performed by AI, or it can be done without AI. For example, the identification unit can input the submission time data of the items into the AI, which will then determine the identification priority.

[0062] The identification unit can adjust the identification order based on the relevance of items during identification. For example, it may prioritize identifying highly relevant items, or postpone identifying less relevant items. Furthermore, the identification unit can dynamically adjust the identification order based on the relevance of items. Thus, the identification order can be adjusted according to the relevance of items. Some or all of the above processing in the identification unit can be performed by AI, or it can be performed without AI. For example, the identification unit can input the relevance data of the items into the AI, which will then adjust the identification order.

[0063] The recognition unit can adjust the use of technical terms in the recognition results based on the user's level of expertise. For example, it might use detailed technical terms for users with high expertise, or simpler terms for users with low expertise. Furthermore, the recognition unit can dynamically adjust the use of technical terms based on the user's level of expertise. Thus, it can adjust the use of technical terms in the recognition results according to the user's level of expertise. Some or all of the above processing in the recognition unit can be performed by AI, or it can be done without AI. For example, the recognition unit can input the user's level of expertise data into the AI, which will then execute the use of technical terms.

[0064] The reference unit can adjust the reference accuracy based on the importance of the project during reference. For example, the reference unit references detailed information for projects of high importance. Conversely, it references simplified information for projects of low importance. Furthermore, the reference unit can dynamically adjust the reference accuracy based on the project's importance. Thus, the reference accuracy can be adjusted according to the project's importance. Some or all of the above processing in the reference unit can be performed by AI, or it can be done without AI. For example, the reference unit can input the project's importance data into the AI, which will then perform the adjustment of the reference accuracy.

[0065] The reference unit can apply different reference algorithms based on the category of the item during reference. For example, it can apply a fashion reference algorithm specifically for clothing items. It can also apply a technical reference algorithm specifically for electronic devices. Furthermore, it can apply a food reference algorithm specifically for food items. Thus, it is possible to apply reference algorithms according to the category of the item. Some or all of the above processing in the reference unit can be performed by AI, or it can be performed without AI. For example, the reference unit can input the category data of the item into the AI, and the AI ​​can then execute the application of the reference algorithm.

[0066] The reference unit can improve reference accuracy by referencing the user's past reference results. For example, it can improve accuracy based on data from projects the user has previously referenced. For instance, the reference unit can adjust the reference accuracy for specific projects based on the user's past reference results. Furthermore, the reference unit can analyze the user's reference history and optimize the reference algorithm. Thus, it can improve reference accuracy based on past reference results. Some or all of the above processing in the reference unit can be performed by AI, or it can be done without AI. For example, the reference unit can input the user's past reference result data into the AI, which can then perform the accuracy improvement.

[0067] The reference department can determine the reference priority based on the submission time of a project. For example, it may prioritize projects that have been recently clicked. Alternatively, it may postpone projects that haven't been clicked for a long time. Furthermore, the reference department can dynamically adjust the reference priority based on the submission time. Thus, it can determine the reference priority based on the submission time. Some or all of the above processing in the reference department can be performed by AI, or it can be done without AI. For example, the reference department can input the project's submission time data into the AI, which will then determine the reference priority.

[0068] The reference unit can adjust the reference order based on the relevance of projects during the reference process. For example, it may prioritize projects with high relevance, or place projects with low relevance later. Furthermore, the reference unit can dynamically adjust the reference order based on project relevance. Thus, the reference order can be adjusted according to the relevance of projects. Some or all of the above processing in the reference unit can be performed using AI, or it can be done without AI. For example, the reference unit can input project relevance data into AI, which will then perform the adjustment of the reference order.

[0069] The reference unit can adjust the use of technical terminology in the reference results based on the user's level of expertise. For example, it might use detailed technical terminology for users with high expertise, or simpler terminology for users with low expertise. Furthermore, the reference unit can dynamically adjust the use of technical terminology based on the user's level of expertise. Thus, it can adjust the use of technical terminology in the reference results according to the user's level of expertise. Some or all of the above processing in the reference unit can be performed by AI, or it can be done without AI. For example, the reference unit can input the user's level of expertise data into AI, which can then execute the use of technical terminology.

[0070] The providing unit can adjust the accuracy of information delivery based on its importance. For example, it might provide detailed information for highly important information, or simplified information for less important information. Furthermore, the providing unit can dynamically adjust the accuracy based on the importance of the information. Thus, the accuracy can be adjusted according to the importance of the information. Some or all of the above processing in the providing unit can be performed using AI, or it can be done without AI. For example, the providing unit can input the importance data of the information into AI, which will then adjust the accuracy of the delivery.

[0071] The provisioning department can apply different provisioning algorithms based on the category of information during provision. For example, it can apply a fashion-specific provisioning algorithm for clothing information, a technology-specific provisioning algorithm for electronic device information, and a food-specific provisioning algorithm for food information. Thus, it is possible to apply provisioning algorithms based on the category of information. Some or all of the above processing in the provisioning department can be performed using AI, or it can be performed without AI. For example, the provisioning department can input the category data of the information into AI, which will then execute the application of the provisioning algorithm.

[0072] The provisioning department can improve provisioning accuracy by referencing the user's past provisioning results. For example, it can improve accuracy based on information data previously provided to the user. Specifically, it can adjust the provisioning accuracy of specific information based on the user's past provisioning results. Furthermore, the provisioning department can analyze the user's provisioning history and optimize the provisioning algorithm. Thus, it can improve provisioning accuracy based on past results. Some or all of the above processing in the provisioning department can be performed using AI, or it can be done without AI. For example, the provisioning department can input the user's past provisioning result data into AI, which can then perform the accuracy improvement.

[0073] The provisioning department can determine the priority of information provision based on the timing of information submission. For example, it may prioritize providing recently acquired information, or postpone providing information that has not been acquired for a long time. Furthermore, the provisioning department can dynamically adjust the provisioning priority based on the submission timing. Thus, the provisioning priority can be determined according to the submission timing. Some or all of the above processing in the provisioning department can be performed by AI, or it can be performed without AI. For example, the provisioning department can input the information submission timing data into AI, which will then execute the decision on the provisioning priority.

[0074] The provisioning department can adjust the delivery order based on the relevance of information during delivery. For example, the provisioning department may prioritize the delivery of highly relevant information, or postpone the delivery of less relevant information. Furthermore, the provisioning department can dynamically adjust the delivery order based on the relevance of information. Thus, the delivery order can be adjusted according to the relevance of information. Some or all of the above processing in the provisioning department can be performed by AI, or it may not require AI. For example, the provisioning department can input the relevance data of the information into the AI, which will then adjust the delivery order.

[0075] The provision department can adjust the use of technical terminology in the information based on the user's level of expertise. For example, it might use detailed technical terms for users with high expertise, or simple terms for users with low expertise. Furthermore, the provision department can dynamically adjust the use of technical terminology based on the user's level of expertise. Thus, it can adjust the use of technical terminology according to the user's level of expertise. Some or all of the above processing in the provision department can be performed using AI, or it can be done without AI. For example, the provision department can input the user's level of expertise data into AI, which can then execute the use of technical terminology.

[0076] The system described in this embodiment is not limited to the examples above. For example, various modifications can be made as follows.

[0077] The acquisition department can analyze users' past purchase history and prioritize information on related items. For example, based on data from users' past purchases, the acquisition department can prioritize click information for similar items. Furthermore, the acquisition department can analyze users' purchase history to identify and prioritize interests in specific brands or categories. Moreover, based on users' purchase history, the acquisition department can prioritize information on items that align with seasons or trends. Thus, it can efficiently acquire relevant information based on users' purchase history.

[0078] The recognition unit can utilize user gaze tracking data to prioritize identifying items where the user's gaze lingers. For example, it can prioritize items where the user's gaze remains for an extended period. Furthermore, the unit can analyze gaze movement patterns to prioritize items the user might be interested in. Moreover, the unit can adjust recognition accuracy based on gaze concentration, improving the accuracy of identifying items where the user's gaze is focused. Thus, it can efficiently identify items of interest using user gaze data.

[0079] The reference unit can analyze users' activities on social media to obtain relevant information. For example, it can prioritize information about the brands or influencers users follow on social media. Furthermore, it can analyze users' posted content or comments to obtain information related to topics of interest. Additionally, it can refer to the activities of users' friends on social media to obtain relevant information. Thus, it can efficiently obtain relevant information based on social media activity.

[0080] The information delivery unit can adjust the way information is delivered based on the user's device usage. For example, when a user is using a smartphone, the information is delivered in a way that fits the screen size. Furthermore, when a user is using a tablet, the information can be delivered using a larger screen. Moreover, when a user is using a desktop computer, the information can be delivered using multiple windows. Thus, information can be delivered in the optimal way according to the user's device usage.

[0081] The acquisition department can take into account the user's geographical location information and prioritize acquiring highly relevant information. For example, when the user is located in a specific region, the acquisition department prioritizes acquiring click information for items related to that region. Furthermore, when the user is traveling, the acquisition department can also prioritize acquiring click information for items related to the travel destination. Moreover, when the user is at home, the acquisition department can also prioritize acquiring click information for items related to nearby stores or services. Thus, it is possible to acquire highly relevant information by taking geographical location information into account.

[0082] The following is a brief description of the processing flow of Implementation Method 1.

[0083] Step 1: The acquisition department obtains click information by observing viewers watching video content and clicking on items of interest on their devices. For example, the acquisition department can obtain information such as the location, time, and intensity of the click. Furthermore, the acquisition department can infer the user's emotions and adjust the timing of acquiring click information based on these inferred emotions.

[0084] Step 2: The recognition unit identifies the clicked item based on the click information acquired by the acquisition unit. For example, the recognition unit can use image recognition technology to identify the clicked item. Furthermore, the recognition unit can infer the user's sentiment and adjust the accuracy of item recognition based on the inferred user sentiment.

[0085] Step 3: The reference unit obtains relevant information based on the item information identified by the identification unit. For example, the reference unit can refer to the EC database and other relevant databases to obtain detailed product information, reviews, prices, etc. Furthermore, the reference unit can infer user sentiment and select a reference database based on the inferred user sentiment.

[0086] Step 4: The provision department provides the audience with information obtained from the reference department. For example, the provision department can provide information through pop-up windows, notifications, emails, etc. In addition, the provision department can infer the user's emotions and adjust the way information is provided based on the inferred user emotions.

[0087] Implementation Method 2

[0088] The information provision system of this invention is an information provision system in which viewers click on items appearing in video content on their devices, and AI obtains and provides this information to the viewers. The information provision system obtains information by viewers watching video content and clicking on items of interest. For example, clicking on an actress displays her profile, and clicking on clothing provides information about the sales website. Even with low image resolution, the AI ​​can provide accurate information by referring to EC data. Thus, viewers can easily obtain information about items of interest while watching videos. Through this information provision system, viewers can easily obtain information about items of interest while enjoying video content, increasing their desire to purchase. Furthermore, businesses can also use video content to provide product information and guide viewers to EC websites, thereby expecting to achieve marketing results.

[0089] The information providing system of this embodiment includes an acquisition unit, an identification unit, a reference unit, and a providing unit. The acquisition unit acquires click information by having viewers watch video content on a device and click on items of interest. For example, the acquisition unit can acquire information such as the location, time, and intensity of the click. Furthermore, the acquisition unit can infer the user's emotions and adjust the timing of acquiring click information based on the inferred user emotions. The identification unit identifies the clicked items based on the click information acquired by the acquisition unit. For example, the identification unit can use image recognition technology to identify the clicked items. Furthermore, the identification unit can infer the user's emotions and adjust the accuracy of item recognition based on the inferred user emotions. The reference unit acquires relevant information based on the item information identified by the identification unit. For example, the reference unit can refer to EC databases and other relevant databases to acquire detailed product information, reviews, prices, etc. Furthermore, the reference unit can infer the user's emotions and select a reference database based on the inferred user emotions. The providing unit provides the information acquired by the reference unit to the viewer. For example, the providing unit can provide information through pop-up displays, notifications, emails, etc. Furthermore, the providing unit can infer the user's emotions and adjust the way information is provided based on the inferred user emotions. Therefore, the information providing system of this embodiment enables viewers to easily obtain information about items of interest while watching video content. For example, clicking on an actress will display her profile, and clicking on clothing will provide information about the sales website. Even with low image resolution, AI can provide accurate information by referring to EC data. Thus, viewers can easily obtain information about items of interest while watching videos.

[0090] The reference unit can refer to EC databases. EC databases include, but are not limited to, databases such as Yahoo! Shopping and Amazon. The reference unit can, for example, refer to EC databases to obtain detailed product information, reviews, prices, etc. Thus, by referring to EC databases, accurate product information can be obtained. Some or all of the above processing in the reference unit can be performed by AI, or AI can be used without it. For example, the reference unit can input EC databases into AI, which can then retrieve detailed product information, reviews, prices, etc.

[0091] The reference unit can reference other relevant databases. Relevant databases include, but are not limited to, industry-specific databases and open databases. For example, the reference unit can reference relevant databases to obtain a wide range of information. By referencing relevant databases, a wide range of information can be obtained. Some or all of the above processing in the reference unit can be performed by AI, or it can be performed without AI. For example, the reference unit can input relevant databases into AI, which will then perform the acquisition of a wide range of information.

[0092] The recognition unit is capable of identifying clicked items. For example, the recognition unit can utilize image recognition technology to identify clicked items. For instance, the recognition unit can analyze the image of the clicked item, extract its features, and perform recognition. Furthermore, the recognition unit can also utilize location information to identify clicked items. For instance, the recognition unit can determine the location of the item and obtain that information based on the location information of the click. In addition, the recognition unit can infer the user's sentiment and adjust the accuracy of item recognition based on the inferred user sentiment. Thus, the clicked item can be accurately identified. Some or all of the above processing in the recognition unit can be performed by AI, or AI can be used without it. For example, the recognition unit can input the image data of the clicked item into AI, and AI can perform item recognition.

[0093] The information delivery department can provide information to the audience. This can be done through methods such as pop-ups, notifications, and emails. For example, the department can provide information about the clicked item to the audience via a pop-up. Additionally, it can provide this information via notifications or emails. This allows for rapid information delivery to the audience. Some or all of the above processing in the delivery department can be performed using AI, or it can be done without AI. For example, the department can input the clicked item information into AI, which will then execute the information delivery method.

[0094] The acquisition unit can infer a user's emotions and adjust the timing of acquiring click information based on the inferred emotions. For example, the acquisition unit can capture the user's facial expressions using a camera and use emotion inference algorithms to infer emotions. For instance, the acquisition unit can calculate an emotion score based on facial expression changes. Furthermore, the acquisition unit can record the user's voice and use speech analysis technology to infer emotions. For instance, the acquisition unit can analyze the pitch and speed of the voice to calculate an emotion score. Additionally, the acquisition unit can collect the user's biometric data (heart rate or skin conductance) using sensors and use emotion inference algorithms to infer emotions. For instance, the acquisition unit can calculate an emotion score based on heart rate changes. Thus, the timing of acquiring click information can be optimized based on the user's emotions. Emotion inference can be achieved, for example, through emotion inference functions such as emotion engines or generative AI. Generative AI can be text-generated AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the acquisition unit can be performed by AI, or AI can be omitted. For example, the acquisition unit can input user image data captured by a camera into the generative AI, which will then perform the inference of the user's emotions.

[0095] The acquisition department can analyze a user's past click history and select appropriate acquisition methods. For example, it can analyze frequently clicked item patterns and prioritize acquiring similar items. Alternatively, it can analyze click trends over specific time periods from a user's click history and select the optimal acquisition method for that period. Furthermore, the acquisition department can customize acquisition methods for specific content types based on a user's click history. Thus, it can select the optimal acquisition method based on past click history. Some or all of the above processing in the acquisition department can be performed using AI, or it can be done without AI. For example, the acquisition department can input the user's click history data into AI, which will then select the optimal acquisition method.

[0096] The acquisition department can filter click information based on the user's current viewing content and areas of interest. For example, it can prioritize acquiring click information for items related to the type of content the user is currently viewing. Alternatively, it can filter and acquire click information for highly relevant items based on the user's areas of interest. Furthermore, the acquisition department can acquire click information at appropriate times based on the context of the user's viewing. Thus, it can filter click information based on viewing content and areas of interest. Some or all of the above processing in the acquisition department can be performed using AI, or it can be done without AI. For example, the acquisition department can input the user's viewing content and areas of interest data into AI, which can then perform the filtering of click information.

[0097] The acquisition unit can select the appropriate acquisition method based on the user's input method when acquiring click information. For example, when the user uses voice input, the acquisition unit acquires click information based on voice commands. Similarly, when the user uses gesture input, the acquisition unit acquires click information based on gesture actions. Furthermore, when the user uses text input, the acquisition unit can also acquire click information based on the input text. Thus, the optimal acquisition method can be selected based on the user's input method. Some or all of the above processing in the acquisition unit can be performed by AI, or it can be performed without AI. For example, the acquisition unit can input the user's input method data into AI, which can then select the optimal acquisition method.

[0098] The acquisition unit can infer the user's emotions and determine the priority of acquired click information based on the inferred user emotions. For example, when the user is excited, the acquisition unit prioritizes acquiring click information for items that arouse interest. Similarly, when the user is relaxed, the acquisition unit prioritizes acquiring click information for items with a relaxing effect. Furthermore, when the user is stressed, the acquisition unit can also prioritize acquiring click information for items that help relieve stress. Thus, the priority of click information can be determined based on the user's emotions. Emotion inference can be achieved, for example, through emotion inference functions such as an emotion engine or generative AI. Generative AI can be text-generating AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the acquisition unit can be performed by AI, or AI can be omitted. For example, the acquisition unit can input the user's emotion data into the generative AI, which then performs the decision on the priority of click information.

[0099] The acquisition unit can consider the user's geographical location information when acquiring click information, prioritizing the acquisition of highly relevant information. For example, when the user is located in a specific region, the acquisition unit prioritizes acquiring click information for items related to that region. Similarly, when the user is traveling, the acquisition unit prioritizes acquiring click information for items related to the travel destination. Furthermore, when the user is at home, the acquisition unit can also prioritize acquiring click information for items related to nearby stores or services. Thus, it is possible to consider geographical location information to acquire highly relevant information. Some or all of the above processing in the acquisition unit can be performed by AI, or it can be performed without AI. For example, the acquisition unit can input the user's geographical location information into AI, which can then perform the acquisition of highly relevant information.

[0100] The acquisition department can analyze users' social media activities to obtain relevant information when acquiring click information. For example, it can acquire click information related to users' check-in locations on social media. For instance, it can analyze users' social media posts to acquire click information related to those locations. Furthermore, it can refer to the activity of users' friends on social media to acquire click information related to those friends. Thus, it can acquire relevant information based on social media activity. Some or all of the above processing in the acquisition department can be performed by AI, or it can be done without AI. For example, the acquisition department can input users' social media activity data into AI, which will then perform the acquisition of relevant information.

[0101] The acquisition department can incorporate past user feedback when acquiring click information, allowing for customized acquisition methods. For example, it can optimize acquisition methods based on past user feedback. Specifically, it can customize acquisition methods for specific projects based on past user feedback. Furthermore, the acquisition department can analyze user feedback to identify areas for improvement in the acquisition methods. Thus, it can customize acquisition methods based on past feedback. Some or all of the above processing in the acquisition department can be performed using AI, or it can be done without AI. For example, the acquisition department can input past user feedback data into AI, which can then customize the acquisition methods.

[0102] The recognition unit can infer the user's emotions and adjust the accuracy of item recognition based on the inferred emotions. For example, when the user is excited, the recognition unit increases the recognition accuracy and immediately recognizes items. Conversely, when the user is relaxed, the recognition unit appropriately decreases the recognition accuracy to recognize items naturally. Furthermore, when the user is stressed, the recognition unit can optimize the recognition accuracy to reduce the user's burden. Thus, the accuracy of item recognition can be adjusted according to the user's emotions. Emotion inference can be achieved, for example, through emotion inference functions such as emotion engines or generative AI. Generative AI can be text-generated AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the recognition unit can be performed by AI, or AI can be omitted. For example, the recognition unit can input the user's emotion data into the generative AI, which then performs the adjustment of item recognition accuracy.

[0103] The recognition unit can adjust the recognition accuracy based on the importance of the items during recognition. For example, it may perform detailed recognition of highly important items, or simplified recognition of less important items. Furthermore, the recognition unit can dynamically adjust the recognition accuracy according to the importance of the items. Thus, the recognition accuracy can be adjusted based on the importance of the items. Some or all of the above processing in the recognition unit can be performed by AI, or it can be performed without AI. For example, the recognition unit can input the importance data of the items into the AI, which will then adjust the recognition accuracy.

[0104] The recognition unit can apply different recognition algorithms based on the category of the item during recognition. For example, it can apply a dedicated fashion recognition algorithm for clothing items. It can also apply a dedicated technology recognition algorithm for electronic devices. Furthermore, it can apply a dedicated food recognition algorithm for food items. Thus, it is possible to apply recognition algorithms based on the category of the item. Some or all of the above processing in the recognition unit can be performed by AI, or it can be performed without AI. For example, the recognition unit can input the category data of the item into the AI, which will then execute the application of the recognition algorithm.

[0105] The recognition unit can improve recognition accuracy by referencing the user's past recognition results. For example, it can improve accuracy based on the user's past recognition data. Specifically, it can adjust the recognition accuracy for specific items based on the user's past results. Furthermore, the recognition unit can analyze the user's recognition history to optimize the recognition algorithm. Thus, it can improve recognition accuracy based on past results. Some or all of the above processing in the recognition unit can be performed by AI, or it can be done without AI. For example, the recognition unit can input the user's past recognition result data into the AI, which will then perform the accuracy improvement.

[0106] The recognition unit can infer the user's emotions and determine the recognition priority based on the inferred emotions. For example, when the user is excited, the recognition unit prioritizes recognizing items that arouse interest. Similarly, when the user is relaxed, the recognition unit prioritizes recognizing items with a relaxing effect. Furthermore, when the user is stressed, the recognition unit can also prioritize recognizing items that help relieve stress. Thus, the recognition priority can be determined based on the user's emotions. Emotion inference can be achieved, for example, through emotion inference functions such as an emotion engine or generative AI. Generative AI can be text-generated AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the recognition unit can be performed by AI, or AI can be omitted. For example, the recognition unit can input the user's emotion data into the generative AI, which then performs the determination of recognition priorities.

[0107] The identification unit can determine the identification priority based on the submission time of the items. For example, it may prioritize identifying items that have been recently clicked. Alternatively, it may postpone identifying items that haven't been clicked for a long time. Furthermore, the identification unit can dynamically adjust the identification priority based on the submission time. Thus, it can determine the identification priority based on the submission time. Some or all of the above processing in the identification unit can be performed by AI, or it can be done without AI. For example, the identification unit can input the submission time data of the items into the AI, which will then determine the identification priority.

[0108] The identification unit can adjust the identification order based on the relevance of items during identification. For example, it may prioritize identifying highly relevant items, or postpone identifying less relevant items. Furthermore, the identification unit can dynamically adjust the identification order based on the relevance of items. Thus, the identification order can be adjusted according to the relevance of items. Some or all of the above processing in the identification unit can be performed by AI, or it can be performed without AI. For example, the identification unit can input the relevance data of the items into the AI, which will then adjust the identification order.

[0109] The recognition unit can adjust the use of technical terms in the recognition results based on the user's level of expertise. For example, it might use detailed technical terms for users with high expertise, or simpler terms for users with low expertise. Furthermore, the recognition unit can dynamically adjust the use of technical terms based on the user's level of expertise. Thus, it can adjust the use of technical terms in the recognition results according to the user's level of expertise. Some or all of the above processing in the recognition unit can be performed by AI, or it can be done without AI. For example, the recognition unit can input the user's level of expertise data into the AI, which will then execute the use of technical terms.

[0110] The reference unit can infer the user's emotions and select a reference database based on the inferred emotions. For example, when the user is excited, the reference unit prioritizes referencing entertainment-related databases. Similarly, when the user is relaxed, the reference unit prioritizes referencing databases with relaxing effects. Furthermore, when the user is stressed, the reference unit can prioritize referencing databases that help reduce stress. Thus, the reference database can be selected based on the user's emotions. Emotion inference can be achieved, for example, through an emotion engine or generative AI. The generative AI can be text-generating AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the reference unit can be performed by AI, or AI can be omitted. For example, the reference unit can input the user's emotion data into the generative AI, which then performs the database selection.

[0111] The reference unit can adjust the reference accuracy based on the importance of the project during reference. For example, the reference unit references detailed information for projects of high importance. Conversely, it references simplified information for projects of low importance. Furthermore, the reference unit can dynamically adjust the reference accuracy based on the project's importance. Thus, the reference accuracy can be adjusted according to the project's importance. Some or all of the above processing in the reference unit can be performed by AI, or it can be done without AI. For example, the reference unit can input the project's importance data into the AI, which will then perform the adjustment of the reference accuracy.

[0112] The reference unit can apply different reference algorithms based on the category of the item during reference. For example, it can apply a fashion reference algorithm specifically for clothing items. It can also apply a technical reference algorithm specifically for electronic devices. Furthermore, it can apply a food reference algorithm specifically for food items. Thus, it is possible to apply reference algorithms according to the category of the item. Some or all of the above processing in the reference unit can be performed by AI, or it can be performed without AI. For example, the reference unit can input the category data of the item into the AI, and the AI ​​can then execute the application of the reference algorithm.

[0113] The reference unit can improve reference accuracy by referencing the user's past reference results. For example, it can improve accuracy based on data from projects the user has previously referenced. For instance, the reference unit can adjust the reference accuracy for specific projects based on the user's past reference results. Furthermore, the reference unit can analyze the user's reference history and optimize the reference algorithm. Thus, it can improve reference accuracy based on past reference results. Some or all of the above processing in the reference unit can be performed by AI, or it can be done without AI. For example, the reference unit can input the user's past reference result data into the AI, which can then perform the accuracy improvement.

[0114] The reference unit can infer a user's emotions and determine the priority of references based on the inferred emotions. For example, when a user is excited, the reference unit prioritizes items that arouse interest. Similarly, when a user is relaxed, the reference unit prioritizes items with a relaxing effect. Furthermore, when a user is stressed, the reference unit can prioritize items that help relieve stress. Thus, the reference priority can be determined based on the user's emotions. Emotion inference can be achieved, for example, through emotion inference functions such as an emotion engine or generative AI. Generative AI can be text-generating AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the reference unit can be performed by AI, or AI can be omitted. For example, the reference unit can input the user's emotion data into the generative AI, which then performs the determination of reference priorities.

[0115] The reference department can determine the reference priority based on the submission time of a project. For example, it may prioritize projects that have been recently clicked. Alternatively, it may postpone projects that haven't been clicked for a long time. Furthermore, the reference department can dynamically adjust the reference priority based on the submission time. Thus, it can determine the reference priority based on the submission time. Some or all of the above processing in the reference department can be performed by AI, or it can be done without AI. For example, the reference department can input the project's submission time data into the AI, which will then determine the reference priority.

[0116] The reference unit can adjust the reference order based on the relevance of projects during the reference process. For example, it may prioritize projects with high relevance, or place projects with low relevance later. Furthermore, the reference unit can dynamically adjust the reference order based on project relevance. Thus, the reference order can be adjusted according to the relevance of projects. Some or all of the above processing in the reference unit can be performed using AI, or it can be done without AI. For example, the reference unit can input project relevance data into AI, which will then perform the adjustment of the reference order.

[0117] The reference unit can adjust the use of technical terminology in the reference results based on the user's level of expertise. For example, it might use detailed technical terminology for users with high expertise, or simpler terminology for users with low expertise. Furthermore, the reference unit can dynamically adjust the use of technical terminology based on the user's level of expertise. Thus, it can adjust the use of technical terminology in the reference results according to the user's level of expertise. Some or all of the above processing in the reference unit can be performed by AI, or it can be done without AI. For example, the reference unit can input the user's level of expertise data into AI, which can then execute the use of technical terminology.

[0118] The information delivery unit can infer the user's emotions and adjust the way information is delivered based on the inferred emotions. For example, when the user is excited, the information delivery unit may provide information in a visually stimulating manner. Conversely, when the user is relaxed, the information delivery unit may provide information in a calming manner. Furthermore, when the user is stressed, the information delivery unit may provide information in a concise and highly visual manner. Thus, the information delivery method can be adjusted according to the user's emotions. Emotion inference can be achieved, for example, through emotion inference functions such as emotion engines or generative AI. Generative AI can be text-generating AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the delivery unit can be performed by AI, or AI may not be used. For example, the delivery unit can input the user's emotion data into the generative AI, which will then adjust the way information is delivered.

[0119] The providing unit can adjust the accuracy of information delivery based on its importance. For example, it might provide detailed information for highly important information, or simplified information for less important information. Furthermore, the providing unit can dynamically adjust the accuracy based on the importance of the information. Thus, the accuracy can be adjusted according to the importance of the information. Some or all of the above processing in the providing unit can be performed using AI, or it can be done without AI. For example, the providing unit can input the importance data of the information into AI, which will then adjust the accuracy of the delivery.

[0120] The provisioning department can apply different provisioning algorithms based on the category of information during provision. For example, it can apply a fashion-specific provisioning algorithm for clothing information, a technology-specific provisioning algorithm for electronic device information, and a food-specific provisioning algorithm for food information. Thus, it is possible to apply provisioning algorithms based on the category of information. Some or all of the above processing in the provisioning department can be performed using AI, or it can be performed without AI. For example, the provisioning department can input the category data of the information into AI, which will then execute the application of the provisioning algorithm.

[0121] The provisioning department can improve provisioning accuracy by referencing the user's past provisioning results. For example, it can improve accuracy based on information data previously provided to the user. Specifically, it can adjust the provisioning accuracy of specific information based on the user's past provisioning results. Furthermore, the provisioning department can analyze the user's provisioning history and optimize the provisioning algorithm. Thus, it can improve provisioning accuracy based on past results. Some or all of the above processing in the provisioning department can be performed using AI, or it can be done without AI. For example, the provisioning department can input the user's past provisioning result data into AI, which can then perform the accuracy improvement.

[0122] The information delivery unit can infer the user's emotions and determine the priority of the information provided based on the inferred emotions. For example, when the user is excited, the delivery unit prioritizes providing information that arouses interest. Similarly, when the user is relaxed, the delivery unit prioritizes providing information with a relaxing effect. Furthermore, when the user is stressed, the delivery unit can also prioritize providing information that helps reduce stress. Thus, the priority of the information provided can be determined based on the user's emotions. Emotion inference can be achieved, for example, through emotion inference functions such as emotion engines or generative AI. Generative AI can be text-generating AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the delivery unit can be performed by AI, or AI can be omitted. For example, the delivery unit can input the user's emotion data into the generative AI, which then performs the information prioritization decision.

[0123] The provisioning department can determine the priority of information provision based on the timing of information submission. For example, it may prioritize providing recently acquired information, or postpone providing information that has not been acquired for a long time. Furthermore, the provisioning department can dynamically adjust the provisioning priority based on the submission timing. Thus, the provisioning priority can be determined according to the submission timing. Some or all of the above processing in the provisioning department can be performed by AI, or it can be performed without AI. For example, the provisioning department can input the information submission timing data into AI, which will then execute the decision on the provisioning priority.

[0124] The provisioning department can adjust the delivery order based on the relevance of information during delivery. For example, the provisioning department may prioritize the delivery of highly relevant information, or postpone the delivery of less relevant information. Furthermore, the provisioning department can dynamically adjust the delivery order based on the relevance of information. Thus, the delivery order can be adjusted according to the relevance of information. Some or all of the above processing in the provisioning department can be performed by AI, or it may not require AI. For example, the provisioning department can input the relevance data of the information into the AI, which will then adjust the delivery order.

[0125] The provision department can adjust the use of technical terminology in the information based on the user's level of expertise. For example, it might use detailed technical terms for users with high expertise, or simple terms for users with low expertise. Furthermore, the provision department can dynamically adjust the use of technical terminology based on the user's level of expertise. Thus, it can adjust the use of technical terminology according to the user's level of expertise. Some or all of the above processing in the provision department can be performed using AI, or it can be done without AI. For example, the provision department can input the user's level of expertise data into AI, which can then execute the use of technical terminology.

[0126] ===Hardware Implementation 1-1===

[0127] Each of the aforementioned elements—acquisition unit, recognition unit, reference unit, and provision unit—is implemented, for example, by at least one of the smart device 14 and the data processing device 12. For example, the acquisition unit acquires viewer click information using the touch panel 38A of the smart device 14. The recognition unit, for example, is implemented by the specific processing unit 290 of the data processing device 12, using image recognition technology to identify the clicked item. The reference unit, for example, is implemented by the specific processing unit 290 of the data processing device 12, referencing the EC database 24 to acquire relevant information. The provision unit, for example, provides information to the viewer using the output device 40 of the smart device 14.

[0128] ===Hardware Implementation 1-2===

[0129] Each of the aforementioned elements—acquisition unit, recognition unit, reference unit, and providing unit—is implemented, for example, by at least one of the smart glasses 214 and the data processing device 12. For example, the acquisition unit acquires the viewer's click information using the touch panel of the smart glasses 214. The recognition unit, for example, is implemented by the specific processing unit 290 of the data processing device 12, using image recognition technology to identify the clicked item. The reference unit, for example, is implemented by the specific processing unit 290 of the data processing device 12, referencing the EC database 24 to acquire relevant information. The providing unit, for example, provides information to the viewer using the display of the smart glasses 214.

[0130] ===Hardware Implementation 1-3===

[0131] Each of the aforementioned elements—acquisition unit, recognition unit, reference unit, and provision unit—is implemented, for example, by at least one of the head-mounted terminal 314 and the data processing device 12. For example, the acquisition unit acquires viewer click information using the touch panel of the head-mounted terminal 314. The recognition unit, for example, is implemented by the specific processing unit 290 of the data processing device 12, using image recognition technology to identify the clicked item. The reference unit, for example, is implemented by the specific processing unit 290 of the data processing device 12, referencing the EC database 24 to acquire relevant information. The provision unit, for example, provides information to the viewer using the display of the head-mounted terminal 314.

[0132] ===Hardware Implementation 1-4===

[0133] Each of the aforementioned elements—acquisition unit, recognition unit, reference unit, and provision unit—is implemented, for example, by at least one of the robot 414 and the data processing device 12. For example, the acquisition unit acquires viewer click information using the touch panel of the robot 414. The recognition unit, for example, is implemented by a specific processing unit 290 of the data processing device 12, using image recognition technology to identify the clicked item. The reference unit, for example, is implemented by the specific processing unit 290 of the data processing device 12, referencing the EC database 24 to acquire relevant information. The provision unit, for example, provides information to the viewer using the display of the robot 414.

[0134] The system described in this embodiment is not limited to the examples above. For example, various modifications can be made as follows.

[0135] The acquisition department can analyze users' past purchase history and prioritize information on related items. For example, based on data from users' past purchases, the acquisition department can prioritize click information for similar items. Furthermore, the acquisition department can analyze users' purchase history to identify and prioritize interests in specific brands or categories. Moreover, based on users' purchase history, the acquisition department can prioritize information on items that align with seasons or trends. Thus, it can efficiently acquire relevant information based on users' purchase history.

[0136] The recognition unit can utilize user gaze tracking data to prioritize identifying items where the user's gaze lingers. For example, it can prioritize items where the user's gaze remains for an extended period. Furthermore, the unit can analyze gaze movement patterns to prioritize items the user might be interested in. Moreover, the unit can adjust recognition accuracy based on gaze concentration, improving the accuracy of identifying items where the user's gaze is focused. Thus, it can efficiently identify items of interest using user gaze data.

[0137] The reference unit can analyze users' activities on social media to obtain relevant information. For example, it can prioritize information about the brands or influencers users follow on social media. Furthermore, it can analyze users' posted content or comments to obtain information related to topics of interest. Additionally, it can refer to the activities of users' friends on social media to obtain relevant information. Thus, it can efficiently obtain relevant information based on social media activity.

[0138] The information delivery unit can adjust the way information is delivered based on the user's device usage. For example, when a user is using a smartphone, the information is delivered in a way that fits the screen size. Furthermore, when a user is using a tablet, the information can be delivered using a larger screen. Moreover, when a user is using a desktop computer, the information can be delivered using multiple windows. Thus, information can be delivered in the optimal way according to the user's device usage.

[0139] The acquisition unit can infer a user's emotions and determine the type of information to acquire based on these inferred emotions. For example, when a user is excited, the acquisition unit prioritizes acquiring entertainment-related information. Furthermore, when a user is relaxed, the acquisition unit can prioritize acquiring information with a relaxing effect. Moreover, when a user is stressed, the acquisition unit can prioritize acquiring information that helps reduce stress. Thus, the type of information acquired can be optimized according to the user's emotions.

[0140] The recognition unit can infer the user's emotions and determine the recognition priority based on the inferred emotions. For example, when the user is excited, the recognition unit prioritizes recognizing items that arouse interest. Furthermore, when the user is relaxed, the recognition unit can also prioritize recognizing items with a relaxing effect. Moreover, when the user is stressed, the recognition unit can also prioritize recognizing items that help relieve stress. Thus, the recognition priority can be determined based on the user's emotions.

[0141] The reference unit can infer a user's emotions and select a reference database based on those inferences. For example, when a user is excited, the reference unit prioritizes referencing entertainment-related databases. Furthermore, when a user is relaxed, the reference unit can prioritize referencing databases with relaxing effects. Moreover, when a user is stressed, the reference unit can prioritize referencing databases that help reduce stress. Thus, it is possible to select a reference database based on the user's emotions.

[0142] The information delivery department can infer the user's emotions and adjust the way information is delivered based on these inferences. For example, when the user is excited, the department can provide information in a visually stimulating manner. Furthermore, when the user is relaxed, the department can provide information in a calming manner. Moreover, when the user is stressed, the department can provide information in a concise and highly visual manner. Thus, the department can adjust the way information is delivered according to the user's emotions.

[0143] The information delivery unit can infer the user's emotions and prioritize the information provided based on these inferences. For example, when the user is excited, the unit prioritizes providing information that arouses interest. Furthermore, when the user is relaxed, the unit can prioritize providing information with a relaxing effect. Moreover, when the user is stressed, the unit can prioritize providing information that helps reduce stress. Thus, the unit can determine the priority of the information provided based on the user's emotions.

[0144] The acquisition department can take into account the user's geographical location information and prioritize acquiring highly relevant information. For example, when the user is located in a specific region, the acquisition department prioritizes acquiring click information for items related to that region. Furthermore, when the user is traveling, the acquisition department can also prioritize acquiring click information for items related to the travel destination. Moreover, when the user is at home, the acquisition department can also prioritize acquiring click information for items related to nearby stores or services. Thus, it is possible to acquire highly relevant information by taking geographical location information into account.

[0145] The following is a brief description of the processing flow of Implementation Method 2.

[0146] Step 1: The acquisition department obtains click information by observing viewers watching video content and clicking on items of interest on their devices. For example, the acquisition department can obtain information such as the location, time, and intensity of the click. Furthermore, the acquisition department can infer the user's emotions and adjust the timing of acquiring click information based on these inferred emotions.

[0147] Step 2: The recognition unit identifies the clicked item based on the click information acquired by the acquisition unit. For example, the recognition unit can use image recognition technology to identify the clicked item. Furthermore, the recognition unit can infer the user's sentiment and adjust the accuracy of item recognition based on the inferred user sentiment.

[0148] Step 3: The reference unit obtains relevant information based on the item information identified by the identification unit. For example, the reference unit can refer to the EC database and other relevant databases to obtain detailed product information, reviews, prices, etc. Furthermore, the reference unit can infer user sentiment and select a reference database based on the inferred user sentiment.

[0149] Step 4: The provision department provides the audience with information obtained from the reference department. For example, the provision department can provide information through pop-up windows, notifications, emails, etc. In addition, the provision department can infer the user's emotions and adjust the way information is provided based on the inferred user emotions.

[0150] The specific processing unit 290 sends the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires voice representing the user's input to the result of the specific processing. The control unit 46A sends the voice data representing the user's input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0151] Data generation model 58 is what is known as generative AI (Artificial Intelligence). An example of data generation model 58 includes ChatGPT (registered trademark) (Internet search).<URL:https: / / openai.com / blog / chatgpt> Generative AI, such as data generation model 58, is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, as well as inference data such as speech data (representing speech), text data (representing text), and image data (representing images). The data generation model 58 infers the input inference data according to the instructions shown in the prompts and outputs the inference results in the form of speech data, text data, etc. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the aforementioned specific processing while using the data generation model 58. The data generation model 58 can also be a fine-tuned model to output inference results from prompts without instructions; in this case, the data generation model 58 can output inference results from prompts without instructions. The data processing device 12, etc., includes various data generation models 58, including AI other than generative AI. AI other than generative AI includes, but is not limited to, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes. Furthermore, AI can also act as an AI agent. Additionally, when the processing described above is performed by AI, this processing may be partially or entirely performed by AI, but is not limited to this example. Moreover, processing performed by AI, including generative AI, can be replaced by rule-based processing, and vice versa.

[0152] Furthermore, the processing performed by the aforementioned data processing system 10 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but it can also be executed jointly by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Additionally, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart device 14 or external devices, and the smart device 14 acquires or collects information required for processing from the data processing device 12 or external devices.

[0153] The correspondence between the various parts and the devices or control units is not limited to the examples mentioned above, and various changes can be made.

[0154] Second Implementation Method

[0155] Figure 3 An example of the configuration of the data processing system 210 according to the second embodiment is shown.

[0156] like Figure 3 As shown, the data processing system 210 includes a data processing device 12 and smart glasses 214. One example of the data processing device 12 is a server.

[0157] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. An example of the network 54 includes a WAN and / or LAN, etc.

[0158] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. Furthermore, the microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0159] Microphone 238 receives user commands by receiving the user's voice. Microphone 238 captures the user's voice, converts the captured sound into speech data, and outputs it to processor 46. Speaker 240 outputs sound according to commands from processor 46.

[0160] Camera 42 is a small digital camera equipped with an optical system such as a lens, aperture and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge-Coupled Device) image sensor, used to capture the user's surroundings (e.g., the shooting range defined by an angle of view equivalent to the field of vision of an average healthy person).

[0161] Communication I / F 44 is connected to network 54. Communication I / F 44 and 26 are responsible for the transmission and reception of various information between processor 46 and processor 28 via network 54. The transmission and reception of various information between processor 46 and processor 28 using communication I / F 44 and 26 is performed in a secure state.

[0162] Figure 4 An example of the main functions of the data processing device 12 and the smart glasses 214 is shown. Figure 4 As shown, in the data processing device 12, specific processing is performed by the processor 28. The memory 32 stores a specific processing program 56.

[0163] The processor 28 reads a specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is implemented by the processor 28 operating as a specific processing unit 290 based on the specific processing program 56 executed on the RAM 30.

[0164] The memory 32 stores a data generation model 58 and an emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290. The specific processing unit 290 can use the emotion-specific model 59 to infer the user's emotions and perform specific processing based on the user's emotions. The emotion inference function using the emotion-specific model 59 (emotion-specific function) includes inferring and predicting the user's emotions, performing various inferences and predictions related to the user's emotions, but is not limited to this example. Furthermore, emotion inference and prediction also include, for example, emotion analysis (analysis).

[0165] In the smart glasses 214, specific processing is performed by the processor 46. A specific processing program 60 is stored in the memory 50. The processor 46 reads the specific processing program 60 from the memory 50 and executes the read specific processing program 60 on the RAM 48. Specific processing is implemented by the processor 46 operating as a control unit 46A based on the specific processing program 60 executed on the RAM 48. Furthermore, the smart glasses 214 may also have the same data generation model and emotion-specific model as the data generation model 58 and the emotion-specific model 59, and use these models to perform the same processing as the specific processing unit 290.

[0166] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains the processing results (prediction results, etc.) using the data generation model 58 by communicating with the server device that has the data generation model 58. In addition, the data processing device 12 may be a server device or a user-owned terminal device (e.g., a mobile phone, robot, home appliance, etc.).

[0167] The specific processing unit 290 sends the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires voice input representing the user's input to the specific processing result. The control unit 46A sends the voice data representing the user's input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0168] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 includes generative AIs such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing instructions is input to the data generation model 58, along with inference data such as speech data representing speech, text data representing text, and image data representing images. The data generation model 58 infers the input inference data according to the instructions shown in the prompt and outputs the inference result in the form of data such as speech data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the aforementioned specific processing while using the data generation model 58. The data generation model 58 can also be a fine-tuned model to output inference results from prompts without instructions; in this case, the data generation model 58 can output inference results from prompts without instructions. The data processing device 12, etc., includes various data generation models 58, including AI other than generative AI. AI other than generative AI includes, but is not limited to, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes. Furthermore, AI can also act as an AI agent. Additionally, when the processing described above is performed by AI, this processing may be partially or entirely performed by AI, but is not limited to this example. Moreover, processing performed by AI, including generative AI, can be replaced by rule-based processing, and vice versa.

[0169] The data processing system 210 of the second embodiment performs the same processing as the data processing system 10 of the first embodiment. The processing performed by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but it can also be executed jointly by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or external devices, and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or external devices.

[0170] The correspondence between the various parts and the devices or control units is not limited to the examples mentioned above, and various changes can be made.

[0171] Third Implementation Method

[0172] Figure 5 An example of the configuration of the data processing system 310 according to the third embodiment is shown.

[0173] like Figure 5 As shown, the data processing system 310 includes a data processing device 12 and a head-mounted terminal 314. An example of the data processing device 12 is a server.

[0174] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. An example of the network 54 includes a WAN and / or LAN, etc.

[0175] The head-mounted terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. Furthermore, the microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0176] Microphone 238 receives user commands by receiving the user's voice. Microphone 238 captures the user's voice, converts the captured sound into speech data, and outputs it to processor 46. Speaker 240 outputs sound according to commands from processor 46.

[0177] Camera 42 is a small digital camera equipped with an optical system such as a lens, aperture and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge-Coupled Device) image sensor, used to capture the user's surroundings (e.g., the shooting range defined by an angle of view equivalent to the field of vision of an average healthy person).

[0178] Communication I / F 44 is connected to network 54. Communication I / F 44 and 26 are responsible for the transmission and reception of various information between processor 46 and processor 28 via network 54. The transmission and reception of various information between processor 46 and processor 28 using communication I / F 44 and 26 is performed in a secure state.

[0179] Figure 6 An example of the main functions of the data processing device 12 and the head-mounted terminal 314 is shown. Figure 6 As shown, in the data processing device 12, specific processing is performed by the processor 28. The memory 32 stores a specific processing program 56.

[0180] The processor 28 reads a specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is implemented by the processor 28 operating as a specific processing unit 290 based on the specific processing program 56 executed on the RAM 30.

[0181] The memory 32 stores a data generation model 58 and an emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290. The specific processing unit 290 can use the emotion-specific model 59 to infer the user's emotions and perform specific processing based on the user's emotions. The emotion inference function using the emotion-specific model 59 (emotion-specific function) includes inferring and predicting the user's emotions, performing various inferences and predictions related to the user's emotions, but is not limited to this example. Furthermore, emotion inference and prediction also include, for example, emotion analysis (analysis).

[0182] In the head-mounted terminal 314, specific processing is performed by the processor 46. A specific program 60 is stored in the memory 50. The processor 46 reads the specific program 60 from the memory 50 and executes the read specific program 60 on the RAM 48. Specific processing is implemented by the processor 46 operating as a control unit 46A based on the specific program 60 executed on the RAM 48. Furthermore, the head-mounted terminal 314 may also have the same data generation model and emotion-specific model as the data generation model 58 and the emotion-specific model 59, and use these models to perform the same processing as the specific processing unit 290.

[0183] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains the processing results (prediction results, etc.) using the data generation model 58 by communicating with the server device that has the data generation model 58. In addition, the data processing device 12 may be a server device or a user-owned terminal device (e.g., a mobile phone, robot, home appliance, etc.).

[0184] The specific processing unit 290 sends the result of the specific processing to the head-mounted terminal 314. In the head-mounted terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires voice representing the user's input to the specific processing result. The control unit 46A sends the voice data representing the user's input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0185] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 includes generative AIs such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing instructions is input to the data generation model 58, along with inference data such as speech data representing speech, text data representing text, and image data representing images. The data generation model 58 infers the input inference data according to the instructions shown in the prompt and outputs the inference result in the form of data such as speech data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the aforementioned specific processing while using the data generation model 58. The data generation model 58 can also be a fine-tuned model to output inference results from prompts without instructions; in this case, the data generation model 58 can output inference results from prompts without instructions. The data processing device 12, etc., includes various data generation models 58, including AI other than generative AI. AI other than generative AI includes, but is not limited to, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes. Furthermore, AI can also act as an AI agent. Additionally, when the processing described above is performed by AI, this processing may be partially or entirely performed by AI, but is not limited to this example. Moreover, processing performed by AI, including generative AI, can be replaced by rule-based processing, and vice versa.

[0186] The data processing system 310 of the third embodiment performs the same processing as the data processing system 10 of the first embodiment. The processing performed by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the head-mounted terminal 314, but it can also be executed jointly by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the head-mounted terminal 314. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the head-mounted terminal 314 or external devices, and the head-mounted terminal 314 acquires or collects information required for processing from the data processing device 12 or external devices.

[0187] The correspondence between the various parts and the devices or control units is not limited to the examples mentioned above, and various changes can be made.

[0188] Fourth Implementation Method

[0189] Figure 7 An example of the configuration of the data processing system 410 according to the fourth embodiment is shown.

[0190] like Figure 7 As shown, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0191] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. An example of the network 54 includes a WAN and / or LAN, etc.

[0192] Robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control object 443. The computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. Furthermore, the microphone 238, speaker 240, camera 42, and control object 443 are also connected to the bus 52.

[0193] Microphone 238 receives user commands by receiving the user's voice. Microphone 238 captures the user's voice, converts the captured sound into speech data, and outputs it to processor 46. Speaker 240 outputs sound according to commands from processor 46.

[0194] The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture and shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, used to photograph the user's surroundings (e.g., the shooting range defined by an angle of view equivalent to the field of vision of an average healthy person).

[0195] Communication I / F 44 is connected to network 54. Communication I / F 44 and 26 are responsible for the transmission and reception of various information between processor 46 and processor 28 via network 54. The transmission and reception of various information between processor 46 and processor 28 using communication I / F 44 and 26 is performed in a secure state.

[0196] The controlled object 443 includes a display device, LEDs for the eyes, and motors for driving the arms, hands, and feet. The posture and movements of the robot 414 are controlled by controlling the motors for the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. In addition, facial expressions of the robot 414 can also be expressed by controlling the illumination state of the LEDs for the robot 414's eyes.

[0197] Figure 8An example of the main functions of the data processing device 12 and the robot 414 is shown. For example... Figure 8 As shown, in the data processing device 12, specific processing is performed by the processor 28. The memory 32 stores a specific processing program 56.

[0198] The processor 28 reads a specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is implemented by the processor 28 operating as a specific processing unit 290 based on the specific processing program 56 executed on the RAM 30.

[0199] The memory 32 stores a data generation model 58 and an emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290. The specific processing unit 290 can use the emotion-specific model 59 to infer the user's emotions and perform specific processing based on the user's emotions. The emotion inference function using the emotion-specific model 59 (emotion-specific function) includes inferring and predicting the user's emotions, performing various inferences and predictions related to the user's emotions, but is not limited to this example. Furthermore, emotion inference and prediction also include, for example, emotion analysis (analysis).

[0200] In robot 414, specific processing is performed by processor 46. A specific program 60 is stored in memory 50. Processor 46 reads the specific program 60 from memory 50 and executes the read specific program 60 on RAM 48. Specific processing is achieved by processor 46 acting as control unit 46A based on the specific program 60 executed on RAM 48. Furthermore, robot 414 may also have the same data generation model and emotion-specific model as data generation model 58 and emotion-specific model 59, and use these models to perform the same processing as specific processing unit 290.

[0201] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains the processing results (prediction results, etc.) using the data generation model 58 by communicating with the server device that has the data generation model 58. In addition, the data processing device 12 may be a server device or a user-owned terminal device (e.g., a mobile phone, robot, home appliance, etc.).

[0202] The specific processing unit 290 sends the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires voice representing the user's input regarding the result of the specific processing. The control unit 46A sends the voice data representing the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0203] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 includes generative AIs such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing instructions is input to the data generation model 58, along with inference data such as speech data representing speech, text data representing text, and image data representing images. The data generation model 58 infers the input inference data according to the instructions shown in the prompt and outputs the inference result in the form of data such as speech data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the aforementioned specific processing while using the data generation model 58. The data generation model 58 can also be a fine-tuned model to output inference results from prompts without instructions; in this case, the data generation model 58 can output inference results from prompts without instructions. The data processing device 12, etc., includes various data generation models 58, including AI other than generative AI. AI other than generative AI includes, but is not limited to, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes. Furthermore, AI can also act as an AI agent. Additionally, when the processing described above is performed by AI, this processing may be partially or entirely performed by AI, but is not limited to this example. Moreover, processing performed by AI, including generative AI, can be replaced by rule-based processing, and vice versa.

[0204] The data processing system 410 of the fourth embodiment performs the same processing as the data processing system 10 of the first embodiment. The processing performed by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but it can also be executed jointly by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or external devices, and the robot 414 acquires or collects information required for processing from the data processing device 12 or external devices.

[0205] The correspondence between the various parts and the devices or control units is not limited to the examples mentioned above, and various changes can be made.

[0206] Furthermore, the emotion-specific model 59, serving as an emotion engine, can determine the user's emotion based on a specific mapping. Specifically, the emotion-specific model 59 can determine the user's emotion based on an emotion graph that serves as a specific mapping (see...). Figure 9 The robot's emotions can be determined by the emotion-specific model 59. In addition, the emotion-specific model 59 can also determine the robot's emotions in the same way, and the specific processing unit 290 can also perform specific processing using the robot's emotions.

[0207] Figure 9 This is a diagram representing an emotion map 400 that maps various emotions. In the emotion map 400, emotions are arranged radially from the center in concentric circles. The closer to the center of the concentric circles, the more primitive the emotion is. Further out on the concentric circles, emotions are arranged representing states or actions arising from mood. Emotion is a concept that includes both feelings and mental states. To the left of the concentric circles, emotions generated by reactions occurring in the brain are arranged roughly. To the right of the concentric circles, emotions guided by situational judgments are arranged roughly. Above and below the concentric circles, emotions generated by reactions occurring in the brain and guided by situational judgments are arranged roughly. Furthermore, the emotion of "pleasure" is arranged above the concentric circles, and the emotion of "unpleasantness" is arranged below. Thus, in the emotion map 400, various emotions are mapped according to the structure of emotion generation, while easily generated emotions are mapped nearby.

[0208] These emotions are distributed at the 3 o'clock position on the Emotion Chart 400, and usually fluctuate between peace and unease. In the right half of the Emotion Chart 400, because situational awareness is more dominant than internal feelings, it gives a sense of calm.

[0209] The inner side of the emotion diagram 400 represents the mind, and the outer side of the emotion diagram 400 represents actions. Therefore, the further you go to the outer side of the emotion diagram 400, the more the emotion can be seen (manifested in actions).

[0210] Here, human emotions are based on a balance of various factors such as posture and blood sugar levels. When these balances deviate from the ideal, it indicates unhappiness; when they approach the ideal, it indicates pleasure. In robots, cars, and motorcycles, emotions can also be created based on a balance of factors such as posture and remaining battery power. When these balances deviate from the ideal, it indicates unhappiness; when they approach the ideal, it indicates pleasure. Emotion maps can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Speech Emotion Recognition and Brain Physiological Signal Analysis Systems for Emotion, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). In the left half of the emotion map, emotions belonging to the "Reaction" domain, where sensation is dominant, are arranged. Furthermore, in the right half of the emotion map, emotions belonging to the "Situation" domain, where situational cognition is dominant, are arranged.

[0211] The emotion map defines two types of emotions that promote learning. One is a negative emotion located near the middle of "repentance" or "reflection" on the situation side. That is, when the robot experiences negative emotions such as "I never want to feel this way again" or "I never want to be scolded again." The other is a positive emotion located near "desire" on the response side. That is, when the robot experiences positive feelings such as "wanting more" or "wanting to know more."

[0212] The emotion-specific model 59 feeds user input into a pre-learned neural network to obtain emotion values ​​representing each emotion shown in the emotion graph 400, and determines the user's emotion. This neural network is pre-learned based on multiple learning data sets that combine user input with emotion values ​​representing each emotion shown in the emotion graph 400. Furthermore, this neural network is learned to... Figure 10 As shown in sentiment graph 900, sentiment values ​​in nearby configurations are similar to each other. Figure 10 Examples show that multiple emotions such as "peace of mind", "stability", and "reassurance" have similar emotional values.

[0213] In the above embodiments, a specific processing is described by a single computer 22, but the technology disclosed herein is not limited to this, and distributed processing by multiple computers, including computer 22, is also possible.

[0214] In the above embodiments, an example of storing a specific processing program 56 in memory 32 is illustrated, but the technology disclosed herein is not limited thereto. For example, the specific processing program 56 may also be stored in a portable computer-readable non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed into the computer 22 of the data processing device 12. The processor 28 performs specific processing according to the specific processing program 56.

[0215] Alternatively, the specific processing program 56 can be stored in a storage device such as a server connected to the data processing device 12 via a network 54, and the specific processing program 56 can be downloaded and installed into the computer 22 upon request from the data processing device 12.

[0216] Furthermore, it is not necessary to store the entire specific process 56 in a storage device such as a server connected to the data processing device 12 via the network 54, nor is it necessary to store the entire specific process 56 in the memory 32; a portion of the specific process 56 may also be stored.

[0217] As a hardware resource for performing specific processing, various processors can be used. For example, a CPU is a general-purpose processor that functions as a hardware resource for performing specific processing by executing software, i.e., programs. Additionally, a dedicated circuit can be listed as a processor; it is a processor with a circuit structure specifically designed for performing specific processing, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit). Every processor has built-in or connected memory, and every processor executes specific processing by using memory.

[0218] The hardware resources for performing a specific process can consist of one of these various processors, or a combination of two or more processors of the same or different types (e.g., a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Furthermore, the hardware resources for performing a specific process can also be a single processor.

[0219] As an example of a single processor, the first type consists of a combination of one or more CPUs and software, which functions as a hardware resource to perform specific processing. The second type uses a processor, such as a System-on-a-chip (SoC), which implements the entire system functionality, including multiple hardware resources for performing specific processing, using a single IC chip. In this case, the specific processing is implemented using one or more of the aforementioned processors as hardware resources.

[0220] Furthermore, as the hardware architecture of these various processors, more specifically, circuits combining semiconductor elements and other circuit components can be used. Moreover, the specific process described above is merely an example. Therefore, it goes without saying that, without departing from the main point, unnecessary steps can be removed, new steps can be added, or the processing order can be changed.

[0221] Furthermore, although the above examples have been described in terms of first to fourth embodiments, some or all of these embodiments can be combined. Additionally, the smart device 14, smart glasses 214, head-mounted terminal 314, and robot 414 are just examples and can be combined separately, or other devices may be used. Furthermore, although the above examples have been described in terms of morphological example 1 and morphological example 2, these can also be combined.

[0222] The foregoing descriptions and illustrations are detailed explanations of the parts covered by this disclosure and are merely one example of this disclosure. For instance, the descriptions of the above-described structure, function, role, and effect are just one example of the structure, function, role, and effect of the parts covered by this disclosure. Therefore, it goes without saying that, without departing from the spirit of this disclosure, unnecessary parts can be deleted, new elements can be added, or replacements can be made to the foregoing descriptions and illustrations. Furthermore, to avoid confusion and facilitate understanding of the parts covered by this disclosure, explanations of technical common sense that does not require special explanation for implementing this disclosure have been omitted from the foregoing descriptions and illustrations.

[0223] All documents, patent applications and technical standards described in this specification are incorporated herein by reference as if they were specifically and individually described as incorporated by reference.

Claims

1. A system, characterized in that, include: The acquisition department is used to acquire click information; The identification unit is used to identify items based on the click information acquired by the acquisition unit; The reference unit is used to obtain relevant information based on the project information identified by the identification unit; A providing unit is used to provide information obtained from the reference unit.

2. The system as claimed in claim 1, wherein, The reference section refers to the EC database.

3. The system as described in claim 1, wherein, The reference section refers to other relevant databases.

4. The system as claimed in claim 1, wherein, The recognition unit identifies the clicked item.

5. The system as claimed in claim 1, wherein, The provision department provides information to the audience.

6. The system as claimed in claim 1, wherein, The acquisition unit infers the user's emotions and adjusts the timing of acquiring click information based on the inferred user emotions.

7. The system as claimed in claim 1, wherein, The acquisition unit analyzes the user's past click history and selects an appropriate acquisition method.

8. The system as claimed in claim 1, wherein, When acquiring click information, the acquisition unit filters based on the user's current viewing content and areas of interest.

9. The system as claimed in claim 1, wherein, When acquiring click information, the acquisition unit selects an appropriate acquisition method based on the user's input method.

Images