System

Abstract

To search for a product based on an object that comes into view, and generate and display an image of the product when it is worn.SOLUTION: The system according to an embodiment includes a recognition unit, a voice analysis unit, a retrieval unit, an image generation unit, and a display. The recognition unit recognizes an object that comes into view. The voice analysis unit analyzes a voice instruction based on the object recognized by the recognition unit. The retrieval unit searches for a product based on the instruction analyzed by the voice analysis unit. The image generation unit, on the basis of the product retrieved by the retrieval unit, generates an image of the product when it is worn. The display shows the image of the product when it is worn that is generated by the image generation unit.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional technologies do not allow for product search or generation of wearing images based on objects in view, and there is room for improvement.

[0005] The system according to the embodiment aims to search for products based on objects that come into view, and generate and display images of how the products would look when worn. [Means for solving the problem]

[0006] The system according to the embodiment includes a recognition unit, a voice analysis unit, a search unit, an image generation unit, and a display unit. The recognition unit recognizes an object that comes into view. The voice analysis unit analyzes a voice instruction based on the object recognized by the recognition unit. The search unit searches for a product based on the instruction analyzed by the voice analysis unit. The image generation unit generates a wearing image based on the product searched for by the search unit. The display unit displays the wearing image generated by the image generation unit. Effect of the Invention

[0007] The system according to the embodiment can search for products based on objects that come into view, and generate and display images of how the products would look when worn. [Brief description of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10]1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., voice and / or text). Display 40A displays visible information such as text and images according to instructions from processor 46. Speaker 40B outputs voice according to instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Also, the data processing device 12 may be a server device, or may be a terminal device (e.g., a mobile phone, a robot, a home appliance, etc.) owned by a user. Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) The E-commerce system according to the embodiment of the present invention is a system that realizes E-commerce starting from an object that comes into the field of view through AR glasses. This E-commerce system operates by a user wearing AR glasses and giving voice instructions while looking at an object that comes into the field of view. For example, when a user calls out, "Find me some shorts that go well with this T-shirt!", this voice instruction is analyzed by a language generation AI. Next, the language generation AI searches for a corresponding product from an E-commerce service based on the analysis result. For example, it searches for shorts from an E-commerce service. This search result is generated as a wearing image by an image generation AI. The generated wearing image is displayed on the display of the AR glasses. The user can consider purchasing while comparing the object that comes into the field of view with the generated wearing image. This realizes E-commerce that blends into a real environment. For example, while looking at a T-shirt on a shelf in his / her room, the user can easily find shorts that go well with the T-shirt. In addition, since the user can check how the product actually looks through the generated wearing image, it becomes easier to make a purchase decision. This allows the E-commerce system to search for a product through voice instructions starting from an object that comes into the field of view of the user, and generate and display a wearing image.

[0029] The E-commerce system according to the embodiment includes a recognition unit, a voice analysis unit, a search unit, an image generation unit, and a display unit. The recognition unit recognizes an object that has entered the field of view. The object that has entered the field of view includes, for example, an object within a specific distance and an object of a specific size or shape, but is not limited to such examples. The recognition unit, for example, takes a picture of the object that has entered the field of view using a camera, and identifies the object using image recognition technology. The recognition unit can also obtain position information of the object that has entered the field of view and use it to identify the object. The voice analysis unit analyzes a voice instruction based on the object recognized by the recognition unit. The voice instruction includes, for example, a voice command and an instruction in natural language, but is not limited to such examples. The voice analysis unit converts the voice instruction into text data using, for example, a voice recognition technology, and analyzes the instruction content using a natural language processing technology. The search unit searches for a product based on the instruction analyzed by the voice analysis unit. The search unit searches for a product from, for example, an E-commerce service. The E-commerce service includes, for example, categories such as clothing, electronic devices, and food, but is not limited to such examples. The search unit searches a product database using, for example, an API of an e-commerce service to acquire a corresponding product. The image generation unit generates a wearing image based on the product searched by the search unit. The image generation unit generates a wearing image using, for example, a generation AI. The generation AI includes, for example, a GAN (generative adversarial network) and a VAE (variational autoencoder), but is not limited to such examples. The image generation unit generates a wearing image by combining, for example, a product image and an image of a user. The display unit displays the wearing image generated by the image generation unit. The display unit displays, for example, the wearing image on a display of the AR glasses. This allows the user to consider purchasing while comparing an object in view with the generated wearing image. This allows the e-commerce system according to the embodiment to search for a product through a voice instruction starting from an object in view, and generate and display a wearing image.

[0030] The recognition unit recognizes an object that has entered the field of view. The object that has entered the field of view includes, for example, an object within a specific distance and an object of a specific size and shape, but is not limited to such examples. For example, the recognition unit uses a camera to capture an object that has entered the field of view, and identifies the object using image recognition technology. Specifically, the camera acquires a high-resolution image and extracts features such as the object's contour, color, and texture. These features are analyzed by an image recognition algorithm using deep learning, and the type and attributes of the object are identified. For example, if the object that has entered the field of view is clothing, its type (shirt, pants, dress, etc.), color, brand, etc. are recognized. The recognition unit can also acquire position information of the object that has entered the field of view and use it to identify the object. The position information is measured using the viewing angle of the camera and a distance sensor, and the three-dimensional position of the object is identified. This allows the user to accurately grasp where the object that has entered the field of view is located. Furthermore, the recognition unit can combine multiple cameras and sensors to achieve more accurate object recognition. For example, a stereo camera is used to acquire depth information of the object and recognize the three-dimensional shape of the object. In addition, the infrared sensor can be used to accurately recognize objects even in dark places, backlit areas, etc. This enables the recognition unit to achieve highly accurate object recognition in a variety of environments and conditions, improving the reliability of the entire system.

[0031] The voice analysis unit analyzes the voice instruction based on the object recognized by the recognition unit. The voice instruction includes, for example, a voice command or an instruction in natural language, but is not limited to such examples. The voice analysis unit converts the voice instruction into text data using, for example, a voice recognition technology, and analyzes the instruction content using a natural language processing technology. Specifically, the voice recognition technology analyzes the user's utterance in real time and converts the voice signal into text data. At this time, noise reduction and voice enhancement technologies are used to remove environmental sounds and noise and improve the clarity of the voice. The converted text data is analyzed using a natural language processing technology, and the user's intention and request are identified. For example, a voice instruction such as "Show me different colors of this shirt" is analyzed, and an instruction to search for products with different colors of the shirt is generated. The voice analysis unit can perform more accurate analysis by taking into account the context of the user's utterance and past search history. For example, the voice analysis unit learns the user's preferences and tendencies based on products searched for by the user in the past and purchase history, and provides more appropriate search results. In addition, the voice analysis unit can handle multiple languages ​​and dialects, and can analyze voice instructions with high accuracy even for users around the world. This allows the voice analysis unit to quickly and accurately analyze the user's voice instructions, improving the operability and convenience of the entire system.

[0032] The search unit searches for products based on the instruction analyzed by the voice analysis unit. The search unit searches for products from, for example, an e-commerce service. The e-commerce service includes categories such as clothing, electronic devices, and food, but is not limited to such examples. The search unit, for example, uses an API of the e-commerce service to search a product database and acquires a corresponding product. Specifically, based on the instruction received from the voice analysis unit, a query is generated for the product database and the corresponding product is searched for. The search query may include conditions such as product category, brand, price range, color, and size. For example, a product is searched for based on specific conditions such as "blue shirt" or "latest smartphone". The search unit can acquire search results in real time and provide them to the user. Furthermore, the search unit can provide personalized search results by taking into account the user's past search history and purchase history. For example, the search unit preferentially displays products of a brand that the user has purchased in the past, or recommends products that match the user's preferences. Furthermore, the search unit can search multiple e-commerce services across the board and provide the user with the most suitable product. As a result, the search unit can quickly and accurately respond to the user's request and improve the convenience and satisfaction of the entire system.

[0033] The image generation unit generates a wearing image based on the product searched by the search unit. The image generation unit generates the wearing image using, for example, a generating AI. The generating AI includes, for example, a GAN (generative adversarial network) and a VAE (variational autoencoder), but is not limited to such examples. Specifically, the generating AI receives a product image and a user's image as input, and combines them to generate a realistic wearing image. For example, the user provides a full-body photo of himself / herself, and combines it with the searched product image of the clothing to generate an image of the user wearing the clothing. The generating AI can generate a natural wearing image that matches the user's body type and posture using deep learning. Furthermore, the generating AI can also generate a wearing image at different angles and under different lighting conditions, providing the user with an opportunity to check the product from various perspectives. In addition, the generating AI can learn the user's preferences and past selection history and generate a more personalized wearing image. As a result, the image generation unit can provide the user with a realistic and attractive wearing image, thereby increasing the user's desire to purchase.

[0034] The display unit displays the wearing image generated by the image generation unit. The display unit displays the wearing image on, for example, the display of the AR glasses. Specifically, the display of the AR glasses displays the wearing image superimposed on the user's field of vision, providing an experience as if the user were actually wearing the product. The AR glasses can track the movement of the user's head and line of sight, and update the wearing image in real time according to the field of vision. This allows the user to consider purchasing while comparing the object in view with the generated wearing image. Furthermore, the display unit can also display the wearing image on the display of a smartphone or tablet. This allows even users who do not have AR glasses to easily check the wearing image. The display unit supports operations such as zooming in and out and rotating the wearing image according to the user's operation, enabling detailed confirmation. In addition, the display unit simultaneously displays information, prices, reviews, etc. of products related to the wearing image, providing reference information when the user considers purchasing. This allows the display unit to provide the user with a visually attractive and information-rich experience, improving the convenience and satisfaction of the entire E-commerce system.

[0035] The E-commerce system includes a reception unit that receives voice instructions. The reception unit receives the voice instructions. The voice instructions include, for example, voice commands and instructions in natural language, but are not limited to such examples. The reception unit, for example, collects the user's voice using a microphone and converts the voice instructions into text data using a voice recognition technology. The voice recognition technology includes, for example, a voice recognition model using deep learning, but is not limited to such examples. The reception unit receives the voice instructions, thereby facilitating the user's operation. As a result, the E-commerce system receives the voice instructions, thereby facilitating the user's operation. A part or all of the above-mentioned processing in the reception unit may be performed, for example, using AI, or may be performed without using AI. For example, the reception unit can input the voice instructions to AI and cause the AI ​​to analyze the voice instructions.

[0036] The image generation unit can generate a wearing image by a generating AI. The image generation unit generates a wearing image by using, for example, a generating AI. The generating AI includes, for example, a GAN (generative adversarial network) and a VAE (variational autoencoder), but is not limited to such examples. The image generation unit generates a wearing image by combining, for example, a product image and a user's image. For example, the generating AI receives a product image as an input and combines it with the user's image to generate a wearing image. The generating AI fuses the product image and the user's image by using, for example, a GAN to generate a realistic wearing image. The generating AI can also encode the product image and the user's image by using a VAE, and decode it to generate a wearing image. As a result, the generation accuracy of the wearing image is improved by using the generating AI. A part or all of the above-mentioned processing in the image generation unit may be performed by using, for example, an AI, or may be performed without using an AI. For example, the image generation unit can input the product image and the user's image to the AI ​​and cause the AI ​​to generate the wearing image.

[0037] The search unit can search for products from an e-commerce service. The search unit, for example, searches for products from an e-commerce service. The e-commerce service includes categories such as clothing, electronic devices, and food, but is not limited to such examples. The search unit, for example, uses an API of the e-commerce service to search a product database and acquires a corresponding product. For example, the search unit transmits a query to the API of the e-commerce service based on a voice instruction of the user, and acquires a list of corresponding products. The search unit can also analyze the acquired list of products and select a product that meets the needs of the user. In this way, a product that meets the needs of the user can be provided by searching for a product from the e-commerce service. A part or all of the above-mentioned processing in the search unit may be performed, for example, using an AI, or may be performed without using an AI. For example, the search unit can input a voice instruction of the user to an AI and cause the AI ​​to execute a product search.

[0038] The display unit can display the generated wearing image on the display of the AR glasses. The display unit, for example, displays the generated wearing image on the display of the AR glasses. The AR glasses include, for example, HoloLens and Magic Leap, but are not limited to such examples. The display unit, for example, displays the generated wearing image on the display of the AR glasses in real time, so that the user can visually confirm it. For example, the display unit overlays the generated wearing image on the display of the AR glasses, so that the user can compare it with an object that has entered the field of view. The display unit can also display the generated wearing image by enlarging / reducing or rotating it. As a result, the user can visually confirm it by displaying it on the display of the AR glasses. A part or all of the above-mentioned processing in the display unit may be performed, for example, using AI, or may be performed without using AI. For example, the display unit can input the generated wearing image to AI and cause AI to adjust the display method.

[0039] The recognition unit can recognize an object that has entered the field of view. The recognition unit, for example, uses a camera to capture an object that has entered the field of view, and identifies the object using image recognition technology. The object that has entered the field of view includes, for example, an object within a specific distance and an object of a specific size or shape, but is not limited to such examples. The recognition unit, for example, analyzes features such as the shape, color, and texture of the object to identify the object. The recognition unit can also obtain position information of the object that has entered the field of view and use it to identify the object. For example, the recognition unit identifies the position of the object based on the field of view information of the camera, improving the recognition accuracy of the object. As a result, by recognizing the object that has entered the field of view, the user's operation becomes intuitive. A part or all of the above-mentioned processing in the recognition unit may be performed, for example, using AI, or may be performed without using AI. For example, the recognition unit can input image data of the object that has entered the field of view to AI, and cause the AI ​​to recognize the object.

[0040] The recognition unit can set the recognition priority based on the frequency of use of the object at the time of recognition. The recognition unit sets the recognition priority based on, for example, the frequency of use of the object. The frequency of use of the object includes, for example, past use history and sensor data, but is not limited to such examples. The recognition unit, for example, preferentially recognizes objects that are used frequently based on past use history. For example, the recognition unit preferentially recognizes objects that are used frequently and postpones objects that are used less frequently. The recognition unit can also measure the frequency of use based on sensor data and dynamically change the recognition priority. For example, the recognition unit analyzes sensor data in real time and preferentially recognizes objects that are used frequently. As a result, important objects can be preferentially recognized by setting the recognition priority based on the frequency of use of the object. A part or all of the above-mentioned processing in the recognition unit may be performed using, for example, AI, or may be performed without using AI. For example, the recognition unit can input the frequency of use data of the object to AI and cause the AI ​​to execute the setting of the recognition priority.

[0041] The recognition unit can improve the recognition accuracy based on the state of the object at the time of recognition. The recognition unit improves the recognition accuracy based on, for example, the state of the object. The state of the object includes, for example, newness, dirtiness, degree of damage, etc., but is not limited to such examples. For example, the recognition unit recognizes a new object with high accuracy, and recognizes a dirty object by adjusting the recognition accuracy. For example, the recognition unit evaluates the state of the object in real time and dynamically adjusts the recognition accuracy. For example, the recognition unit photographs the state of the object with a camera and evaluates the state using image analysis technology. The recognition unit can also adjust the recognition algorithm based on the state of the object to improve the recognition accuracy. As a result, the recognition accuracy is improved by taking the state of the object into consideration. A part or all of the above-mentioned processing in the recognition unit may be performed using, for example, AI, or may be performed without using AI. For example, the recognition unit can input state data of the object to AI and cause AI to adjust the recognition accuracy.

[0042] The recognition unit can improve the recognition accuracy based on the position information of the object at the time of recognition. The recognition unit improves the recognition accuracy based on, for example, the position information of the object. The position information of the object includes, for example, GPS data and field of view information of the camera, but is not limited to such examples. The recognition unit adjusts the recognition accuracy based on, for example, the position information of the object. For example, the recognition unit improves the recognition accuracy when the object is in a specific position. The recognition unit can also acquire the position information of the object in real time and dynamically adjust the recognition accuracy. For example, the recognition unit identifies the position of the object based on the field of view information of the camera and improves the recognition accuracy. As a result, the recognition accuracy is improved by taking into account the position information of the object. A part or all of the above-mentioned processing in the recognition unit may be performed using, for example, AI, or may be performed without using AI. For example, the recognition unit can input the position information of the object to the AI ​​and cause the AI ​​to adjust the recognition accuracy.

[0043] The recognition unit can improve the recognition accuracy based on the related information of the object at the time of recognition. The recognition unit improves the recognition accuracy based on, for example, the related information of the object. The related information of the object includes, for example, the manufacturer, the purchase date, the use history, etc., but is not limited to such examples. The recognition unit adjusts the recognition accuracy based on, for example, the manufacturer information of the object. For example, the recognition unit adjusts the recognition accuracy based on the purchase date information of the object. The recognition unit can also acquire the related information of the object in real time and dynamically adjust the recognition accuracy. For example, the recognition unit improves the recognition accuracy based on the use history of the object. As a result, the recognition accuracy is improved by referring to the related information of the object. A part or all of the above-mentioned processing in the recognition unit may be performed using, for example, AI, or may be performed without using AI. For example, the recognition unit can input the related information of the object to the AI ​​and cause the AI ​​to adjust the recognition accuracy.

[0044] The voice analysis unit can improve the analysis accuracy by referring to the user's past voice instruction history during voice analysis. The voice analysis unit improves the analysis accuracy by referring to the user's past voice instruction history, for example. The user's past voice instruction history includes, but is not limited to, log data and voice records. The voice analysis unit adjusts the analysis accuracy based on the user's past voice instruction history, for example. For example, the voice analysis unit preferentially analyzes voice instructions frequently used by the user. The voice analysis unit can also dynamically adjust the analysis accuracy by referring to the user's voice instruction history in real time. For example, the voice analysis unit improves the analysis accuracy of a specific voice instruction based on the past voice instruction history. As a result, the analysis accuracy is improved by referring to the user's past voice instruction history. A part or all of the above-mentioned processing in the voice analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the voice analysis unit can input the user's past voice instruction history to AI and cause the AI ​​to improve the analysis accuracy.

[0045] The voice analysis unit can improve the analysis accuracy based on the user's speech speed and tone during voice analysis. The voice analysis unit improves the analysis accuracy based on, for example, the user's speech speed and tone. The user's speech speed and tone include, for example, a voice analysis algorithm and an acoustic feature, but are not limited to such examples. The voice analysis unit adjusts the analysis accuracy based on, for example, the user's speech speed. For example, the voice analysis unit adjusts the analysis accuracy based on the user's tone. The voice analysis unit can also analyze the user's speech speed and tone in real time and dynamically adjust the analysis accuracy. For example, the voice analysis unit analyzes the user's speech speed and tone to improve the analysis accuracy of a specific voice instruction. As a result, the analysis accuracy is improved by taking into account the user's speech speed and tone. A part or all of the above-mentioned processing in the voice analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the voice analysis unit can input data on the user's speech speed and tone to AI and cause the AI ​​to improve the analysis accuracy.

[0046] The voice analysis unit can improve the analysis accuracy based on the geographical location information of the user during voice analysis. The voice analysis unit improves the analysis accuracy based on, for example, the geographical location information of the user. The geographical location information of the user includes, for example, GPS data and a location sensor, but is not limited to such examples. The voice analysis unit adjusts the analysis accuracy based on, for example, the geographical location information of the user. For example, the voice analysis unit improves the analysis accuracy when the user is in a specific location. In addition, the voice analysis unit can acquire the geographical location information of the user in real time and dynamically adjust the analysis accuracy. For example, the voice analysis unit improves the analysis accuracy of a specific voice instruction based on the geographical location information of the user. As a result, the analysis accuracy is improved by taking into account the geographical location information of the user. A part or all of the above-mentioned processing in the voice analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the voice analysis unit can input the geographical location information of the user to AI and cause the AI ​​to improve the analysis accuracy.

[0047] The voice analysis unit can analyze the social media activity of the user during the voice analysis and analyze the related voice instructions preferentially. The voice analysis unit, for example, analyzes the social media activity of the user and analyzes the related voice instructions preferentially. The social media activity of the user includes, for example, the content of posts, the number of likes, the number of followers, etc., but is not limited to such examples. The voice analysis unit adjusts the analysis accuracy, for example, based on the social media activity of the user. For example, the voice analysis unit preferentially analyzes the voice instructions frequently used by the user. The voice analysis unit can also analyze the social media activity of the user in real time and dynamically adjust the analysis accuracy. For example, the voice analysis unit improves the analysis accuracy of a specific voice instruction based on the social media activity of the user. As a result, the related voice instructions can be analyzed preferentially by analyzing the social media activity of the user. A part or all of the above-mentioned processing in the voice analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the voice analysis unit can input the social media activity data of the user to AI and cause the AI ​​to improve the analysis accuracy.

[0048] The search unit can improve search accuracy based on the user's past purchase history at the time of search. The search unit improves search accuracy based on the user's past purchase history, for example. The user's past purchase history includes, but is not limited to, purchase history data and receipt information. The search unit adjusts search accuracy based on the user's past purchase history, for example. For example, the search unit preferentially searches for products that the user frequently purchases. The search unit can also dynamically adjust search accuracy by referring to the user's purchase history in real time. For example, the search unit improves search accuracy for a specific product category based on the past purchase history. As a result, the search accuracy is improved by referring to the user's past purchase history. A part or all of the above-mentioned processing in the search unit may be performed using, for example, AI, or may be performed without using AI. For example, the search unit can input the user's past purchase history into AI and cause the AI ​​to improve search accuracy.

[0049] The search unit can filter search results based on the user's current field of interest at the time of search. The search unit, for example, filters search results based on the user's current field of interest. The user's current field of interest includes, for example, search history, browsing history, survey results, etc., but is not limited to such examples. The search unit, for example, filters search results based on the user's current field of interest. For example, if the user is interested in a specific field of interest, the search unit preferentially searches for products in that field. The search unit can also obtain the user's field of interest in real time and dynamically filter search results. For example, the search unit preferentially displays highly relevant products based on the user's search history and browsing history. As a result, highly relevant results can be provided by filtering search results based on the user's current field of interest. A part or all of the above-mentioned processing in the search unit may be performed using, for example, AI, or may be performed without using AI. For example, the search unit can input the user's field of interest data to AI and cause the AI ​​to filter the search results.

[0050] The search unit can search for highly relevant products preferentially based on the geographical location information of the user at the time of search. The search unit preferentially searches for highly relevant products based on the geographical location information of the user, for example. The geographical location information of the user includes, but is not limited to, GPS data and a location sensor. The search unit adjusts the search accuracy, for example, based on the geographical location information of the user. For example, when the user is in a specific location, the search unit preferentially searches for products related to the location. The search unit can also acquire the geographical location information of the user in real time and dynamically adjust the search accuracy. For example, the search unit improves the search accuracy of products related to a specific area based on the geographical location information of the user. This makes it possible to preferentially search for highly relevant products by considering the geographical location information of the user. A part or all of the above-mentioned processing in the search unit may be performed, for example, using AI, or may be performed without using AI. For example, the search unit can input the geographical location information of the user to AI and cause the AI ​​to improve the search accuracy.

[0051] The search unit can analyze the user's social media activity at the time of search and search for related products with priority. The search unit, for example, analyzes the user's social media activity and searches for related products with priority. The user's social media activity includes, for example, the content of posts, the number of likes, the number of followers, etc., but is not limited to such examples. The search unit, for example, adjusts search accuracy based on the user's social media activity. For example, the search unit searches for products in which the user is frequently interested with priority. The search unit can also analyze the user's social media activity in real time and dynamically adjust search accuracy. For example, the search unit improves search accuracy for a specific product category based on the user's social media activity. As a result, related products can be searched with priority by analyzing the user's social media activity. A part or all of the above-mentioned processing in the search unit may be performed using, for example, AI, or may be performed without using AI. For example, the search unit can input the user's social media activity data into AI and cause the AI ​​to improve search accuracy.

[0052] The image generation unit can improve the generation accuracy based on the user's past image generation history when generating an image. The image generation unit improves the generation accuracy based on the user's past image generation history, for example. The user's past image generation history includes, for example, data of generated images and a log of the generation history, but is not limited to such examples. The image generation unit adjusts the generation accuracy based on the user's past image generation history, for example. For example, the image generation unit preferentially generates images in a style that the user frequently generates. The image generation unit can also dynamically adjust the generation accuracy by referring to the user's image generation history in real time. For example, the image generation unit improves the image generation accuracy of a specific style based on the past image generation history. As a result, the generation accuracy is improved by referring to the user's past image generation history. A part or all of the above-mentioned processing in the image generation unit may be performed using, for example, AI, or may be performed without using AI. For example, the image generation unit can input the user's past image generation history to AI and cause the AI ​​to execute the improvement of the generation accuracy.

[0053] The image generating unit can customize the image to be generated based on the user's current fashion trend at the time of image generation. The image generating unit customizes the image to be generated based on the user's current fashion trend, for example. The user's current fashion trend includes, but is not limited to, for example, a subscription history of a fashion magazine and a post content of a social media. The image generating unit customizes the image based on, for example, the user's current fashion trend. For example, if the user is interested in a particular fashion trend, the image generating unit generates an image that matches the trend. The image generating unit can also obtain the user's fashion trend in real time and dynamically customize the image. For example, the image generating unit generates a highly relevant image based on the user's subscription history of a fashion magazine and a post content of a social media. As a result, a more appropriate image can be generated by customizing the image based on the user's current fashion trend. A part or all of the above-mentioned processing in the image generating unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the image generating unit can input the user's fashion trend data into the AI ​​and cause the AI ​​to execute image customization.

[0054] The image generating unit can customize the image to be generated based on the geographical location information of the user when generating the image. The image generating unit customizes the image to be generated based on, for example, the geographical location information of the user. The geographical location information of the user includes, for example, GPS data and a location sensor, but is not limited to such examples. The image generating unit customizes the image based on, for example, the geographical location information of the user. For example, when the user is in a specific location, the image generating unit generates an image related to the location. In addition, the image generating unit can obtain the geographical location information of the user in real time and dynamically customize the image. For example, the image generating unit generates an image related to a specific area based on the geographical location information of the user. As a result, a more appropriate image can be generated by taking into account the geographical location information of the user. A part or all of the above-mentioned processing in the image generating unit may be performed using, for example, AI, or may be performed without using AI. For example, the image generating unit can input the geographical location information of the user to the AI ​​and cause the AI ​​to execute the customization of the image.

[0055] The image generation unit can analyze the social media activity of the user and generate a related image when generating an image. The image generation unit, for example, analyzes the social media activity of the user and generates a related image. The social media activity of the user includes, for example, the content of posts, the number of likes, the number of followers, etc., but is not limited to such examples. The image generation unit, for example, customizes an image based on the social media activity of the user. For example, the image generation unit generates an image related to a theme in which the user is frequently interested. The image generation unit can also analyze the social media activity of the user in real time and dynamically customize the image. For example, the image generation unit generates an image related to a specific theme based on the social media activity of the user. In this way, a related image can be generated by analyzing the social media activity of the user. A part or all of the above-mentioned processing in the image generation unit may be performed, for example, using AI, or may be performed without using AI. For example, the image generation unit can input the social media activity data of the user to AI and cause the AI ​​to generate an image.

[0056] The display unit can select an optimal display method based on the user's past display history at the time of display. The display unit selects an optimal display method based on the user's past display history, for example. The user's past display history includes, for example, a log of the displayed content and data of the display history, but is not limited to such an example. The display unit adjusts the display method based on the user's past display history, for example. For example, the display unit preferentially provides a display method that the user frequently uses. The display unit can also dynamically adjust the display method by referring to the user's display history in real time. For example, the display unit improves the accuracy of a specific display method based on the past display history. As a result, the optimal display method can be selected by referring to the user's past display history. A part or all of the above-mentioned processing in the display unit may be performed using, for example, AI, or may be performed without using AI. For example, the display unit can input the user's past display history to AI and cause the AI ​​to select the display method.

[0057] The display unit can adjust the display method based on the user's current visual state at the time of display. The display unit adjusts the display method based on the user's current visual state, for example. The user's current visual state includes, but is not limited to, for example, a visual acuity test result and an evaluation of eye fatigue. For example, the display unit provides a display method with high visibility when the user is tired. For example, the display unit provides a normal display method when the user is relaxed. The display unit can also acquire the user's visual state in real time and dynamically adjust the display method. For example, the display unit improves the accuracy of a specific display method based on the user's visual state. This enables a display with high visibility by taking the user's visual state into consideration. A part or all of the above-mentioned processing in the display unit may be performed, for example, using AI, or may be performed without using AI. For example, the display unit can input the user's visual state data to AI and cause the AI ​​to adjust the display method.

[0058] The display unit can select an optimal display method based on the user's device information at the time of display. The display unit selects an optimal display method based on, for example, the user's device information. The user's device information includes, for example, the type of device, the OS version, and the usage status, but is not limited to such examples. For example, when the user is using a smartphone, the display unit provides a display method that matches the screen size. For example, when the user is using a tablet, the display unit provides a display method optimized for a large screen. In addition, when the user is using a smart watch, the display unit can also provide a simple and highly visible display method. For example, the display unit selects a display method optimized for a specific device based on the user's device information. This makes it possible to select an optimal display method by considering the user's device information. A part or all of the above-mentioned processing in the display unit may be performed, for example, using AI, or may be performed without using AI. For example, the display unit can input the user's device information to AI and cause the AI ​​to select the display method.

[0059] The display unit can analyze the user's social media activity at the time of display and preferentially display related display content. The display unit, for example, analyzes the user's social media activity and preferentially displays related display content. The user's social media activity includes, for example, the content of posts, the number of likes, the number of followers, etc., but is not limited to such examples. The display unit adjusts the display content, for example, based on the user's social media activity. For example, the display unit preferentially displays display content related to a theme in which the user is frequently interested. The display unit can also analyze the user's social media activity in real time and dynamically adjust the display content. For example, the display unit preferentially displays display content related to a specific theme based on the user's social media activity. As a result, the display unit can preferentially display related display content by analyzing the user's social media activity. A part or all of the above-mentioned processing in the display unit may be performed, for example, using AI, or may be performed without using AI. For example, the display unit can input the user's social media activity data into AI and cause AI to adjust the display content.

[0060] The reception unit can select the optimal reception method based on the user's past reception history at the time of reception. The reception unit selects the optimal reception method based on the user's past reception history, for example. The user's past reception history includes, for example, a log of reception contents and reception history data, but is not limited to such examples. The reception unit adjusts the reception method based on the user's past reception history, for example. For example, the reception unit preferentially provides a reception method that the user uses frequently. The reception unit can also dynamically adjust the reception method by referring to the user's reception history in real time. For example, the reception unit improves the accuracy of a specific reception method based on the past reception history. As a result, the optimal reception method can be selected by referring to the user's past reception history. A part or all of the above-mentioned processing in the reception unit may be performed using, for example, AI, or may be performed without using AI. For example, the reception unit can input the user's past reception history into AI and have the AI ​​select the reception method.

[0061] The reception unit can select the optimal reception method based on the device information of the user at the time of reception. The reception unit selects the optimal reception method based on, for example, the device information of the user. The device information of the user includes, for example, the type of device, the version of the OS, the usage status, etc., but is not limited to such examples. For example, when the user is using a smartphone, the reception unit provides a reception method that matches the screen size. For example, when the user is using a tablet, the reception unit provides a reception method optimized for a large screen. In addition, when the user is using a smart watch, the reception unit can also provide a simple and highly visible reception method. For example, the reception unit selects a reception method optimized for a specific device based on the device information of the user. This makes it possible to select the optimal reception method by considering the device information of the user. A part or all of the above-mentioned processing in the reception unit may be performed using, for example, AI, or may be performed without using AI. For example, the reception unit can input the device information of the user to AI and cause the AI ​​to select the reception method.

[0062] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0063] The recognition unit can also analyze the material of an object that comes into view and make product suggestions based on the material. For example, the recognition unit analyzes an image of an object captured by a camera and identifies the material of the object, such as cloth, metal, or plastic. Next, the search unit searches for related products based on the identified material. For example, if a cloth object is recognized, the search unit preferentially searches for products related to cloth products. The recognition unit can also suggest appropriate maintenance methods to the user based on the material of the object. This allows the user to obtain products and information according to the material of the object.

[0064] The reception unit can also analyze background sounds of the user's voice instructions and improve the accuracy of the voice instructions based on the background sounds. For example, the reception unit can analyze noise or music in the background of the voice instructions and use noise canceling technology to make the voice instructions clear. The reception unit can also increase the sensitivity of the voice instructions when the background sounds are quiet. Furthermore, the reception unit can estimate the user's situation based on the background sounds and select an appropriate method of analyzing the voice instructions. This allows the user to give accurate voice instructions in any environment.

[0065] The search unit can also display reviews and ratings of related products based on the user's purchasing history. For example, the search unit collects reviews and ratings of products previously purchased by the user and displays them in the search results. This allows the user to make better purchasing decisions by referring to the opinions of other users. The search unit can also filter reviews and ratings of related products based on the user's purchasing history to provide reliable information. This allows the user to select products based on reliable information.

[0066] The recognition unit can also analyze the movement of an object that comes into view and identify the type of object based on the movement. For example, the recognition unit can analyze the movement of an object captured by a camera and identify the type of object, such as an animal, a vehicle, or a human. Next, the search unit searches for related products and information based on the identified type of object. For example, if an animal is recognized, the search unit preferentially searches for products related to pet supplies. The recognition unit can also suggest appropriate actions to the user based on the movement of the object. This allows the user to obtain products and information according to the movement.

[0067] The recognition unit can also analyze the temperature of an object that comes into view and identify the state of the object based on the temperature. For example, the recognition unit can measure the temperature of the object using an infrared camera and identify whether the object is cold, warm, hot, or other such state. The search unit then searches for related products and information based on the identified temperature. For example, if a hot object is recognized, the search unit will preferentially search for products related to cooling products. The recognition unit can also suggest appropriate actions to the user based on the temperature of the object. This allows the user to obtain products and information according to the temperature.

[0068] The recognition unit can also analyze the sound of an object that comes into view and identify the type of the object based on the sound. For example, the recognition unit collects sounds emitted from the object using a microphone and identifies the type of the object using voice analysis technology. Next, the search unit searches for related products and information based on the identified type of object. For example, if a musical instrument is recognized, the search unit preferentially searches for products related to the musical instrument. The recognition unit can also suggest appropriate actions to the user based on the sound of the object. This allows the user to obtain products and information according to the sound.

[0069] The process flow of the first embodiment will be briefly described below.

[0070] Step 1: The recognition unit recognizes an object that comes into view. Examples of objects that come into view include, but are not limited to, objects within a certain distance or objects of a certain size or shape. The recognition unit, for example, uses a camera to capture an image of the object that comes into view and identifies the object using image recognition technology. The recognition unit can also obtain position information of the object that comes into view and use it to identify the object. Step 2: The voice analysis unit analyzes the voice instruction based on the object recognized by the recognition unit. The voice instruction includes, but is not limited to, a voice command or an instruction in natural language. The voice analysis unit converts the voice instruction into text data using, for example, a voice recognition technology, and analyzes the instruction content using a natural language processing technology. Step 3: The search unit searches for products based on the instruction analyzed by the voice analysis unit. The search unit searches for products from, for example, an e-commerce service. The e-commerce service includes categories such as, but is not limited to, clothing, electronic devices, and food. The search unit searches a product database using, for example, an API of the e-commerce service to obtain a corresponding product. Step 4: The image generation unit generates a wearing image based on the product searched by the search unit. The image generation unit generates a wearing image using, for example, a generation AI. The generation AI includes, for example, a GAN (generative adversarial network) and a VAE (variational autoencoder), but is not limited to such examples. The image generation unit generates a wearing image by combining, for example, a product image and a user image. Step 5: The display unit displays the wearing image generated by the image generation unit. The display unit displays the wearing image on, for example, the display of the AR glasses. This allows the user to consider purchasing while comparing the object in view with the generated wearing image.

[0071] (Example 2) The E-commerce system according to the embodiment of the present invention is a system that realizes E-commerce starting from an object that comes into the field of view through AR glasses. This E-commerce system operates by a user wearing AR glasses and giving voice instructions while looking at an object that comes into the field of view. For example, when a user calls out, "Find me some shorts that go well with this T-shirt!", this voice instruction is analyzed by a language generation AI. Next, the language generation AI searches for a corresponding product from an E-commerce service based on the analysis result. For example, it searches for shorts from an E-commerce service. This search result is generated as a wearing image by an image generation AI. The generated wearing image is displayed on the display of the AR glasses. The user can consider purchasing while comparing the object that comes into the field of view with the generated wearing image. This realizes E-commerce that blends into a real environment. For example, while looking at a T-shirt on a shelf in his / her room, the user can easily find shorts that go well with the T-shirt. In addition, since the user can check how the product actually looks through the generated wearing image, it becomes easier to make a purchase decision. This allows the E-commerce system to search for a product through voice instructions starting from an object that comes into the field of view of the user, and generate and display a wearing image.

[0072] The E-commerce system according to the embodiment includes a recognition unit, a voice analysis unit, a search unit, an image generation unit, and a display unit. The recognition unit recognizes an object that has entered the field of view. The object that has entered the field of view includes, for example, an object within a specific distance and an object of a specific size or shape, but is not limited to such examples. The recognition unit, for example, takes a picture of the object that has entered the field of view using a camera, and identifies the object using image recognition technology. The recognition unit can also obtain position information of the object that has entered the field of view and use it to identify the object. The voice analysis unit analyzes a voice instruction based on the object recognized by the recognition unit. The voice instruction includes, for example, a voice command and an instruction in natural language, but is not limited to such examples. The voice analysis unit converts the voice instruction into text data using, for example, a voice recognition technology, and analyzes the instruction content using a natural language processing technology. The search unit searches for a product based on the instruction analyzed by the voice analysis unit. The search unit searches for a product from, for example, an E-commerce service. The E-commerce service includes, for example, categories such as clothing, electronic devices, and food, but is not limited to such examples. The search unit searches a product database using, for example, an API of an e-commerce service to acquire a corresponding product. The image generation unit generates a wearing image based on the product searched by the search unit. The image generation unit generates a wearing image using, for example, a generation AI. The generation AI includes, for example, a GAN (generative adversarial network) and a VAE (variational autoencoder), but is not limited to such examples. The image generation unit generates a wearing image by combining, for example, a product image and an image of a user. The display unit displays the wearing image generated by the image generation unit. The display unit displays, for example, the wearing image on a display of the AR glasses. This allows the user to consider purchasing while comparing an object in view with the generated wearing image. This allows the e-commerce system according to the embodiment to search for a product through a voice instruction starting from an object in view, and generate and display a wearing image.

[0073] The recognition unit recognizes an object that has entered the field of view. The object that has entered the field of view includes, for example, an object within a specific distance and an object of a specific size and shape, but is not limited to such examples. For example, the recognition unit uses a camera to capture an object that has entered the field of view, and identifies the object using image recognition technology. Specifically, the camera acquires a high-resolution image and extracts features such as the object's contour, color, and texture. These features are analyzed by an image recognition algorithm using deep learning, and the type and attributes of the object are identified. For example, if the object that has entered the field of view is clothing, its type (shirt, pants, dress, etc.), color, brand, etc. are recognized. The recognition unit can also acquire position information of the object that has entered the field of view and use it to identify the object. The position information is measured using the viewing angle of the camera and a distance sensor, and the three-dimensional position of the object is identified. This allows the user to accurately grasp where the object that has entered the field of view is located. Furthermore, the recognition unit can combine multiple cameras and sensors to achieve more accurate object recognition. For example, a stereo camera is used to acquire depth information of the object and recognize the three-dimensional shape of the object. In addition, the infrared sensor can be used to accurately recognize objects even in dark places, backlit areas, etc. This enables the recognition unit to achieve highly accurate object recognition in a variety of environments and conditions, improving the reliability of the entire system.

[0074] The voice analysis unit analyzes the voice instruction based on the object recognized by the recognition unit. The voice instruction includes, for example, a voice command or an instruction in natural language, but is not limited to such examples. The voice analysis unit converts the voice instruction into text data using, for example, a voice recognition technology, and analyzes the instruction content using a natural language processing technology. Specifically, the voice recognition technology analyzes the user's utterance in real time and converts the voice signal into text data. At this time, noise reduction and voice enhancement technologies are used to remove environmental sounds and noise and improve the clarity of the voice. The converted text data is analyzed using a natural language processing technology, and the user's intention and request are identified. For example, a voice instruction such as "Show me different colors of this shirt" is analyzed, and an instruction to search for products with different colors of the shirt is generated. The voice analysis unit can perform more accurate analysis by taking into account the context of the user's utterance and past search history. For example, the voice analysis unit learns the user's preferences and tendencies based on products searched for by the user in the past and purchase history, and provides more appropriate search results. In addition, the voice analysis unit can handle multiple languages ​​and dialects, and can analyze voice instructions with high accuracy even for users around the world. This allows the voice analysis unit to quickly and accurately analyze the user's voice instructions, improving the operability and convenience of the entire system.

[0075] The search unit searches for products based on the instruction analyzed by the voice analysis unit. The search unit searches for products from, for example, an e-commerce service. The e-commerce service includes categories such as clothing, electronic devices, and food, but is not limited to such examples. The search unit, for example, uses an API of the e-commerce service to search a product database and acquires a corresponding product. Specifically, based on the instruction received from the voice analysis unit, a query is generated for the product database and the corresponding product is searched for. The search query may include conditions such as product category, brand, price range, color, and size. For example, a product is searched for based on specific conditions such as "blue shirt" or "latest smartphone". The search unit can acquire search results in real time and provide them to the user. Furthermore, the search unit can provide personalized search results by taking into account the user's past search history and purchase history. For example, the search unit preferentially displays products of a brand that the user has purchased in the past, or recommends products that match the user's preferences. Furthermore, the search unit can search multiple e-commerce services across the board and provide the user with the most suitable product. As a result, the search unit can quickly and accurately respond to the user's request and improve the convenience and satisfaction of the entire system.

[0076] The image generation unit generates a wearing image based on the product searched by the search unit. The image generation unit generates the wearing image using, for example, a generating AI. The generating AI includes, for example, a GAN (generative adversarial network) and a VAE (variational autoencoder), but is not limited to such examples. Specifically, the generating AI receives a product image and a user's image as input, and combines them to generate a realistic wearing image. For example, the user provides a full-body photo of himself / herself, and combines it with the searched product image of the clothing to generate an image of the user wearing the clothing. The generating AI can generate a natural wearing image that matches the user's body type and posture using deep learning. Furthermore, the generating AI can also generate a wearing image at different angles and under different lighting conditions, providing the user with an opportunity to check the product from various perspectives. In addition, the generating AI can learn the user's preferences and past selection history and generate a more personalized wearing image. As a result, the image generation unit can provide the user with a realistic and attractive wearing image, thereby increasing the user's desire to purchase.

[0077] The display unit displays the wearing image generated by the image generation unit. The display unit displays the wearing image on, for example, the display of the AR glasses. Specifically, the display of the AR glasses displays the wearing image superimposed on the user's field of vision, providing an experience as if the user were actually wearing the product. The AR glasses can track the movement of the user's head and line of sight, and update the wearing image in real time according to the field of vision. This allows the user to consider purchasing while comparing the object in view with the generated wearing image. Furthermore, the display unit can also display the wearing image on the display of a smartphone or tablet. This allows even users who do not have AR glasses to easily check the wearing image. The display unit supports operations such as zooming in and out and rotating the wearing image according to the user's operation, enabling detailed confirmation. In addition, the display unit simultaneously displays information, prices, reviews, etc. of products related to the wearing image, providing reference information when the user considers purchasing. This allows the display unit to provide the user with a visually attractive and information-rich experience, improving the convenience and satisfaction of the entire E-commerce system.

[0078] The E-commerce system includes a reception unit that receives voice instructions. The reception unit receives the voice instructions. The voice instructions include, for example, voice commands and instructions in natural language, but are not limited to such examples. The reception unit, for example, collects the user's voice using a microphone and converts the voice instructions into text data using a voice recognition technology. The voice recognition technology includes, for example, a voice recognition model using deep learning, but is not limited to such examples. The reception unit receives the voice instructions, thereby facilitating the user's operation. As a result, the E-commerce system receives the voice instructions, thereby facilitating the user's operation. A part or all of the above-mentioned processing in the reception unit may be performed, for example, using AI, or may be performed without using AI. For example, the reception unit can input the voice instructions to AI and cause the AI ​​to analyze the voice instructions.

[0079] The image generation unit can generate a wearing image by a generating AI. The image generation unit generates a wearing image by using, for example, a generating AI. The generating AI includes, for example, a GAN (generative adversarial network) and a VAE (variational autoencoder), but is not limited to such examples. The image generation unit generates a wearing image by combining, for example, a product image and a user's image. For example, the generating AI receives a product image as an input and combines it with the user's image to generate a wearing image. The generating AI fuses the product image and the user's image by using, for example, a GAN to generate a realistic wearing image. The generating AI can also encode the product image and the user's image by using a VAE, and decode it to generate a wearing image. As a result, the generation accuracy of the wearing image is improved by using the generating AI. A part or all of the above-mentioned processing in the image generation unit may be performed by using, for example, an AI, or may be performed without using an AI. For example, the image generation unit can input the product image and the user's image to the AI ​​and cause the AI ​​to generate the wearing image.

[0080] The search unit can search for products from an e-commerce service. The search unit, for example, searches for products from an e-commerce service. The e-commerce service includes categories such as clothing, electronic devices, and food, but is not limited to such examples. The search unit, for example, uses an API of the e-commerce service to search a product database and acquires a corresponding product. For example, the search unit transmits a query to the API of the e-commerce service based on a voice instruction of the user, and acquires a list of corresponding products. The search unit can also analyze the acquired list of products and select a product that meets the needs of the user. In this way, a product that meets the needs of the user can be provided by searching for a product from the e-commerce service. A part or all of the above-mentioned processing in the search unit may be performed, for example, using an AI, or may be performed without using an AI. For example, the search unit can input a voice instruction of the user to an AI and cause the AI ​​to execute a product search.

[0081] The display unit can display the generated wearing image on the display of the AR glasses. The display unit, for example, displays the generated wearing image on the display of the AR glasses. The AR glasses include, for example, HoloLens and Magic Leap, but are not limited to such examples. The display unit, for example, displays the generated wearing image on the display of the AR glasses in real time, so that the user can visually confirm it. For example, the display unit overlays the generated wearing image on the display of the AR glasses, so that the user can compare it with an object that has entered the field of view. The display unit can also display the generated wearing image by enlarging / reducing or rotating it. As a result, the user can visually confirm it by displaying it on the display of the AR glasses. A part or all of the above-mentioned processing in the display unit may be performed, for example, using AI, or may be performed without using AI. For example, the display unit can input the generated wearing image to AI and cause AI to adjust the display method.

[0082] The recognition unit can recognize an object that has entered the field of view. The recognition unit, for example, uses a camera to capture an object that has entered the field of view, and identifies the object using image recognition technology. The object that has entered the field of view includes, for example, an object within a specific distance and an object of a specific size or shape, but is not limited to such examples. The recognition unit, for example, analyzes features such as the shape, color, and texture of the object to identify the object. The recognition unit can also obtain position information of the object that has entered the field of view and use it to identify the object. For example, the recognition unit identifies the position of the object based on the field of view information of the camera, improving the recognition accuracy of the object. As a result, by recognizing the object that has entered the field of view, the user's operation becomes intuitive. A part or all of the above-mentioned processing in the recognition unit may be performed, for example, using AI, or may be performed without using AI. For example, the recognition unit can input image data of the object that has entered the field of view to AI, and cause the AI ​​to recognize the object.

[0083] The recognition unit can estimate the user's emotion and adjust the recognition accuracy based on the estimated user's emotion. The recognition unit, for example, estimates the user's emotion and adjusts the recognition accuracy based on the estimated user's emotion. The user's emotion includes, for example, excitement, relaxation, stress, etc., but is not limited to such examples. The recognition unit, for example, estimates the user's emotion using facial expression recognition technology. For example, the recognition unit analyzes a facial expression image of the user taken by a camera and calculates an emotion score. The recognition unit can also estimate the user's emotion using voice analysis technology. For example, the recognition unit analyzes the user's voice data and calculates an emotion score based on the tone and speed of the voice. Furthermore, the recognition unit can estimate the user's emotion using biometric technology. For example, the recognition unit collects the user's heart rate and electrodermal activity with a sensor and calculates an emotion score. As a result, the recognition accuracy is improved by adjusting the recognition accuracy according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI may be, for example, a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-mentioned processing in the recognition unit may be performed, for example, using an AI, or may be performed without using an AI. For example, the recognition unit may input user emotion data to the AI ​​and cause the AI ​​to adjust the recognition accuracy.

[0084] The recognition unit can set the recognition priority based on the frequency of use of the object at the time of recognition. The recognition unit sets the recognition priority based on, for example, the frequency of use of the object. The frequency of use of the object includes, for example, past use history and sensor data, but is not limited to such examples. The recognition unit, for example, preferentially recognizes objects that are used frequently based on past use history. For example, the recognition unit preferentially recognizes objects that are used frequently and postpones objects that are used less frequently. The recognition unit can also measure the frequency of use based on sensor data and dynamically change the recognition priority. For example, the recognition unit analyzes sensor data in real time and preferentially recognizes objects that are used frequently. As a result, important objects can be preferentially recognized by setting the recognition priority based on the frequency of use of the object. A part or all of the above-mentioned processing in the recognition unit may be performed using, for example, AI, or may be performed without using AI. For example, the recognition unit can input the frequency of use data of the object to AI and cause the AI ​​to execute the setting of the recognition priority.

[0085] The recognition unit can improve the recognition accuracy based on the state of the object at the time of recognition. The recognition unit improves the recognition accuracy based on, for example, the state of the object. The state of the object includes, for example, newness, dirtiness, degree of damage, etc., but is not limited to such examples. For example, the recognition unit recognizes a new object with high accuracy, and recognizes a dirty object by adjusting the recognition accuracy. For example, the recognition unit evaluates the state of the object in real time and dynamically adjusts the recognition accuracy. For example, the recognition unit photographs the state of the object with a camera and evaluates the state using image analysis technology. The recognition unit can also adjust the recognition algorithm based on the state of the object to improve the recognition accuracy. As a result, the recognition accuracy is improved by taking the state of the object into consideration. A part or all of the above-mentioned processing in the recognition unit may be performed using, for example, AI, or may be performed without using AI. For example, the recognition unit can input state data of the object to AI and cause AI to adjust the recognition accuracy.

[0086] The recognition unit can estimate the user's emotion and adjust the display method of the recognition result based on the estimated user's emotion. The recognition unit, for example, estimates the user's emotion and adjusts the display method of the recognition result based on the estimated user's emotion. The user's emotion includes, for example, excitement, relaxation, stress, etc., but is not limited to such examples. The recognition unit, for example, estimates the user's emotion using facial expression recognition technology. For example, the recognition unit analyzes a facial expression image of the user taken by a camera and calculates an emotion score. The recognition unit can also estimate the user's emotion using voice analysis technology. For example, the recognition unit analyzes the user's voice data and calculates an emotion score based on the tone and speed of the voice. Furthermore, the recognition unit can estimate the user's emotion using biometric technology. For example, the recognition unit collects the user's heart rate and electrodermal activity with a sensor and calculates an emotion score. This allows the display to be easy for the user to see by adjusting the display method of the recognition result according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI may be, for example, a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-mentioned processing in the recognition unit may be performed, for example, using AI, or may be performed without using AI. For example, the recognition unit may input user emotion data to the AI ​​and cause the AI ​​to adjust the display method.

[0087] The recognition unit can improve the recognition accuracy based on the position information of the object at the time of recognition. The recognition unit improves the recognition accuracy based on, for example, the position information of the object. The position information of the object includes, for example, GPS data and field of view information of the camera, but is not limited to such examples. The recognition unit adjusts the recognition accuracy based on, for example, the position information of the object. For example, the recognition unit improves the recognition accuracy when the object is in a specific position. The recognition unit can also acquire the position information of the object in real time and dynamically adjust the recognition accuracy. For example, the recognition unit identifies the position of the object based on the field of view information of the camera and improves the recognition accuracy. As a result, the recognition accuracy is improved by taking into account the position information of the object. A part or all of the above-mentioned processing in the recognition unit may be performed using, for example, AI, or may be performed without using AI. For example, the recognition unit can input the position information of the object to the AI ​​and cause the AI ​​to adjust the recognition accuracy.

[0088] The recognition unit can improve the recognition accuracy based on the related information of the object at the time of recognition. The recognition unit improves the recognition accuracy based on, for example, the related information of the object. The related information of the object includes, for example, the manufacturer, the purchase date, the use history, etc., but is not limited to such examples. The recognition unit adjusts the recognition accuracy based on, for example, the manufacturer information of the object. For example, the recognition unit adjusts the recognition accuracy based on the purchase date information of the object. The recognition unit can also acquire the related information of the object in real time and dynamically adjust the recognition accuracy. For example, the recognition unit improves the recognition accuracy based on the use history of the object. As a result, the recognition accuracy is improved by referring to the related information of the object. A part or all of the above-mentioned processing in the recognition unit may be performed using, for example, AI, or may be performed without using AI. For example, the recognition unit can input the related information of the object to the AI ​​and cause the AI ​​to adjust the recognition accuracy.

[0089] The voice analysis unit can estimate the user's emotion and adjust the accuracy of the voice analysis based on the estimated user's emotion. The voice analysis unit, for example, estimates the user's emotion and adjusts the accuracy of the voice analysis based on the estimated user's emotion. The user's emotion includes, for example, excitement, relaxation, stress, etc., but is not limited to such examples. The voice analysis unit, for example, estimates the user's emotion using facial expression recognition technology. For example, the voice analysis unit analyzes a facial expression image of the user taken by a camera and calculates an emotion score. The voice analysis unit can also estimate the user's emotion using voice analysis technology. For example, the voice analysis unit analyzes the user's voice data and calculates an emotion score based on the tone and speed of the voice. Furthermore, the voice analysis unit can estimate the user's emotion using biometric technology. For example, the voice analysis unit collects the user's heart rate and electrodermal activity with a sensor and calculates an emotion score. As a result, the accuracy of the voice analysis is adjusted according to the user's emotion, thereby improving the accuracy of the analysis. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI may be, for example, a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-mentioned processing in the voice analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the voice analysis unit may input user emotion data to the AI ​​and cause the AI ​​to adjust the analysis accuracy.

[0090] The voice analysis unit can improve the analysis accuracy by referring to the user's past voice instruction history during voice analysis. The voice analysis unit improves the analysis accuracy by referring to the user's past voice instruction history, for example. The user's past voice instruction history includes, but is not limited to, log data and voice records. The voice analysis unit adjusts the analysis accuracy based on the user's past voice instruction history, for example. For example, the voice analysis unit preferentially analyzes voice instructions frequently used by the user. The voice analysis unit can also dynamically adjust the analysis accuracy by referring to the user's voice instruction history in real time. For example, the voice analysis unit improves the analysis accuracy of a specific voice instruction based on the past voice instruction history. As a result, the analysis accuracy is improved by referring to the user's past voice instruction history. A part or all of the above-mentioned processing in the voice analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the voice analysis unit can input the user's past voice instruction history to AI and cause the AI ​​to improve the analysis accuracy.

[0091] The voice analysis unit can improve the analysis accuracy based on the user's speech speed and tone during voice analysis. The voice analysis unit improves the analysis accuracy based on, for example, the user's speech speed and tone. The user's speech speed and tone include, for example, a voice analysis algorithm and an acoustic feature, but are not limited to such examples. The voice analysis unit adjusts the analysis accuracy based on, for example, the user's speech speed. For example, the voice analysis unit adjusts the analysis accuracy based on the user's tone. The voice analysis unit can also analyze the user's speech speed and tone in real time and dynamically adjust the analysis accuracy. For example, the voice analysis unit analyzes the user's speech speed and tone to improve the analysis accuracy of a specific voice instruction. As a result, the analysis accuracy is improved by taking into account the user's speech speed and tone. A part or all of the above-mentioned processing in the voice analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the voice analysis unit can input data on the user's speech speed and tone to AI and cause the AI ​​to improve the analysis accuracy.

[0092] The voice analysis unit can estimate the user's emotion and determine the priority of voice instructions based on the estimated user's emotion. The voice analysis unit, for example, estimates the user's emotion and determines the priority of voice instructions based on the estimated user's emotion. The user's emotion includes, for example, excitement, relaxation, stress, etc., but is not limited to such examples. The voice analysis unit, for example, estimates the user's emotion using facial expression recognition technology. For example, the voice analysis unit analyzes a facial expression image of the user taken by a camera and calculates an emotion score. The voice analysis unit can also estimate the user's emotion using voice analysis technology. For example, the voice analysis unit analyzes the user's voice data and calculates an emotion score based on the tone and speed of the voice. Furthermore, the voice analysis unit can estimate the user's emotion using biometric technology. For example, the voice analysis unit collects the user's heart rate and electrodermal activity with a sensor and calculates an emotion score. As a result, important instructions can be preferentially processed by determining the priority of voice instructions according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI may be, for example, a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-mentioned processing in the voice analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the voice analysis unit may input user emotion data to the AI ​​and cause the AI ​​to determine the priority order of voice instructions.

[0093] The voice analysis unit can improve the analysis accuracy based on the geographical location information of the user during voice analysis. The voice analysis unit improves the analysis accuracy based on, for example, the geographical location information of the user. The geographical location information of the user includes, for example, GPS data and a location sensor, but is not limited to such examples. The voice analysis unit adjusts the analysis accuracy based on, for example, the geographical location information of the user. For example, the voice analysis unit improves the analysis accuracy when the user is in a specific location. In addition, the voice analysis unit can acquire the geographical location information of the user in real time and dynamically adjust the analysis accuracy. For example, the voice analysis unit improves the analysis accuracy of a specific voice instruction based on the geographical location information of the user. As a result, the analysis accuracy is improved by taking into account the geographical location information of the user. A part or all of the above-mentioned processing in the voice analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the voice analysis unit can input the geographical location information of the user to AI and cause the AI ​​to improve the analysis accuracy.

[0094] The voice analysis unit can analyze the social media activity of the user during the voice analysis and analyze the related voice instructions preferentially. The voice analysis unit, for example, analyzes the social media activity of the user and analyzes the related voice instructions preferentially. The social media activity of the user includes, for example, the content of posts, the number of likes, the number of followers, etc., but is not limited to such examples. The voice analysis unit adjusts the analysis accuracy, for example, based on the social media activity of the user. For example, the voice analysis unit preferentially analyzes the voice instructions frequently used by the user. The voice analysis unit can also analyze the social media activity of the user in real time and dynamically adjust the analysis accuracy. For example, the voice analysis unit improves the analysis accuracy of a specific voice instruction based on the social media activity of the user. As a result, the related voice instructions can be analyzed preferentially by analyzing the social media activity of the user. A part or all of the above-mentioned processing in the voice analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the voice analysis unit can input the social media activity data of the user to AI and cause the AI ​​to improve the analysis accuracy.

[0095] The search unit can estimate the user's emotion and adjust the display method of the search results based on the estimated user's emotion. The search unit, for example, estimates the user's emotion and adjusts the display method of the search results based on the estimated user's emotion. The user's emotion includes, for example, excitement, relaxation, stress, etc., but is not limited to such examples. The search unit, for example, estimates the user's emotion using facial expression recognition technology. For example, the search unit analyzes the user's facial expression image taken by a camera and calculates an emotion score. The search unit can also estimate the user's emotion using voice analysis technology. For example, the search unit analyzes the user's voice data and calculates an emotion score based on the tone and speed of the voice. Furthermore, the search unit can estimate the user's emotion using biometric technology. For example, the search unit collects the user's heart rate and electrodermal activity with a sensor and calculates an emotion score. This allows the display of the search results to be easily viewed by the user by adjusting the display method according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI may be, for example, a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-mentioned processing in the search unit may be performed, for example, using AI, or may be performed without using AI. For example, the search unit may input user emotion data to the AI ​​and cause the AI ​​to adjust the display method.

[0096] The search unit can improve search accuracy based on the user's past purchase history at the time of search. The search unit improves search accuracy based on the user's past purchase history, for example. The user's past purchase history includes, but is not limited to, purchase history data and receipt information. The search unit adjusts search accuracy based on the user's past purchase history, for example. For example, the search unit preferentially searches for products that the user frequently purchases. The search unit can also dynamically adjust search accuracy by referring to the user's purchase history in real time. For example, the search unit improves search accuracy for a specific product category based on the past purchase history. As a result, the search accuracy is improved by referring to the user's past purchase history. A part or all of the above-mentioned processing in the search unit may be performed using, for example, AI, or may be performed without using AI. For example, the search unit can input the user's past purchase history into AI and cause the AI ​​to improve search accuracy.

[0097] The search unit can filter search results based on the user's current field of interest at the time of search. The search unit, for example, filters search results based on the user's current field of interest. The user's current field of interest includes, for example, search history, browsing history, survey results, etc., but is not limited to such examples. The search unit, for example, filters search results based on the user's current field of interest. For example, if the user is interested in a specific field of interest, the search unit preferentially searches for products in that field. The search unit can also obtain the user's field of interest in real time and dynamically filter search results. For example, the search unit preferentially displays highly relevant products based on the user's search history and browsing history. As a result, highly relevant results can be provided by filtering search results based on the user's current field of interest. A part or all of the above-mentioned processing in the search unit may be performed using, for example, AI, or may be performed without using AI. For example, the search unit can input the user's field of interest data to AI and cause the AI ​​to filter the search results.

[0098] The search unit can estimate the user's emotion and determine the priority of the search results based on the estimated user's emotion. The search unit, for example, estimates the user's emotion and determines the priority of the search results based on the estimated user's emotion. The user's emotion includes, for example, excitement, relaxation, stress, etc., but is not limited to such examples. The search unit, for example, estimates the user's emotion using facial expression recognition technology. For example, the search unit analyzes a facial expression image of the user taken by a camera and calculates an emotion score. The search unit can also estimate the user's emotion using voice analysis technology. For example, the search unit analyzes the user's voice data and calculates an emotion score based on the tone and speed of the voice. Furthermore, the search unit can estimate the user's emotion using biometric technology. For example, the search unit collects the user's heart rate and electrodermal activity with a sensor and calculates an emotion score. As a result, important results can be preferentially displayed by determining the priority of the search results according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI may be, for example, a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-mentioned processing in the search unit may be performed, for example, using AI, or may be performed without using AI. For example, the search unit may input user emotion data to the AI ​​and have the AI ​​determine the priority order of search results.

[0099] The search unit can search for highly relevant products preferentially based on the geographical location information of the user at the time of search. The search unit preferentially searches for highly relevant products based on the geographical location information of the user, for example. The geographical location information of the user includes, but is not limited to, GPS data and a location sensor. The search unit adjusts the search accuracy, for example, based on the geographical location information of the user. For example, when the user is in a specific location, the search unit preferentially searches for products related to the location. The search unit can also acquire the geographical location information of the user in real time and dynamically adjust the search accuracy. For example, the search unit improves the search accuracy of products related to a specific area based on the geographical location information of the user. This makes it possible to preferentially search for highly relevant products by considering the geographical location information of the user. A part or all of the above-mentioned processing in the search unit may be performed, for example, using AI, or may be performed without using AI. For example, the search unit can input the geographical location information of the user to AI and cause the AI ​​to improve the search accuracy.

[0100] The search unit can analyze the user's social media activity at the time of search and search for related products with priority. The search unit, for example, analyzes the user's social media activity and searches for related products with priority. The user's social media activity includes, for example, the content of posts, the number of likes, the number of followers, etc., but is not limited to such examples. The search unit, for example, adjusts search accuracy based on the user's social media activity. For example, the search unit searches for products in which the user is frequently interested with priority. The search unit can also analyze the user's social media activity in real time and dynamically adjust search accuracy. For example, the search unit improves search accuracy for a specific product category based on the user's social media activity. As a result, related products can be searched with priority by analyzing the user's social media activity. A part or all of the above-mentioned processing in the search unit may be performed using, for example, AI, or may be performed without using AI. For example, the search unit can input the user's social media activity data into AI and cause the AI ​​to improve search accuracy.

[0101] The image generating unit can estimate the user's emotion and adjust the style of the image to be generated based on the estimated user's emotion. The image generating unit, for example, estimates the user's emotion and adjusts the style of the image to be generated based on the estimated user's emotion. The user's emotion includes, for example, excitement, relaxation, stress, etc., but is not limited to such examples. The image generating unit, for example, estimates the user's emotion using facial expression recognition technology. For example, the image generating unit analyzes the user's facial expression image captured by a camera and calculates an emotion score. The image generating unit can also estimate the user's emotion using voice analysis technology. For example, the image generating unit analyzes the user's voice data and calculates an emotion score based on the tone and speed of the voice. Furthermore, the image generating unit can estimate the user's emotion using biometric technology. For example, the image generating unit collects the user's heart rate and electrodermal activity with a sensor and calculates an emotion score. As a result, a visually appropriate image can be generated by adjusting the style of the image according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI may be, for example, a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-mentioned processing in the image generation unit may be performed, for example, using AI, or may be performed without using AI. For example, the image generation unit may input user emotion data to the AI ​​and cause the AI ​​to adjust the style of the image.

[0102] The image generation unit can improve the generation accuracy based on the user's past image generation history when generating an image. The image generation unit improves the generation accuracy based on the user's past image generation history, for example. The user's past image generation history includes, for example, data of generated images and a log of the generation history, but is not limited to such examples. The image generation unit adjusts the generation accuracy based on the user's past image generation history, for example. For example, the image generation unit preferentially generates images in a style that the user frequently generates. The image generation unit can also dynamically adjust the generation accuracy by referring to the user's image generation history in real time. For example, the image generation unit improves the image generation accuracy of a specific style based on the past image generation history. As a result, the generation accuracy is improved by referring to the user's past image generation history. A part or all of the above-mentioned processing in the image generation unit may be performed using, for example, AI, or may be performed without using AI. For example, the image generation unit can input the user's past image generation history to AI and cause the AI ​​to execute the improvement of the generation accuracy.

[0103] The image generating unit can customize the image to be generated based on the user's current fashion trend at the time of image generation. The image generating unit customizes the image to be generated based on the user's current fashion trend, for example. The user's current fashion trend includes, but is not limited to, for example, a subscription history of a fashion magazine and a post content of a social media. The image generating unit customizes the image based on, for example, the user's current fashion trend. For example, if the user is interested in a particular fashion trend, the image generating unit generates an image that matches the trend. The image generating unit can also obtain the user's fashion trend in real time and dynamically customize the image. For example, the image generating unit generates a highly relevant image based on the user's subscription history of a fashion magazine and a post content of a social media. As a result, a more appropriate image can be generated by customizing the image based on the user's current fashion trend. A part or all of the above-mentioned processing in the image generating unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the image generating unit can input the user's fashion trend data into the AI ​​and cause the AI ​​to execute image customization.

[0104] The image generating unit can estimate the user's emotion and determine the priority of the images to be generated based on the estimated user's emotion. The image generating unit, for example, estimates the user's emotion and determines the priority of the images to be generated based on the estimated user's emotion. The user's emotion includes, for example, excitement, relaxation, stress, etc., but is not limited to such examples. The image generating unit, for example, estimates the user's emotion using facial expression recognition technology. For example, the image generating unit analyzes the user's facial expression image captured by a camera and calculates an emotion score. The image generating unit can also estimate the user's emotion using voice analysis technology. For example, the image generating unit analyzes the user's voice data and calculates an emotion score based on the tone and speed of the voice. Furthermore, the image generating unit can estimate the user's emotion using biometric technology. For example, the image generating unit collects the user's heart rate and electrodermal activity with a sensor and calculates an emotion score. As a result, important images can be preferentially generated by determining the priority of images according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI may be, for example, a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-mentioned processing in the image generation unit may be performed, for example, using AI, or may be performed without using AI. For example, the image generation unit may input user emotion data to the AI ​​and cause the AI ​​to determine the priority order of images.

[0105] The image generating unit can customize the image to be generated based on the geographical location information of the user when generating the image. The image generating unit customizes the image to be generated based on, for example, the geographical location information of the user. The geographical location information of the user includes, for example, GPS data and a location sensor, but is not limited to such examples. The image generating unit customizes the image based on, for example, the geographical location information of the user. For example, when the user is in a specific location, the image generating unit generates an image related to the location. In addition, the image generating unit can obtain the geographical location information of the user in real time and dynamically customize the image. For example, the image generating unit generates an image related to a specific area based on the geographical location information of the user. As a result, a more appropriate image can be generated by taking into account the geographical location information of the user. A part or all of the above-mentioned processing in the image generating unit may be performed using, for example, AI, or may be performed without using AI. For example, the image generating unit can input the geographical location information of the user to the AI ​​and cause the AI ​​to execute the customization of the image.

[0106] The image generation unit can analyze the social media activity of the user and generate a related image when generating an image. The image generation unit, for example, analyzes the social media activity of the user and generates a related image. The social media activity of the user includes, for example, the content of posts, the number of likes, the number of followers, etc., but is not limited to such examples. The image generation unit, for example, customizes an image based on the social media activity of the user. For example, the image generation unit generates an image related to a theme in which the user is frequently interested. The image generation unit can also analyze the social media activity of the user in real time and dynamically customize the image. For example, the image generation unit generates an image related to a specific theme based on the social media activity of the user. In this way, a related image can be generated by analyzing the social media activity of the user. A part or all of the above-mentioned processing in the image generation unit may be performed, for example, using AI, or may be performed without using AI. For example, the image generation unit can input the social media activity data of the user to AI and cause the AI ​​to generate an image.

[0107] The display unit can estimate the user's emotion and adjust the display method based on the estimated user's emotion. The display unit, for example, estimates the user's emotion and adjusts the display method based on the estimated user's emotion. The user's emotion includes, for example, excitement, relaxation, stress, etc., but is not limited to such examples. The display unit, for example, estimates the user's emotion using facial expression recognition technology. For example, the display unit analyzes a facial expression image of the user taken by a camera and calculates an emotion score. The display unit can also estimate the user's emotion using voice analysis technology. For example, the display unit analyzes the user's voice data and calculates an emotion score based on the tone and speed of the voice. Furthermore, the display unit can estimate the user's emotion using biometric technology. For example, the display unit collects the user's heart rate and electrodermal activity with a sensor and calculates an emotion score. This allows for visually appropriate display by adjusting the display method according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI may be, for example, a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-mentioned processing in the display unit may be performed, for example, using AI, or may be performed without using AI. For example, the display unit may input the user's emotion data to the AI ​​and cause the AI ​​to adjust the display method.

[0108] The display unit can select an optimal display method based on the user's past display history at the time of display. The display unit selects an optimal display method based on the user's past display history, for example. The user's past display history includes, for example, a log of the displayed content and data of the display history, but is not limited to such an example. The display unit adjusts the display method based on the user's past display history, for example. For example, the display unit preferentially provides a display method that the user frequently uses. The display unit can also dynamically adjust the display method by referring to the user's display history in real time. For example, the display unit improves the accuracy of a specific display method based on the past display history. As a result, the optimal display method can be selected by referring to the user's past display history. A part or all of the above-mentioned processing in the display unit may be performed using, for example, AI, or may be performed without using AI. For example, the display unit can input the user's past display history to AI and cause the AI ​​to select the display method.

[0109] The display unit can adjust the display method based on the user's current visual state at the time of display. The display unit adjusts the display method based on the user's current visual state, for example. The user's current visual state includes, but is not limited to, for example, a visual acuity test result and an evaluation of eye fatigue. For example, the display unit provides a display method with high visibility when the user is tired. For example, the display unit provides a normal display method when the user is relaxed. The display unit can also acquire the user's visual state in real time and dynamically adjust the display method. For example, the display unit improves the accuracy of a specific display method based on the user's visual state. This enables a display with high visibility by taking the user's visual state into consideration. A part or all of the above-mentioned processing in the display unit may be performed, for example, using AI, or may be performed without using AI. For example, the display unit can input the user's visual state data to AI and cause the AI ​​to adjust the display method.

[0110] The display unit can estimate the user's emotion and determine the priority of the display contents based on the estimated user's emotion. The display unit, for example, estimates the user's emotion and determines the priority of the display contents based on the estimated user's emotion. The user's emotion includes, for example, excitement, relaxation, stress, etc., but is not limited to such examples. The display unit, for example, estimates the user's emotion using facial expression recognition technology. For example, the display unit analyzes a facial expression image of the user taken by a camera and calculates an emotion score. The display unit can also estimate the user's emotion using voice analysis technology. For example, the display unit analyzes the user's voice data and calculates an emotion score based on the tone and speed of the voice. Furthermore, the display unit can estimate the user's emotion using biometric technology. For example, the display unit collects the user's heart rate and electrodermal activity with a sensor and calculates an emotion score. As a result, important content can be preferentially displayed by determining the priority of the display contents according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI may be, for example, a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-mentioned processing in the display unit may be performed, for example, using AI, or may be performed without using AI. For example, the display unit may input user emotion data to the AI ​​and cause the AI ​​to determine the priority order of display contents.

[0111] The display unit can select an optimal display method based on the user's device information at the time of display. The display unit selects an optimal display method based on, for example, the user's device information. The user's device information includes, for example, the type of device, the OS version, and the usage status, but is not limited to such examples. For example, when the user is using a smartphone, the display unit provides a display method that matches the screen size. For example, when the user is using a tablet, the display unit provides a display method optimized for a large screen. In addition, when the user is using a smart watch, the display unit can also provide a simple and highly visible display method. For example, the display unit selects a display method optimized for a specific device based on the user's device information. This makes it possible to select an optimal display method by considering the user's device information. A part or all of the above-mentioned processing in the display unit may be performed, for example, using AI, or may be performed without using AI. For example, the display unit can input the user's device information to AI and cause the AI ​​to select the display method.

[0112] The display unit can analyze the user's social media activity at the time of display and preferentially display related display content. The display unit, for example, analyzes the user's social media activity and preferentially displays related display content. The user's social media activity includes, for example, the content of posts, the number of likes, the number of followers, etc., but is not limited to such examples. The display unit adjusts the display content, for example, based on the user's social media activity. For example, the display unit preferentially displays display content related to a theme in which the user is frequently interested. The display unit can also analyze the user's social media activity in real time and dynamically adjust the display content. For example, the display unit preferentially displays display content related to a specific theme based on the user's social media activity. As a result, the display unit can preferentially display related display content by analyzing the user's social media activity. A part or all of the above-mentioned processing in the display unit may be performed, for example, using AI, or may be performed without using AI. For example, the display unit can input the user's social media activity data into AI and cause AI to adjust the display content.

[0113] The reception unit can estimate the user's emotion and adjust the reception method based on the estimated user's emotion. The reception unit, for example, estimates the user's emotion and adjusts the reception method based on the estimated user's emotion. The user's emotion includes, for example, excitement, relaxation, stress, etc., but is not limited to such examples. The reception unit, for example, estimates the user's emotion using facial expression recognition technology. For example, the reception unit analyzes a facial expression image of the user taken by a camera and calculates an emotion score. The reception unit can also estimate the user's emotion using voice analysis technology. For example, the reception unit analyzes the user's voice data and calculates an emotion score based on the tone and speed of the voice. Furthermore, the reception unit can estimate the user's emotion using biometric technology. For example, the reception unit collects the user's heart rate and electrodermal activity with a sensor and calculates an emotion score. This allows appropriate reception by adjusting the reception method according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI may be, for example, a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-mentioned processing in the reception unit may be performed, for example, using an AI, or may be performed without using an AI. For example, the reception unit may input the user's emotion data to the AI ​​and cause the AI ​​to adjust the reception method.

[0114] The reception unit can select the optimal reception method based on the user's past reception history at the time of reception. The reception unit selects the optimal reception method based on the user's past reception history, for example. The user's past reception history includes, for example, a log of reception contents and reception history data, but is not limited to such examples. The reception unit adjusts the reception method based on the user's past reception history, for example. For example, the reception unit preferentially provides a reception method that the user uses frequently. The reception unit can also dynamically adjust the reception method by referring to the user's reception history in real time. For example, the reception unit improves the accuracy of a specific reception method based on the past reception history. As a result, the optimal reception method can be selected by referring to the user's past reception history. A part or all of the above-mentioned processing in the reception unit may be performed using, for example, AI, or may be performed without using AI. For example, the reception unit can input the user's past reception history into AI and have the AI ​​select the reception method.

[0115] The reception unit can estimate the user's emotion and determine the priority of the reception contents based on the estimated user's emotion. The reception unit, for example, estimates the user's emotion and determines the priority of the reception contents based on the estimated user's emotion. The user's emotion includes, for example, excitement, relaxation, stress, etc., but is not limited to such examples. The reception unit, for example, estimates the user's emotion using facial expression recognition technology. For example, the reception unit analyzes the user's facial expression image taken by a camera and calculates an emotion score. The reception unit can also estimate the user's emotion using voice analysis technology. For example, the reception unit analyzes the user's voice data and calculates an emotion score based on the tone and speed of the voice. Furthermore, the reception unit can estimate the user's emotion using biometric technology. For example, the reception unit collects the user's heart rate and skin electrodermal activity with a sensor and calculates an emotion score. As a result, important content can be preferentially processed by determining the priority of the reception contents according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI may be, for example, a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-mentioned processing in the reception unit may be performed, for example, using AI, or may be performed without using AI. For example, the reception unit may input the user's emotion data to the AI ​​and cause the AI ​​to determine the priority order of the reception contents.

[0116] The reception unit can select the optimal reception method based on the device information of the user at the time of reception. The reception unit selects the optimal reception method based on, for example, the device information of the user. The device information of the user includes, for example, the type of device, the version of the OS, the usage status, etc., but is not limited to such examples. For example, when the user is using a smartphone, the reception unit provides a reception method that matches the screen size. For example, when the user is using a tablet, the reception unit provides a reception method optimized for a large screen. In addition, when the user is using a smart watch, the reception unit can also provide a simple and highly visible reception method. For example, the reception unit selects a reception method optimized for a specific device based on the device information of the user. This makes it possible to select the optimal reception method by considering the device information of the user. A part or all of the above-mentioned processing in the reception unit may be performed using, for example, AI, or may be performed without using AI. For example, the reception unit can input the device information of the user to AI and cause the AI ​​to select the reception method.

[0117] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0118] The recognition unit can also analyze the material of an object that comes into view and make product suggestions based on the material. For example, the recognition unit analyzes an image of an object captured by a camera and identifies the material of the object, such as cloth, metal, or plastic. Next, the search unit searches for related products based on the identified material. For example, if a cloth object is recognized, the search unit preferentially searches for products related to cloth products. The recognition unit can also suggest appropriate maintenance methods to the user based on the material of the object. This allows the user to obtain products and information according to the material of the object.

[0119] The reception unit can also analyze background sounds of the user's voice instructions and improve the accuracy of the voice instructions based on the background sounds. For example, the reception unit can analyze noise or music in the background of the voice instructions and use noise canceling technology to make the voice instructions clear. The reception unit can also increase the sensitivity of the voice instructions when the background sounds are quiet. Furthermore, the reception unit can estimate the user's situation based on the background sounds and select an appropriate method of analyzing the voice instructions. This allows the user to give accurate voice instructions in any environment.

[0120] The image generating unit can also reflect the user's preferred color tones in the generated image. For example, the image generating unit analyzes the user's past selection history and social media postings to identify the user's preferred color tones. Then, the image generating unit reflects the color tones in the generated image to provide a more attractive image to the user. The image generating unit can also adjust the color tones based on the user's current mood. This allows the user to easily obtain an image that matches his or her preferences.

[0121] The search unit can also display reviews and ratings of related products based on the user's purchasing history. For example, the search unit collects reviews and ratings of products previously purchased by the user and displays them in the search results. This allows the user to make better purchasing decisions by referring to the opinions of other users. The search unit can also filter reviews and ratings of related products based on the user's purchasing history to provide reliable information. This allows the user to select products based on reliable information.

[0122] The display unit can also estimate the user's visual fatigue level and adjust the display method based on the fatigue level. For example, the display unit analyzes the user's eye tracking data and blink frequency to estimate the visual fatigue level. Next, if the fatigue level is high, the display unit adjusts the brightness and contrast of the screen to improve visibility. The display unit can also simplify the display content based on the fatigue level to reduce the burden on the user. This allows the user to comfortably view information even when using the device for long periods of time.

[0123] The recognition unit can also analyze the movement of an object that comes into view and identify the type of object based on the movement. For example, the recognition unit can analyze the movement of an object captured by a camera and identify the type of object, such as an animal, a vehicle, or a human. Next, the search unit searches for related products and information based on the identified type of object. For example, if an animal is recognized, the search unit preferentially searches for products related to pet supplies. The recognition unit can also suggest appropriate actions to the user based on the movement of the object. This allows the user to obtain products and information according to the movement.

[0124] The recognition unit can also estimate the user's emotions and feed back the object recognition results based on the estimated emotions. For example, when the user is excited, the recognition unit can provide the recognition results quickly to maintain the user's excitement. When the user is relaxed, the recognition unit can provide the recognition results in detail to promote the user's relaxation. Furthermore, the recognition unit can adjust the display method of the recognition results based on the user's emotions to provide the user with the most suitable information. This allows the user to obtain information according to his or her emotions.

[0125] The recognition unit can also analyze the temperature of an object that comes into view and identify the state of the object based on the temperature. For example, the recognition unit can measure the temperature of the object using an infrared camera and identify whether the object is cold, warm, hot, or other such state. The search unit then searches for related products and information based on the identified temperature. For example, if a hot object is recognized, the search unit will preferentially search for products related to cooling products. The recognition unit can also suggest appropriate actions to the user based on the temperature of the object. This allows the user to obtain products and information according to the temperature.

[0126] The recognition unit can also analyze the sound of an object that comes into view and identify the type of the object based on the sound. For example, the recognition unit collects sounds emitted from the object using a microphone and identifies the type of the object using voice analysis technology. Next, the search unit searches for related products and information based on the identified type of object. For example, if a musical instrument is recognized, the search unit preferentially searches for products related to the musical instrument. The recognition unit can also suggest appropriate actions to the user based on the sound of the object. This allows the user to obtain products and information according to the sound.

[0127] The recognition unit can also estimate the user's emotions and dynamically adjust the object recognition accuracy based on the estimated emotions. For example, when the user is stressed, the recognition unit can increase the recognition accuracy to provide quick results and reduce the user's stress. When the user is relaxed, the recognition unit can return the recognition accuracy to normal and provide detailed information. Furthermore, the recognition unit can adjust the recognition algorithm based on the user's emotions to provide optimal recognition results. This allows the user to obtain recognition results according to their emotions.

[0128] The process flow of the second embodiment will be briefly described below.

[0129] Step 1: The recognition unit recognizes an object that comes into view. Examples of objects that come into view include, but are not limited to, objects within a certain distance or objects of a certain size or shape. The recognition unit, for example, uses a camera to capture an image of the object that comes into view and identifies the object using image recognition technology. The recognition unit can also obtain position information of the object that comes into view and use it to identify the object. Step 2: The voice analysis unit analyzes the voice instruction based on the object recognized by the recognition unit. The voice instruction includes, but is not limited to, a voice command or an instruction in natural language. The voice analysis unit converts the voice instruction into text data using, for example, a voice recognition technology, and analyzes the instruction content using a natural language processing technology. Step 3: The search unit searches for products based on the instruction analyzed by the voice analysis unit. The search unit searches for products from, for example, an e-commerce service. The e-commerce service includes categories such as, but is not limited to, clothing, electronic devices, and food. The search unit searches a product database using, for example, an API of the e-commerce service to obtain a corresponding product. Step 4: The image generation unit generates a wearing image based on the product searched by the search unit. The image generation unit generates a wearing image using, for example, a generation AI. The generation AI includes, for example, a GAN (generative adversarial network) and a VAE (variational autoencoder), but is not limited to such examples. The image generation unit generates a wearing image by combining, for example, a product image and a user image. Step 5: The display unit displays the wearing image generated by the image generation unit. The display unit displays the wearing image on, for example, the display of the AR glasses. This allows the user to consider purchasing while comparing the object in view with the generated wearing image.

[0130] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0131] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) are input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may also be an AI agent. In addition, when the processes of each of the above-mentioned parts are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples.In addition, processing performed by AI including the generation AI may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI including the generation AI.

[0132] Furthermore, the above-described processing by the data processing system 10 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information required for processing from the data processing device 12 or an external device.

[0133] Each of the multiple elements including the above-mentioned recognition unit, voice analysis unit, search unit, image generation unit, display unit, and reception unit is realized, for example, in at least one of the smart device 14 and the data processing device 12. For example, the recognition unit photographs an object that comes into view using the camera 42 of the smart device 14, and identifies the object using the control unit 46A. The voice analysis unit collects voice instructions using the microphone 38B of the smart device 14, and analyzes the voice instructions using the control unit 46A. The search unit searches for products using the specific processing unit 290 of the data processing device 12. The image generation unit generates a wearing image using the specific processing unit 290 of the data processing device 12. The display unit displays the wearing image generated by the display 40A of the smart device 14. The reception unit receives voice instructions using the microphone 38B of the smart device 14, and analyzes the voice instructions using the control unit 46A. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0134] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0135] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0136] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0137] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0138] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0139] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0140] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0141] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0142] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0143] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0144] In the smart glasses 214, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart glasses 214 have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0145] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0146] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0147] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0148] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device.

[0149] Each of the multiple elements including the above-mentioned recognition unit, voice analysis unit, search unit, image generation unit, display unit, and reception unit is realized, for example, by at least one of the smart glasses 214 and the data processing device 12. For example, the recognition unit photographs an object that comes into view using the camera 42 of the smart glasses 214, and identifies the object by the control unit 46A. The voice analysis unit collects voice instructions using the microphone 238 of the smart glasses 214, and analyzes the voice instructions by the control unit 46A. The search unit searches for products by the specific processing unit 290 of the data processing device 12. The image generation unit generates a wearing image by the specific processing unit 290 of the data processing device 12. The display unit displays the wearing image generated by the display of the smart glasses 214. The reception unit receives voice instructions using the microphone 238 of the smart glasses 214, and analyzes the voice instructions by the control unit 46A. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0150] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0151] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0152] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0153] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0154] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0155] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0156] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0157] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0158] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0159] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0160] In the headset type terminal 314, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50, and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the identification processing unit 290 using these models.

[0161] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0162] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0163] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0164] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device.

[0165] Each of the multiple elements including the above-mentioned recognition unit, voice analysis unit, search unit, image generation unit, display unit, and reception unit is realized, for example, in at least one of the headset type terminal 314 and the data processing device 12. For example, the recognition unit photographs an object that comes into view using the camera 42 of the headset type terminal 314, and identifies the object using the control unit 46A. The voice analysis unit collects voice instructions using the microphone 238 of the headset type terminal 314, and analyzes the voice instructions using the control unit 46A. The search unit searches for products using the specific processing unit 290 of the data processing device 12. The image generation unit generates a wearing image using the specific processing unit 290 of the data processing device 12. The display unit displays the wearing image generated by the display 343 of the headset type terminal 314. The reception unit receives voice instructions using the microphone 238 of the headset type terminal 314, and analyzes the voice instructions using the control unit 46A. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0166] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0167] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0168] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0169] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0170] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0171] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of a typical healthy person).

[0172] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0173] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and legs. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, and legs. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0174] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0175] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0176] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0177] In the robot 414, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads out the specific program 60 from the storage 50, and executes the specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. The robot 414 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0178] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0179] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0180] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0181] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0182] Each of the multiple elements including the above-mentioned recognition unit, voice analysis unit, search unit, image generation unit, display unit, and reception unit is realized, for example, by at least one of the robot 414 and the data processing device 12. For example, the recognition unit photographs an object that comes into view using the camera 42 of the robot 414, and identifies the object by the control unit 46A. The voice analysis unit collects voice instructions using the microphone 238 of the robot 414, and analyzes the voice instructions by the control unit 46A. The search unit searches for a product by the specific processing unit 290 of the data processing device 12. The image generation unit generates a wearing image by the specific processing unit 290 of the data processing device 12. The display unit displays the wearing image generated by the display of the robot 414. The reception unit receives voice instructions using the microphone 238 of the robot 414, and analyzes the voice instructions by the control unit 46A. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0183] The emotion identification model 59 as an emotion engine may determine the emotion of the user according to a specific mapping. Specifically, the emotion identification model 59 may determine the emotion of the user according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the emotion of the robot, and the identification processing unit 290 may perform identification processing using the emotion of the robot.

[0184] FIG. 9 is a diagram showing an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive emotions are arranged. The more outside the concentric circles, the more emotions that represent states and actions that arise from a state of mind are arranged. Emotions are a concept that includes emotions and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions that occur in the brain are arranged. On the right side of the concentric circles, emotions that are generally induced by situational judgment are arranged. On the upper and lower sides of the concentric circles, emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged. In addition, on the upper side of the concentric circles, emotions of "pleasure" are arranged, and on the lower side, emotions of "discomfort" are arranged. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0185] These emotions are distributed in the three o'clock direction of emotion map 400 and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0186] The inside of emotion map 400 represents what is going on inside one's mind, and the outside of emotion map 400 represents behavior, so the further out on emotion map 400 you go, the more visible the emotions become (the more they are expressed in behavior).

[0187] Here, human emotions are based on various balances such as posture and blood sugar level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and battery level, so that when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. The emotion map may be generated, for example, based on the emotion map of Dr. Mitsuyoshi (Research on speech emotion recognition and emotion brain physiological signal analysis system, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). On the left half of the emotion map, emotions belonging to an area called "reaction" where sensation is dominant are lined up. On the right half of the emotion map, emotions belonging to an area called "situation" where situation recognition is dominant are lined up.

[0188] The emotion map defines two emotions that promote learning. The first is the negative emotion around the middle of "repentance" or "remorse" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot has positive feelings such as "I want more" or "I want to know more."

[0189] The emotion identification model 59 inputs the user input to a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the emotion of the user. This neural network is pre-trained based on multiple learning data that are combinations of the user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in Fig. 10. Fig. 10 shows an example in which multiple emotions, "relief," "calm," and "encouraging," have similar emotion values.

[0190] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers, including computer 22.

[0191] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable computer-readable non-transitory storage medium such as a Universal Serial Bus (USB) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0192] In addition, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 upon request from the data processing device 12.

[0193] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0194] As the hardware resource for executing the specific process, various processors as shown below can be used. An example of the processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the specific process by executing software, i.e., a program. Another example of the processor is a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing a specific process, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), or an Application Specific Integrated Circuit (ASIC). Each processor has a built-in or connected memory, and each processor executes the specific process by using the memory.

[0195] The hardware resource that executes the specific process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0196] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0197] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. The specific processes described above are merely examples. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed without departing from the spirit of the invention.

[0198] In the above example, the first to fourth embodiments are described separately, but some or all of these embodiments may be combined. The smart device 14, the smart glasses 214, the headset terminal 314, and the robot 414 are only examples, and they may be combined with each other, or may be other devices. In the above example, the first and second embodiments are described separately, but they may be combined.

[0199] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replaced with respect to the description and illustrations shown above, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the description and illustrations shown above omit explanations of technical common sense that do not require particular explanation in enabling the implementation of the technology of the present disclosure.

[0200] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, and standard was specifically and individually indicated to be incorporated by reference.

[0201] (Appendix 1) A recognition unit that recognizes objects that come into view; a voice analysis unit that analyzes a voice instruction based on the object recognized by the recognition unit; a search unit that searches for products based on the instruction analyzed by the voice analysis unit; an image generating unit that generates a wearing image based on the product searched for by the search unit; A display unit that displays the wearing image generated by the image generating unit. A system characterized by: (Appendix 2) Equipped with a reception unit that accepts voice instructions 2. The system of claim 1, (Appendix 3) The image generating unit includes: Generates a wearing image using generative AI 2. The system of claim 1, (Appendix 4) The search unit is Search for products from e-commerce services 2. The system of claim 1, (Appendix 5) The display unit is The generated image of the glasses being worn is displayed on the AR glasses display. 2. The system of claim 1, (Appendix 6) The recognition unit is Recognize objects in view 2. The system of claim 1, (Appendix 7) The recognition unit is Estimate the user's emotion and adjust the recognition accuracy based on the estimated user's emotion. 2. The system of claim 1, (Appendix 8) The recognition unit is During recognition, set recognition priority based on the frequency of use of objects 2. The system of claim 1, (Appendix 9) The recognition unit is During recognition, improve recognition accuracy based on the object's state 2. The system of claim 1, (Appendix 10) The recognition unit is Estimating a user's emotion and adjusting the display method of the recognition result based on the estimated user's emotion 2. The system of claim 1, (Appendix 11) The recognition unit is During recognition, the accuracy of recognition is improved based on object position information. 2. The system of claim 1, (Appendix 12) The recognition unit is During recognition, the recognition accuracy is improved based on the related information of the object. 2. The system of claim 1, (Appendix 13) The voice analysis unit Estimate user emotions and adjust the accuracy of voice analysis based on the estimated user emotions 2. The system of claim 1, (Appendix 14) The voice analysis unit When analyzing voice, the accuracy of the analysis can be improved by referring to the user's past voice instruction history. 2. The system of claim 1, (Appendix 15) The voice analysis unit Improve accuracy of voice analysis based on the user's speech rate and tone 2. The system of claim 1, (Appendix 16) The voice analysis unit Estimating a user's emotion and determining the priority of voice instructions based on the estimated user's emotion 2. The system of claim 1, (Appendix 17) The voice analysis unit Improve accuracy of voice analysis based on the user's geographic location 2. The system of claim 1, (Appendix 18) The voice analysis unit During voice analysis, the user's social media activity is analyzed and relevant voice instructions are prioritized for analysis. 2. The system of claim 1, (Appendix 19) The search unit is Infer user sentiment and adjust how search results are displayed based on the inferred user sentiment 2. The system of claim 1, (Appendix 20) The search unit is When searching, improve search accuracy based on the user's past purchase history 2. The system of claim 1, (Appendix 21) The search unit is When searching, filter search results based on the user's current areas of interest 2. The system of claim 1, (Appendix 22) The search unit is Infer user sentiment and prioritize search results based on the inferred user sentiment 2. The system of claim 1, (Appendix 23) The search unit is When searching, prioritize relevant products based on the user's geographic location 2. The system of claim 1, (Appendix 24) The search unit is When searching, analyze users' social media activity to prioritize relevant products 2. The system of claim 1, (Appendix 25) The image generating unit includes: Estimate the user's emotion and adjust the style of the generated image based on the estimated user's emotion. 2. The system of claim 1, (Appendix 26) The image generating unit includes: When generating images, the accuracy of the generation is improved based on the user's past image generation history. 2. The system of claim 1, (Appendix 27) The image generating unit includes: Customize the image to be generated based on the user's current fashion trends when generating the image 2. The system of claim 1, (Appendix 28) The image generating unit includes: Estimate the user's emotions and prioritize the images to be generated based on the estimated user's emotions. 2. The system of claim 1, (Appendix 29) The image generating unit includes: Customize the images generated based on the user's geographic location when generating the images 2. The system of claim 1, (Appendix 30) The image generating unit includes: When generating images, the app analyzes the user's social media activity and generates relevant images. 2. The system of claim 1, (Appendix 31) The display unit is Estimate user's emotions and adjust the display method based on the estimated user's emotions 2. The system of claim 1, (Appendix 32) The display unit is When displaying, the optimal display method is selected based on the user's past display history. 2. The system of claim 1, (Appendix 33) The display unit is At display time, it adjusts its appearance to take into account the user's current visual state 2. The system of claim 1, (Appendix 34) The display unit is Estimate the user's emotions and prioritize the content to be displayed based on the estimated user's emotions. 2. The system of claim 1, (Appendix 35) The display unit is When displaying, the optimal display method is selected based on the user's device information. 2. The system of claim 1, (Appendix 36) The display unit is When displaying content, the app analyzes the user's social media activity and prioritizes relevant content. 2. The system of claim 1, (Appendix 37) The reception unit is To estimate a user's emotion and adjust a reception method based on the estimated user's emotion. 3. The system of claim 2, (Appendix 38) The reception unit is At the time of reception, the optimal reception method is selected based on the user's past reception history. 3. The system of claim 2, (Appendix 39) The reception unit is Estimate the user's emotions and prioritize the reception contents based on the estimated user's emotions 3. The system of claim 2, (Appendix 40) The reception unit is At the time of reception, the optimal reception method is selected based on the user's device information. 3. The system of claim 2, [Explanation of symbols]

[0202] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. A recognition unit that recognizes objects that come into view; a voice analysis unit that analyzes a voice instruction based on the object recognized by the recognition unit; a search unit that searches for products based on the instruction analyzed by the voice analysis unit; an image generating unit that generates a wearing image based on the product searched for by the search unit; A display unit that displays the wearing image generated by the image generating unit. A system characterized by:

2. Equipped with a reception unit that accepts voice instructions 2. The system of claim 1 .

3. The image generating unit includes: Generate a wearing image using AI 2. The system of claim 1 .

4. The search unit is Search for products from e-commerce services 2. The system of claim 1 .

5. The display unit is The generated image of the glasses being worn is displayed on the AR glasses display.

2. The system of claim 1 .

6. The recognition unit is Estimate the user's emotion and adjust the recognition accuracy based on the estimated user's emotion.

2. The system of claim 1 .

7. The recognition unit is During recognition, set recognition priority based on the frequency of use of objects 2. The system of claim 1 .

Images