system

Abstract

The system according to this embodiment aims to analyze the shogi board in real time and provide next move suggestions and strategic advice. [Solution] The system according to the embodiment comprises a capture unit, an analysis unit, a provision unit, and a projection unit. The capture unit captures the state of the board. The analysis unit analyzes the information captured by the capture unit. The provision unit provides the next move and strategic advice identified by the analysis unit. The projection unit projects the evaluation value of the position and the best move.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the conventional technology, there is a problem that it is difficult to analyze a shogi board in real time and provide the next move or strategic advice.

[0005] The system according to the embodiment aims to analyze a shogi board in real time and provide the next move or strategic advice.

Means for Solving the Problems

[0006] The system according to this embodiment comprises a capture unit, an analysis unit, a provision unit, and a projection unit. The capture unit captures the state of the board. The analysis unit analyzes the information captured by the capture unit. The provision unit provides the next move and strategic advice identified by the analysis unit. The projection unit projects the evaluation value of the position and the best move. [Effects of the Invention]

[0007] The system according to this embodiment can analyze the shogi board in real time and provide next move suggestions and strategic advice. [Brief explanation of the drawing]

[0008] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Modes for carrying out the invention]

[0009] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0010] First, let's explain the terminology used in the following explanation.

[0011] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), or TPU (Tensor Processing Unit).

[0012] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0013] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0014] In the following embodiments, the numbered communication I / F (Interface) is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it may be only A, only B, or a combination of A and B. Also, in this specification, when expressing three or more matters connected by "and / or", the same concept as "A and / or B" is applied.

[0016] [First Embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] As shown in FIG. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. Also, the database 24 and the communication I / F 26 are connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 comprises a computer 36, a receiving device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The receiving device 38, output device 40, and camera 42 are also connected to the bus 52.

[0020] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, and accepts user input. The touch panel 38A accepts user input via touch by detecting contact with an object (e.g., a pen or finger). The microphone 38B accepts user input via voice by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 (see Figure 2) acquires the data indicating the user input.

[0021] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user by outputting the data in a form perceptible to the user (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0023] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0025] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0026] In the smart device 14, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart device 14 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0027] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device having the data generation model 58. The data processing device 12 may also be a server device or a terminal device owned by a user (e.g., a mobile phone, robot, home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example of form 1) The Shogi support agent according to an embodiment of the present invention is a glasses-type device that helps Shogi players deeply understand strategy and enjoy the game. This device analyzes the board situation in real time and provides next move suggestions and strategic advice via a bone conduction speaker. The glasses project the evaluation value of the board and the best move, providing visual support. It supports skill improvement for players from beginners to advanced players and brings out the appeal of Shogi. The generating AI analyzes the board and provides optimal advice, enabling more strategic play. For example, when a user starts a Shogi game, the glasses-type device captures the board situation in real time. This information is sent to the generating AI, which analyzes the board situation. The generating AI analyzes the position and movement patterns of each piece and identifies candidate next moves. Next, the next move and strategic advice identified by the generating AI are provided to the user via a bone conduction speaker. For example, specific advice such as "It would be good to move the rook to the right next" is provided by voice. This allows the user to receive advice in real time during the game. Furthermore, the evaluation value of the board and the best move are projected onto the glasses. For example, the evaluation value of the current position is displayed as "+1.5," and the best move is visually supported as "move the bishop to the left." This allows the user to visually confirm the next move. This device is suitable for a wide range of players, from beginners to advanced players. It provides support for beginners to learn basic piece movements and strategies, and provides advanced strategic advice for advanced players. For example, beginners are given basic advice such as "In this position, it would be good to move the gold piece forward," while advanced players are given advanced advice such as "In this position, it would be good to move the rook to the right and attack the bishop on the next move." In this way, the Shogi Support Agent aims to help players improve their skills and bring out the appeal of Shogi by utilizing generative AI. Players can receive advice in real time during a match and enjoy more strategic gameplay. In this way, the Shogi Support Agent can help players improve their skills and bring out the appeal of Shogi.

[0029] The shogi support agent according to this embodiment comprises a capture unit, an analysis unit, a provision unit, and a projection unit. The capture unit captures the state of the board. The capture unit is, for example, mounted on a glasses-type device and captures the state of the board in real time. The capture unit acquires an image of the board using a camera and transmits the image to the analysis unit. The capture unit may also incorporate filtering technology to minimize the effects of light reflection and shadows. The analysis unit analyzes the information captured by the capture unit using a generative AI. The analysis unit identifies the position and movement patterns of each piece, for example. The analysis unit identifies candidate moves by having the generative AI analyze the state of the board. The analysis unit may also identify similar positions by referring to past game data and provide them as reference information. The provision unit provides the next move and strategic advice identified by the analysis unit. The provision unit provides the next move and strategic advice to the user, for example, through a bone conduction speaker. The providing unit can, for example, estimate the user's emotions and adjust the way advice is presented based on the estimated emotions. The providing unit can also, for example, repeatedly provide specific advice in response to the user's voice commands. The projection unit projects the evaluation value of the game position and the best move. The projection unit can, for example, project the evaluation value of the game position and the best move onto the glasses portion to provide visual support. The projection unit can also, for example, track the user's gaze and display information in the optimal position. The projection unit can also, for example, estimate the user's emotions and adjust the way the projected content is displayed based on the estimated emotions. As a result, the shogi support agent according to this embodiment can help improve the player's skills and bring out the appeal of shogi.

[0030] The capture unit captures the state of the shogi board. The capture unit is, for example, mounted on a glasses-type device and captures the state of the board in real time. Specifically, the capture unit has a built-in high-resolution camera that can clearly capture the entire shogi board. The camera incorporates filtering technology to minimize the effects of light reflection and shadows, enabling accurate image acquisition under any lighting conditions. The capture unit is equipped with a high-speed communication module to transmit the acquired images to the analysis unit in real time, allowing for data transmission without delay. Furthermore, the capture unit has a function to track the user's gaze, allowing it to prioritize capturing the areas the user is focusing on. This enables detailed capture of positions and piece movements that the user is particularly interested in. The capture unit also has a function to automatically adjust the camera angle to follow the user's head movements, ensuring that the board is always captured from the optimal viewpoint. In this way, the capture unit supports the user in enjoying shogi with natural movements.

[0031] The analysis unit uses a generative AI to analyze the information captured by the capture unit. For example, the analysis unit identifies the position and movement patterns of each piece. Specifically, the generative AI uses image recognition technology to identify the position of each piece from the image of the board and recognizes the type and arrangement of the pieces. Furthermore, the generative AI refers to past game data and identifies positions similar to the current position, presenting multiple candidate moves. The generative AI uses a deep learning algorithm to calculate the evaluation value of each candidate move and identify the most advantageous move. In addition, the generative AI can learn the user's playing style and past game history to provide the user with optimal advice. The analysis unit transmits these analysis results to the provision unit in real time, providing immediate feedback to the user. Furthermore, the analysis unit continuously updates the analysis results in accordance with changes in the position, providing advice based on the latest information. In this way, the analysis unit supports the user in making optimal decisions at all times and helps improve their shogi skills.

[0032] The service provider offers next move suggestions and strategic advice identified by the analysis unit. For example, the service provider delivers these suggestions to the user via a bone conduction speaker. Specifically, the service provider incorporates speech synthesis technology to convey information transmitted from the analysis unit to the user, allowing the user to receive advice while still hearing ambient sounds without blocking their ears. The service provider also features emotion recognition technology to estimate the user's emotions, determining their emotional state from their facial expressions and tone of voice. This allows the service provider to offer appropriate advice depending on the situation, such as providing advice in a calm tone when the user is relaxed and adding words of encouragement when the user is tense. Furthermore, the service provider has a function to repeatedly provide specific advice in response to the user's voice commands, supporting the user's understanding. For example, if the user says "Tell me again," the service provider will repeat the previous advice. Additionally, the service provider can customize the content and presentation of advice according to the user's play style and preferences. This allows the service provider to provide personalized support to the user, enhancing the enjoyment of shogi.

[0033] The projection unit projects the evaluation value of the game position and the best move. For example, it projects the evaluation value and best move onto the glasses portion of the device, providing visual support. Specifically, the projection unit uses eye-tracking technology to display the most relevant information on the part the user is looking at. This allows the user to instantly check the necessary information without moving their eyes. In addition to the evaluation value and best move, the projection unit can also display candidate moves for the next move, the reasons for their evaluation, and information on similar past game positions. Furthermore, the projection unit can estimate the user's emotions and adjust the display method of the projected content based on the estimated emotions. For example, if the user is tense, the information will be displayed simply, while if they are relaxed, detailed information will be provided. The projection unit can also customize the displayed content according to the user's playing style and preferences. For example, if the user is interested in a particular strategy, information related to that strategy will be prioritized. In this way, the projection unit provides users with visually easy-to-understand information, deepening their understanding of shogi. Furthermore, the projection unit can continuously improve the displayed content based on user feedback, providing more effective support. This makes the projection unit a powerful tool for users to improve their skills while enjoying shogi.

[0034] The capture unit is mounted on a glasses-type device and can capture the state of the game board in real time. The capture unit, for example, acquires an image of the game board using a camera and transmits that image to the analysis unit. The capture unit can also introduce filtering technology to minimize the effects of light reflection and shadows. The capture unit can also integrate images from different angles using multiple cameras to acquire more accurate game board information. This allows the glasses-type device to capture the state of the game board in real time. Some or all of the above processing in the capture unit may be performed using AI, for example, or without AI. For example, the capture unit can input image data acquired by the camera into a generating AI and have the generating AI perform the processing of analyzing the state of the game board from the image data.

[0035] The analysis unit can analyze the board state using a generating AI and identify the position and movement patterns of each piece. For example, the analysis unit uses the generating AI to analyze the board state and identify candidate moves. The analysis unit can also, for example, refer to past game data to identify similar positions and provide them as reference information. The analysis unit can also, for example, consider the movement history of each piece to identify candidate moves with greater accuracy. In this way, by using a generating AI, the board state can be accurately analyzed and the next move can be identified. Some or all of the above processing in the analysis unit may be performed using a generating AI, or not. For example, the analysis unit can input image data of the board state into a generating AI and have the generating AI perform the process of identifying candidate moves.

[0036] The service provider can provide the user with next move suggestions and strategic advice through a bone conduction speaker. The service provider can, for example, estimate the user's emotions and adjust the way the advice is presented based on those emotions. The service provider can also, for example, repeatedly provide specific advice in response to the user's voice commands. When providing advice, the service provider can also provide multi-sensory advice using not only voice but also vibrations and light patterns. This allows for real-time advice to be provided to the user using a bone conduction speaker. Some or all of the above processing in the service provider may be performed using AI, for example, or without AI. For example, the service provider can use generative AI to estimate the user's emotions and adjust the way the advice is presented.

[0037] The projection unit can project evaluation values ​​of the game position and the best move onto the glasses portion, providing visual support. The projection unit can also, for example, track the user's gaze and display information in the optimal position. The projection unit can also, for example, estimate the user's emotions and adjust the display method of the projected content based on the estimated emotions of the user. The projection unit can also, for example, customize the displayed content according to the user's play style. This helps the user understand the game position by visually displaying evaluation values ​​and the best move. Some or all of the above processing in the projection unit may be performed using AI, for example, or without AI. For example, the projection unit can use generative AI to estimate the user's emotions and adjust the display method of the projected content.

[0038] The capture unit can incorporate filtering techniques to minimize the effects of light reflection and shadows when capturing the state of the game board. For example, the capture unit can use a polarizing filter to suppress light reflection. The capture unit can also arrange multiple light sources to reduce the effects of shadows and provide uniform illumination. The capture unit can also automatically correct light reflection and shadows using image processing techniques. This allows for the acquisition of more accurate game board information by suppressing the effects of light reflection and shadows. Some or all of the above processing in the capture unit may be performed using AI, for example, or without AI. For example, the capture unit can input image data acquired by a camera into a generating AI and have the generating AI perform processing to correct the effects of light reflection and shadows.

[0039] The capture unit can acquire more accurate board information by integrating images from different angles using multiple cameras when capturing the state of the game board. For example, the capture unit can place cameras at the four corners of the game board and integrate images from different angles. The capture unit can also acquire three-dimensional board information using, for example, a stereo camera. The capture unit can also analyze images from multiple cameras in real time and generate an optimal image. This allows for the integration of images from different angles by using multiple cameras, thereby acquiring accurate board information. Some or all of the above processing in the capture unit may be performed using, for example, AI, or without AI. For example, the capture unit can input video data acquired from multiple cameras into a generating AI and have the generating AI perform the integration of the images.

[0040] The capture unit can use user eye-tracking technology to prioritize capturing the parts of the board that the user is focusing on when capturing the board state. For example, the capture unit may be equipped with an eye-tracking sensor to prioritize capturing the pieces the user is looking at. The capture unit can also analyze the user's eye movements in real time to identify the areas of focus. For example, the capture unit can automatically capture important positions based on eye-tracking data. This allows for the priority capture of areas of focus by tracking the user's gaze. Some or all of the above processing in the capture unit may be performed using AI, for example, or without AI. For example, the capture unit can input data acquired by the eye-tracking sensor into a generating AI and have the generating AI perform the processing of analyzing the user's gaze.

[0041] The capture unit can capture specific pieces or areas based on user instructions using speech recognition technology when capturing the state of the board. For example, if the user instructs the capture unit to "capture the rook," the capture unit will capture the position of the rook. The capture unit can also capture the left side of the board if the user instructs it to "capture the left side of the board." The capture unit can also capture candidate moves if the user instructs it to "capture the next move." In this way, by using speech recognition technology, specific pieces or areas can be captured based on user instructions. Some or all of the above processing in the capture unit may be performed using AI, for example, or without AI. For example, the capture unit can input data acquired by speech recognition technology into a generating AI and have the generating AI perform the processing of analyzing the user's instructions.

[0042] The analysis unit, when analyzing the board state, can identify similar positions by referring to past game data and provide them as reference information. For example, the analysis unit can search a database of past games to identify similar positions. The analysis unit can also provide win / loss data for similar positions. The analysis unit can also provide the best move for similar positions as reference information. In this way, by referring to past game data, reference information for similar positions can be provided. Some or all of the above processing in the analysis unit may be performed using, for example, a generating AI, or without using a generating AI. For example, the analysis unit can input past game data into a generating AI and have the generating AI perform the process of identifying similar positions.

[0043] The analysis unit, when analyzing the board state, can identify candidates for the next move with greater accuracy by considering the movement history of each piece. For example, the analysis unit analyzes the movement history of each piece to identify candidates for the next move. The analysis unit can also analyze the positional relationships of the pieces from the movement history and propose the optimal move. For example, the analysis unit can present multiple candidates for the next move based on the movement history. This allows for more accurate identification of candidates for the next move by considering the movement history of each piece. Some or all of the above processing in the analysis unit may be performed using, for example, a generating AI, or without a generating AI. For example, the analysis unit can input the movement history data of each piece into a generating AI and have the generating AI perform the process of identifying candidates for the next move.

[0044] The analysis unit can learn the user's playing style when analyzing the board state and propose the optimal strategy based on that. For example, the analysis unit can analyze the user's past game data to identify the playing style. The analysis unit can also propose the optimal strategy based on the playing style. The analysis unit can also dynamically adjust the strategy in response to changes in the playing style. In this way, by learning the user's playing style, it can propose the optimal strategy. Some or all of the above processing in the analysis unit may be performed using, for example, a generative AI, or without a generative AI. For example, the analysis unit can input the user's past game data into a generative AI and have the generative AI perform the process of identifying the playing style.

[0045] The analysis unit can incorporate the latest strategies by referencing other players' game data in real time when analyzing the board state. For example, the analysis unit can acquire other players' game data in real time and reflect it in the analysis. The analysis unit can also incorporate the latest strategies into the analysis results and provide them to the user. For example, the analysis unit can suggest candidate moves based on other players' game data. In this way, the latest strategies can be incorporated by referring to other players' game data. Some or all of the above processing in the analysis unit may be performed using, for example, a generating AI, or without a generating AI. For example, the analysis unit can input other players' game data into a generating AI and have the generating AI perform the process of incorporating the latest strategies.

[0046] The service provider can provide individually customized advice by referring to the user's past game data when providing advice. For example, the service provider can analyze the user's past game data and provide individually customized advice. For example, the service provider can also provide advice that strengthens the user's weaknesses based on past game data. For example, the service provider can also provide advice tailored to the user's playing style. This allows for individually customized advice to be provided by referring to the user's past game data. Some or all of the above processing in the service provider may be performed using AI, for example, or without AI. For example, the service provider can input the user's past game data into a generating AI and have the generating AI perform the process of providing individually customized advice.

[0047] The service provider can provide advice in a multi-sensory manner, using not only voice but also vibration and light patterns. For example, the service provider can provide voice advice using a bone conduction speaker. For example, the service provider can use vibration to inform the user of important situations. For example, the service provider can use light patterns to visually indicate the next move. This allows for the provision of advice in a multi-sensory manner, using not only voice but also vibration and light patterns. Some or all of the above processing in the service provider may be performed using AI, for example, or without AI. For example, the service provider can use generative AI to estimate the user's emotions and adjust the way the advice is expressed.

[0048] The service provider can adjust the level of detail and frequency of advice given according to the user's play style. For example, the service provider can provide detailed advice according to the user's play style. The service provider can also adjust the frequency of advice according to the play style. The service provider can also dynamically adjust the content of advice according to changes in the play style. By adjusting the level of detail and frequency of advice according to the user's play style, more appropriate advice can be provided. Some or all of the above processing in the service provider may be performed using AI, for example, or without AI. For example, the service provider can input user play style data into a generating AI and have the generating AI perform the processing of adjusting the level of detail and frequency of advice.

[0049] The service provider can repeatedly provide specific advice in response to the user's voice commands when providing advice. For example, if the user instructs "again," the service provider will repeatedly provide the same advice. The service provider can also provide the next piece of advice if the user instructs "next piece of advice." The service provider can also provide detailed advice if the user instructs "tell me more details." In this way, by repeatedly providing specific advice in response to the user's voice commands, the service provider can provide advice that meets the user's needs. Some or all of the above processing in the service provider may be performed using AI, for example, or without AI. For example, the service provider can input data acquired by voice recognition technology into a generating AI and have the generating AI perform the processing of analyzing the user's instructions.

[0050] The projection unit can track the user's gaze and display information in the optimal position when projecting evaluation values ​​of a board position or the best move. For example, the projection unit may be equipped with an eye-tracking sensor to display evaluation values ​​and the best move at the position the user is looking at. The projection unit can also analyze the user's eye movements in real time and display information in the optimal position. For example, the projection unit can display important information within the user's field of view based on eye-tracking data. This allows information to be displayed in the optimal position by tracking the user's gaze. Some or all of the above processing in the projection unit may be performed using AI, for example, or without AI. For example, the projection unit can input data acquired by the eye-tracking sensor into a generating AI and have the generating AI perform the processing of analyzing the user's gaze.

[0051] The projection unit can customize the displayed content according to the user's playing style when projecting evaluation values ​​and best moves for a given position. For example, the projection unit adjusts the display method for evaluation values ​​and best moves according to the user's playing style. The projection unit can also highlight important information based on the playing style. For example, the projection unit can dynamically adjust the displayed content in response to changes in the playing style. This allows for the provision of more appropriate information by customizing the displayed content according to the user's playing style. Some or all of the above processing in the projection unit may be performed using AI, for example, or without AI. For example, the projection unit can input the user's playing style data into a generating AI and have the generating AI perform the process of customizing the displayed content.

[0052] The projection unit can select the optimal display method by considering the user's device information when projecting evaluation values ​​of a position or the best move. For example, if the user is using a smartphone, the projection unit provides a display method that matches the screen size. For example, if the user is using a tablet, the projection unit can also provide a display method optimized for a larger screen. For example, if the user is using a smartwatch, the projection unit can also provide a concise and highly visible display method. This allows the projection unit to select the optimal display method by considering the user's device information. Some or all of the above processing in the projection unit may be performed using AI, for example, or without AI. For example, the projection unit can input the user's device information into a generating AI and have the generating AI perform the process of selecting the optimal display method.

[0053] The projection unit can change its display content in response to user voice commands when projecting the evaluation value of a position or the best move. For example, if the user instructs the projection unit to "show the next move," it will display the candidates for the next move. The projection unit can also display the evaluation value of the position if the user instructs it to "show the evaluation value." The projection unit can also display detailed information if the user instructs it to "show details." In this way, by changing the display content in response to the user's voice commands, it is possible to provide information that meets the user's needs. Some or all of the above processing in the projection unit may be performed using AI, for example, or without AI. For example, the projection unit can input data acquired by voice recognition technology into a generating AI and have the generating AI perform the processing of analyzing the user's instructions.

[0054] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0055] The capture unit can incorporate filtering techniques to minimize the effects of light reflection and shadows when capturing the state of the game board. For example, a polarizing filter can be used to suppress light reflection. Multiple light sources can also be placed to provide uniform illumination and reduce the effects of shadows. Image processing technology can also be used to automatically correct light reflection and shadows. This allows for the acquisition of more accurate game board information by suppressing the effects of light reflection and shadows.

[0056] The capture unit can capture the state of the game board by using multiple cameras to integrate images from different angles, thereby obtaining more accurate game board information. For example, cameras can be placed at the four corners of the game board to integrate images from different angles. Stereo cameras can also be used to obtain three-dimensional game board information. Multiple camera images can be analyzed in real time to generate the optimal image. As a result, by using multiple cameras, images from different angles can be integrated to obtain accurate game board information.

[0057] The analysis unit, when analyzing the board state, can identify similar positions by referring to past game data and provide them as reference information. For example, it can search a database of past games to identify similar positions. It can also provide win / loss data for similar positions. It can also provide the best move for similar positions as reference information. In this way, by referring to past game data, it can provide reference information for similar positions.

[0058] The analysis unit, when analyzing the board state, can identify potential next moves with greater accuracy by considering the movement history of each piece. For example, it can analyze the movement history of each piece to identify potential next moves. It can also analyze the positional relationships of the pieces from the movement history and suggest the optimal move. Based on the movement history, it can also present multiple potential next moves. In this way, by considering the movement history of each piece, potential next moves can be identified with greater accuracy.

[0059] The system can provide advice not only through voice but also through vibrations and light patterns, offering a multi-sensory approach. For example, it can use bone conduction speakers to provide voice advice. Vibrations can be used to notify the user of important situations. Light patterns can be used to visually indicate the next move. This allows for the provision of advice in a multi-sensory manner, using not only voice but also vibrations and light patterns.

[0060] The following briefly describes the processing flow for example form 1.

[0061] Step 1: The capture unit captures the state of the game board. The capture unit is mounted on a glasses-type device, for example, and captures the state of the game board in real time. The capture unit acquires an image of the game board using a camera, for example, and transmits that image to the analysis unit. The capture unit may also incorporate filtering technology to minimize the effects of light reflection and shadows, for example. Step 2: The analysis unit uses a generating AI to analyze the information captured by the capture unit. The analysis unit identifies, for example, the position and movement patterns of each piece. The analysis unit uses, for example, the generating AI to analyze the board situation and identify candidate moves. The analysis unit can also, for example, refer to past game data to identify similar positions and provide them as reference information. Step 3: The service provider provides the next move or strategic advice identified by the analysis unit. The service provider provides the next move or strategic advice to the user, for example, through a bone conduction speaker. The service provider can also, for example, estimate the user's emotions and adjust the way the advice is expressed based on the estimated user emotions. The service provider can also, for example, repeatedly provide specific advice in response to the user's voice commands. Step 4: The projection unit projects the evaluation value of the position and the best move. The projection unit visually supports the player by projecting the evaluation value of the position and the best move onto, for example, the glasses portion of the device. The projection unit can also track the user's gaze and display the information in the optimal position. The projection unit can also estimate the user's emotions and adjust how the projected content is displayed based on the estimated emotions.

[0062] (Example of form 2) The Shogi support agent according to an embodiment of the present invention is a glasses-type device that helps Shogi players deeply understand strategy and enjoy the game. This device analyzes the board situation in real time and provides next move suggestions and strategic advice via a bone conduction speaker. The glasses project the evaluation value of the board and the best move, providing visual support. It supports skill improvement for players from beginners to advanced players and brings out the appeal of Shogi. The generating AI analyzes the board and provides optimal advice, enabling more strategic play. For example, when a user starts a Shogi game, the glasses-type device captures the board situation in real time. This information is sent to the generating AI, which analyzes the board situation. The generating AI analyzes the position and movement patterns of each piece and identifies candidate next moves. Next, the next move and strategic advice identified by the generating AI are provided to the user via a bone conduction speaker. For example, specific advice such as "It would be good to move the rook to the right next" is provided by voice. This allows the user to receive advice in real time during the game. Furthermore, the evaluation value of the board and the best move are projected onto the glasses. For example, the evaluation value of the current position is displayed as "+1.5," and the best move is visually supported as "move the bishop to the left." This allows the user to visually confirm the next move. This device is suitable for a wide range of players, from beginners to advanced players. It provides support for beginners to learn basic piece movements and strategies, and provides advanced strategic advice for advanced players. For example, beginners are given basic advice such as "In this position, it would be good to move the gold piece forward," while advanced players are given advanced advice such as "In this position, it would be good to move the rook to the right and attack the bishop on the next move." In this way, the Shogi Support Agent aims to help players improve their skills and bring out the appeal of Shogi by utilizing generative AI. Players can receive advice in real time during a match and enjoy more strategic gameplay. In this way, the Shogi Support Agent can help players improve their skills and bring out the appeal of Shogi.

[0063] The shogi support agent according to this embodiment comprises a capture unit, an analysis unit, a provision unit, and a projection unit. The capture unit captures the state of the board. The capture unit is, for example, mounted on a glasses-type device and captures the state of the board in real time. The capture unit acquires an image of the board using a camera and transmits the image to the analysis unit. The capture unit may also incorporate filtering technology to minimize the effects of light reflection and shadows. The analysis unit analyzes the information captured by the capture unit using a generative AI. The analysis unit identifies the position and movement patterns of each piece, for example. The analysis unit identifies candidate moves by having the generative AI analyze the state of the board. The analysis unit may also identify similar positions by referring to past game data and provide them as reference information. The provision unit provides the next move and strategic advice identified by the analysis unit. The provision unit provides the next move and strategic advice to the user, for example, through a bone conduction speaker. The providing unit can, for example, estimate the user's emotions and adjust the way advice is presented based on the estimated emotions. The providing unit can also, for example, repeatedly provide specific advice in response to the user's voice commands. The projection unit projects the evaluation value of the game position and the best move. The projection unit can, for example, project the evaluation value of the game position and the best move onto the glasses portion to provide visual support. The projection unit can also, for example, track the user's gaze and display information in the optimal position. The projection unit can also, for example, estimate the user's emotions and adjust the way the projected content is displayed based on the estimated emotions. As a result, the shogi support agent according to this embodiment can help improve the player's skills and bring out the appeal of shogi.

[0064] The capture unit captures the state of the shogi board. The capture unit is, for example, mounted on a glasses-type device and captures the state of the board in real time. Specifically, the capture unit has a built-in high-resolution camera that can clearly capture the entire shogi board. The camera incorporates filtering technology to minimize the effects of light reflection and shadows, enabling accurate image acquisition under any lighting conditions. The capture unit is equipped with a high-speed communication module to transmit the acquired images to the analysis unit in real time, allowing for data transmission without delay. Furthermore, the capture unit has a function to track the user's gaze, allowing it to prioritize capturing the areas the user is focusing on. This enables detailed capture of positions and piece movements that the user is particularly interested in. The capture unit also has a function to automatically adjust the camera angle to follow the user's head movements, ensuring that the board is always captured from the optimal viewpoint. In this way, the capture unit supports the user in enjoying shogi with natural movements.

[0065] The analysis unit uses a generative AI to analyze the information captured by the capture unit. For example, the analysis unit identifies the position and movement patterns of each piece. Specifically, the generative AI uses image recognition technology to identify the position of each piece from the image of the board and recognizes the type and arrangement of the pieces. Furthermore, the generative AI refers to past game data and identifies positions similar to the current position, presenting multiple candidate moves. The generative AI uses a deep learning algorithm to calculate the evaluation value of each candidate move and identify the most advantageous move. In addition, the generative AI can learn the user's playing style and past game history to provide the user with optimal advice. The analysis unit transmits these analysis results to the provision unit in real time, providing immediate feedback to the user. Furthermore, the analysis unit continuously updates the analysis results in accordance with changes in the position, providing advice based on the latest information. In this way, the analysis unit supports the user in making optimal decisions at all times and helps improve their shogi skills.

[0066] The service provider offers next move suggestions and strategic advice identified by the analysis unit. For example, the service provider delivers these suggestions to the user via a bone conduction speaker. Specifically, the service provider incorporates speech synthesis technology to convey information transmitted from the analysis unit to the user, allowing the user to receive advice while still hearing ambient sounds without blocking their ears. The service provider also features emotion recognition technology to estimate the user's emotions, determining their emotional state from their facial expressions and tone of voice. This allows the service provider to offer appropriate advice depending on the situation, such as providing advice in a calm tone when the user is relaxed and adding words of encouragement when the user is tense. Furthermore, the service provider has a function to repeatedly provide specific advice in response to the user's voice commands, supporting the user's understanding. For example, if the user says "Tell me again," the service provider will repeat the previous advice. Additionally, the service provider can customize the content and presentation of advice according to the user's play style and preferences. This allows the service provider to provide personalized support to the user, enhancing the enjoyment of shogi.

[0067] The projection unit projects the evaluation value of the game position and the best move. For example, it projects the evaluation value and best move onto the glasses portion of the device, providing visual support. Specifically, the projection unit uses eye-tracking technology to display the most relevant information on the part the user is looking at. This allows the user to instantly check the necessary information without moving their eyes. In addition to the evaluation value and best move, the projection unit can also display candidate moves for the next move, the reasons for their evaluation, and information on similar past game positions. Furthermore, the projection unit can estimate the user's emotions and adjust the display method of the projected content based on the estimated emotions. For example, if the user is tense, the information will be displayed simply, while if they are relaxed, detailed information will be provided. The projection unit can also customize the displayed content according to the user's playing style and preferences. For example, if the user is interested in a particular strategy, information related to that strategy will be prioritized. In this way, the projection unit provides users with visually easy-to-understand information, deepening their understanding of shogi. Furthermore, the projection unit can continuously improve the displayed content based on user feedback, providing more effective support. This makes the projection unit a powerful tool for users to improve their skills while enjoying shogi.

[0068] The capture unit is mounted on a glasses-type device and can capture the state of the game board in real time. The capture unit, for example, acquires an image of the game board using a camera and transmits that image to the analysis unit. The capture unit can also introduce filtering technology to minimize the effects of light reflection and shadows. The capture unit can also integrate images from different angles using multiple cameras to acquire more accurate game board information. This allows the glasses-type device to capture the state of the game board in real time. Some or all of the above processing in the capture unit may be performed using AI, for example, or without AI. For example, the capture unit can input image data acquired by the camera into a generating AI and have the generating AI perform the processing of analyzing the state of the game board from the image data.

[0069] The analysis unit can analyze the board state using a generating AI and identify the position and movement patterns of each piece. For example, the analysis unit uses the generating AI to analyze the board state and identify candidate moves. The analysis unit can also, for example, refer to past game data to identify similar positions and provide them as reference information. The analysis unit can also, for example, consider the movement history of each piece to identify candidate moves with greater accuracy. In this way, by using a generating AI, the board state can be accurately analyzed and the next move can be identified. Some or all of the above processing in the analysis unit may be performed using a generating AI, or not. For example, the analysis unit can input image data of the board state into a generating AI and have the generating AI perform the process of identifying candidate moves.

[0070] The service provider can provide the user with next move suggestions and strategic advice through a bone conduction speaker. The service provider can, for example, estimate the user's emotions and adjust the way the advice is presented based on those emotions. The service provider can also, for example, repeatedly provide specific advice in response to the user's voice commands. When providing advice, the service provider can also provide multi-sensory advice using not only voice but also vibrations and light patterns. This allows for real-time advice to be provided to the user using a bone conduction speaker. Some or all of the above processing in the service provider may be performed using AI, for example, or without AI. For example, the service provider can use generative AI to estimate the user's emotions and adjust the way the advice is presented.

[0071] The projection unit can project evaluation values ​​of the game position and the best move onto the glasses portion, providing visual support. The projection unit can also, for example, track the user's gaze and display information in the optimal position. The projection unit can also, for example, estimate the user's emotions and adjust the display method of the projected content based on the estimated emotions of the user. The projection unit can also, for example, customize the displayed content according to the user's play style. This helps the user understand the game position by visually displaying evaluation values ​​and the best move. Some or all of the above processing in the projection unit may be performed using AI, for example, or without AI. For example, the projection unit can use generative AI to estimate the user's emotions and adjust the display method of the projected content.

[0072] The capture unit can estimate the user's emotions and adjust the timing of capturing the board based on the estimated emotions. For example, if the user is concentrating, the capture unit can increase the capture frequency to acquire detailed board information. For example, if the user is relaxed, the capture unit can decrease the capture frequency to support a more natural game. For example, if the user is anxious, the capture unit can capture only important positions to provide advice. By adjusting the capture timing according to the user's emotions, board information can be acquired at a more appropriate time. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or a generative AI. The generative AI is, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above processing in the capture unit may be performed using AI, or not using AI. For example, the capture unit can input image data acquired by the camera into the generative AI and have the generative AI perform the process of estimating the user's emotions.

[0073] The capture unit can incorporate filtering techniques to minimize the effects of light reflection and shadows when capturing the state of the game board. For example, the capture unit can use a polarizing filter to suppress light reflection. The capture unit can also arrange multiple light sources to reduce the effects of shadows and provide uniform illumination. The capture unit can also automatically correct light reflection and shadows using image processing techniques. This allows for the acquisition of more accurate game board information by suppressing the effects of light reflection and shadows. Some or all of the above processing in the capture unit may be performed using AI, for example, or without AI. For example, the capture unit can input image data acquired by a camera into a generating AI and have the generating AI perform processing to correct the effects of light reflection and shadows.

[0074] The capture unit can acquire more accurate board information by integrating images from different angles using multiple cameras when capturing the state of the game board. For example, the capture unit can place cameras at the four corners of the game board and integrate images from different angles. The capture unit can also acquire three-dimensional board information using, for example, a stereo camera. The capture unit can also analyze images from multiple cameras in real time and generate an optimal image. This allows for the integration of images from different angles by using multiple cameras, thereby acquiring accurate board information. Some or all of the above processing in the capture unit may be performed using, for example, AI, or without AI. For example, the capture unit can input video data acquired from multiple cameras into a generating AI and have the generating AI perform the integration of the images.

[0075] The capture unit can estimate the user's emotions and dynamically change the area of ​​the board to capture based on the estimated emotions. For example, if the user is excited, the capture unit may prioritize capturing the area around important pieces. If the user is relaxed, the capture unit may also capture a wide area of ​​the board. If the user is focused, the capture unit may also focus on capturing a specific position. This allows for prioritizing the capture of important parts by changing the area to capture according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or a generative AI. The generative AI may be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above processing in the capture unit may be performed using AI or not using AI. For example, the capture unit can input image data acquired by the camera into the generative AI and have the generative AI perform the process of estimating the user's emotions.

[0076] The capture unit can use user eye-tracking technology to prioritize capturing the parts of the board that the user is focusing on when capturing the board state. For example, the capture unit may be equipped with an eye-tracking sensor to prioritize capturing the pieces the user is looking at. The capture unit can also analyze the user's eye movements in real time to identify the areas of focus. For example, the capture unit can automatically capture important positions based on eye-tracking data. This allows for the priority capture of areas of focus by tracking the user's gaze. Some or all of the above processing in the capture unit may be performed using AI, for example, or without AI. For example, the capture unit can input data acquired by the eye-tracking sensor into a generating AI and have the generating AI perform the processing of analyzing the user's gaze.

[0077] The capture unit can capture specific pieces or areas based on user instructions using speech recognition technology when capturing the state of the board. For example, if the user instructs the capture unit to "capture the rook," the capture unit will capture the position of the rook. The capture unit can also capture the left side of the board if the user instructs it to "capture the left side of the board." The capture unit can also capture candidate moves if the user instructs it to "capture the next move." In this way, by using speech recognition technology, specific pieces or areas can be captured based on user instructions. Some or all of the above processing in the capture unit may be performed using AI, for example, or without AI. For example, the capture unit can input data acquired by speech recognition technology into a generating AI and have the generating AI perform the processing of analyzing the user's instructions.

[0078] The analysis unit can estimate the user's emotions and adjust the presentation method of the analysis results based on the estimated user emotions. For example, if the user is relaxed, the analysis unit can provide detailed analysis results. For example, if the user is anxious, the analysis unit can also provide concise analysis results. For example, if the user is focused, the analysis unit can highlight important points in the analysis results. This allows for the provision of more appropriate information by adjusting the presentation method of the analysis results according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above-described processes in the analysis unit may be performed using a generative AI, or not using a generative AI. For example, the analysis unit can input user emotion data into a generative AI and have the generative AI perform the process of adjusting the presentation method of the analysis results.

[0079] The analysis unit, when analyzing the board state, can identify similar positions by referring to past game data and provide them as reference information. For example, the analysis unit can search a database of past games to identify similar positions. The analysis unit can also provide win / loss data for similar positions. The analysis unit can also provide the best move for similar positions as reference information. In this way, by referring to past game data, reference information for similar positions can be provided. Some or all of the above processing in the analysis unit may be performed using, for example, a generating AI, or without using a generating AI. For example, the analysis unit can input past game data into a generating AI and have the generating AI perform the process of identifying similar positions.

[0080] The analysis unit, when analyzing the board state, can identify candidates for the next move with greater accuracy by considering the movement history of each piece. For example, the analysis unit analyzes the movement history of each piece to identify candidates for the next move. The analysis unit can also analyze the positional relationships of the pieces from the movement history and propose the optimal move. For example, the analysis unit can present multiple candidates for the next move based on the movement history. This allows for more accurate identification of candidates for the next move by considering the movement history of each piece. Some or all of the above processing in the analysis unit may be performed using, for example, a generating AI, or without a generating AI. For example, the analysis unit can input the movement history data of each piece into a generating AI and have the generating AI perform the process of identifying candidates for the next move.

[0081] The analysis unit can estimate the user's emotions and adjust the level of detail in the analysis results based on the estimated emotions. For example, if the user is relaxed, the analysis unit can provide detailed analysis results. For example, if the user is anxious, the analysis unit can also provide concise analysis results. For example, if the user is focused, the analysis unit can highlight important points in the analysis results. This allows for the provision of more appropriate information by adjusting the level of detail in the analysis results according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or a generative AI. The generative AI is, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processes in the analysis unit may be performed using a generative AI, or not. For example, the analysis unit can input user emotion data into a generative AI and have the generative AI perform the process of adjusting the level of detail in the analysis results.

[0082] The analysis unit can learn the user's playing style when analyzing the board state and propose the optimal strategy based on that. For example, the analysis unit can analyze the user's past game data to identify the playing style. The analysis unit can also propose the optimal strategy based on the playing style. The analysis unit can also dynamically adjust the strategy in response to changes in the playing style. In this way, by learning the user's playing style, it can propose the optimal strategy. Some or all of the above processing in the analysis unit may be performed using, for example, a generative AI, or without a generative AI. For example, the analysis unit can input the user's past game data into a generative AI and have the generative AI perform the process of identifying the playing style.

[0083] The analysis unit can incorporate the latest strategies by referencing other players' game data in real time when analyzing the board state. For example, the analysis unit can acquire other players' game data in real time and reflect it in the analysis. The analysis unit can also incorporate the latest strategies into the analysis results and provide them to the user. For example, the analysis unit can suggest candidate moves based on other players' game data. In this way, the latest strategies can be incorporated by referring to other players' game data. Some or all of the above processing in the analysis unit may be performed using, for example, a generating AI, or without a generating AI. For example, the analysis unit can input other players' game data into a generating AI and have the generating AI perform the process of incorporating the latest strategies.

[0084] The service provider can estimate the user's emotions and adjust the way advice is expressed based on the estimated emotions. For example, if the user is relaxed, the service provider can provide detailed advice. If the user is anxious, the service provider can also provide concise advice. If the user is focused, the service provider can also provide advice that emphasizes important points. By adjusting the way advice is expressed according to the user's emotions, more appropriate advice can be provided. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the service provider may be performed using AI or not using AI. For example, the service provider can input user emotion data into a generative AI and have the generative AI perform the process of adjusting the way advice is expressed.

[0085] The service provider can provide individually customized advice by referring to the user's past game data when providing advice. For example, the service provider can analyze the user's past game data and provide individually customized advice. For example, the service provider can also provide advice that strengthens the user's weaknesses based on past game data. For example, the service provider can also provide advice tailored to the user's playing style. This allows for individually customized advice to be provided by referring to the user's past game data. Some or all of the above processing in the service provider may be performed using AI, for example, or without AI. For example, the service provider can input the user's past game data into a generating AI and have the generating AI perform the process of providing individually customized advice.

[0086] The service provider can provide advice in a multi-sensory manner, using not only voice but also vibration and light patterns. For example, the service provider can provide voice advice using a bone conduction speaker. For example, the service provider can use vibration to inform the user of important situations. For example, the service provider can use light patterns to visually indicate the next move. This allows for the provision of advice in a multi-sensory manner, using not only voice but also vibration and light patterns. Some or all of the above processing in the service provider may be performed using AI, for example, or without AI. For example, the service provider can use generative AI to estimate the user's emotions and adjust the way the advice is expressed.

[0087] The service provider can estimate the user's emotions and adjust the timing of advice based on the estimated emotions. For example, if the user is relaxed, the service provider may delay the timing of advice. For example, if the user is anxious, the service provider may advance the timing of advice. For example, if the user is focused, the service provider may provide advice at a crucial moment. By adjusting the timing of advice according to the user's emotions, advice can be provided at a more appropriate time. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the service provider may be performed using AI, for example, or not using AI. For example, the service provider can input user emotion data into a generative AI and have the generative AI perform the process of adjusting the timing of advice.

[0088] The service provider can adjust the level of detail and frequency of advice given according to the user's play style. For example, the service provider can provide detailed advice according to the user's play style. The service provider can also adjust the frequency of advice according to the play style. The service provider can also dynamically adjust the content of advice according to changes in the play style. By adjusting the level of detail and frequency of advice according to the user's play style, more appropriate advice can be provided. Some or all of the above processing in the service provider may be performed using AI, for example, or without AI. For example, the service provider can input user play style data into a generating AI and have the generating AI perform the processing of adjusting the level of detail and frequency of advice.

[0089] The service provider can repeatedly provide specific advice in response to the user's voice commands when providing advice. For example, if the user instructs "again," the service provider will repeatedly provide the same advice. The service provider can also provide the next piece of advice if the user instructs "next piece of advice." The service provider can also provide detailed advice if the user instructs "tell me more details." In this way, by repeatedly providing specific advice in response to the user's voice commands, the service provider can provide advice that meets the user's needs. Some or all of the above processing in the service provider may be performed using AI, for example, or without AI. For example, the service provider can input data acquired by voice recognition technology into a generating AI and have the generating AI perform the processing of analyzing the user's instructions.

[0090] The projection unit can estimate the user's emotions and adjust the display method of the projected content based on the estimated user emotions. For example, if the user is relaxed, the projection unit can display detailed information. For example, if the user is anxious, the projection unit can also display concise information. For example, if the user is focused, the projection unit can highlight important information. In this way, by adjusting the display method of the projected content according to the user's emotions, more appropriate information can be provided. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or a generative AI. The generative AI is, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above processing in the projection unit may be performed using AI, for example, or without AI. For example, the projection unit can input user emotion data into the generative AI and cause the generative AI to perform the process of adjusting the display method of the projected content.

[0091] The projection unit can track the user's gaze and display information in the optimal position when projecting evaluation values ​​of a board position or the best move. For example, the projection unit may be equipped with an eye-tracking sensor to display evaluation values ​​and the best move at the position the user is looking at. The projection unit can also analyze the user's eye movements in real time and display information in the optimal position. For example, the projection unit can display important information within the user's field of view based on eye-tracking data. This allows information to be displayed in the optimal position by tracking the user's gaze. Some or all of the above processing in the projection unit may be performed using AI, for example, or without AI. For example, the projection unit can input data acquired by the eye-tracking sensor into a generating AI and have the generating AI perform the processing of analyzing the user's gaze.

[0092] The projection unit can customize the displayed content according to the user's playing style when projecting evaluation values ​​and best moves for a given position. For example, the projection unit adjusts the display method for evaluation values ​​and best moves according to the user's playing style. The projection unit can also highlight important information based on the playing style. For example, the projection unit can dynamically adjust the displayed content in response to changes in the playing style. This allows for the provision of more appropriate information by customizing the displayed content according to the user's playing style. Some or all of the above processing in the projection unit may be performed using AI, for example, or without AI. For example, the projection unit can input the user's playing style data into a generating AI and have the generating AI perform the process of customizing the displayed content.

[0093] The projection unit can estimate the user's emotions and determine the priority of projected content based on the estimated emotions. For example, if the user is relaxed, the projection unit may prioritize displaying detailed information. If the user is anxious, the projection unit may also prioritize displaying important information. If the user is focused, the projection unit may also prioritize displaying candidates for the next move. This allows for the provision of more appropriate information by prioritizing projected content according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above-described processing in the projection unit may be performed using AI or not. For example, the projection unit can input user emotion data into the generative AI and have the generative AI perform the process of determining the priority of projected content.

[0094] The projection unit can select the optimal display method by considering the user's device information when projecting evaluation values ​​of a position or the best move. For example, if the user is using a smartphone, the projection unit provides a display method that matches the screen size. For example, if the user is using a tablet, the projection unit can also provide a display method optimized for a larger screen. For example, if the user is using a smartwatch, the projection unit can also provide a concise and highly visible display method. This allows the projection unit to select the optimal display method by considering the user's device information. Some or all of the above processing in the projection unit may be performed using AI, for example, or without AI. For example, the projection unit can input the user's device information into a generating AI and have the generating AI perform the process of selecting the optimal display method.

[0095] The projection unit can change its display content in response to user voice commands when projecting the evaluation value of a position or the best move. For example, if the user instructs the projection unit to "show the next move," it will display the candidates for the next move. The projection unit can also display the evaluation value of the position if the user instructs it to "show the evaluation value." The projection unit can also display detailed information if the user instructs it to "show details." In this way, by changing the display content in response to the user's voice commands, it is possible to provide information that meets the user's needs. Some or all of the above processing in the projection unit may be performed using AI, for example, or without AI. For example, the projection unit can input data acquired by voice recognition technology into a generating AI and have the generating AI perform the processing of analyzing the user's instructions.

[0096] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0097] The analysis unit can estimate the user's emotions and adjust the presentation method of the analysis results based on the estimated emotions. For example, if the user is relaxed, it can provide detailed analysis results. If the user is anxious, it can provide concise analysis results. If the user is focused, it can highlight important points in the analysis results. In this way, by adjusting the presentation method of the analysis results according to the user's emotions, more appropriate information can be provided.

[0098] The service provider can estimate the user's emotions and adjust the way advice is presented based on those emotions. For example, if the user is relaxed, detailed advice can be provided. If the user is anxious, concise advice can be provided. If the user is focused, advice can be presented emphasizing key points. By adjusting the way advice is presented according to the user's emotions, more appropriate advice can be provided.

[0099] The projection unit can estimate the user's emotions and adjust how the projected content is displayed based on those emotions. For example, if the user is relaxed, detailed information can be displayed. If the user is anxious, concise information can be displayed. If the user is focused, important information can be highlighted. By adjusting how the projected content is displayed according to the user's emotions, more appropriate information can be provided.

[0100] The capture unit can estimate the user's emotions and adjust the timing of board captures based on those emotions. For example, if the user is concentrating, the capture frequency can be increased to obtain detailed board information. If the user is relaxed, the capture frequency can be decreased to support a more natural game. If the user is anxious, only important positions can be captured to provide advice. In this way, by adjusting the capture timing according to the user's emotions, board information can be obtained at a more appropriate time.

[0101] The service provider can estimate the user's emotions and adjust the timing of advice based on those emotions. For example, if the user is relaxed, the advice can be delayed. If the user is anxious, the advice can be advanced. If the user is focused, advice can be provided at crucial moments. By adjusting the timing of advice according to the user's emotions, advice can be provided at a more appropriate time.

[0102] The capture unit can incorporate filtering techniques to minimize the effects of light reflection and shadows when capturing the state of the game board. For example, a polarizing filter can be used to suppress light reflection. Multiple light sources can also be placed to provide uniform illumination and reduce the effects of shadows. Image processing technology can also be used to automatically correct light reflection and shadows. This allows for the acquisition of more accurate game board information by suppressing the effects of light reflection and shadows.

[0103] The capture unit can capture the state of the game board by using multiple cameras to integrate images from different angles, thereby obtaining more accurate game board information. For example, cameras can be placed at the four corners of the game board to integrate images from different angles. Stereo cameras can also be used to obtain three-dimensional game board information. Multiple camera images can be analyzed in real time to generate the optimal image. As a result, by using multiple cameras, images from different angles can be integrated to obtain accurate game board information.

[0104] The analysis unit, when analyzing the board state, can identify similar positions by referring to past game data and provide them as reference information. For example, it can search a database of past games to identify similar positions. It can also provide win / loss data for similar positions. It can also provide the best move for similar positions as reference information. In this way, by referring to past game data, it can provide reference information for similar positions.

[0105] The analysis unit, when analyzing the board state, can identify potential next moves with greater accuracy by considering the movement history of each piece. For example, it can analyze the movement history of each piece to identify potential next moves. It can also analyze the positional relationships of the pieces from the movement history and suggest the optimal move. Based on the movement history, it can also present multiple potential next moves. In this way, by considering the movement history of each piece, potential next moves can be identified with greater accuracy.

[0106] The system can provide advice not only through voice but also through vibrations and light patterns, offering a multi-sensory approach. For example, it can use bone conduction speakers to provide voice advice. Vibrations can be used to notify the user of important situations. Light patterns can be used to visually indicate the next move. This allows for the provision of advice in a multi-sensory manner, using not only voice but also vibrations and light patterns.

[0107] The following briefly describes the processing flow for example form 2.

[0108] Step 1: The capture unit captures the state of the game board. The capture unit is mounted on a glasses-type device, for example, and captures the state of the game board in real time. The capture unit acquires an image of the game board using a camera, for example, and transmits that image to the analysis unit. The capture unit may also incorporate filtering technology to minimize the effects of light reflection and shadows, for example. Step 2: The analysis unit uses a generating AI to analyze the information captured by the capture unit. The analysis unit identifies, for example, the position and movement patterns of each piece. The analysis unit uses, for example, the generating AI to analyze the board situation and identify candidate moves. The analysis unit can also, for example, refer to past game data to identify similar positions and provide them as reference information. Step 3: The service provider provides the next move or strategic advice identified by the analysis unit. The service provider provides the next move or strategic advice to the user, for example, through a bone conduction speaker. The service provider can also, for example, estimate the user's emotions and adjust the way the advice is expressed based on the estimated user emotions. The service provider can also, for example, repeatedly provide specific advice in response to the user's voice commands. Step 4: The projection unit projects the evaluation value of the position and the best move. The projection unit visually supports the player by projecting the evaluation value of the position and the best move onto, for example, the glasses portion of the device. The projection unit can also track the user's gaze and display the information in the optimal position. The projection unit can also estimate the user's emotions and adjust how the projected content is displayed based on the estimated emotions.

[0109] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0110] Data generation model 58 is a form of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include text generation AI, image generation AI, and multimodal generation AI. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats from audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each of the above parts is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example.Furthermore, processing performed by AI, including generative AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI, including generative AI.

[0111] Furthermore, the processing performed by the data processing system 10 described above is carried out by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but it may also be carried out by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0112] Each of the multiple elements described above, including the capture unit, analysis unit, provision unit, and projection unit, is implemented, for example, by at least one of the smart device 14 and the data processing unit 12. For example, the capture unit captures the state of the board using the camera 42 of the smart device 14, the analysis unit performs analysis using generated AI by the identification processing unit 290 of the data processing unit 12, the provision unit provides advice through the bone conduction speaker 40B of the smart device 14, and the projection unit projects the evaluation value of the position and the best move onto the display 40A of the smart device 14. The correspondence between each unit and the device or control unit is not limited to the example described above, and various modifications are possible.

[0113] [Second Embodiment] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0114] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0115] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0116] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0117] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0118] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0119] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0120] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing by the processor 28. The storage 32 stores the specific processing program 56.

[0121] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0122] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0123] In the smart glasses 214, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0124] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0125] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0126] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0127] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0128] Each of the multiple elements described above, including the capture unit, analysis unit, provision unit, and projection unit, is implemented in at least one of the smart glasses 214 and the data processing unit 12. For example, the capture unit captures the state of the board using the camera 42 of the smart glasses 214, the analysis unit performs analysis using generated AI by the identification processing unit 290 of the data processing unit 12, the provision unit provides advice through the bone conduction speaker 240 of the smart glasses 214, and the projection unit projects the evaluation value of the position and the best move onto the display of the smart glasses 214. The correspondence between each unit and the device or control unit is not limited to the example described above and can be modified in various ways.

[0129] [Third Embodiment] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0130] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0131] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0132] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0133] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0134] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0135] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0136] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0137] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0138] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0139] In the headset terminal 314, specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50 and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific program 60 executed on the RAM 48. The headset terminal 314 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0140] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0141] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0142] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0143] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset terminal 314, but may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the headset terminal 314 or an external device, and the headset terminal 314 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0144] Each of the multiple elements described above, including the capture unit, analysis unit, provision unit, and projection unit, is implemented, for example, by at least one of the headset terminal 314 and the data processing unit 12. For example, the capture unit captures the state of the board using the camera 42 of the headset terminal 314, the analysis unit performs analysis using generated AI by the identification processing unit 290 of the data processing unit 12, the provision unit provides advice through the bone conduction speaker 240 of the headset terminal 314, and the projection unit projects the evaluation value of the position and the best move onto the display 343 of the headset terminal 314. The correspondence between each unit and the device or control unit is not limited to the example described above, and various modifications are possible.

[0145] [Fourth Embodiment] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0146] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0147] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0148] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0149] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0150] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS image sensor or CCD image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0151] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0152] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. The robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0153] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0154] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0155] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0156] In robot 414, specific processing is performed by processor 46. A specific program 60 is stored in storage 50. Processor 46 reads the specific program 60 from storage 50 and executes it on RAM 48. The specific processing is achieved by processor 46 acting as a control unit 46A according to the specific program 60 executed on RAM 48. Robot 414 also has data generation model 58 and emotion identification model 59, similar to those of the robot, and can perform processing similar to that of the specific processing unit 290 using these models.

[0157] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0158] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0159] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0160] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the robot 414 or an external device, and the robot 414 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0161] Each of the multiple elements described above, including the capture unit, analysis unit, provision unit, and projection unit, is implemented, for example, by at least one of the robot 414 and the data processing unit 12. For example, the capture unit captures the state of the board using the camera 42 of the robot 414, the analysis unit performs analysis using generated AI by the identification processing unit 290 of the data processing unit 12, the provision unit provides advice through the bone conduction speaker 240 of the robot 414, and the projection unit projects the evaluation value of the position and the best move onto the display of the robot 414. The correspondence between each unit and the device or control unit is not limited to the example described above, and various modifications are possible.

[0162] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0163] Figure 9 shows the emotion map 400, in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0164] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0165] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0166] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, and motorcycles, emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated based, for example, on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0167] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0168] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0169] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing method for the specific process may be used, which includes computer 22 and multiple other computers.

[0170] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0171] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0172] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0173] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0174] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0175] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0176] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0177] Furthermore, although the above-described examples were divided into four embodiments, some or all of these embodiments may be combined. Also, the smart device 14, smart glasses 214, headset terminal 314, and robot 414 are just examples, and they may be combined, or other devices may be used. Also, although the above-described examples were divided into two embodiments, Embodiment 1 and Embodiment 2, these may be combined.

[0178] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and other things that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0179] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0180] (Note 1) A capture unit that captures the state of the game board, An analysis unit that analyzes the information captured by the aforementioned capture unit, The analysis unit provides the next move and strategic advice identified by the analysis unit, It comprises a projection unit that projects the evaluation value of the position and the best move. A system characterized by the following features. (Note 2) The aforementioned capture unit is It is integrated into a glasses-type device and captures the state of the game board in real time. The system described in Appendix 1, characterized by the features described herein. (Note 3) The aforementioned analysis unit, The AI ​​is used to analyze the board state and identify the position and movement patterns of each piece. The system described in Appendix 1, characterized by the features described herein. (Note 4) The aforementioned supply unit is, Bone conduction speakers provide users with next-step advice and strategic guidance. The system described in Appendix 1, characterized by the features described herein. (Note 5) The projection unit is The system projects evaluation values ​​and the best move onto the glasses portion, providing visual support. The system described in Appendix 1, characterized by the features described herein. (Note 6) The aforementioned capture unit is The system estimates the user's emotions and adjusts the timing of capturing the game board based on those emotions. The system described in Appendix 1, characterized by the features described herein. (Note 7) The aforementioned capture unit is When capturing the state of the game board, filtering technology is introduced to minimize the effects of light reflection and shadows. The system described in Appendix 1, characterized by the features described herein. (Note 8) The aforementioned capture unit is When capturing the state of the game board, multiple cameras are used to integrate footage from different angles, thereby obtaining more accurate information about the game board. The system described in Appendix 1, characterized by the features described herein. (Note 9) The aforementioned capture unit is It estimates the user's emotions and dynamically changes the area of ​​the game board to capture based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 10) The aforementioned capture unit is When capturing the state of the game board, user eye-tracking technology is used to prioritize capturing the area the user is focusing on. The system described in Appendix 1, characterized by the features described herein. (Note 11) The aforementioned capture unit is When capturing the board state, voice recognition technology is used to capture specific pieces or areas based on user instructions. The system described in Appendix 1, characterized by the features described herein. (Note 12) The aforementioned analysis unit, It estimates the user's emotions and adjusts the presentation method of the analysis results based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 13) The aforementioned analysis unit, When analyzing the board state, we refer to past game data to identify similar positions and provide them as reference information. The system described in Appendix 1, characterized by the features described herein. (Note 14) The aforementioned analysis unit, When analyzing the board state, considering the movement history of each piece allows for more accurate identification of potential next moves. The system described in Appendix 1, characterized by the features described herein. (Note 15) The aforementioned analysis unit, It estimates the user's emotions and adjusts the level of detail in the analysis results based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 16) The aforementioned analysis unit, When analyzing the board state, the system learns the user's play style and proposes the optimal strategy based on that. The system described in Appendix 1, characterized by the features described herein. (Note 17) The aforementioned analysis unit, When analyzing the board state, the system references real-time game data from other players and incorporates the latest strategies. The system described in Appendix 1, characterized by the features described herein. (Note 18) The aforementioned supply unit is, It estimates the user's emotions and adjusts the way advice is presented based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 19) The aforementioned supply unit is, When providing advice, we refer to the user's past game data to provide individually customized advice. The system described in Appendix 1, characterized by the features described herein. (Note 20) The aforementioned supply unit is, When providing advice, we offer multi-sensory guidance using not only voice but also vibrations and light patterns. The system described in Appendix 1, characterized by the features described herein. (Note 21) The aforementioned supply unit is, It estimates the user's emotions and adjusts the timing of advice based on those emotions. The system described in Appendix 1, characterized by the features described herein. (Note 22) The aforementioned supply unit is, When providing advice, adjust the level of detail and frequency of the advice according to the user's play style. The system described in Appendix 1, characterized by the features described herein. (Note 23) The aforementioned supply unit is, When providing advice, the system will repeatedly provide specific advice in response to the user's voice commands. The system described in Appendix 1, characterized by the features described herein. (Note 24) The projection unit is It estimates the user's emotions and adjusts how projected content is displayed based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 25) The projection unit is When projecting evaluation values ​​and the best move for a given position, the system tracks the user's gaze to display the information in the optimal position. The system described in Appendix 1, characterized by the features described herein. (Note 26) The projection unit is When projecting the evaluation value of the game position and the best move, the displayed content can be customized according to the user's play style. The system described in Appendix 1, characterized by the features described herein. (Note 27) The projection unit is It estimates the user's emotions and determines the priority of projection content based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 28) The projection unit is When projecting evaluation values ​​and the best move for a given position, the optimal display method is selected considering the user's device information. The system described in Appendix 1, characterized by the features described herein. (Note 29) The projection unit is When projecting the evaluation value of the game position or the best move, the displayed content changes according to the user's voice commands. The system described in Appendix 1, characterized by the features described herein. [Explanation of Symbols]

[0181] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots

Claims

1. A capture unit that captures the state of the game board, An analysis unit that analyzes the information captured by the aforementioned capture unit, The analysis unit provides the next move and strategic advice identified by the analysis unit, It comprises a projection unit that projects the evaluation value of the position and the best move. A system characterized by the following features.

2. The aforementioned capture unit is It is integrated into a glasses-type device and captures the state of the game board in real time. The system according to feature 1.

3. The aforementioned analysis unit, The AI ​​generates data to analyze the board state and identify the position and movement patterns of each piece. The system according to feature 1.

4. The aforementioned supply unit is, Bone conduction speakers provide users with next-step advice and strategic guidance. The system according to feature 1.

5. The projection unit is The system projects evaluation values ​​and the best move onto the glasses portion, providing visual support. The system according to feature 1.

6. The aforementioned capture unit is The system estimates the user's emotions and adjusts the timing of capturing the game board based on those emotions. The system according to feature 1.

7. The aforementioned capture unit is When capturing the state of the game board, filtering technology is introduced to minimize the effects of light reflection and shadows. The system according to feature 1.

8. The aforementioned capture unit is When capturing the state of the game board, multiple cameras are used to integrate footage from different angles, thereby obtaining more accurate information about the game board. The system according to feature 1.

Images