system

Abstract

A system is provided.SOLUTION: A system comprising camera means for recording a game or practice scene, means for uploading the recorded data to a cloud server, means for collecting professional game data and analyzing each scene in terms of positioning and course, means for receiving the user's play data and matching it with professional game data to suggest an optimal strategy based on a machine learning model, and means for actually simulating the suggested strategy using a VR system.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] With conventional training methods, it was difficult for players who had acquired a certain level of skill to develop effective tactics for match play. In particular, the lack of a system that utilizes professional match data to propose tactics suited to individual players made it difficult to acquire efficient tactics. Furthermore, the lack of a comprehensive system that could collect and analyze related data and propose tactics made it difficult to improve overall skills. [Means for solving the problem]

[0005] The present invention provides a system that includes a camera for recording matches and practice sessions, a means for uploading the recorded data to a cloud server, a means for collecting professional match data and analyzing each scene from the perspectives of positioning and course, a means for receiving a user's play data and comparing it with the professional match data to propose optimal tactics based on a machine learning model, and a means for actually simulating the proposed tactics using a VR system. This allows for optimal match management for each player, enabling efficient tactical learning through a realistic match experience. Furthermore, the cloud server receives the user's simulation data and performs further analysis to improve the accuracy of the next tactical proposal. Furthermore, by automatically collecting professional match data and storing the analysis results in a database, comprehensive skill improvement is supported.

[0006] "Camera means" refers to hardware devices such as video cameras or smartphone cameras used to record matches and practice sessions.

[0007] "Recorded data" refers to video data of a match or practice scene captured by a camera means.

[0008] A "cloud server" refers to a remote server that stores and processes data over the Internet.

[0009] "Upload" refers to the process of transferring data from a local device to a cloud server.

[0010] "Professional match data" refers to video and action data relating to professional sports matches.

[0011] "Positioning" refers to the positioning and placement of players during a match.

[0012] A "course" refers to the path that a ball or shuttlecock takes or the location where it is hit in a sports game.

[0013] "Analysis" refers to the process of analyzing collected data using statistical methods and algorithms to extract useful information.

[0014] "User play data" refers to video and action data generated by individual players during matches and practice.

[0015] "Matching" refers to the process of comparing different data sets to find commonalities and differences.

[0016] A "machine learning model" refers to an artificial intelligence algorithm that learns patterns from large amounts of data and makes predictions and classifications.

[0017] "Tactics" refers to a plan or method for effectively aiming to win a game or competition.

[0018] "Suggestion" means showing the user the optimal action or strategy.

[0019] "VR system" refers to a combination of hardware and software for generating virtual reality and providing an immersive experience to a user.

[0020] "Simulation" refers to the process of recreating real-world actions or events in a virtual environment.

[0021] A "database" refers to an information system that enables efficient management and retrieval of large amounts of data. [Brief explanation of the drawings]

[0022] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0023] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0024] First, the terms used in the following description will be explained.

[0025] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0026] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0027] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0028] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0029] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0030] [First embodiment]

[0031] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0032] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0033] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0034] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0035] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0036] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0037] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0038] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0039] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0040] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0041] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0042] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0043] The present invention is a system that allows users to record matches and practice sessions, analyze the recorded data, and learn tactics. A detailed example of the system is described below.

[0044] The system works in the following steps:

[0045] 1. Recording and data uploading

[0046] First, the user sets up a camera to record a match or practice session. The user presses the camera's record button to capture the video. This video data is temporarily saved on the device. When recording is finished, the device automatically uploads the recorded data to the cloud server.

[0047] 2. Professional match data collection and analysis

[0048] The server periodically runs a script to collect professional game data from the Internet. This data includes video footage of professional games, player positioning, ball trajectories, etc. The collected data is analyzed by a video analysis engine, and the extracted information is stored in a database.

[0049] 3. User data collation and strategy proposals

[0050] When the server receives the user's recorded data, it analyzes the user's play data using a video analysis engine. The results of this analysis are compared with professional match data, and the optimal tactics for the user are generated based on a machine learning model. The generated tactics are then suggested to the user as specific positioning and ball trajectory instructions. These suggestions are then notified to the user via their device.

[0051] 4. VR Simulation

[0052] The user puts on VR goggles based on the proposed tactic and launches the application. The device then displays tactical footage generated by the VR system to the user. For example, the user can experience realistically the positioning and movements required to hit a backhand cross shot. Data from the simulation is then uploaded back to the server for further analysis.

[0053] Specific examples

[0054] For example, suppose a user records their tennis practice and uploads the data to a cloud server. When the server analyzes the received data, it finds that the user makes many mistakes with backhand shots. The server then analyzes the movements of professional players in similar situations based on professional match data. It then finds that a cross-court shot is effective when taking a backhand shot. Based on this information, the server suggests to the user, "Try a cross-court shot from your backhand in your next match." The user can then put on VR goggles and simulate the suggested tactic in a virtual environment. Further data from this simulation is collected, and the next suggestion will become more accurate.

[0055] This allows the present invention to help players improve their practical skills and enable them to master optimal tactics. By precisely comparing professional data with user data and proposing tactics suited to each individual's playing style, efficient and effective training can be achieved.

[0056] The processing flow will be explained below.

[0057] Step 1:

[0058] The user sets up a camera in the area where the game or practice is being held, turns the camera on, and presses the start recording button to capture the video.

[0059] Step 2:

[0060] The device temporarily stores the recorded data from the camera in local memory. When recording is finished, the user presses the stop recording button to end the recording.

[0061] Step 3:

[0062] The device uploads the recorded data to the cloud server, where it is divided into multiple segments and uploaded to the cloud storage sequentially.

[0063] Step 4:

[0064] The server periodically runs a script that collects professional match data from the internet, which is obtained via APIs from official websites and streaming services.

[0065] Step 5:

[0066] The server passes the collected professional match data to a video analysis engine, which analyzes each frame to extract player movements, positioning, and ball trajectory.

[0067] Step 6:

[0068] The server stores the analysis results in a database, where the data is categorized by scene and features are extracted using pattern recognition algorithms.

[0069] Step 7:

[0070] The server receives the recorded data uploaded by the user and analyzes it using a video analysis engine, which also analyzes each frame to extract the movement and course of the action.

[0071] Step 8:

[0072] The server compares professional match data with the user's play data, and uses machine learning models to suggest game strategies that suit the user's playing style and level.

[0073] Step 9:

[0074] The server stores the generated tactical suggestions in a database and notifies the user's device. For example, the suggestions may include specific advice such as "At the next point, try a cross shot from your backhand."

[0075] Step 10:

[0076] The user puts on the VR goggles and launches the application, which starts the VR simulation based on the proposed tactics.

[0077] Step 11:

[0078] The device uses a VR system to provide users with tactical footage, which is reflected in real time as the user moves, allowing them to experience actual positioning and movements during a match.

[0079] Step 12:

[0080] The user ends the simulation and receives feedback. The data from the simulation is uploaded to the cloud server again.

[0081] Step 13:

[0082] The server receives the data during the simulation and performs further analysis, which is used to refine future tactical proposals.

[0083] Example 1

[0084] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0085] Previous technology made it difficult for users to analyze their own matches and practice sessions and learn effective tactics. In particular, the lack of comparison with professional match data and tactical simulation functions made it difficult to achieve efficient training and practical skill improvement. Furthermore, existing systems lacked a means to provide realistic simulations using virtual reality, making them insufficient for improving users' actual performance.

[0086] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0087] In this invention, the server includes a video recording means for recording matches and practice scenes, a communication means for uploading the recorded data to a cloud server, a data analysis means for collecting professional match data and analyzing each scene from the perspective of position information and progression paths, a tactic generation means for receiving the user's play data, comparing it with the professional match data, and proposing optimal tactics based on a machine learning model, and a virtual reality display means for actually simulating the proposed tactics using a virtual reality system. This allows the user to compare and learn from their own recorded play with the professional data, and further simulate the proposed tactics using virtual reality.

[0088] "Video recording means" refers to a device or system for recording video of a match or practice session.

[0089] "Communication means" refers to the internet connection or network device used to upload recorded data to the cloud server.

[0090] "Data analysis means" refers to software or algorithms for analyzing collected professional match data from the perspective of location information and progression paths.

[0091] The "tactic generation means" is a system that receives the user's play data, compares it with professional match data, and proposes optimal tactics based on a machine learning model.

[0092] "Virtual reality display means" refers to a device or system for actually simulating a proposed tactic using a virtual reality system.

[0093] A "cloud server" is a remote server used to store, manage, and analyze data over the Internet.

[0094] A "machine learning model" is an algorithm that recognizes patterns based on large amounts of data and makes predictions and decisions.

[0095] The present invention is a system that allows users to record matches and practice sessions, analyze the recorded data, and learn tactics. A detailed example of the system is described below.

[0096] Recording and Data Upload

[0097] A user sets up a camera to record a match or practice session. Specifically, the camera is fixed on a tripod to ensure a good image. When the user presses the record button on the camera, the video is captured and temporarily saved on the device. When the recording is finished, the device uploads the recorded data to the cloud server using a Wi-Fi module. The upload is performed using an HTTP POST request.

[0098] Professional match data collection and analysis

[0099] The server periodically runs a Python script to collect professional match data from the Internet through web scraping. For example, it uses BeautifulSoup and Selenium to retrieve match footage and statistical data from sports sites. The collected data is then analyzed for each frame using OpenCV to identify player positions and ball trajectories. The analysis results are then stored in a MySQL® database.

[0100] User data collation and tactical proposals

[0101] When the server receives the video data uploaded by the user, it analyzes the user's play data using OpenCV and a machine learning model (e.g., TENSORFLOW (registered trademark)). The analyzed user data is compared with previously collected professional match data. Based on the results of this comparison, the machine learning model generates optimal tactics. The generated tactical suggestions are notified to the user via their device in the form of specific positioning and ball trajectory. For example, the app notification function could display a message saying, "Try a backhand crosscourt shot."

[0102] VR Simulation

[0103] Based on the proposed tactics, the user puts on VR goggles (e.g., Oculus Rift) and launches a dedicated VR application. The device then provides the user with tactical footage generated using a VR system based on Unity or Unreal Engine. The user then practices the presented tactics in the virtual environment, experiencing realistic movements in the process. Performance data from the simulation is then uploaded back to a cloud server for further analysis.

[0104] Specific examples

[0105] For example, suppose a user records their tennis practice and uploads the data to a cloud server. When the server analyzes the received data, it finds that the user makes many mistakes with backhand shots. The server then analyzes the movements of professional players in similar situations based on professional match data. It then finds that a cross-court shot is effective when taking a backhand shot. Based on this information, the server suggests to the user, "Try a cross-court shot from your backhand in your next match." The user then puts on VR goggles and simulates the suggested tactic in a virtual environment. Further data from this simulation is collected, and the next suggestion becomes more accurate.

[0106] Prompt Sentence Examples

[0107] Below are some example prompts to input to the generative AI model:

[0108] "Explain how to analyze a user's recorded tennis practice video and suggest improvements for the backhand shot."

[0109] "Please explain the process of proposing optimal tactics to users based on data on professional players' backhand shots."

[0110] "Please explain in detail the specific steps to use VR goggles to simulate tennis tactics."

[0111] This allows the present invention to help players improve their practical skills and enable them to master optimal tactics. By precisely comparing professional data with user data and proposing tactics suited to each individual's playing style, efficient and effective training can be achieved.

[0112] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0113] Step 1:

[0114] A user sets up a camera to record a match or practice session and presses the record button. This causes the camera to capture video and generate recorded data. The recorded data is temporarily stored in the device's internal storage. The input is video data, and the output is a recorded file stored in the internal storage.

[0115] Step 2:

[0116] When recording is finished, the device connects to the Internet using the Wi-Fi module and uploads the recorded data to the cloud server. Specifically, the recorded file is sent to the cloud server via an HTTP POST request. The input is the recorded file in the internal storage, and the output is the recorded data saved on the cloud server.

[0117] Step 3:

[0118] The server periodically runs a Python script to collect professional match data from the Internet, for example, using BeautifulSoup and Selenium to retrieve match footage and statistics from sports sites. The input is the URL of the sports site, and the output is the collected professional match data.

[0119] Step 4:

[0120] The server uses OpenCV to analyze the professional match data collected. Each frame of the video is analyzed to identify the player's position information and the ball's trajectory. The input is the professional match data, and the output is the analyzed position information and ball's trajectory data. The analysis results are stored in a MySQL database.

[0121] Step 5:

[0122] The server receives the recorded data uploaded by the user and analyzes it using a video analysis engine (OpenCV) and a machine learning model (TensorFlow). The server analyzes the user's play data and identifies the player's movements and success rate. The input is the user's recorded data, and the output is the analyzed play data.

[0123] Step 6:

[0124] The server compares the analyzed user data with professional match data. A machine learning model generates optimal tactics based on this data. For example, if the success rate of backhand shots is low, effective tactics and techniques are extracted from a set of professional data. The input is the user data and the professional data, and the output is an optimal tactical proposal.

[0125] Step 7:

[0126] The generated tactical proposal is notified to the user via the device. Specifically, the details of the tactic (e.g., attempt a cross-court shot from the backhand) are communicated to the user using app notifications or email notifications. The input is the generated tactical proposal, and the output is the notification sent to the user.

[0127] Step 8:

[0128] The user puts on VR goggles based on the proposed tactics and launches a dedicated VR application. The device displays tactical footage generated using Unity or Unreal Engine in real time. The input is the tactical proposal and VR system components, and the output is a tactical simulation in a virtual environment.

[0129] Step 9:

[0130] During the simulation, the user's movements and performance data are uploaded from the device to the cloud server. The server receives this data and performs further analysis to improve the accuracy of the next tactical proposal. The input is the simulation data, and the output is the results of further analysis.

[0131] (Application example 1)

[0132] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0133] Conventional sports and work training systems have had difficulty proposing optimal tactics and work procedures based on the user's individual characteristics and movement data. Furthermore, they lacked the means to actually experience the proposed tactics and work procedures and obtain feedback. As a result, it was difficult for users to receive immediate and accurate feedback on their movements, hindering efficient training and work optimization.

[0134] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0135] In this invention, the server includes a camera means for recording matches and practice scenes, a means for uploading the recorded data to a cloud server, a means for collecting professional match data and analyzing each scene from the perspective of positioning and course, a means for receiving a user's play data, comparing it with the professional match data, and proposing optimal tactics based on a machine learning model, a means for actually simulating the proposed tactics using a VR system, a means for recording factory work and analyzing the data to learn and propose efficient movements, and a means for confirming and simulating the work content using smart glasses. This makes it possible to propose optimal tactics and work procedures based on the user's individual data, experience simulations on the spot, and obtain feedback.

[0136] The "camera means" is a device that records scenes of matches and practices, as well as work in the factory.

[0137] A "cloud server" is a system that stores recorded data in a remote location via the Internet and centrally manages the data for analysis.

[0138] "Professional match data" means video footage of matches or activities and related analytical data produced by experts or skilled personnel.

[0139] "Means of analysis from the perspective of positioning and course" refers to technology for analyzing the positioning and direction / flow of movement at specific scenes in a video.

[0140] "User play data" refers to video data of a user's own matches or activities that the user has recorded.

[0141] A "machine learning model" is an algorithm or model that allows a computer to learn from data and make predictions and decisions.

[0142] "Means for suggesting optimal tactics" refers to a method for presenting the most effective tactics and work procedures for the user based on analyzed data.

[0143] A "VR system" is a system that uses virtual reality technology to allow users to experience proposed tactics and work procedures in a virtual environment.

[0144] "Means for recording factory work" refers to devices and technologies that record work procedures and actions within a factory as video.

[0145] The "means for learning and suggesting efficient movements" is a system that analyzes recorded work data and recommends movements that will improve work efficiency.

[0146] "Smart glasses" are wearable devices that can display information in the user's field of vision.

[0147] "Simulation means" refers to technology that provides a virtual environment in which proposed tactics and procedures can be experienced.

[0148] This invention is a system that allows users to record matches, training sessions, or factory work, and analyze the data to learn efficiency and tactics. A specific example of the system is described below.

[0149] First, the user uses a camera to record a match, practice session, or factory work. When recording is finished, the device automatically uploads the recorded data to a cloud server. In this process, a commercially available video camera or wearable camera can be used as the camera. The recorded data is temporarily stored on the device and then transferred to the cloud server via the Internet. In this case, cloud services such as Amazon Web Services (AWS®) or Google® Cloud Platform (GCP) can be used as the cloud server.

[0150] The cloud server periodically collects and analyzes professional match and work data. This process involves using scripts to automatically collect the necessary data from the Internet and storing the results in a database. The analysis can be performed using OpenCV or TensorFlow as a video analysis engine. The collected data also includes information to evaluate positioning, course, and work efficiency.

[0151] When a user uploads recorded data to a cloud server, the server receives the data and analyzes it using a video analysis engine. The analysis results are compared with professional data, and optimal tactics and work procedures are generated based on a machine learning model. This machine learning model is built using Keras and TensorFlow. The generated tactics and work procedures are then suggested to the user as specific positioning and actions. These suggestions are then notified to the user via their device.

[0152] Additionally, users can wear a VR system or smart glasses to simulate proposed tactics and procedures in a virtual environment. For example, Oculus Rift or HTC Vive can be used as VR goggles, while Microsoft® HoloLens® or Google Glass® are suitable for smart glasses. The VR system provides users with a realistic experience, allowing them to practice suggested movements and positions. The data from this simulation is also uploaded back to the cloud server for further analysis, improving the accuracy of future proposals.

[0153] As a concrete example, let's say work in a factory is recorded and the data is uploaded to a cloud server. When the server analyzes the received data, it finds that the worker is inefficient in a particular procedure. Next, the server analyzes the movements of professionals in similar situations based on the work data of the professionals. As a result, it finds that efficiency can be improved by changing a particular procedure. Based on this information, the server suggests to the user, "Next time, try assembling from the right side." The user can then put on smart glasses and simulate the suggested procedure in a virtual environment. Further data from this simulation is collected, and the next suggestion will become more accurate.

[0154] An example of a prompt is:

[0155] "Develop an application that records video of a robot's assembly work and suggests optimal movement patterns. This application will cover everything from recording the video to uploading it to the cloud, analyzing the data, and simulating tactics. It should also include a function that allows users to check and simulate the suggestions using smart glasses."

[0156] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0157] Step 1:

[0158] A user uses a camera to record a game, practice, or factory work. The user presses the record button on the camera to capture the video. This video data is temporarily saved on the device. After recording is complete, the device automatically uploads the recorded data to a cloud server. The input is the recorded video data, and the output is the data uploaded to the cloud server.

[0159] Step 2:

[0160] The server receives video data uploaded to the cloud. This data includes footage of games, practice sessions, and factory work recorded by users. The server receives and stores this data. The input is the video data uploaded by the user, and the output is the data stored in the server.

[0161] Step 3:

[0162] The server periodically collects professional match and performance data. The method used here is to collect data using internet scripts and APIs. This allows us to obtain information such as positioning and movement patterns. The input is the professional data collected via the internet, and the output is data stored in a database for analysis.

[0163] Step 4:

[0164] The server uses a video analysis engine to analyze the collected professional match and work data. Software such as OpenCV and TensorFlow are used. The analyzed data is broken down into individual frames, and positional information and movement patterns are extracted. The input is the collected professional data, and the output is analyzed positioning and movement pattern data.

[0165] Step 5:

[0166] The server uses a video analysis engine to analyze the play data uploaded by the user and compares it with professional match data and work data. Based on a machine learning model (using Keras and TensorFlow), it generates the optimal tactics and work procedures for the user's actions. The input is the user's play data and professional data, and the output is the optimal tactics and work procedures suggested to the user.

[0167] Step 6:

[0168] The device notifies the user of the generated optimal tactics and work procedures. Notification methods include a smartphone app and a notification system. At this point, the user can confirm the proposal and then perform an actual simulation in the next step. The input is the proposal information from the server, and the output is a notification to the user.

[0169] Step 7:

[0170] Users wear a VR system or smart glasses and simulate proposed tactics and work procedures in a virtual environment. VR systems such as Oculus Rift and HTC Vive are used, while smart glasses such as Microsoft HoloLens and Google Glass are used. Data from the simulation is also uploaded to a server for further analysis. The input is the proposed tactics and work procedures, and the output is the data from the simulation.

[0171] Step 8:

[0172] The server receives the data from the simulation and performs further analysis. This data is used to improve the accuracy of the next proposal. A video analysis engine and machine learning models are used for data analysis. The input is the data from the simulation, and the output is an updated proposal.

[0173] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0174] The present invention relates to a system that allows users to record matches and practice sessions, analyze the data, and learn tactics, and further combines it with an emotion engine that recognizes the user's emotional state and adjusts tactical suggestions accordingly. A specific example of this system is shown below.

[0175] Basic configuration

[0176] The system consists of the following main components:

[0177] 1. Camera means: A device that allows users to record games and practice sessions.

[0178] 2. Uploading recorded data: A function that allows the device to upload recorded data to the cloud server.

[0179] 3. Professional match data collection means: The server collects professional match data from the Internet.

[0180] 4. Data analysis means: The server has the ability to analyze professional match data and user play data.

[0181] 5. Tactical suggestion means: A function in which the server suggests optimal tactics to the user based on the analysis results.

[0182] 6. VR System: A virtual reality environment for practical simulation of the proposed tactics.

[0183] 7. Emotion Engine: The ability to recognize user emotions and tailor suggestions accordingly.

[0184] Program processing

[0185] First, the user sets up the camera to record a match or practice session, then presses the record button to capture the footage. The recorded data is temporarily saved on the device and uploaded to the cloud server after recording is complete.

[0186] The server periodically collects professional match data from the Internet, analyzes each frame with a video analysis engine, and extracts player movements, positioning, and ball trajectory, then stores this data in a database.

[0187] The server receives the recorded data uploaded by the user and analyzes it using a video analysis engine. The results of this analysis are compared with professional match data and the optimal tactics are suggested to the user based on a machine learning model.

[0188] The proposed tactics are notified to the user via their device, and the user receives specific advice through the application. At this time, the emotion engine identifies the user's emotional state from their facial expressions and tone of voice, and adjusts the suggestions based on this information. For example, if the user is nervous, the tactical suggestions will also include mental advice.

[0189] The user puts on the VR goggles and launches the application. The device plays back the tactical footage generated using the VR system, providing the user with a realistic match experience. Data is also collected during the simulation and uploaded back to the cloud server for further analysis. This data is used to improve the accuracy of future tactical suggestions.

[0190] Specific examples

[0191] For example, suppose a user records a basketball practice and uploads the data to a cloud server. When the server analyzes the received data, it finds that the user has a low success rate in jump shots. After comparing the data with that of professional players, it concludes that the user needs to improve their posture and positioning when taking a jump shot. Based on this information, the server suggests to the user to "pay attention to the position of their feet and the rotation of their shoulders when taking a jump shot."

[0192] Furthermore, if the emotion engine detects nervousness from the user's facial expression, it will add mental advice such as "Relax and shoot." The user then puts on VR goggles and simulates the proposed tactic in a virtual environment. The simulation results are then uploaded back to the cloud server for further analysis.

[0193] This allows the present invention to support more personalized tactical suggestions and practical skill improvement, taking into account the user's emotional state. By precisely matching professional data, user data, and emotional data, the present invention can suggest tactics suited to each individual's playing style, enabling efficient and effective training.

[0194] The processing flow will be explained below.

[0195] Step 1:

[0196] Users set up a camera to record their matches or practice sessions. They turn on the camera and press the record start button to capture footage. The camera records the user's movements in high resolution.

[0197] Step 2:

[0198] The device temporarily stores the video data recorded by the camera in local memory. When recording is finished, the user presses the stop recording button to end the recording.

[0199] Step 3:

[0200] The device uploads the temporarily saved recording data to the cloud server, where it is divided into segments and uploaded to the cloud storage sequentially.

[0201] Step 4:

[0202] The server periodically runs a script that collects professional match data from the internet, and the collected data is obtained from official websites and streaming services using APIs.

[0203] Step 5:

[0204] The server passes professional match data to a video analysis engine, which analyzes each frame to extract player movements, positioning, and ball trajectory, which are then stored in a database.

[0205] Step 6:

[0206] The server receives the user's recorded data and analyzes it using the same video analysis engine. As a result, the user's movements and the course of the ball are extracted.

[0207] Step 7:

[0208] The server compares professional match data with the user's playing data, and based on machine learning models, generates optimal tactics for the user, providing specific instructions on positioning and ball trajectory.

[0209] Step 8:

[0210] The server stores the generated tactical suggestions in a database and notifies the device, which may include specific advice such as "try a cross shot from your backhand."

[0211] Step 9:

[0212] The device receives the tactical proposal and activates the emotion engine, which analyzes the user's facial expressions and tone of voice to identify their emotional state.

[0213] Step 10:

[0214] The emotion engine adjusts tactical suggestions based on detected emotional data. For example, if the user is nervous, the suggestion will include mental advice.

[0215] Step 11:

[0216] The user puts on the VR goggles and launches the application based on the tactical proposal. The user then practices the proposed tactical proposal in the VR simulation.

[0217] Step 12:

[0218] The device uses a VR system to provide users with tactical footage that responds to the user's movements in real time, providing an experience similar to that of a real match.

[0219] Step 13:

[0220] The user finishes the simulation and checks the feedback obtained. The simulation data is then uploaded to the cloud server again.

[0221] Step 14:

[0222] The server receives the simulation data and performs further analysis, which is then stored in a database and used to improve the accuracy of the next tactical proposal.

[0223] This allows for personalized tactical suggestions that take into account the user's emotional state, resulting in efficient and effective training.

[0224] Example 2

[0225] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0226] Conventional sports analysis systems analyze recorded data and propose tactics without taking into account the user's emotional state, which means they are unable to propose tactics that are appropriate for the user's mental state. Additionally, there is a lack of an iterative analysis system to improve the accuracy of tactical proposals.

[0227] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0228] In this invention, the server includes a recording device for recording game and practice scenes, a means for uploading the recorded data to the data server, a means for collecting expert game data and analyzing each scene from the perspective of position information and movement, a means for receiving user movement data, comparing it with the expert game data, and proposing optimal tactics based on a machine learning algorithm, a means for actually simulating the proposed tactics using a virtual reality device, and an emotion analysis means for recognizing the user's emotional state and adjusting the proposed content. This enables individual tactical proposals suited to the user's emotional state. Furthermore, by reanalyzing the simulation data, the accuracy of the tactical proposals can be continuously improved.

[0229] "Recording equipment means" refers to a device for recording footage of matches, practice sessions, etc.

[0230] "Data server" refers to a server that stores recorded data and analysis results and exchanges data with other devices.

[0231] "Expert match data" refers to records of matches and practice sessions of professional or specialized athletes, and is data used to analyze their movements and tactics.

[0232] "Means for analyzing from the perspective of positional information and movements" refers to technology for analyzing recorded data and collected data to extract and evaluate players' positional information and movements.

[0233] "Motion data" refers to data that quantifies or quantifies the movements and actions of a user while playing.

[0234] "Machine learning algorithms" refers to a collection of computer programs and mathematical models for analyzing data and creating predictive models.

[0235] "Means for suggesting tactics" refers to a system that suggests optimal tactics and moves to users based on the analysis results.

[0236] "Virtual reality device" refers to a device or system that allows a user to simulate an experience in a virtual environment.

[0237] "Emotion analysis means" refers to technology that recognizes and analyzes a user's emotional state from their facial expressions, tone of voice, and manner of operation.

[0238] The present invention is a system that allows users to record sports matches and practice sessions, analyze the data, and learn tactics. It also incorporates an emotion engine that recognizes the user's emotional state and adjusts tactical suggestions accordingly.

[0239] System configuration

[0240] The system includes the following main components and specific hardware and software:

[0241] 1. Recording equipment means:

[0242] This is a device that allows users to record matches and practice sessions. For example, it uses a smartphone camera or a dedicated recording camera.

[0243] 2. How to upload recording data:

[0244] This function allows the device to upload recorded data to a cloud server. The data is stored in cloud storage, such as AWS S3.

[0245] 3. Expert match data collection methods:

[0246] The server collects expert match data from the Internet. The collected data is obtained using, for example, a web scraping tool (e.g., BeautifulSoup) and analyzed by a video analysis engine (e.g., OpenCV).

[0247] 4. Data analysis methods:

[0248] The server receives the video data uploaded by users and analyzes it with a video analysis engine (e.g., OpenCV). The analysis results are then compared with expert match data using machine learning algorithms (e.g., TensorFlow).

[0249] 5. Tactical Proposal Methods:

[0250] This function allows the server to propose optimal strategies to users based on the analysis results. The proposals are saved in a database and sent to the user via their device.

[0251] 6. Virtual Reality Devices:

[0252] It is a device that allows users to simulate proposed tactics in a virtual environment, for example, using VR goggles such as Oculus Rift.

[0253] 7. Emotion analysis means:

[0254] The device analyzes the user's facial expressions and tone of voice and uses an emotion engine (e.g., Microsoft Azure® Emotion API) to identify their emotional state, then tailors tactical suggestions based on this information.

[0255] Example of operation

[0256] A user records their basketball practice and uploads the data to a cloud server. The server analyzes the data and finds that the user's jump shot success rate is low. After comparing it with expert game data, it is determined that the user's posture and positioning during jump shots need to be improved. Based on this, the server suggests, "Pay attention to the position of your feet and shoulder rotation when taking a jump shot."

[0257] Furthermore, if the emotion analysis means detects nervousness from the user's facial expression, the system adds mental advice such as "Relax and shoot." The user then puts on the VR goggles and simulates the proposed tactics in a virtual environment. The results of this simulation are uploaded back to the server for further analysis and to improve the accuracy of the tactical suggestions.

[0258] Prompt Sentence Examples

[0259] "We recorded basketball practice and uploaded the data to a cloud server. The results showed that the jump shot success rate was low. We compared it with expert player data and would like to know what improvements can be made. Also, please provide any mental advice that would be useful if the user is nervous."

[0260] The present invention makes it possible to propose individual tactics that take into account the user's emotional state, and furthermore, the accuracy of the tactics proposals is continuously improved by reanalyzing the simulation data.

[0261] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0262] Step 1:

[0263] Start and stop recording

[0264] The user sets up a camera and presses the record button in the application to record a match or practice. The scene to be recorded is the input, and the recorded data is generated as the output.

[0265] When you have finished recording, press the stop recording button to end the recording. Specifically, you can perform the recording operation within the smartphone or dedicated camera app.

[0266] Step 2:

[0267] Uploading recording data

[0268] The device uploads the recorded data to the data server. The input is the recorded data stored in the device, and the output is the data safely stored in the cloud.

[0269] The data is encrypted and transferred securely, and once the recording is complete, the app automatically uploads the data to a cloud storage service such as AWS S3.

[0270] Step 3:

[0271] Expert match data collection

[0272] The server collects expert match data from the Internet. The input is match data on the Internet, and the output is match data stored in a database in the server.

[0273] Analyze the video using a video analysis engine (e.g., OpenCV) to extract location information and behavior. Specifically, use a web scraping tool (e.g., BeautifulSoup) to obtain and analyze the data.

[0274] Step 4:

[0275] Analysis of recorded data

[0276] The server receives the recorded data uploaded by the user. The input is the recorded data uploaded to the cloud, and the output is the analyzed play data.

[0277] A video analysis engine (e.g., OpenCV) analyzes each frame and extracts information such as player movements, positioning, and ball trajectory. Specifically, the server analyzes the recorded data using a machine learning model (e.g., TensorFlow).

[0278] Step 5:

[0279] Generate tactical suggestions

[0280] The server proposes optimal tactics based on the analysis results. The input is the analysis results, and the output is tactical proposals to the user.

[0281] The proposed content is stored in a database and notified to the user via the device. Specifically, the server uses a machine learning algorithm to calculate and propose tactics based on the analyzed data.

[0282] Step 6:

[0283] Emotion recognition and tactical suggestion adjustment

[0284] The device analyzes the user's facial expressions and tone of voice to identify their emotional state. The input is the user's facial and voice data, and the output is tailored tactical suggestions.

[0285] The app uses an emotion engine (e.g., Microsoft Azure Emotion API) to analyze the user's emotional state and adjust the suggestions accordingly. Specifically, the app uses the camera and microphone to capture and analyze the user's emotional data.

[0286] Step 7:

[0287] VR Simulation

[0288] The user puts on the VR goggles and launches the application to simulate tactics. The input is the tactical proposal data, and the output is the simulation results.

[0289] The device plays the generated tactical footage in a VR environment, providing the user with a realistic match experience. Specifically, the user puts on VR goggles such as Oculus Rift and launches the app.

[0290] Step 8:

[0291] Re-uploading and analyzing simulation data

[0292] The terminal uploads the data collected during the simulation back to the data server. The input is the simulation data, and the output is the analyzable data on the server.

[0293] The server performs the analysis again and stores the analysis results in a database. Specifically, once the simulation is complete, the application automatically uploads the data to the cloud for further analysis.

[0294] The above steps enable individual tactical suggestions and continuous analysis based on the user's emotional state.

[0295] (Application example 2)

[0296] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0297] The objective of this invention is to propose driving tactics that take into account not only the analysis of game and practice scenes, but also the analysis of vehicle driving conditions and the driver's emotional state. Conventional systems have difficulty integrating these complex elements to analyze and propose driving tactics, and there is a lack of technology to improve driver safety and efficient driving.

[0298] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0299] In this invention, the server includes a camera means for recording game and practice scenes, a means for uploading the recorded data to a remote server, a means for collecting professional game data and analyzing each scene in terms of positioning and course, a means for receiving user play data and comparing it with the professional game data to propose optimal tactics based on a machine learning model, a means for actually simulating the proposed tactics using a virtual reality system, a means for collecting vehicle driving situation data using a camera and sensor devices, a means for analyzing the driving data and generating optimal driving tactics, and a means for recognizing the driver's emotional state and adjusting the content of the tactical proposal. This makes it possible to propose safe and effective driving tactics to the driver through detailed data collection using the camera and sensor devices, and analysis and emotion recognition based on the data.

[0300] "Photography equipment" refers to equipment for recording video and images of games, practice scenes, vehicle driving conditions, and the like.

[0301] A "remote server" is a remotely located computer system for storing, analyzing, and providing data.

[0302] "Professional match data" refers to data collected from matches involving professional players, and includes information on positioning, movements, strategies, and the like.

[0303] A "machine learning model" is an algorithm or system that is trained using data to automatically learn and perform specific tasks.

[0304] "Virtual reality system" is a general term for devices and software that allow users to experience a virtual environment visually and with other senses.

[0305] A "sensor device" is a device that detects the driving conditions of a vehicle and the surrounding environment and collects the data.

[0306] "Driving data" refers to information about the driving conditions of a vehicle, including detailed data such as speed, braking, and steering.

[0307] "Operation tactics" is a general term for plans and methods for achieving safe and efficient vehicle operation.

[0308] "Emotional state" refers to the psychological state or mood of the driver or user, and includes emotions such as tension, relaxation, and fatigue.

[0309] The present invention relates to a system that records and analyzes game and practice scenes, and proposes tactics that take into account the driver's emotional state. It also has the function of analyzing the vehicle's driving situation and proposing driving tactics based on the driver's emotional state. Specific embodiments are described below.

[0310] Program Overview

[0311] The server includes the following means: a camera means for recording matches and practice scenes, a means for uploading the recorded data to a remote server, a means for collecting professional match data and analyzing each scene in terms of positioning and course, a means for receiving the user's play data, comparing it with the professional match data, and proposing optimal tactics based on a machine learning model, a means for actually simulating the proposed tactics using a virtual reality system, a means for collecting vehicle driving situation data using a camera device and sensor devices, a means for analyzing the driving data and generating optimal driving tactics, and a means for recognizing the driver's emotional state and adjusting the content of the proposed tactics. This makes it possible to collect detailed data using the camera device and sensor devices, and then propose safe and effective driving tactics to the driver through analysis and emotion recognition based on the collected data.

[0312] Hardware and Software Use

[0313] Camera means: The camera uses NVIDIA's Jetson AGX Xavier development kit.

[0314] Cloud servers: AWS or Google Cloud are used for data storage and analysis.

[0315] VPN system: Oculus Rift or HTC Vive are used for virtual reality experiences.

[0316] Sensor devices: LiDAR and various sensors are used to collect data on vehicle driving conditions.

[0317] Emotion recognition: Using software such as OpenFace and Google Speech-to-Text API.

[0318] Data analysis: TensorFlow and PyTorch are used to analyze data and implement machine learning models.

[0319] Program processing

[0320] First, the user uses the camera means to record the game or practice scene, and saves the recorded data in the terminal.

[0321] The device automatically uploads the recorded data to a cloud server, which collects professional match data from the internet and analyzes it using a video analysis engine.

[0322] It uses cameras and sensors to collect data on vehicle driving conditions, then uses AWS Lambda and Google Cloud Functions to analyze the data and generate safe and efficient driving tactics.

[0323] The emotion recognition engine recognizes the driver's emotional state from their facial expressions and tone of voice, and adjusts the tactical suggestions based on this information.

[0324] Specific examples

[0325] For example, if a driver is tired after a long drive, the emotion engine will recognize this state and suggest that the driver take a rest at the next rest area. This suggestion can also be experienced visually through the virtual reality system.

[0326] Prompt Sentence Examples

[0327] Please provide the user's camera footage and driving data to develop a system that performs driving analysis and emotion recognition.

[0328] Include the following elements:

[0329] 1. Recording devices and their data upload methods

[0330] 2. Professional driving data collection methods

[0331] 3. Data analysis approach

[0332] 4. How to propose operational tactics

[0333] 5. Emotion Recognition Algorithms

[0334] 6. How to run a VR simulation

[0335] This invention integrates driving and emotion recognition technology to improve driver safety and efficiency by providing improved driving instructions in real time.

[0336] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0337] Step 1:

[0338] A user uses a camera to capture video data of a match or practice session. The video data is input and temporarily saved.

[0339] Step 2:

[0340] The terminal automatically uploads the temporarily stored video data to a remote server. The input is the temporarily stored video data, and the output is the video data on the remote server.

[0341] Step 3:

[0342] The server collects professional match data from the Internet and uses a video analysis engine (e.g., TensorFlow or PyTorch) to analyze each scene in terms of positioning and course. The input is the professional match data, and the output is the analyzed data.

[0343] Step 4:

[0344] The server receives the user's play data, compares it with the professional match data, and proposes optimal tactics based on the machine learning model. The input is the user's play data and the analyzed professional match data, and the output is a tactical proposal.

[0345] Step 5:

[0346] The user uses the virtual reality system to actually simulate the tactics proposed by the server. The input is the tactical proposal from the server, and the output is the simulation data.

[0347] Step 6:

[0348] The user acquires driving situation data using the camera and sensor devices mounted on the vehicle. The input is the driving situation data, and the output is the temporarily saved data.

[0349] Step 7:

[0350] The terminal uploads the temporarily saved driving situation data to the cloud server. The input is the temporarily saved driving situation data, and the output is the data on the cloud server.

[0351] Step 8:

[0352] The server analyzes the driving situation data and the user's emotional state data and generates optimal driving tactics using a machine learning model (e.g., TensorFlow or PyTorch). The input is the driving situation data and the emotional state data, and the output is a proposed driving tactic.

[0353] Step 9:

[0354] The server uses an emotion recognition engine (e.g., OpenFace or Google Speech-to-Text API) to analyze the user's facial expressions and tone of voice to recognize their emotional state. The input is the user's facial expression data and voice data, and the output is emotional state data.

[0355] Step 10:

[0356] The server provides the generated driving tactics to the user and continues driving based on the proposed tactics. The input is the proposed driving tactics and emotional state data, and the output is safe and efficient driving.

[0357] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0358] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0359] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0360] [Second embodiment]

[0361] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0362] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0363] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0364] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0365] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0366] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0367] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0368] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0369] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0370] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0371] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0372] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0373] The present invention is a system that allows users to record matches and practice sessions, analyze the recorded data, and learn tactics. A detailed example of the system is described below.

[0374] The system works in the following steps:

[0375] 1. Recording and data uploading

[0376] First, the user sets up a camera to record a match or practice session. The user presses the camera's record button to capture the video. This video data is temporarily saved on the device. When recording is finished, the device automatically uploads the recorded data to the cloud server.

[0377] 2. Professional match data collection and analysis

[0378] The server periodically runs a script to collect professional game data from the Internet. This data includes video footage of professional games, player positioning, ball trajectories, etc. The collected data is analyzed by a video analysis engine, and the extracted information is stored in a database.

[0379] 3. User data collation and strategy proposals

[0380] When the server receives the user's recorded data, it analyzes the user's play data using a video analysis engine. The results of this analysis are compared with professional match data, and the optimal tactics for the user are generated based on a machine learning model. The generated tactics are then suggested to the user as specific positioning and ball trajectory instructions. These suggestions are then notified to the user via their device.

[0381] 4. VR Simulation

[0382] The user puts on VR goggles based on the proposed tactic and launches the application. The device then displays tactical footage generated by the VR system to the user. For example, the user can experience realistically the positioning and movements required to hit a backhand cross shot. Data from the simulation is then uploaded back to the server for further analysis.

[0383] Specific examples

[0384] For example, suppose a user records their tennis practice and uploads the data to a cloud server. When the server analyzes the received data, it finds that the user makes many mistakes with backhand shots. The server then analyzes the movements of professional players in similar situations based on professional match data. It then finds that a cross-court shot is effective when taking a backhand shot. Based on this information, the server suggests to the user, "Try a cross-court shot from your backhand in your next match." The user can then put on VR goggles and simulate the suggested tactic in a virtual environment. Further data from this simulation is collected, and the next suggestion will become more accurate.

[0385] This allows the present invention to help players improve their practical skills and enable them to master optimal tactics. By precisely comparing professional data with user data and proposing tactics suited to each individual's playing style, efficient and effective training can be achieved.

[0386] The processing flow will be explained below.

[0387] Step 1:

[0388] The user sets up a camera in the area where the game or practice is being held, turns the camera on, and presses the start recording button to capture the video.

[0389] Step 2:

[0390] The device temporarily stores the recorded data from the camera in local memory. When recording is finished, the user presses the stop recording button to end the recording.

[0391] Step 3:

[0392] The device uploads the recorded data to the cloud server, where it is divided into multiple segments and uploaded to the cloud storage sequentially.

[0393] Step 4:

[0394] The server periodically runs a script that collects professional match data from the internet, which is obtained via APIs from official websites and streaming services.

[0395] Step 5:

[0396] The server passes the collected professional match data to a video analysis engine, which analyzes each frame to extract player movements, positioning, and ball trajectory.

[0397] Step 6:

[0398] The server stores the analysis results in a database, where the data is categorized by scene and features are extracted using pattern recognition algorithms.

[0399] Step 7:

[0400] The server receives the recorded data uploaded by the user and analyzes it using a video analysis engine, which also analyzes each frame to extract the movement and course of the action.

[0401] Step 8:

[0402] The server compares professional match data with the user's play data, and uses machine learning models to suggest game strategies that suit the user's playing style and level.

[0403] Step 9:

[0404] The server stores the generated tactical suggestions in a database and notifies the user's device. For example, the suggestions may include specific advice such as "At the next point, try a cross shot from your backhand."

[0405] Step 10:

[0406] The user puts on the VR goggles and launches the application, which starts the VR simulation based on the proposed tactics.

[0407] Step 11:

[0408] The device uses a VR system to provide users with tactical footage, which is reflected in real time as the user moves, allowing them to experience actual positioning and movements during a match.

[0409] Step 12:

[0410] The user ends the simulation and receives feedback. The data from the simulation is uploaded to the cloud server again.

[0411] Step 13:

[0412] The server receives the data during the simulation and performs further analysis, which is used to refine future tactical proposals.

[0413] Example 1

[0414] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0415] Previous technology made it difficult for users to analyze their own matches and practice sessions and learn effective tactics. In particular, the lack of comparison with professional match data and tactical simulation functions made it difficult to achieve efficient training and practical skill improvement. Furthermore, existing systems lacked a means to provide realistic simulations using virtual reality, making them insufficient for improving users' actual performance.

[0416] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0417] In this invention, the server includes a video recording means for recording matches and practice scenes, a communication means for uploading the recorded data to a cloud server, a data analysis means for collecting professional match data and analyzing each scene from the perspective of position information and progression paths, a tactic generation means for receiving the user's play data, comparing it with the professional match data, and proposing optimal tactics based on a machine learning model, and a virtual reality display means for actually simulating the proposed tactics using a virtual reality system. This allows the user to compare and learn from their own recorded play with the professional data, and further simulate the proposed tactics using virtual reality.

[0418] "Video recording means" refers to a device or system for recording video of a match or practice session.

[0419] "Communication means" refers to the internet connection or network device used to upload recorded data to the cloud server.

[0420] "Data analysis means" refers to software or algorithms for analyzing collected professional match data from the perspective of location information and progression paths.

[0421] The "tactic generation means" is a system that receives the user's play data, compares it with professional match data, and proposes optimal tactics based on a machine learning model.

[0422] "Virtual reality display means" refers to a device or system for actually simulating a proposed tactic using a virtual reality system.

[0423] A "cloud server" is a remote server used to store, manage, and analyze data over the Internet.

[0424] A "machine learning model" is an algorithm that recognizes patterns based on large amounts of data and makes predictions and decisions.

[0425] The present invention is a system that allows users to record matches and practice sessions, analyze the recorded data, and learn tactics. A detailed example of the system is described below.

[0426] Recording and Data Upload

[0427] A user sets up a camera to record a match or practice session. Specifically, the camera is fixed on a tripod to ensure a good image. When the user presses the record button on the camera, the video is captured and temporarily saved on the device. When the recording is finished, the device uploads the recorded data to the cloud server using a Wi-Fi module. The upload is performed using an HTTP POST request.

[0428] Professional match data collection and analysis

[0429] The server periodically runs a Python script to collect professional match data from the Internet through web scraping. For example, it uses BeautifulSoup and Selenium to retrieve match footage and statistical data from sports sites. The collected data is then analyzed for each frame using OpenCV to identify player positions and ball trajectories. The analysis results are then stored in a MySQL database.

[0430] User data collation and tactical proposals

[0431] When the server receives the video data uploaded by the user, it uses OpenCV and a machine learning model (e.g., TensorFlow) to analyze the user's play data. The analyzed user data is then compared with previously collected professional match data. Based on the results of this comparison, the machine learning model generates optimal tactics. The generated tactical suggestions are notified to the user via their device in the form of specific positioning and ball trajectory. For example, the app's notification function could display a message such as, "Try a backhand cross-court shot."

[0432] VR Simulation

[0433] Based on the proposed tactics, the user puts on VR goggles (e.g., Oculus Rift) and launches a dedicated VR application. The device then provides the user with tactical footage generated using a VR system based on Unity or Unreal Engine. The user then practices the presented tactics in the virtual environment, experiencing realistic movements in the process. Performance data from the simulation is then uploaded back to a cloud server for further analysis.

[0434] Specific examples

[0435] For example, suppose a user records their tennis practice and uploads the data to a cloud server. When the server analyzes the received data, it finds that the user makes many mistakes with backhand shots. The server then analyzes the movements of professional players in similar situations based on professional match data. It then finds that a cross-court shot is effective when taking a backhand shot. Based on this information, the server suggests to the user, "Try a cross-court shot from your backhand in your next match." The user then puts on VR goggles and simulates the suggested tactic in a virtual environment. Further data from this simulation is collected, and the next suggestion becomes more accurate.

[0436] Prompt Sentence Examples

[0437] Below are some example prompts to input to the generative AI model:

[0438] "Explain how to analyze a user's recorded tennis practice video and suggest improvements for the backhand shot."

[0439] "Please explain the process of proposing optimal tactics to users based on data on professional players' backhand shots."

[0440] "Please explain in detail the specific steps to use VR goggles to simulate tennis tactics."

[0441] This allows the present invention to help players improve their practical skills and enable them to master optimal tactics. By precisely comparing professional data with user data and proposing tactics suited to each individual's playing style, efficient and effective training can be achieved.

[0442] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0443] Step 1:

[0444] A user sets up a camera to record a match or practice session and presses the record button. This causes the camera to capture video and generate recorded data. The recorded data is temporarily stored in the device's internal storage. The input is video data, and the output is a recorded file stored in the internal storage.

[0445] Step 2:

[0446] When recording is finished, the device connects to the Internet using the Wi-Fi module and uploads the recorded data to the cloud server. Specifically, the recorded file is sent to the cloud server via an HTTP POST request. The input is the recorded file in the internal storage, and the output is the recorded data saved on the cloud server.

[0447] Step 3:

[0448] The server periodically runs a Python script to collect professional match data from the Internet, for example, using BeautifulSoup and Selenium to retrieve match footage and statistics from sports sites. The input is the URL of the sports site, and the output is the collected professional match data.

[0449] Step 4:

[0450] The server uses OpenCV to analyze the professional match data collected. Each frame of the video is analyzed to identify the player's position information and the ball's trajectory. The input is the professional match data, and the output is the analyzed position information and ball's trajectory data. The analysis results are stored in a MySQL database.

[0451] Step 5:

[0452] The server receives the recorded data uploaded by the user and analyzes it using a video analysis engine (OpenCV) and a machine learning model (TensorFlow). The server analyzes the user's play data and identifies the player's movements and success rate. The input is the user's recorded data, and the output is the analyzed play data.

[0453] Step 6:

[0454] The server compares the analyzed user data with professional match data. A machine learning model generates optimal tactics based on this data. For example, if the success rate of backhand shots is low, effective tactics and techniques are extracted from a set of professional data. The input is the user data and the professional data, and the output is an optimal tactical proposal.

[0455] Step 7:

[0456] The generated tactical proposal is notified to the user via the device. Specifically, the details of the tactic (e.g., attempt a cross-court shot from the backhand) are communicated to the user using app notifications or email notifications. The input is the generated tactical proposal, and the output is the notification sent to the user.

[0457] Step 8:

[0458] The user puts on VR goggles based on the proposed tactics and launches a dedicated VR application. The device displays tactical footage generated using Unity or Unreal Engine in real time. The input is the tactical proposal and VR system components, and the output is a tactical simulation in a virtual environment.

[0459] Step 9:

[0460] During the simulation, the user's movements and performance data are uploaded from the device to the cloud server. The server receives this data and performs further analysis to improve the accuracy of the next tactical proposal. The input is the simulation data, and the output is the results of further analysis.

[0461] (Application example 1)

[0462] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0463] Conventional sports and work training systems have had difficulty proposing optimal tactics and work procedures based on the user's individual characteristics and movement data. Furthermore, they lacked the means to actually experience the proposed tactics and work procedures and obtain feedback. As a result, it was difficult for users to receive immediate and accurate feedback on their movements, hindering efficient training and work optimization.

[0464] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0465] In this invention, the server includes a camera means for recording matches and practice scenes, a means for uploading the recorded data to a cloud server, a means for collecting professional match data and analyzing each scene from the perspective of positioning and course, a means for receiving a user's play data, comparing it with the professional match data, and proposing optimal tactics based on a machine learning model, a means for actually simulating the proposed tactics using a VR system, a means for recording factory work and analyzing the data to learn and propose efficient movements, and a means for confirming and simulating the work content using smart glasses. This makes it possible to propose optimal tactics and work procedures based on the user's individual data, experience simulations on the spot, and obtain feedback.

[0466] The "camera means" is a device that records scenes of matches and practices, as well as work in the factory.

[0467] A "cloud server" is a system that stores recorded data in a remote location via the Internet and centrally manages the data for analysis.

[0468] "Professional match data" means video footage of matches or activities and related analytical data produced by experts or skilled personnel.

[0469] "Means of analysis from the perspective of positioning and course" refers to technology for analyzing the positioning and direction / flow of movement at specific scenes in a video.

[0470] "User play data" refers to video data of a user's own matches or activities that the user has recorded.

[0471] A "machine learning model" is an algorithm or model that allows a computer to learn from data and make predictions and decisions.

[0472] "Means for suggesting optimal tactics" refers to a method for presenting the most effective tactics and work procedures for the user based on analyzed data.

[0473] A "VR system" is a system that uses virtual reality technology to allow users to experience proposed tactics and work procedures in a virtual environment.

[0474] "Means for recording factory work" refers to devices and technologies that record work procedures and actions within a factory as video.

[0475] The "means for learning and suggesting efficient movements" is a system that analyzes recorded work data and recommends movements that will improve work efficiency.

[0476] "Smart glasses" are wearable devices that can display information in the user's field of vision.

[0477] "Simulation means" refers to technology that provides a virtual environment in which proposed tactics and procedures can be experienced.

[0478] This invention is a system that allows users to record matches, training sessions, or factory work, and analyze the data to learn efficiency and tactics. A specific example of the system is described below.

[0479] First, the user uses a camera to record a match, practice session, or factory work. Once recording is complete, the device automatically uploads the recorded data to a cloud server. This process can be performed using a commercially available video camera or wearable camera. The recorded data is temporarily stored on the device and then transferred via the Internet to a cloud server. Cloud services such as Amazon Web Services (AWS) and Google Cloud Platform (GCP) can be used as the cloud server.

[0480] The cloud server periodically collects and analyzes professional match and work data. This process involves using scripts to automatically collect the necessary data from the Internet and storing the results in a database. The analysis can be performed using OpenCV or TensorFlow as a video analysis engine. The collected data also includes information to evaluate positioning, course, and work efficiency.

[0481] When a user uploads recorded data to a cloud server, the server receives the data and analyzes it using a video analysis engine. The analysis results are compared with professional data, and optimal tactics and work procedures are generated based on a machine learning model. This machine learning model is built using Keras and TensorFlow. The generated tactics and work procedures are then suggested to the user as specific positioning and actions. These suggestions are then notified to the user via their device.

[0482] Additionally, users can wear a VR system or smart glasses to simulate proposed tactics and procedures in a virtual environment. For example, Oculus Rift or HTC Vive can be used as VR goggles, while Microsoft HoloLens or Google Glass are suitable for smart glasses. The VR system provides users with a realistic experience, allowing them to practice the proposed movements and positions. The data from this simulation is also uploaded back to the cloud server for further analysis, improving the accuracy of future proposals.

[0483] As a concrete example, let's say work in a factory is recorded and the data is uploaded to a cloud server. When the server analyzes the received data, it finds that the worker is inefficient in a particular procedure. Next, the server analyzes the movements of professionals in similar situations based on the work data of the professionals. As a result, it finds that efficiency can be improved by changing a particular procedure. Based on this information, the server suggests to the user, "Next time, try assembling from the right side." The user can then put on smart glasses and simulate the suggested procedure in a virtual environment. Further data from this simulation is collected, and the next suggestion will become more accurate.

[0484] An example of a prompt is:

[0485] "Develop an application that records video of a robot's assembly work and suggests optimal movement patterns. This application will cover everything from recording the video to uploading it to the cloud, analyzing the data, and simulating tactics. It should also include a function that allows users to check and simulate the suggestions using smart glasses."

[0486] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0487] Step 1:

[0488] A user uses a camera to record a game, practice, or factory work. The user presses the record button on the camera to capture the video. This video data is temporarily saved on the device. After recording is complete, the device automatically uploads the recorded data to a cloud server. The input is the recorded video data, and the output is the data uploaded to the cloud server.

[0489] Step 2:

[0490] The server receives video data uploaded to the cloud. This data includes footage of games, practice sessions, and factory work recorded by users. The server receives and stores this data. The input is the video data uploaded by the user, and the output is the data stored in the server.

[0491] Step 3:

[0492] The server periodically collects professional match and performance data. The method used here is to collect data using internet scripts and APIs. This allows us to obtain information such as positioning and movement patterns. The input is the professional data collected via the internet, and the output is data stored in a database for analysis.

[0493] Step 4:

[0494] The server uses a video analysis engine to analyze the collected professional match and work data. Software such as OpenCV and TensorFlow are used. The analyzed data is broken down into individual frames, and positional information and movement patterns are extracted. The input is the collected professional data, and the output is analyzed positioning and movement pattern data.

[0495] Step 5:

[0496] The server uses a video analysis engine to analyze the play data uploaded by the user and compares it with professional match data and work data. Based on a machine learning model (using Keras and TensorFlow), it generates the optimal tactics and work procedures for the user's actions. The input is the user's play data and professional data, and the output is the optimal tactics and work procedures suggested to the user.

[0497] Step 6:

[0498] The device notifies the user of the generated optimal tactics and work procedures. Notification methods include a smartphone app and a notification system. At this point, the user can confirm the proposal and then perform an actual simulation in the next step. The input is the proposal information from the server, and the output is a notification to the user.

[0499] Step 7:

[0500] Users wear a VR system or smart glasses and simulate proposed tactics and work procedures in a virtual environment. VR systems such as Oculus Rift and HTC Vive are used, while smart glasses such as Microsoft HoloLens and Google Glass are used. Data from the simulation is also uploaded to a server for further analysis. The input is the proposed tactics and work procedures, and the output is the data from the simulation.

[0501] Step 8:

[0502] The server receives the data from the simulation and performs further analysis. This data is used to improve the accuracy of the next proposal. A video analysis engine and machine learning models are used for data analysis. The input is the data from the simulation, and the output is an updated proposal.

[0503] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0504] The present invention relates to a system that allows users to record matches and practice sessions, analyze the data, and learn tactics, and further combines it with an emotion engine that recognizes the user's emotional state and adjusts tactical suggestions accordingly. A specific example of this system is shown below.

[0505] Basic configuration

[0506] The system consists of the following main components:

[0507] 1. Camera means: A device that allows users to record games and practice sessions.

[0508] 2. Uploading recorded data: A function that allows the device to upload recorded data to the cloud server.

[0509] 3. Professional match data collection means: The server collects professional match data from the Internet.

[0510] 4. Data analysis means: The server has the ability to analyze professional match data and user play data.

[0511] 5. Tactical suggestion means: A function in which the server suggests optimal tactics to the user based on the analysis results.

[0512] 6. VR System: A virtual reality environment for practical simulation of the proposed tactics.

[0513] 7. Emotion Engine: The ability to recognize user emotions and tailor suggestions accordingly.

[0514] Program processing

[0515] First, the user sets up the camera to record a match or practice session, then presses the record button to capture the footage. The recorded data is temporarily saved on the device and uploaded to the cloud server after recording is complete.

[0516] The server periodically collects professional match data from the Internet, analyzes each frame with a video analysis engine, and extracts player movements, positioning, and ball trajectory, then stores this data in a database.

[0517] The server receives the recorded data uploaded by the user and analyzes it using a video analysis engine. The results of this analysis are compared with professional match data and the optimal tactics are suggested to the user based on a machine learning model.

[0518] The proposed tactics are notified to the user via their device, and the user receives specific advice through the application. At this time, the emotion engine identifies the user's emotional state from their facial expressions and tone of voice, and adjusts the suggestions based on this information. For example, if the user is nervous, the tactical suggestions will also include mental advice.

[0519] The user puts on the VR goggles and launches the application. The device plays back the tactical footage generated using the VR system, providing the user with a realistic match experience. Data is also collected during the simulation and uploaded back to the cloud server for further analysis. This data is used to improve the accuracy of future tactical suggestions.

[0520] Specific examples

[0521] For example, suppose a user records a basketball practice and uploads the data to a cloud server. When the server analyzes the received data, it finds that the user has a low success rate in jump shots. After comparing the data with that of professional players, it concludes that the user needs to improve their posture and positioning when taking a jump shot. Based on this information, the server suggests to the user to "pay attention to the position of their feet and the rotation of their shoulders when taking a jump shot."

[0522] Furthermore, if the emotion engine detects nervousness from the user's facial expression, it will add mental advice such as "Relax and shoot." The user then puts on VR goggles and simulates the proposed tactic in a virtual environment. The simulation results are then uploaded back to the cloud server for further analysis.

[0523] This allows the present invention to support more personalized tactical suggestions and practical skill improvement, taking into account the user's emotional state. By precisely matching professional data, user data, and emotional data, the present invention can suggest tactics suited to each individual's playing style, enabling efficient and effective training.

[0524] The processing flow will be explained below.

[0525] Step 1:

[0526] Users set up a camera to record their matches or practice sessions. They turn on the camera and press the record start button to capture footage. The camera records the user's movements in high resolution.

[0527] Step 2:

[0528] The device temporarily stores the video data recorded by the camera in local memory. When recording is finished, the user presses the stop recording button to end the recording.

[0529] Step 3:

[0530] The device uploads the temporarily saved recording data to the cloud server, where it is divided into segments and uploaded to the cloud storage sequentially.

[0531] Step 4:

[0532] The server periodically runs a script that collects professional match data from the internet, and the collected data is obtained from official websites and streaming services using APIs.

[0533] Step 5:

[0534] The server passes professional match data to a video analysis engine, which analyzes each frame to extract player movements, positioning, and ball trajectory, which are then stored in a database.

[0535] Step 6:

[0536] The server receives the user's recorded data and analyzes it using the same video analysis engine. As a result, the user's movements and the course of the ball are extracted.

[0537] Step 7:

[0538] The server compares professional match data with the user's playing data, and based on machine learning models, generates optimal tactics for the user, providing specific instructions on positioning and ball trajectory.

[0539] Step 8:

[0540] The server stores the generated tactical suggestions in a database and notifies the device, which may include specific advice such as "try a cross shot from your backhand."

[0541] Step 9:

[0542] The device receives the tactical proposal and activates the emotion engine, which analyzes the user's facial expressions and tone of voice to identify their emotional state.

[0543] Step 10:

[0544] The emotion engine adjusts tactical suggestions based on detected emotional data. For example, if the user is nervous, the suggestion will include mental advice.

[0545] Step 11:

[0546] The user puts on the VR goggles and launches the application based on the tactical proposal. The user then practices the proposed tactical proposal in the VR simulation.

[0547] Step 12:

[0548] The device uses a VR system to provide users with tactical footage that responds to the user's movements in real time, providing an experience similar to that of a real match.

[0549] Step 13:

[0550] The user finishes the simulation and checks the feedback obtained. The simulation data is then uploaded to the cloud server again.

[0551] Step 14:

[0552] The server receives the simulation data and performs further analysis, which is then stored in a database and used to improve the accuracy of the next tactical proposal.

[0553] This allows for personalized tactical suggestions that take into account the user's emotional state, resulting in efficient and effective training.

[0554] Example 2

[0555] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0556] Conventional sports analysis systems analyze recorded data and propose tactics without taking into account the user's emotional state, which means they are unable to propose tactics that are appropriate for the user's mental state. Additionally, there is a lack of an iterative analysis system to improve the accuracy of tactical proposals.

[0557] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0558] In this invention, the server includes a recording device for recording game and practice scenes, a means for uploading the recorded data to the data server, a means for collecting expert game data and analyzing each scene from the perspective of position information and movement, a means for receiving user movement data, comparing it with the expert game data, and proposing optimal tactics based on a machine learning algorithm, a means for actually simulating the proposed tactics using a virtual reality device, and an emotion analysis means for recognizing the user's emotional state and adjusting the proposed content. This enables individual tactical proposals suited to the user's emotional state. Furthermore, by reanalyzing the simulation data, the accuracy of the tactical proposals can be continuously improved.

[0559] "Recording equipment means" refers to a device for recording footage of matches, practice sessions, etc.

[0560] "Data server" refers to a server that stores recorded data and analysis results and exchanges data with other devices.

[0561] "Expert match data" refers to records of matches and practice sessions of professional or specialized athletes, and is data used to analyze their movements and tactics.

[0562] "Means for analyzing from the perspective of positional information and movements" refers to technology for analyzing recorded data and collected data to extract and evaluate players' positional information and movements.

[0563] "Motion data" refers to data that quantifies or quantifies the movements and actions of a user while playing.

[0564] "Machine learning algorithms" refers to a collection of computer programs and mathematical models for analyzing data and creating predictive models.

[0565] "Means for suggesting tactics" refers to a system that suggests optimal tactics and moves to users based on the analysis results.

[0566] "Virtual reality device" refers to a device or system that allows a user to simulate an experience in a virtual environment.

[0567] "Emotion analysis means" refers to technology that recognizes and analyzes a user's emotional state from their facial expressions, tone of voice, and manner of operation.

[0568] The present invention is a system that allows users to record sports matches and practice sessions, analyze the data, and learn tactics. It also incorporates an emotion engine that recognizes the user's emotional state and adjusts tactical suggestions accordingly.

[0569] System configuration

[0570] The system includes the following main components and specific hardware and software:

[0571] 1. Recording equipment means:

[0572] This is a device that allows users to record matches and practice sessions. For example, it uses a smartphone camera or a dedicated recording camera.

[0573] 2. How to upload recording data:

[0574] This function allows the device to upload recorded data to a cloud server. The data is stored in cloud storage, such as AWS S3.

[0575] 3. Expert match data collection methods:

[0576] The server collects expert match data from the Internet. The collected data is obtained using, for example, a web scraping tool (e.g., BeautifulSoup) and analyzed by a video analysis engine (e.g., OpenCV).

[0577] 4. Data analysis methods:

[0578] The server receives the video data uploaded by users and analyzes it with a video analysis engine (e.g., OpenCV). The analysis results are then compared with expert match data using machine learning algorithms (e.g., TensorFlow).

[0579] 5. Tactical Proposal Methods:

[0580] This function allows the server to propose optimal strategies to users based on the analysis results. The proposals are saved in a database and sent to the user via their device.

[0581] 6. Virtual Reality Devices:

[0582] It is a device that allows users to simulate proposed tactics in a virtual environment, for example, using VR goggles such as Oculus Rift.

[0583] 7. Emotion analysis means:

[0584] The device analyzes the user's facial expressions and tone of voice and uses an emotion engine (e.g., Microsoft Azure Emotion API) to identify their emotional state, and then tailors tactical suggestions based on this information.

[0585] Example of operation

[0586] A user records their basketball practice and uploads the data to a cloud server. The server analyzes the data and finds that the user's jump shot success rate is low. After comparing it with expert game data, it is determined that the user's posture and positioning during jump shots need to be improved. Based on this, the server suggests, "Pay attention to the position of your feet and shoulder rotation when taking a jump shot."

[0587] Furthermore, if the emotion analysis means detects nervousness from the user's facial expression, the system adds mental advice such as "Relax and shoot." The user then puts on the VR goggles and simulates the proposed tactics in a virtual environment. The results of this simulation are uploaded back to the server for further analysis and to improve the accuracy of the tactical suggestions.

[0588] Prompt Sentence Examples

[0589] "We recorded basketball practice and uploaded the data to a cloud server. The results showed that the jump shot success rate was low. We compared it with expert player data and would like to know what improvements can be made. Also, please provide any mental advice that would be useful if the user is nervous."

[0590] The present invention makes it possible to propose individual tactics that take into account the user's emotional state, and furthermore, the accuracy of the tactics proposals is continuously improved by reanalyzing the simulation data.

[0591] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0592] Step 1:

[0593] Start and stop recording

[0594] The user sets up a camera and presses the record button in the application to record a match or practice. The scene to be recorded is the input, and the recorded data is generated as the output.

[0595] When you have finished recording, press the stop recording button to end the recording. Specifically, you can perform the recording operation within the smartphone or dedicated camera app.

[0596] Step 2:

[0597] Uploading recording data

[0598] The device uploads the recorded data to the data server. The input is the recorded data stored in the device, and the output is the data safely stored in the cloud.

[0599] The data is encrypted and transferred securely, and once the recording is complete, the app automatically uploads the data to a cloud storage service such as AWS S3.

[0600] Step 3:

[0601] Expert match data collection

[0602] The server collects expert match data from the Internet. The input is match data on the Internet, and the output is match data stored in a database in the server.

[0603] Analyze the video using a video analysis engine (e.g., OpenCV) to extract location information and behavior. Specifically, use a web scraping tool (e.g., BeautifulSoup) to obtain and analyze the data.

[0604] Step 4:

[0605] Analysis of recorded data

[0606] The server receives the recorded data uploaded by the user. The input is the recorded data uploaded to the cloud, and the output is the analyzed play data.

[0607] A video analysis engine (e.g., OpenCV) analyzes each frame and extracts information such as player movements, positioning, and ball trajectory. Specifically, the server analyzes the recorded data using a machine learning model (e.g., TensorFlow).

[0608] Step 5:

[0609] Generate tactical suggestions

[0610] The server proposes optimal tactics based on the analysis results. The input is the analysis results, and the output is tactical proposals to the user.

[0611] The proposed content is stored in a database and notified to the user via the device. Specifically, the server uses a machine learning algorithm to calculate and propose tactics based on the analyzed data.

[0612] Step 6:

[0613] Emotion recognition and tactical suggestion adjustment

[0614] The device analyzes the user's facial expressions and tone of voice to identify their emotional state. The input is the user's facial and voice data, and the output is tailored tactical suggestions.

[0615] The app uses an emotion engine (e.g., Microsoft Azure Emotion API) to analyze the user's emotional state and adjust the suggestions accordingly. Specifically, the app uses the camera and microphone to capture and analyze the user's emotional data.

[0616] Step 7:

[0617] VR Simulation

[0618] The user puts on the VR goggles and launches the application to simulate tactics. The input is the tactical proposal data, and the output is the simulation results.

[0619] The device plays the generated tactical footage in a VR environment, providing the user with a realistic match experience. Specifically, the user puts on VR goggles such as Oculus Rift and launches the app.

[0620] Step 8:

[0621] Re-uploading and analyzing simulation data

[0622] The terminal uploads the data collected during the simulation back to the data server. The input is the simulation data, and the output is the analyzable data on the server.

[0623] The server performs the analysis again and stores the analysis results in a database. Specifically, once the simulation is complete, the application automatically uploads the data to the cloud for further analysis.

[0624] The above steps enable individual tactical suggestions and continuous analysis based on the user's emotional state.

[0625] (Application example 2)

[0626] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0627] The objective of this invention is to propose driving tactics that take into account not only the analysis of game and practice scenes, but also the analysis of vehicle driving conditions and the driver's emotional state. Conventional systems have difficulty integrating these complex elements to analyze and propose driving tactics, and there is a lack of technology to improve driver safety and efficient driving.

[0628] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0629] In this invention, the server includes a camera means for recording game and practice scenes, a means for uploading the recorded data to a remote server, a means for collecting professional game data and analyzing each scene in terms of positioning and course, a means for receiving user play data and comparing it with the professional game data to propose optimal tactics based on a machine learning model, a means for actually simulating the proposed tactics using a virtual reality system, a means for collecting vehicle driving situation data using a camera and sensor devices, a means for analyzing the driving data and generating optimal driving tactics, and a means for recognizing the driver's emotional state and adjusting the content of the tactical proposal. This makes it possible to propose safe and effective driving tactics to the driver through detailed data collection using the camera and sensor devices, and analysis and emotion recognition based on the data.

[0630] "Photography equipment" refers to equipment for recording video and images of games, practice scenes, vehicle driving conditions, and the like.

[0631] A "remote server" is a remotely located computer system for storing, analyzing, and providing data.

[0632] "Professional match data" refers to data collected from matches involving professional players, and includes information on positioning, movements, strategies, and the like.

[0633] A "machine learning model" is an algorithm or system that is trained using data to automatically learn and perform specific tasks.

[0634] "Virtual reality system" is a general term for devices and software that allow users to experience a virtual environment visually and with other senses.

[0635] A "sensor device" is a device that detects the driving conditions of a vehicle and the surrounding environment and collects the data.

[0636] "Driving data" refers to information about the driving conditions of a vehicle, including detailed data such as speed, braking, and steering.

[0637] "Operation tactics" is a general term for plans and methods for achieving safe and efficient vehicle operation.

[0638] "Emotional state" refers to the psychological state or mood of the driver or user, and includes emotions such as tension, relaxation, and fatigue.

[0639] The present invention relates to a system that records and analyzes game and practice scenes, and proposes tactics that take into account the driver's emotional state. It also has the function of analyzing the vehicle's driving situation and proposing driving tactics based on the driver's emotional state. Specific embodiments are described below.

[0640] Program Overview

[0641] The server includes the following means: a camera means for recording matches and practice scenes, a means for uploading the recorded data to a remote server, a means for collecting professional match data and analyzing each scene in terms of positioning and course, a means for receiving the user's play data, comparing it with the professional match data, and proposing optimal tactics based on a machine learning model, a means for actually simulating the proposed tactics using a virtual reality system, a means for collecting vehicle driving situation data using a camera device and sensor devices, a means for analyzing the driving data and generating optimal driving tactics, and a means for recognizing the driver's emotional state and adjusting the content of the proposed tactics. This makes it possible to collect detailed data using the camera device and sensor devices, and then propose safe and effective driving tactics to the driver through analysis and emotion recognition based on the collected data.

[0642] Hardware and Software Use

[0643] Camera means: The camera uses NVIDIA's Jetson AGX Xavier development kit.

[0644] Cloud servers: AWS or Google Cloud are used for data storage and analysis.

[0645] VPN system: Oculus Rift or HTC Vive are used for virtual reality experiences.

[0646] Sensor devices: LiDAR and various sensors are used to collect data on vehicle driving conditions.

[0647] Emotion recognition: Using software such as OpenFace and Google Speech-to-Text API.

[0648] Data analysis: TensorFlow and PyTorch are used to analyze data and implement machine learning models.

[0649] Program processing

[0650] First, the user uses the camera means to record the game or practice scene, and saves the recorded data in the terminal.

[0651] The device automatically uploads the recorded data to a cloud server, which collects professional match data from the internet and analyzes it using a video analysis engine.

[0652] It uses cameras and sensors to collect data on vehicle driving conditions, then uses AWS Lambda and Google Cloud Functions to analyze the data and generate safe and efficient driving tactics.

[0653] The emotion recognition engine recognizes the driver's emotional state from their facial expressions and tone of voice, and adjusts the tactical suggestions based on this information.

[0654] Specific examples

[0655] For example, if a driver is tired after a long drive, the emotion engine will recognize this state and suggest that the driver take a rest at the next rest area. This suggestion can also be experienced visually through the virtual reality system.

[0656] Prompt Sentence Examples

[0657] Please provide the user's camera footage and driving data to develop a system that performs driving analysis and emotion recognition.

[0658] Include the following elements:

[0659] 1. Recording devices and their data upload methods

[0660] 2. Professional driving data collection methods

[0661] 3. Data analysis approach

[0662] 4. How to propose operational tactics

[0663] 5. Emotion Recognition Algorithms

[0664] 6. How to run a VR simulation

[0665] This invention integrates driving and emotion recognition technology to improve driver safety and efficiency by providing improved driving instructions in real time.

[0666] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0667] Step 1:

[0668] A user uses a camera to capture video data of a match or practice session. The video data is input and temporarily saved.

[0669] Step 2:

[0670] The terminal automatically uploads the temporarily stored video data to a remote server. The input is the temporarily stored video data, and the output is the video data on the remote server.

[0671] Step 3:

[0672] The server collects professional match data from the Internet and uses a video analysis engine (e.g., TensorFlow or PyTorch) to analyze each scene in terms of positioning and course. The input is the professional match data, and the output is the analyzed data.

[0673] Step 4:

[0674] The server receives the user's play data, compares it with the professional match data, and proposes optimal tactics based on the machine learning model. The input is the user's play data and the analyzed professional match data, and the output is a tactical proposal.

[0675] Step 5:

[0676] The user uses the virtual reality system to actually simulate the tactics proposed by the server. The input is the tactical proposal from the server, and the output is the simulation data.

[0677] Step 6:

[0678] The user acquires driving situation data using the camera and sensor devices mounted on the vehicle. The input is the driving situation data, and the output is the temporarily saved data.

[0679] Step 7:

[0680] The terminal uploads the temporarily saved driving situation data to the cloud server. The input is the temporarily saved driving situation data, and the output is the data on the cloud server.

[0681] Step 8:

[0682] The server analyzes the driving situation data and the user's emotional state data and generates optimal driving tactics using a machine learning model (e.g., TensorFlow or PyTorch). The input is the driving situation data and the emotional state data, and the output is a proposed driving tactic.

[0683] Step 9:

[0684] The server uses an emotion recognition engine (e.g., OpenFace or Google Speech-to-Text API) to analyze the user's facial expressions and tone of voice to recognize their emotional state. The input is the user's facial expression data and voice data, and the output is emotional state data.

[0685] Step 10:

[0686] The server provides the generated driving tactics to the user and continues driving based on the proposed tactics. The input is the proposed driving tactics and emotional state data, and the output is safe and efficient driving.

[0687] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0688] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0689] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0690] [Third embodiment]

[0691] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0692] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0693] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0694] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0695] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0696] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0697] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0698] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0699] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0700] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0701] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0702] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0703] The present invention is a system that allows users to record matches and practice sessions, analyze the recorded data, and learn tactics. A detailed example of the system is described below.

[0704] The system works in the following steps:

[0705] 1. Recording and data uploading

[0706] First, the user sets up a camera to record a match or practice session. The user presses the camera's record button to capture the video. This video data is temporarily saved on the device. When recording is finished, the device automatically uploads the recorded data to the cloud server.

[0707] 2. Professional match data collection and analysis

[0708] The server periodically runs a script to collect professional game data from the Internet. This data includes video footage of professional games, player positioning, ball trajectories, etc. The collected data is analyzed by a video analysis engine, and the extracted information is stored in a database.

[0709] 3. User data collation and strategy proposals

[0710] When the server receives the user's recorded data, it analyzes the user's play data using a video analysis engine. The results of this analysis are compared with professional match data, and the optimal tactics for the user are generated based on a machine learning model. The generated tactics are then suggested to the user as specific positioning and ball trajectory instructions. These suggestions are then notified to the user via their device.

[0711] 4. VR Simulation

[0712] The user puts on VR goggles based on the proposed tactic and launches the application. The device then displays tactical footage generated by the VR system to the user. For example, the user can experience realistically the positioning and movements required to hit a backhand cross shot. Data from the simulation is then uploaded back to the server for further analysis.

[0713] Specific examples

[0714] For example, suppose a user records their tennis practice and uploads the data to a cloud server. When the server analyzes the received data, it finds that the user makes many mistakes with backhand shots. The server then analyzes the movements of professional players in similar situations based on professional match data. It then finds that a cross-court shot is effective when taking a backhand shot. Based on this information, the server suggests to the user, "Try a cross-court shot from your backhand in your next match." The user can then put on VR goggles and simulate the suggested tactic in a virtual environment. Further data from this simulation is collected, and the next suggestion will become more accurate.

[0715] This allows the present invention to help players improve their practical skills and enable them to master optimal tactics. By precisely comparing professional data with user data and proposing tactics suited to each individual's playing style, efficient and effective training can be achieved.

[0716] The processing flow will be explained below.

[0717] Step 1:

[0718] The user sets up a camera in the area where the game or practice is being held, turns the camera on, and presses the start recording button to capture the video.

[0719] Step 2:

[0720] The device temporarily stores the recorded data from the camera in local memory. When recording is finished, the user presses the stop recording button to end the recording.

[0721] Step 3:

[0722] The device uploads the recorded data to the cloud server, where it is divided into multiple segments and uploaded to the cloud storage sequentially.

[0723] Step 4:

[0724] The server periodically runs a script that collects professional match data from the internet, which is obtained via APIs from official websites and streaming services.

[0725] Step 5:

[0726] The server passes the collected professional match data to a video analysis engine, which analyzes each frame to extract player movements, positioning, and ball trajectory.

[0727] Step 6:

[0728] The server stores the analysis results in a database, where the data is categorized by scene and features are extracted using pattern recognition algorithms.

[0729] Step 7:

[0730] The server receives the recorded data uploaded by the user and analyzes it using a video analysis engine, which also analyzes each frame to extract the movement and course of the action.

[0731] Step 8:

[0732] The server compares professional match data with the user's play data, and uses machine learning models to suggest game strategies that suit the user's playing style and level.

[0733] Step 9:

[0734] The server stores the generated tactical suggestions in a database and notifies the user's device. For example, the suggestions may include specific advice such as "At the next point, try a cross shot from your backhand."

[0735] Step 10:

[0736] The user puts on the VR goggles and launches the application, which starts the VR simulation based on the proposed tactics.

[0737] Step 11:

[0738] The device uses a VR system to provide users with tactical footage, which is reflected in real time as the user moves, allowing them to experience actual positioning and movements during a match.

[0739] Step 12:

[0740] The user ends the simulation and receives feedback. The data from the simulation is uploaded to the cloud server again.

[0741] Step 13:

[0742] The server receives the data during the simulation and performs further analysis, which is used to refine future tactical proposals.

[0743] Example 1

[0744] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0745] Previous technology made it difficult for users to analyze their own matches and practice sessions and learn effective tactics. In particular, the lack of comparison with professional match data and tactical simulation functions made it difficult to achieve efficient training and practical skill improvement. Furthermore, existing systems lacked a means to provide realistic simulations using virtual reality, making them insufficient for improving users' actual performance.

[0746] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0747] In this invention, the server includes a video recording means for recording matches and practice scenes, a communication means for uploading the recorded data to a cloud server, a data analysis means for collecting professional match data and analyzing each scene from the perspective of position information and progression paths, a tactic generation means for receiving the user's play data, comparing it with the professional match data, and proposing optimal tactics based on a machine learning model, and a virtual reality display means for actually simulating the proposed tactics using a virtual reality system. This allows the user to compare and learn from their own recorded play with the professional data, and further simulate the proposed tactics using virtual reality.

[0748] "Video recording means" refers to a device or system for recording video of a match or practice session.

[0749] "Communication means" refers to the internet connection or network device used to upload recorded data to the cloud server.

[0750] "Data analysis means" refers to software or algorithms for analyzing collected professional match data from the perspective of location information and progression paths.

[0751] The "tactic generation means" is a system that receives the user's play data, compares it with professional match data, and proposes optimal tactics based on a machine learning model.

[0752] "Virtual reality display means" refers to a device or system for actually simulating a proposed tactic using a virtual reality system.

[0753] A "cloud server" is a remote server used to store, manage, and analyze data over the Internet.

[0754] A "machine learning model" is an algorithm that recognizes patterns based on large amounts of data and makes predictions and decisions.

[0755] The present invention is a system that allows users to record matches and practice sessions, analyze the recorded data, and learn tactics. A detailed example of the system is described below.

[0756] Recording and Data Upload

[0757] A user sets up a camera to record a match or practice session. Specifically, the camera is fixed on a tripod to ensure a good image. When the user presses the record button on the camera, the video is captured and temporarily saved on the device. When the recording is finished, the device uploads the recorded data to the cloud server using a Wi-Fi module. The upload is performed using an HTTP POST request.

[0758] Professional match data collection and analysis

[0759] The server periodically runs a Python script to collect professional match data from the Internet through web scraping. For example, it uses BeautifulSoup and Selenium to retrieve match footage and statistical data from sports sites. The collected data is then analyzed for each frame using OpenCV to identify player positions and ball trajectories. The analysis results are then stored in a MySQL database.

[0760] User data collation and tactical proposals

[0761] When the server receives the video data uploaded by the user, it uses OpenCV and a machine learning model (e.g., TensorFlow) to analyze the user's play data. The analyzed user data is then compared with previously collected professional match data. Based on the results of this comparison, the machine learning model generates optimal tactics. The generated tactical suggestions are notified to the user via their device in the form of specific positioning and ball trajectory. For example, the app's notification function could display a message such as, "Try a backhand cross-court shot."

[0762] VR Simulation

[0763] Based on the proposed tactics, the user puts on VR goggles (e.g., Oculus Rift) and launches a dedicated VR application. The device then provides the user with tactical footage generated using a VR system based on Unity or Unreal Engine. The user then practices the presented tactics in the virtual environment, experiencing realistic movements in the process. Performance data from the simulation is then uploaded back to a cloud server for further analysis.

[0764] Specific examples

[0765] For example, suppose a user records their tennis practice and uploads the data to a cloud server. When the server analyzes the received data, it finds that the user makes many mistakes with backhand shots. The server then analyzes the movements of professional players in similar situations based on professional match data. It then finds that a cross-court shot is effective when taking a backhand shot. Based on this information, the server suggests to the user, "Try a cross-court shot from your backhand in your next match." The user then puts on VR goggles and simulates the suggested tactic in a virtual environment. Further data from this simulation is collected, and the next suggestion becomes more accurate.

[0766] Prompt Sentence Examples

[0767] Below are some example prompts to input to the generative AI model:

[0768] "Explain how to analyze a user's recorded tennis practice video and suggest improvements for the backhand shot."

[0769] "Please explain the process of proposing optimal tactics to users based on data on professional players' backhand shots."

[0770] "Please explain in detail the specific steps to use VR goggles to simulate tennis tactics."

[0771] This allows the present invention to help players improve their practical skills and enable them to master optimal tactics. By precisely comparing professional data with user data and proposing tactics suited to each individual's playing style, efficient and effective training can be achieved.

[0772] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0773] Step 1:

[0774] A user sets up a camera to record a match or practice session and presses the record button. This causes the camera to capture video and generate recorded data. The recorded data is temporarily stored in the device's internal storage. The input is video data, and the output is a recorded file stored in the internal storage.

[0775] Step 2:

[0776] When recording is finished, the device connects to the Internet using the Wi-Fi module and uploads the recorded data to the cloud server. Specifically, the recorded file is sent to the cloud server via an HTTP POST request. The input is the recorded file in the internal storage, and the output is the recorded data saved on the cloud server.

[0777] Step 3:

[0778] The server periodically runs a Python script to collect professional match data from the Internet, for example, using BeautifulSoup and Selenium to retrieve match footage and statistics from sports sites. The input is the URL of the sports site, and the output is the collected professional match data.

[0779] Step 4:

[0780] The server uses OpenCV to analyze the professional match data collected. Each frame of the video is analyzed to identify the player's position information and the ball's trajectory. The input is the professional match data, and the output is the analyzed position information and ball's trajectory data. The analysis results are stored in a MySQL database.

[0781] Step 5:

[0782] The server receives the recorded data uploaded by the user and analyzes it using a video analysis engine (OpenCV) and a machine learning model (TensorFlow). The server analyzes the user's play data and identifies the player's movements and success rate. The input is the user's recorded data, and the output is the analyzed play data.

[0783] Step 6:

[0784] The server compares the analyzed user data with professional match data. A machine learning model generates optimal tactics based on this data. For example, if the success rate of backhand shots is low, effective tactics and techniques are extracted from a set of professional data. The input is the user data and the professional data, and the output is an optimal tactical proposal.

[0785] Step 7:

[0786] The generated tactical proposal is notified to the user via the device. Specifically, the details of the tactic (e.g., attempt a cross-court shot from the backhand) are communicated to the user using app notifications or email notifications. The input is the generated tactical proposal, and the output is the notification sent to the user.

[0787] Step 8:

[0788] The user puts on VR goggles based on the proposed tactics and launches a dedicated VR application. The device displays tactical footage generated using Unity or Unreal Engine in real time. The input is the tactical proposal and VR system components, and the output is a tactical simulation in a virtual environment.

[0789] Step 9:

[0790] During the simulation, the user's movements and performance data are uploaded from the device to the cloud server. The server receives this data and performs further analysis to improve the accuracy of the next tactical proposal. The input is the simulation data, and the output is the results of further analysis.

[0791] (Application example 1)

[0792] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0793] Conventional sports and work training systems have had difficulty proposing optimal tactics and work procedures based on the user's individual characteristics and movement data. Furthermore, they lacked the means to actually experience the proposed tactics and work procedures and obtain feedback. As a result, it was difficult for users to receive immediate and accurate feedback on their movements, hindering efficient training and work optimization.

[0794] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0795] In this invention, the server includes a camera means for recording matches and practice scenes, a means for uploading the recorded data to a cloud server, a means for collecting professional match data and analyzing each scene from the perspective of positioning and course, a means for receiving a user's play data, comparing it with the professional match data, and proposing optimal tactics based on a machine learning model, a means for actually simulating the proposed tactics using a VR system, a means for recording factory work and analyzing the data to learn and propose efficient movements, and a means for confirming and simulating the work content using smart glasses. This makes it possible to propose optimal tactics and work procedures based on the user's individual data, experience simulations on the spot, and obtain feedback.

[0796] The "camera means" is a device that records scenes of matches and practices, as well as work in the factory.

[0797] A "cloud server" is a system that stores recorded data in a remote location via the Internet and centrally manages the data for analysis.

[0798] "Professional match data" means video footage of matches or activities and related analytical data produced by experts or skilled personnel.

[0799] "Means of analysis from the perspective of positioning and course" refers to technology for analyzing the positioning and direction / flow of movement at specific scenes in a video.

[0800] "User play data" refers to video data of a user's own matches or activities that the user has recorded.

[0801] A "machine learning model" is an algorithm or model that allows a computer to learn from data and make predictions and decisions.

[0802] "Means for suggesting optimal tactics" refers to a method for presenting the most effective tactics and work procedures for the user based on analyzed data.

[0803] A "VR system" is a system that uses virtual reality technology to allow users to experience proposed tactics and work procedures in a virtual environment.

[0804] "Means for recording factory work" refers to devices and technologies that record work procedures and actions within a factory as video.

[0805] The "means for learning and suggesting efficient movements" is a system that analyzes recorded work data and recommends movements that will improve work efficiency.

[0806] "Smart glasses" are wearable devices that can display information in the user's field of vision.

[0807] "Simulation means" refers to technology that provides a virtual environment in which proposed tactics and procedures can be experienced.

[0808] This invention is a system that allows users to record matches, training sessions, or factory work, and analyze the data to learn efficiency and tactics. A specific example of the system is described below.

[0809] First, the user uses a camera to record a match, practice session, or factory work. Once recording is complete, the device automatically uploads the recorded data to a cloud server. This process can be performed using a commercially available video camera or wearable camera. The recorded data is temporarily stored on the device and then transferred via the Internet to a cloud server. Cloud services such as Amazon Web Services (AWS) and Google Cloud Platform (GCP) can be used as the cloud server.

[0810] The cloud server periodically collects and analyzes professional match and work data. This process involves using scripts to automatically collect the necessary data from the Internet and storing the results in a database. The analysis can be performed using OpenCV or TensorFlow as a video analysis engine. The collected data also includes information to evaluate positioning, course, and work efficiency.

[0811] When a user uploads recorded data to a cloud server, the server receives the data and analyzes it using a video analysis engine. The analysis results are compared with professional data, and optimal tactics and work procedures are generated based on a machine learning model. This machine learning model is built using Keras and TensorFlow. The generated tactics and work procedures are then suggested to the user as specific positioning and actions. These suggestions are then notified to the user via their device.

[0812] Additionally, users can wear a VR system or smart glasses to simulate proposed tactics and procedures in a virtual environment. For example, Oculus Rift or HTC Vive can be used as VR goggles, while Microsoft HoloLens or Google Glass are suitable for smart glasses. The VR system provides users with a realistic experience, allowing them to practice the proposed movements and positions. The data from this simulation is also uploaded back to the cloud server for further analysis, improving the accuracy of future proposals.

[0813] As a concrete example, let's say work in a factory is recorded and the data is uploaded to a cloud server. When the server analyzes the received data, it finds that the worker is inefficient in a particular procedure. Next, the server analyzes the movements of professionals in similar situations based on the work data of the professionals. As a result, it finds that efficiency can be improved by changing a particular procedure. Based on this information, the server suggests to the user, "Next time, try assembling from the right side." The user can then put on smart glasses and simulate the suggested procedure in a virtual environment. Further data from this simulation is collected, and the next suggestion will become more accurate.

[0814] An example of a prompt is:

[0815] "Develop an application that records video of a robot's assembly work and suggests optimal movement patterns. This application will cover everything from recording the video to uploading it to the cloud, analyzing the data, and simulating tactics. It should also include a function that allows users to check and simulate the suggestions using smart glasses."

[0816] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0817] Step 1:

[0818] A user uses a camera to record a game, practice, or factory work. The user presses the record button on the camera to capture the video. This video data is temporarily saved on the device. After recording is complete, the device automatically uploads the recorded data to a cloud server. The input is the recorded video data, and the output is the data uploaded to the cloud server.

[0819] Step 2:

[0820] The server receives video data uploaded to the cloud. This data includes footage of games, practice sessions, and factory work recorded by users. The server receives and stores this data. The input is the video data uploaded by the user, and the output is the data stored in the server.

[0821] Step 3:

[0822] The server periodically collects professional match and performance data. The method used here is to collect data using internet scripts and APIs. This allows us to obtain information such as positioning and movement patterns. The input is the professional data collected via the internet, and the output is data stored in a database for analysis.

[0823] Step 4:

[0824] The server uses a video analysis engine to analyze the collected professional match and work data. Software such as OpenCV and TensorFlow are used. The analyzed data is broken down into individual frames, and positional information and movement patterns are extracted. The input is the collected professional data, and the output is analyzed positioning and movement pattern data.

[0825] Step 5:

[0826] The server uses a video analysis engine to analyze the play data uploaded by the user and compares it with professional match data and work data. Based on a machine learning model (using Keras and TensorFlow), it generates the optimal tactics and work procedures for the user's actions. The input is the user's play data and professional data, and the output is the optimal tactics and work procedures suggested to the user.

[0827] Step 6:

[0828] The device notifies the user of the generated optimal tactics and work procedures. Notification methods include a smartphone app and a notification system. At this point, the user can confirm the proposal and then perform an actual simulation in the next step. The input is the proposal information from the server, and the output is a notification to the user.

[0829] Step 7:

[0830] Users wear a VR system or smart glasses and simulate proposed tactics and work procedures in a virtual environment. VR systems such as Oculus Rift and HTC Vive are used, while smart glasses such as Microsoft HoloLens and Google Glass are used. Data from the simulation is also uploaded to a server for further analysis. The input is the proposed tactics and work procedures, and the output is the data from the simulation.

[0831] Step 8:

[0832] The server receives the data from the simulation and performs further analysis. This data is used to improve the accuracy of the next proposal. A video analysis engine and machine learning models are used for data analysis. The input is the data from the simulation, and the output is an updated proposal.

[0833] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0834] The present invention relates to a system that allows users to record matches and practice sessions, analyze the data, and learn tactics, and further combines it with an emotion engine that recognizes the user's emotional state and adjusts tactical suggestions accordingly. A specific example of this system is shown below.

[0835] Basic configuration

[0836] The system consists of the following main components:

[0837] 1. Camera means: A device that allows users to record games and practice sessions.

[0838] 2. Uploading recorded data: A function that allows the device to upload recorded data to the cloud server.

[0839] 3. Professional match data collection means: The server collects professional match data from the Internet.

[0840] 4. Data analysis means: The server has the ability to analyze professional match data and user play data.

[0841] 5. Tactical suggestion means: A function in which the server suggests optimal tactics to the user based on the analysis results.

[0842] 6. VR System: A virtual reality environment for practical simulation of the proposed tactics.

[0843] 7. Emotion Engine: The ability to recognize user emotions and tailor suggestions accordingly.

[0844] Program processing

[0845] First, the user sets up the camera to record a match or practice session, then presses the record button to capture the footage. The recorded data is temporarily saved on the device and uploaded to the cloud server after recording is complete.

[0846] The server periodically collects professional match data from the Internet, analyzes each frame with a video analysis engine, and extracts player movements, positioning, and ball trajectory, then stores this data in a database.

[0847] The server receives the recorded data uploaded by the user and analyzes it using a video analysis engine. The results of this analysis are compared with professional match data and the optimal tactics are suggested to the user based on a machine learning model.

[0848] The proposed tactics are notified to the user via their device, and the user receives specific advice through the application. At this time, the emotion engine identifies the user's emotional state from their facial expressions and tone of voice, and adjusts the suggestions based on this information. For example, if the user is nervous, the tactical suggestions will also include mental advice.

[0849] The user puts on the VR goggles and launches the application. The device plays back the tactical footage generated using the VR system, providing the user with a realistic match experience. Data is also collected during the simulation and uploaded back to the cloud server for further analysis. This data is used to improve the accuracy of future tactical suggestions.

[0850] Specific examples

[0851] For example, suppose a user records a basketball practice and uploads the data to a cloud server. When the server analyzes the received data, it finds that the user has a low success rate in jump shots. After comparing the data with that of professional players, it concludes that the user needs to improve their posture and positioning when taking a jump shot. Based on this information, the server suggests to the user to "pay attention to the position of their feet and the rotation of their shoulders when taking a jump shot."

[0852] Furthermore, if the emotion engine detects nervousness from the user's facial expression, it will add mental advice such as "Relax and shoot." The user then puts on VR goggles and simulates the proposed tactic in a virtual environment. The simulation results are then uploaded back to the cloud server for further analysis.

[0853] This allows the present invention to support more personalized tactical suggestions and practical skill improvement, taking into account the user's emotional state. By precisely matching professional data, user data, and emotional data, the present invention can suggest tactics suited to each individual's playing style, enabling efficient and effective training.

[0854] The processing flow will be explained below.

[0855] Step 1:

[0856] Users set up a camera to record their matches or practice sessions. They turn on the camera and press the record start button to capture footage. The camera records the user's movements in high resolution.

[0857] Step 2:

[0858] The device temporarily stores the video data recorded by the camera in local memory. When recording is finished, the user presses the stop recording button to end the recording.

[0859] Step 3:

[0860] The device uploads the temporarily saved recording data to the cloud server, where it is divided into segments and uploaded to the cloud storage sequentially.

[0861] Step 4:

[0862] The server periodically runs a script that collects professional match data from the internet, and the collected data is obtained from official websites and streaming services using APIs.

[0863] Step 5:

[0864] The server passes professional match data to a video analysis engine, which analyzes each frame to extract player movements, positioning, and ball trajectory, which are then stored in a database.

[0865] Step 6:

[0866] The server receives the user's recorded data and analyzes it using the same video analysis engine. As a result, the user's movements and the course of the ball are extracted.

[0867] Step 7:

[0868] The server compares professional match data with the user's playing data, and based on machine learning models, generates optimal tactics for the user, providing specific instructions on positioning and ball trajectory.

[0869] Step 8:

[0870] The server stores the generated tactical suggestions in a database and notifies the device, which may include specific advice such as "try a cross shot from your backhand."

[0871] Step 9:

[0872] The device receives the tactical proposal and activates the emotion engine, which analyzes the user's facial expressions and tone of voice to identify their emotional state.

[0873] Step 10:

[0874] The emotion engine adjusts tactical suggestions based on detected emotional data. For example, if the user is nervous, the suggestion will include mental advice.

[0875] Step 11:

[0876] The user puts on the VR goggles and launches the application based on the tactical proposal. The user then practices the proposed tactical proposal in the VR simulation.

[0877] Step 12:

[0878] The device uses a VR system to provide users with tactical footage that responds to the user's movements in real time, providing an experience similar to that of a real match.

[0879] Step 13:

[0880] The user finishes the simulation and checks the feedback obtained. The simulation data is then uploaded to the cloud server again.

[0881] Step 14:

[0882] The server receives the simulation data and performs further analysis, which is then stored in a database and used to improve the accuracy of the next tactical proposal.

[0883] This allows for personalized tactical suggestions that take into account the user's emotional state, resulting in efficient and effective training.

[0884] Example 2

[0885] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0886] Conventional sports analysis systems analyze recorded data and propose tactics without taking into account the user's emotional state, which means they are unable to propose tactics that are appropriate for the user's mental state. Additionally, there is a lack of an iterative analysis system to improve the accuracy of tactical proposals.

[0887] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0888] In this invention, the server includes a recording device for recording game and practice scenes, a means for uploading the recorded data to the data server, a means for collecting expert game data and analyzing each scene from the perspective of position information and movement, a means for receiving user movement data, comparing it with the expert game data, and proposing optimal tactics based on a machine learning algorithm, a means for actually simulating the proposed tactics using a virtual reality device, and an emotion analysis means for recognizing the user's emotional state and adjusting the proposed content. This enables individual tactical proposals suited to the user's emotional state. Furthermore, by reanalyzing the simulation data, the accuracy of the tactical proposals can be continuously improved.

[0889] "Recording equipment means" refers to a device for recording footage of matches, practice sessions, etc.

[0890] "Data server" refers to a server that stores recorded data and analysis results and exchanges data with other devices.

[0891] "Expert match data" refers to records of matches and practice sessions of professional or specialized athletes, and is data used to analyze their movements and tactics.

[0892] "Means for analyzing from the perspective of positional information and movements" refers to technology for analyzing recorded data and collected data to extract and evaluate players' positional information and movements.

[0893] "Motion data" refers to data that quantifies or quantifies the movements and actions of a user while playing.

[0894] "Machine learning algorithms" refers to a collection of computer programs and mathematical models for analyzing data and creating predictive models.

[0895] "Means for suggesting tactics" refers to a system that suggests optimal tactics and moves to users based on the analysis results.

[0896] "Virtual reality device" refers to a device or system that allows a user to simulate an experience in a virtual environment.

[0897] "Emotion analysis means" refers to technology that recognizes and analyzes a user's emotional state from their facial expressions, tone of voice, and manner of operation.

[0898] The present invention is a system that allows users to record sports matches and practice sessions, analyze the data, and learn tactics. It also incorporates an emotion engine that recognizes the user's emotional state and adjusts tactical suggestions accordingly.

[0899] System configuration

[0900] The system includes the following main components and specific hardware and software:

[0901] 1. Recording equipment means:

[0902] This is a device that allows users to record matches and practice sessions. For example, it uses a smartphone camera or a dedicated recording camera.

[0903] 2. How to upload recording data:

[0904] This function allows the device to upload recorded data to a cloud server. The data is stored in cloud storage, such as AWS S3.

[0905] 3. Expert match data collection methods:

[0906] The server collects expert match data from the Internet. The collected data is obtained using, for example, a web scraping tool (e.g., BeautifulSoup) and analyzed by a video analysis engine (e.g., OpenCV).

[0907] 4. Data analysis methods:

[0908] The server receives the video data uploaded by users and analyzes it with a video analysis engine (e.g., OpenCV). The analysis results are then compared with expert match data using machine learning algorithms (e.g., TensorFlow).

[0909] 5. Tactical Proposal Methods:

[0910] This function allows the server to propose optimal strategies to users based on the analysis results. The proposals are saved in a database and sent to the user via their device.

[0911] 6. Virtual Reality Devices:

[0912] It is a device that allows users to simulate proposed tactics in a virtual environment, for example, using VR goggles such as Oculus Rift.

[0913] 7. Emotion analysis means:

[0914] The device analyzes the user's facial expressions and tone of voice and uses an emotion engine (e.g., Microsoft Azure Emotion API) to identify their emotional state, and then tailors tactical suggestions based on this information.

[0915] Example of operation

[0916] A user records their basketball practice and uploads the data to a cloud server. The server analyzes the data and finds that the user's jump shot success rate is low. After comparing it with expert game data, it is determined that the user's posture and positioning during jump shots need to be improved. Based on this, the server suggests, "Pay attention to the position of your feet and shoulder rotation when taking a jump shot."

[0917] Furthermore, if the emotion analysis means detects nervousness from the user's facial expression, the system adds mental advice such as "Relax and shoot." The user then puts on the VR goggles and simulates the proposed tactics in a virtual environment. The results of this simulation are uploaded back to the server for further analysis and to improve the accuracy of the tactical suggestions.

[0918] Prompt Sentence Examples

[0919] "We recorded basketball practice and uploaded the data to a cloud server. The results showed that the jump shot success rate was low. We compared it with expert player data and would like to know what improvements can be made. Also, please provide any mental advice that would be useful if the user is nervous."

[0920] The present invention makes it possible to propose individual tactics that take into account the user's emotional state, and furthermore, the accuracy of the tactics proposals is continuously improved by reanalyzing the simulation data.

[0921] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0922] Step 1:

[0923] Start and stop recording

[0924] The user sets up a camera and presses the record button in the application to record a match or practice. The scene to be recorded is the input, and the recorded data is generated as the output.

[0925] When you have finished recording, press the stop recording button to end the recording. Specifically, you can perform the recording operation within the smartphone or dedicated camera app.

[0926] Step 2:

[0927] Uploading recording data

[0928] The device uploads the recorded data to the data server. The input is the recorded data stored in the device, and the output is the data safely stored in the cloud.

[0929] The data is encrypted and transferred securely, and once the recording is complete, the app automatically uploads the data to a cloud storage service such as AWS S3.

[0930] Step 3:

[0931] Expert match data collection

[0932] The server collects expert match data from the Internet. The input is match data on the Internet, and the output is match data stored in a database in the server.

[0933] Analyze the video using a video analysis engine (e.g., OpenCV) to extract location information and behavior. Specifically, use a web scraping tool (e.g., BeautifulSoup) to obtain and analyze the data.

[0934] Step 4:

[0935] Analysis of recorded data

[0936] The server receives the recorded data uploaded by the user. The input is the recorded data uploaded to the cloud, and the output is the analyzed play data.

[0937] A video analysis engine (e.g., OpenCV) analyzes each frame and extracts information such as player movements, positioning, and ball trajectory. Specifically, the server analyzes the recorded data using a machine learning model (e.g., TensorFlow).

[0938] Step 5:

[0939] Generate tactical suggestions

[0940] The server proposes optimal tactics based on the analysis results. The input is the analysis results, and the output is tactical proposals to the user.

[0941] The proposed content is stored in a database and notified to the user via the device. Specifically, the server uses a machine learning algorithm to calculate and propose tactics based on the analyzed data.

[0942] Step 6:

[0943] Emotion recognition and tactical suggestion adjustment

[0944] The device analyzes the user's facial expressions and tone of voice to identify their emotional state. The input is the user's facial and voice data, and the output is tailored tactical suggestions.

[0945] The app uses an emotion engine (e.g., Microsoft Azure Emotion API) to analyze the user's emotional state and adjust the suggestions accordingly. Specifically, the app uses the camera and microphone to capture and analyze the user's emotional data.

[0946] Step 7:

[0947] VR Simulation

[0948] The user puts on the VR goggles and launches the application to simulate tactics. The input is the tactical proposal data, and the output is the simulation results.

[0949] The device plays the generated tactical footage in a VR environment, providing the user with a realistic match experience. Specifically, the user puts on VR goggles such as Oculus Rift and launches the app.

[0950] Step 8:

[0951] Re-uploading and analyzing simulation data

[0952] The terminal uploads the data collected during the simulation back to the data server. The input is the simulation data, and the output is the analyzable data on the server.

[0953] The server performs the analysis again and stores the analysis results in a database. Specifically, once the simulation is complete, the application automatically uploads the data to the cloud for further analysis.

[0954] The above steps enable individual tactical suggestions and continuous analysis based on the user's emotional state.

[0955] (Application example 2)

[0956] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0957] The objective of this invention is to propose driving tactics that take into account not only the analysis of game and practice scenes, but also the analysis of vehicle driving conditions and the driver's emotional state. Conventional systems have difficulty integrating these complex elements to analyze and propose driving tactics, and there is a lack of technology to improve driver safety and efficient driving.

[0958] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0959] In this invention, the server includes a camera means for recording game and practice scenes, a means for uploading the recorded data to a remote server, a means for collecting professional game data and analyzing each scene in terms of positioning and course, a means for receiving user play data and comparing it with the professional game data to propose optimal tactics based on a machine learning model, a means for actually simulating the proposed tactics using a virtual reality system, a means for collecting vehicle driving situation data using a camera and sensor devices, a means for analyzing the driving data and generating optimal driving tactics, and a means for recognizing the driver's emotional state and adjusting the content of the tactical proposal. This makes it possible to propose safe and effective driving tactics to the driver through detailed data collection using the camera and sensor devices, and analysis and emotion recognition based on the data.

[0960] "Photography equipment" refers to equipment for recording video and images of games, practice scenes, vehicle driving conditions, and the like.

[0961] A "remote server" is a remotely located computer system for storing, analyzing, and providing data.

[0962] "Professional match data" refers to data collected from matches involving professional players, and includes information on positioning, movements, strategies, and the like.

[0963] A "machine learning model" is an algorithm or system that is trained using data to automatically learn and perform specific tasks.

[0964] "Virtual reality system" is a general term for devices and software that allow users to experience a virtual environment visually and with other senses.

[0965] A "sensor device" is a device that detects the driving conditions of a vehicle and the surrounding environment and collects the data.

[0966] "Driving data" refers to information about the driving conditions of a vehicle, including detailed data such as speed, braking, and steering.

[0967] "Operation tactics" is a general term for plans and methods for achieving safe and efficient vehicle operation.

[0968] "Emotional state" refers to the psychological state or mood of the driver or user, and includes emotions such as tension, relaxation, and fatigue.

[0969] The present invention relates to a system that records and analyzes game and practice scenes, and proposes tactics that take into account the driver's emotional state. It also has the function of analyzing the vehicle's driving situation and proposing driving tactics based on the driver's emotional state. Specific embodiments are described below.

[0970] Program Overview

[0971] The server includes the following means: a camera means for recording matches and practice scenes, a means for uploading the recorded data to a remote server, a means for collecting professional match data and analyzing each scene in terms of positioning and course, a means for receiving the user's play data, comparing it with the professional match data, and proposing optimal tactics based on a machine learning model, a means for actually simulating the proposed tactics using a virtual reality system, a means for collecting vehicle driving situation data using a camera device and sensor devices, a means for analyzing the driving data and generating optimal driving tactics, and a means for recognizing the driver's emotional state and adjusting the content of the proposed tactics. This makes it possible to collect detailed data using the camera device and sensor devices, and then propose safe and effective driving tactics to the driver through analysis and emotion recognition based on the collected data.

[0972] Hardware and Software Use

[0973] Camera means: The camera uses NVIDIA's Jetson AGX Xavier development kit.

[0974] Cloud servers: AWS or Google Cloud are used for data storage and analysis.

[0975] VPN system: Oculus Rift or HTC Vive are used for virtual reality experiences.

[0976] Sensor devices: LiDAR and various sensors are used to collect data on vehicle driving conditions.

[0977] Emotion recognition: Using software such as OpenFace and Google Speech-to-Text API.

[0978] Data analysis: TensorFlow and PyTorch are used to analyze data and implement machine learning models.

[0979] Program processing

[0980] First, the user uses the camera means to record the game or practice scene, and saves the recorded data in the terminal.

[0981] The device automatically uploads the recorded data to a cloud server, which collects professional match data from the internet and analyzes it using a video analysis engine.

[0982] It uses cameras and sensors to collect data on vehicle driving conditions, then uses AWS Lambda and Google Cloud Functions to analyze the data and generate safe and efficient driving tactics.

[0983] The emotion recognition engine recognizes the driver's emotional state from their facial expressions and tone of voice, and adjusts the tactical suggestions based on this information.

[0984] Specific examples

[0985] For example, if a driver is tired after a long drive, the emotion engine will recognize this state and suggest that the driver take a rest at the next rest area. This suggestion can also be experienced visually through the virtual reality system.

[0986] Prompt Sentence Examples

[0987] Please provide the user's camera footage and driving data to develop a system that performs driving analysis and emotion recognition.

[0988] Include the following elements:

[0989] 1. Recording devices and their data upload methods

[0990] 2. Professional driving data collection methods

[0991] 3. Data analysis approach

[0992] 4. How to propose operational tactics

[0993] 5. Emotion Recognition Algorithms

[0994] 6. How to run a VR simulation

[0995] This invention integrates driving and emotion recognition technology to improve driver safety and efficiency by providing improved driving instructions in real time.

[0996] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0997] Step 1:

[0998] A user uses a camera to capture video data of a match or practice session. The video data is input and temporarily saved.

[0999] Step 2:

[1000] The terminal automatically uploads the temporarily stored video data to a remote server. The input is the temporarily stored video data, and the output is the video data on the remote server.

[1001] Step 3:

[1002] The server collects professional match data from the Internet and uses a video analysis engine (e.g., TensorFlow or PyTorch) to analyze each scene in terms of positioning and course. The input is the professional match data, and the output is the analyzed data.

[1003] Step 4:

[1004] The server receives the user's play data, compares it with the professional match data, and proposes optimal tactics based on the machine learning model. The input is the user's play data and the analyzed professional match data, and the output is a tactical proposal.

[1005] Step 5:

[1006] The user uses the virtual reality system to actually simulate the tactics proposed by the server. The input is the tactical proposal from the server, and the output is the simulation data.

[1007] Step 6:

[1008] The user acquires driving situation data using the camera and sensor devices mounted on the vehicle. The input is the driving situation data, and the output is the temporarily saved data.

[1009] Step 7:

[1010] The terminal uploads the temporarily saved driving situation data to the cloud server. The input is the temporarily saved driving situation data, and the output is the data on the cloud server.

[1011] Step 8:

[1012] The server analyzes the driving situation data and the user's emotional state data and generates optimal driving tactics using a machine learning model (e.g., TensorFlow or PyTorch). The input is the driving situation data and the emotional state data, and the output is a proposed driving tactic.

[1013] Step 9:

[1014] The server uses an emotion recognition engine (e.g., OpenFace or Google Speech-to-Text API) to analyze the user's facial expressions and tone of voice to recognize their emotional state. The input is the user's facial expression data and voice data, and the output is emotional state data.

[1015] Step 10:

[1016] The server provides the generated driving tactics to the user and continues driving based on the proposed tactics. The input is the proposed driving tactics and emotional state data, and the output is safe and efficient driving.

[1017] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1018] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1019] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1020] [Fourth embodiment]

[1021] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1022] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1023] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1024] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1025] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1026] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1027] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1028] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1029] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1030] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1031] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1032] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1033] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1034] The present invention is a system that allows users to record matches and practice sessions, analyze the recorded data, and learn tactics. A detailed example of the system is described below.

[1035] The system works in the following steps:

[1036] 1. Recording and data uploading

[1037] First, the user sets up a camera to record a match or practice session. The user presses the camera's record button to capture the video. This video data is temporarily saved on the device. When recording is finished, the device automatically uploads the recorded data to the cloud server.

[1038] 2. Professional match data collection and analysis

[1039] The server periodically runs a script to collect professional game data from the Internet. This data includes video footage of professional games, player positioning, ball trajectories, etc. The collected data is analyzed by a video analysis engine, and the extracted information is stored in a database.

[1040] 3. User data collation and strategy proposals

[1041] When the server receives the user's recorded data, it analyzes the user's play data using a video analysis engine. The results of this analysis are compared with professional match data, and the optimal tactics for the user are generated based on a machine learning model. The generated tactics are then suggested to the user as specific positioning and ball trajectory instructions. These suggestions are then notified to the user via their device.

[1042] 4. VR Simulation

[1043] The user puts on VR goggles based on the proposed tactic and launches the application. The device then displays tactical footage generated by the VR system to the user. For example, the user can experience realistically the positioning and movements required to hit a backhand cross shot. Data from the simulation is then uploaded back to the server for further analysis.

[1044] Specific examples

[1045] For example, suppose a user records their tennis practice and uploads the data to a cloud server. When the server analyzes the received data, it finds that the user makes many mistakes with backhand shots. The server then analyzes the movements of professional players in similar situations based on professional match data. It then finds that a cross-court shot is effective when taking a backhand shot. Based on this information, the server suggests to the user, "Try a cross-court shot from your backhand in your next match." The user can then put on VR goggles and simulate the suggested tactic in a virtual environment. Further data from this simulation is collected, and the next suggestion will become more accurate.

[1046] This allows the present invention to help players improve their practical skills and enable them to master optimal tactics. By precisely comparing professional data with user data and proposing tactics suited to each individual's playing style, efficient and effective training can be achieved.

[1047] The processing flow will be explained below.

[1048] Step 1:

[1049] The user sets up a camera in the area where the game or practice is being held, turns the camera on, and presses the start recording button to capture the video.

[1050] Step 2:

[1051] The device temporarily stores the recorded data from the camera in local memory. When recording is finished, the user presses the stop recording button to end the recording.

[1052] Step 3:

[1053] The device uploads the recorded data to the cloud server, where it is divided into multiple segments and uploaded to the cloud storage sequentially.

[1054] Step 4:

[1055] The server periodically runs a script that collects professional match data from the internet, which is obtained via APIs from official websites and streaming services.

[1056] Step 5:

[1057] The server passes the collected professional match data to a video analysis engine, which analyzes each frame to extract player movements, positioning, and ball trajectory.

[1058] Step 6:

[1059] The server stores the analysis results in a database, where the data is categorized by scene and features are extracted using pattern recognition algorithms.

[1060] Step 7:

[1061] The server receives the recorded data uploaded by the user and analyzes it using a video analysis engine, which also analyzes each frame to extract the movement and course of the action.

[1062] Step 8:

[1063] The server compares professional match data with the user's play data, and uses machine learning models to suggest game strategies that suit the user's playing style and level.

[1064] Step 9:

[1065] The server stores the generated tactical suggestions in a database and notifies the user's device. For example, the suggestions may include specific advice such as "At the next point, try a cross shot from your backhand."

[1066] Step 10:

[1067] The user puts on the VR goggles and launches the application, which starts the VR simulation based on the proposed tactics.

[1068] Step 11:

[1069] The device uses a VR system to provide users with tactical footage, which is reflected in real time as the user moves, allowing them to experience actual positioning and movements during a match.

[1070] Step 12:

[1071] The user ends the simulation and receives feedback. The data from the simulation is uploaded to the cloud server again.

[1072] Step 13:

[1073] The server receives the data during the simulation and performs further analysis, which is used to refine future tactical proposals.

[1074] Example 1

[1075] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1076] Previous technology made it difficult for users to analyze their own matches and practice sessions and learn effective tactics. In particular, the lack of comparison with professional match data and tactical simulation functions made it difficult to achieve efficient training and practical skill improvement. Furthermore, existing systems lacked a means to provide realistic simulations using virtual reality, making them insufficient for improving users' actual performance.

[1077] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1078] In this invention, the server includes a video recording means for recording matches and practice scenes, a communication means for uploading the recorded data to a cloud server, a data analysis means for collecting professional match data and analyzing each scene from the perspective of position information and progression paths, a tactic generation means for receiving the user's play data, comparing it with the professional match data, and proposing optimal tactics based on a machine learning model, and a virtual reality display means for actually simulating the proposed tactics using a virtual reality system. This allows the user to compare and learn from their own recorded play with the professional data, and further simulate the proposed tactics using virtual reality.

[1079] "Video recording means" refers to a device or system for recording video of a match or practice session.

[1080] "Communication means" refers to the internet connection or network device used to upload recorded data to the cloud server.

[1081] "Data analysis means" refers to software or algorithms for analyzing collected professional match data from the perspective of location information and progression paths.

[1082] The "tactic generation means" is a system that receives the user's play data, compares it with professional match data, and proposes optimal tactics based on a machine learning model.

[1083] "Virtual reality display means" refers to a device or system for actually simulating a proposed tactic using a virtual reality system.

[1084] A "cloud server" is a remote server used to store, manage, and analyze data over the Internet.

[1085] A "machine learning model" is an algorithm that recognizes patterns based on large amounts of data and makes predictions and decisions.

[1086] The present invention is a system that allows users to record matches and practice sessions, analyze the recorded data, and learn tactics. A detailed example of the system is described below.

[1087] Recording and Data Upload

[1088] A user sets up a camera to record a match or practice session. Specifically, the camera is fixed on a tripod to ensure a good image. When the user presses the record button on the camera, the video is captured and temporarily saved on the device. When the recording is finished, the device uploads the recorded data to the cloud server using a Wi-Fi module. The upload is performed using an HTTP POST request.

[1089] Professional match data collection and analysis

[1090] The server periodically runs a Python script to collect professional match data from the Internet through web scraping. For example, it uses BeautifulSoup and Selenium to retrieve match footage and statistical data from sports sites. The collected data is then analyzed for each frame using OpenCV to identify player positions and ball trajectories. The analysis results are then stored in a MySQL database.

[1091] User data collation and tactical proposals

[1092] When the server receives the video data uploaded by the user, it uses OpenCV and a machine learning model (e.g., TensorFlow) to analyze the user's play data. The analyzed user data is then compared with previously collected professional match data. Based on the results of this comparison, the machine learning model generates optimal tactics. The generated tactical suggestions are notified to the user via their device in the form of specific positioning and ball trajectory. For example, the app's notification function could display a message such as, "Try a backhand cross-court shot."

[1093] VR Simulation

[1094] Based on the proposed tactics, the user puts on VR goggles (e.g., Oculus Rift) and launches a dedicated VR application. The device then provides the user with tactical footage generated using a VR system based on Unity or Unreal Engine. The user then practices the presented tactics in the virtual environment, experiencing realistic movements in the process. Performance data from the simulation is then uploaded back to a cloud server for further analysis.

[1095] Specific examples

[1096] For example, suppose a user records their tennis practice and uploads the data to a cloud server. When the server analyzes the received data, it finds that the user makes many mistakes with backhand shots. The server then analyzes the movements of professional players in similar situations based on professional match data. It then finds that a cross-court shot is effective when taking a backhand shot. Based on this information, the server suggests to the user, "Try a cross-court shot from your backhand in your next match." The user then puts on VR goggles and simulates the suggested tactic in a virtual environment. Further data from this simulation is collected, and the next suggestion becomes more accurate.

[1097] Prompt Sentence Examples

[1098] Below are some example prompts to input to the generative AI model:

[1099] "Explain how to analyze a user's recorded tennis practice video and suggest improvements for the backhand shot."

[1100] "Please explain the process of proposing optimal tactics to users based on data on professional players' backhand shots."

[1101] "Please explain in detail the specific steps to use VR goggles to simulate tennis tactics."

[1102] This allows the present invention to help players improve their practical skills and enable them to master optimal tactics. By precisely comparing professional data with user data and proposing tactics suited to each individual's playing style, efficient and effective training can be achieved.

[1103] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1104] Step 1:

[1105] A user sets up a camera to record a match or practice session and presses the record button. This causes the camera to capture video and generate recorded data. The recorded data is temporarily stored in the device's internal storage. The input is video data, and the output is a recorded file stored in the internal storage.

[1106] Step 2:

[1107] When recording is finished, the device connects to the Internet using the Wi-Fi module and uploads the recorded data to the cloud server. Specifically, the recorded file is sent to the cloud server via an HTTP POST request. The input is the recorded file in the internal storage, and the output is the recorded data saved on the cloud server.

[1108] Step 3:

[1109] The server periodically runs a Python script to collect professional match data from the Internet, for example, using BeautifulSoup and Selenium to retrieve match footage and statistics from sports sites. The input is the URL of the sports site, and the output is the collected professional match data.

[1110] Step 4:

[1111] The server uses OpenCV to analyze the professional match data collected. Each frame of the video is analyzed to identify the player's position information and the ball's trajectory. The input is the professional match data, and the output is the analyzed position information and ball's trajectory data. The analysis results are stored in a MySQL database.

[1112] Step 5:

[1113] The server receives the recorded data uploaded by the user and analyzes it using a video analysis engine (OpenCV) and a machine learning model (TensorFlow). The server analyzes the user's play data and identifies the player's movements and success rate. The input is the user's recorded data, and the output is the analyzed play data.

[1114] Step 6:

[1115] The server compares the analyzed user data with professional match data. A machine learning model generates optimal tactics based on this data. For example, if the success rate of backhand shots is low, effective tactics and techniques are extracted from a set of professional data. The input is the user data and the professional data, and the output is an optimal tactical proposal.

[1116] Step 7:

[1117] The generated tactical proposal is notified to the user via the device. Specifically, the details of the tactic (e.g., attempt a cross-court shot from the backhand) are communicated to the user using app notifications or email notifications. The input is the generated tactical proposal, and the output is the notification sent to the user.

[1118] Step 8:

[1119] The user puts on VR goggles based on the proposed tactics and launches a dedicated VR application. The device displays tactical footage generated using Unity or Unreal Engine in real time. The input is the tactical proposal and VR system components, and the output is a tactical simulation in a virtual environment.

[1120] Step 9:

[1121] During the simulation, the user's movements and performance data are uploaded from the device to the cloud server. The server receives this data and performs further analysis to improve the accuracy of the next tactical proposal. The input is the simulation data, and the output is the results of further analysis.

[1122] (Application example 1)

[1123] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1124] Conventional sports and work training systems have had difficulty proposing optimal tactics and work procedures based on the user's individual characteristics and movement data. Furthermore, they lacked the means to actually experience the proposed tactics and work procedures and obtain feedback. As a result, it was difficult for users to receive immediate and accurate feedback on their movements, hindering efficient training and work optimization.

[1125] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1126] In this invention, the server includes a camera means for recording matches and practice scenes, a means for uploading the recorded data to a cloud server, a means for collecting professional match data and analyzing each scene from the perspective of positioning and course, a means for receiving a user's play data, comparing it with the professional match data, and proposing optimal tactics based on a machine learning model, a means for actually simulating the proposed tactics using a VR system, a means for recording factory work and analyzing the data to learn and propose efficient movements, and a means for confirming and simulating the work content using smart glasses. This makes it possible to propose optimal tactics and work procedures based on the user's individual data, experience simulations on the spot, and obtain feedback.

[1127] The "camera means" is a device that records scenes of matches and practices, as well as work in the factory.

[1128] A "cloud server" is a system that stores recorded data in a remote location via the Internet and centrally manages the data for analysis.

[1129] "Professional match data" means video footage of matches or activities and related analytical data produced by experts or skilled personnel.

[1130] "Means of analysis from the perspective of positioning and course" refers to technology for analyzing the positioning and direction / flow of movement at specific scenes in a video.

[1131] "User play data" refers to video data of a user's own matches or activities that the user has recorded.

[1132] A "machine learning model" is an algorithm or model that allows a computer to learn from data and make predictions and decisions.

[1133] "Means for suggesting optimal tactics" refers to a method for presenting the most effective tactics and work procedures for the user based on analyzed data.

[1134] A "VR system" is a system that uses virtual reality technology to allow users to experience proposed tactics and work procedures in a virtual environment.

[1135] "Means for recording factory work" refers to devices and technologies that record work procedures and actions within a factory as video.

[1136] The "means for learning and suggesting efficient movements" is a system that analyzes recorded work data and recommends movements that will improve work efficiency.

[1137] "Smart glasses" are wearable devices that can display information in the user's field of vision.

[1138] "Simulation means" refers to technology that provides a virtual environment in which proposed tactics and procedures can be experienced.

[1139] This invention is a system that allows users to record matches, training sessions, or factory work, and analyze the data to learn efficiency and tactics. A specific example of the system is described below.

[1140] First, the user uses a camera to record a match, practice session, or factory work. Once recording is complete, the device automatically uploads the recorded data to a cloud server. This process can be performed using a commercially available video camera or wearable camera. The recorded data is temporarily stored on the device and then transferred via the Internet to a cloud server. Cloud services such as Amazon Web Services (AWS) and Google Cloud Platform (GCP) can be used as the cloud server.

[1141] The cloud server periodically collects and analyzes professional match and work data. This process involves using scripts to automatically collect the necessary data from the Internet and storing the results in a database. The analysis can be performed using OpenCV or TensorFlow as a video analysis engine. The collected data also includes information to evaluate positioning, course, and work efficiency.

[1142] When a user uploads recorded data to a cloud server, the server receives the data and analyzes it using a video analysis engine. The analysis results are compared with professional data, and optimal tactics and work procedures are generated based on a machine learning model. This machine learning model is built using Keras and TensorFlow. The generated tactics and work procedures are then suggested to the user as specific positioning and actions. These suggestions are then notified to the user via their device.

[1143] Additionally, users can wear a VR system or smart glasses to simulate proposed tactics and procedures in a virtual environment. For example, Oculus Rift or HTC Vive can be used as VR goggles, while Microsoft HoloLens or Google Glass are suitable for smart glasses. The VR system provides users with a realistic experience, allowing them to practice the proposed movements and positions. The data from this simulation is also uploaded back to the cloud server for further analysis, improving the accuracy of future proposals.

[1144] As a concrete example, let's say work in a factory is recorded and the data is uploaded to a cloud server. When the server analyzes the received data, it finds that the worker is inefficient in a particular procedure. Next, the server analyzes the movements of professionals in similar situations based on the work data of the professionals. As a result, it finds that efficiency can be improved by changing a particular procedure. Based on this information, the server suggests to the user, "Next time, try assembling from the right side." The user can then put on smart glasses and simulate the suggested procedure in a virtual environment. Further data from this simulation is collected, and the next suggestion will become more accurate.

[1145] An example of a prompt is:

[1146] "Develop an application that records video of a robot's assembly work and suggests optimal movement patterns. This application will cover everything from recording the video to uploading it to the cloud, analyzing the data, and simulating tactics. It should also include a function that allows users to check and simulate the suggestions using smart glasses."

[1147] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1148] Step 1:

[1149] A user uses a camera to record a game, practice, or factory work. The user presses the record button on the camera to capture the video. This video data is temporarily saved on the device. After recording is complete, the device automatically uploads the recorded data to a cloud server. The input is the recorded video data, and the output is the data uploaded to the cloud server.

[1150] Step 2:

[1151] The server receives video data uploaded to the cloud. This data includes footage of games, practice sessions, and factory work recorded by users. The server receives and stores this data. The input is the video data uploaded by the user, and the output is the data stored in the server.

[1152] Step 3:

[1153] The server periodically collects professional match and performance data. The method used here is to collect data using internet scripts and APIs. This allows us to obtain information such as positioning and movement patterns. The input is the professional data collected via the internet, and the output is data stored in a database for analysis.

[1154] Step 4:

[1155] The server uses a video analysis engine to analyze the collected professional match and work data. Software such as OpenCV and TensorFlow are used. The analyzed data is broken down into individual frames, and positional information and movement patterns are extracted. The input is the collected professional data, and the output is analyzed positioning and movement pattern data.

[1156] Step 5:

[1157] The server uses a video analysis engine to analyze the play data uploaded by the user and compares it with professional match data and work data. Based on a machine learning model (using Keras and TensorFlow), it generates the optimal tactics and work procedures for the user's actions. The input is the user's play data and professional data, and the output is the optimal tactics and work procedures suggested to the user.

[1158] Step 6:

[1159] The device notifies the user of the generated optimal tactics and work procedures. Notification methods include a smartphone app and a notification system. At this point, the user can confirm the proposal and then perform an actual simulation in the next step. The input is the proposal information from the server, and the output is a notification to the user.

[1160] Step 7:

[1161] Users wear a VR system or smart glasses and simulate proposed tactics and work procedures in a virtual environment. VR systems such as Oculus Rift and HTC Vive are used, while smart glasses such as Microsoft HoloLens and Google Glass are used. Data from the simulation is also uploaded to a server for further analysis. The input is the proposed tactics and work procedures, and the output is the data from the simulation.

[1162] Step 8:

[1163] The server receives the data from the simulation and performs further analysis. This data is used to improve the accuracy of the next proposal. A video analysis engine and machine learning models are used for data analysis. The input is the data from the simulation, and the output is an updated proposal.

[1164] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1165] The present invention relates to a system that allows users to record matches and practice sessions, analyze the data, and learn tactics, and further combines it with an emotion engine that recognizes the user's emotional state and adjusts tactical suggestions accordingly. A specific example of this system is shown below.

[1166] Basic configuration

[1167] The system consists of the following main components:

[1168] 1. Camera means: A device that allows users to record games and practice sessions.

[1169] 2. Uploading recorded data: A function that allows the device to upload recorded data to the cloud server.

[1170] 3. Professional match data collection means: The server collects professional match data from the Internet.

[1171] 4. Data analysis means: The server has the ability to analyze professional match data and user play data.

[1172] 5. Tactical suggestion means: A function in which the server suggests optimal tactics to the user based on the analysis results.

[1173] 6. VR System: A virtual reality environment for practical simulation of the proposed tactics.

[1174] 7. Emotion Engine: The ability to recognize user emotions and tailor suggestions accordingly.

[1175] Program processing

[1176] First, the user sets up the camera to record a match or practice session, then presses the record button to capture the footage. The recorded data is temporarily saved on the device and uploaded to the cloud server after recording is complete.

[1177] The server periodically collects professional match data from the Internet, analyzes each frame with a video analysis engine, and extracts player movements, positioning, and ball trajectory, then stores this data in a database.

[1178] The server receives the recorded data uploaded by the user and analyzes it using a video analysis engine. The results of this analysis are compared with professional match data and the optimal tactics are suggested to the user based on a machine learning model.

[1179] The proposed tactics are notified to the user via their device, and the user receives specific advice through the application. At this time, the emotion engine identifies the user's emotional state from their facial expressions and tone of voice, and adjusts the suggestions based on this information. For example, if the user is nervous, the tactical suggestions will also include mental advice.

[1180] The user puts on the VR goggles and launches the application. The device plays back the tactical footage generated using the VR system, providing the user with a realistic match experience. Data is also collected during the simulation and uploaded back to the cloud server for further analysis. This data is used to improve the accuracy of future tactical suggestions.

[1181] Specific examples

[1182] For example, suppose a user records a basketball practice and uploads the data to a cloud server. When the server analyzes the received data, it finds that the user has a low success rate in jump shots. After comparing the data with that of professional players, it concludes that the user needs to improve their posture and positioning when taking a jump shot. Based on this information, the server suggests to the user to "pay attention to the position of their feet and the rotation of their shoulders when taking a jump shot."

[1183] Furthermore, if the emotion engine detects nervousness from the user's facial expression, it will add mental advice such as "Relax and shoot." The user then puts on VR goggles and simulates the proposed tactic in a virtual environment. The simulation results are then uploaded back to the cloud server for further analysis.

[1184] This allows the present invention to support more personalized tactical suggestions and practical skill improvement, taking into account the user's emotional state. By precisely matching professional data, user data, and emotional data, the present invention can suggest tactics suited to each individual's playing style, enabling efficient and effective training.

[1185] The processing flow will be explained below.

[1186] Step 1:

[1187] Users set up a camera to record their matches or practice sessions. They turn on the camera and press the record start button to capture footage. The camera records the user's movements in high resolution.

[1188] Step 2:

[1189] The device temporarily stores the video data recorded by the camera in local memory. When recording is finished, the user presses the stop recording button to end the recording.

[1190] Step 3:

[1191] The device uploads the temporarily saved recording data to the cloud server, where it is divided into segments and uploaded to the cloud storage sequentially.

[1192] Step 4:

[1193] The server periodically runs a script that collects professional match data from the internet, and the collected data is obtained from official websites and streaming services using APIs.

[1194] Step 5:

[1195] The server passes professional match data to a video analysis engine, which analyzes each frame to extract player movements, positioning, and ball trajectory, which are then stored in a database.

[1196] Step 6:

[1197] The server receives the user's recorded data and analyzes it using the same video analysis engine. As a result, the user's movements and the course of the ball are extracted.

[1198] Step 7:

[1199] The server compares professional match data with the user's playing data, and based on machine learning models, generates optimal tactics for the user, providing specific instructions on positioning and ball trajectory.

[1200] Step 8:

[1201] The server stores the generated tactical suggestions in a database and notifies the device, which may include specific advice such as "try a cross shot from your backhand."

[1202] Step 9:

[1203] The device receives the tactical proposal and activates the emotion engine, which analyzes the user's facial expressions and tone of voice to identify their emotional state.

[1204] Step 10:

[1205] The emotion engine adjusts tactical suggestions based on detected emotional data. For example, if the user is nervous, the suggestion will include mental advice.

[1206] Step 11:

[1207] The user puts on the VR goggles and launches the application based on the tactical proposal. The user then practices the proposed tactical proposal in the VR simulation.

[1208] Step 12:

[1209] The device uses a VR system to provide users with tactical footage that responds to the user's movements in real time, providing an experience similar to that of a real match.

[1210] Step 13:

[1211] The user finishes the simulation and checks the feedback obtained. The simulation data is then uploaded to the cloud server again.

[1212] Step 14:

[1213] The server receives the simulation data and performs further analysis, which is then stored in a database and used to improve the accuracy of the next tactical proposal.

[1214] This allows for personalized tactical suggestions that take into account the user's emotional state, resulting in efficient and effective training.

[1215] Example 2

[1216] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1217] Conventional sports analysis systems analyze recorded data and propose tactics without taking into account the user's emotional state, which means they are unable to propose tactics that are appropriate for the user's mental state. Additionally, there is a lack of an iterative analysis system to improve the accuracy of tactical proposals.

[1218] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1219] In this invention, the server includes a recording device for recording game and practice scenes, a means for uploading the recorded data to the data server, a means for collecting expert game data and analyzing each scene from the perspective of position information and movement, a means for receiving user movement data, comparing it with the expert game data, and proposing optimal tactics based on a machine learning algorithm, a means for actually simulating the proposed tactics using a virtual reality device, and an emotion analysis means for recognizing the user's emotional state and adjusting the proposed content. This enables individual tactical proposals suited to the user's emotional state. Furthermore, by reanalyzing the simulation data, the accuracy of the tactical proposals can be continuously improved.

[1220] "Recording equipment means" refers to a device for recording footage of matches, practice sessions, etc.

[1221] "Data server" refers to a server that stores recorded data and analysis results and exchanges data with other devices.

[1222] "Expert match data" refers to records of matches and practice sessions of professional or specialized athletes, and is data used to analyze their movements and tactics.

[1223] "Means for analyzing from the perspective of positional information and movements" refers to technology for analyzing recorded data and collected data to extract and evaluate players' positional information and movements.

[1224] "Motion data" refers to data that quantifies or quantifies the movements and actions of a user while playing.

[1225] "Machine learning algorithms" refers to a collection of computer programs and mathematical models for analyzing data and creating predictive models.

[1226] "Means for suggesting tactics" refers to a system that suggests optimal tactics and moves to users based on the analysis results.

[1227] "Virtual reality device" refers to a device or system that allows a user to simulate an experience in a virtual environment.

[1228] "Emotion analysis means" refers to technology that recognizes and analyzes a user's emotional state from their facial expressions, tone of voice, and manner of operation.

[1229] The present invention is a system that allows users to record sports matches and practice sessions, analyze the data, and learn tactics. It also incorporates an emotion engine that recognizes the user's emotional state and adjusts tactical suggestions accordingly.

[1230] System configuration

[1231] The system includes the following main components and specific hardware and software:

[1232] 1. Recording equipment means:

[1233] This is a device that allows users to record matches and practice sessions. For example, it uses a smartphone camera or a dedicated recording camera.

[1234] 2. How to upload recording data:

[1235] This function allows the device to upload recorded data to a cloud server. The data is stored in cloud storage, such as AWS S3.

[1236] 3. Expert match data collection methods:

[1237] The server collects expert match data from the Internet. The collected data is obtained using, for example, a web scraping tool (e.g., BeautifulSoup) and analyzed by a video analysis engine (e.g., OpenCV).

[1238] 4. Data analysis methods:

[1239] The server receives the video data uploaded by users and analyzes it with a video analysis engine (e.g., OpenCV). The analysis results are then compared with expert match data using machine learning algorithms (e.g., TensorFlow).

[1240] 5. Tactical Proposal Methods:

[1241] This function allows the server to propose optimal strategies to users based on the analysis results. The proposals are saved in a database and sent to the user via their device.

[1242] 6. Virtual Reality Devices:

[1243] It is a device that allows users to simulate proposed tactics in a virtual environment, for example, using VR goggles such as Oculus Rift.

[1244] 7. Emotion analysis means:

[1245] The device analyzes the user's facial expressions and tone of voice and uses an emotion engine (e.g., Microsoft Azure Emotion API) to identify their emotional state, and then tailors tactical suggestions based on this information.

[1246] Example of operation

[1247] A user records their basketball practice and uploads the data to a cloud server. The server analyzes the data and finds that the user's jump shot success rate is low. After comparing it with expert game data, it is determined that the user's posture and positioning during jump shots need to be improved. Based on this, the server suggests, "Pay attention to the position of your feet and shoulder rotation when taking a jump shot."

[1248] Furthermore, if the emotion analysis means detects nervousness from the user's facial expression, the system adds mental advice such as "Relax and shoot." The user then puts on the VR goggles and simulates the proposed tactics in a virtual environment. The results of this simulation are uploaded back to the server for further analysis and to improve the accuracy of the tactical suggestions.

[1249] Prompt Sentence Examples

[1250] "We recorded basketball practice and uploaded the data to a cloud server. The results showed that the jump shot success rate was low. We compared it with expert player data and would like to know what improvements can be made. Also, please provide any mental advice that would be useful if the user is nervous."

[1251] The present invention makes it possible to propose individual tactics that take into account the user's emotional state, and furthermore, the accuracy of the tactics proposals is continuously improved by reanalyzing the simulation data.

[1252] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1253] Step 1:

[1254] Start and stop recording

[1255] The user sets up a camera and presses the record button in the application to record a match or practice. The scene to be recorded is the input, and the recorded data is generated as the output.

[1256] When you have finished recording, press the stop recording button to end the recording. Specifically, you can perform the recording operation within the smartphone or dedicated camera app.

[1257] Step 2:

[1258] Uploading recording data

[1259] The device uploads the recorded data to the data server. The input is the recorded data stored in the device, and the output is the data safely stored in the cloud.

[1260] The data is encrypted and transferred securely, and once the recording is complete, the app automatically uploads the data to a cloud storage service such as AWS S3.

[1261] Step 3:

[1262] Expert match data collection

[1263] The server collects expert match data from the Internet. The input is match data on the Internet, and the output is match data stored in a database in the server.

[1264] Analyze the video using a video analysis engine (e.g., OpenCV) to extract location information and behavior. Specifically, use a web scraping tool (e.g., BeautifulSoup) to obtain and analyze the data.

[1265] Step 4:

[1266] Analysis of recorded data

[1267] The server receives the recorded data uploaded by the user. The input is the recorded data uploaded to the cloud, and the output is the analyzed play data.

[1268] A video analysis engine (e.g., OpenCV) analyzes each frame and extracts information such as player movements, positioning, and ball trajectory. Specifically, the server analyzes the recorded data using a machine learning model (e.g., TensorFlow).

[1269] Step 5:

[1270] Generate tactical suggestions

[1271] The server proposes optimal tactics based on the analysis results. The input is the analysis results, and the output is tactical proposals to the user.

[1272] The proposed content is stored in a database and notified to the user via the device. Specifically, the server uses a machine learning algorithm to calculate and propose tactics based on the analyzed data.

[1273] Step 6:

[1274] Emotion recognition and tactical suggestion adjustment

[1275] The device analyzes the user's facial expressions and tone of voice to identify their emotional state. The input is the user's facial and voice data, and the output is tailored tactical suggestions.

[1276] The app uses an emotion engine (e.g., Microsoft Azure Emotion API) to analyze the user's emotional state and adjust the suggestions accordingly. Specifically, the app uses the camera and microphone to capture and analyze the user's emotional data.

[1277] Step 7:

[1278] VR Simulation

[1279] The user puts on the VR goggles and launches the application to simulate tactics. The input is the tactical proposal data, and the output is the simulation results.

[1280] The device plays the generated tactical footage in a VR environment, providing the user with a realistic match experience. Specifically, the user puts on VR goggles such as Oculus Rift and launches the app.

[1281] Step 8:

[1282] Re-uploading and analyzing simulation data

[1283] The terminal uploads the data collected during the simulation back to the data server. The input is the simulation data, and the output is the analyzable data on the server.

[1284] The server performs the analysis again and stores the analysis results in a database. Specifically, once the simulation is complete, the application automatically uploads the data to the cloud for further analysis.

[1285] The above steps enable individual tactical suggestions and continuous analysis based on the user's emotional state.

[1286] (Application example 2)

[1287] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1288] The objective of this invention is to propose driving tactics that take into account not only the analysis of game and practice scenes, but also the analysis of vehicle driving conditions and the driver's emotional state. Conventional systems have difficulty integrating these complex elements to analyze and propose driving tactics, and there is a lack of technology to improve driver safety and efficient driving.

[1289] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1290] In this invention, the server includes a camera means for recording game and practice scenes, a means for uploading the recorded data to a remote server, a means for collecting professional game data and analyzing each scene in terms of positioning and course, a means for receiving user play data and comparing it with the professional game data to propose optimal tactics based on a machine learning model, a means for actually simulating the proposed tactics using a virtual reality system, a means for collecting vehicle driving situation data using a camera and sensor devices, a means for analyzing the driving data and generating optimal driving tactics, and a means for recognizing the driver's emotional state and adjusting the content of the tactical proposal. This makes it possible to propose safe and effective driving tactics to the driver through detailed data collection using the camera and sensor devices, and analysis and emotion recognition based on the data.

[1291] "Photography equipment" refers to equipment for recording video and images of games, practice scenes, vehicle driving conditions, and the like.

[1292] A "remote server" is a remotely located computer system for storing, analyzing, and providing data.

[1293] "Professional match data" refers to data collected from matches involving professional players, and includes information on positioning, movements, strategies, and the like.

[1294] A "machine learning model" is an algorithm or system that is trained using data to automatically learn and perform specific tasks.

[1295] "Virtual reality system" is a general term for devices and software that allow users to experience a virtual environment visually and with other senses.

[1296] A "sensor device" is a device that detects the driving conditions of a vehicle and the surrounding environment and collects the data.

[1297] "Driving data" refers to information about the driving conditions of a vehicle, including detailed data such as speed, braking, and steering.

[1298] "Operation tactics" is a general term for plans and methods for achieving safe and efficient vehicle operation.

[1299] "Emotional state" refers to the psychological state or mood of the driver or user, and includes emotions such as tension, relaxation, and fatigue.

[1300] The present invention relates to a system that records and analyzes game and practice scenes, and proposes tactics that take into account the driver's emotional state. It also has the function of analyzing the vehicle's driving situation and proposing driving tactics based on the driver's emotional state. Specific embodiments are described below.

[1301] Program Overview

[1302] The server includes the following means: a camera means for recording matches and practice scenes, a means for uploading the recorded data to a remote server, a means for collecting professional match data and analyzing each scene in terms of positioning and course, a means for receiving the user's play data, comparing it with the professional match data, and proposing optimal tactics based on a machine learning model, a means for actually simulating the proposed tactics using a virtual reality system, a means for collecting vehicle driving situation data using a camera device and sensor devices, a means for analyzing the driving data and generating optimal driving tactics, and a means for recognizing the driver's emotional state and adjusting the content of the proposed tactics. This makes it possible to collect detailed data using the camera device and sensor devices, and then propose safe and effective driving tactics to the driver through analysis and emotion recognition based on the collected data.

[1303] Hardware and Software Use

[1304] Camera means: The camera uses NVIDIA's Jetson AGX Xavier development kit.

[1305] Cloud servers: AWS or Google Cloud are used for data storage and analysis.

[1306] VPN system: Oculus Rift or HTC Vive are used for virtual reality experiences.

[1307] Sensor devices: LiDAR and various sensors are used to collect data on vehicle driving conditions.

[1308] Emotion recognition: Using software such as OpenFace and Google Speech-to-Text API.

[1309] Data analysis: TensorFlow and PyTorch are used to analyze data and implement machine learning models.

[1310] Program processing

[1311] First, the user uses the camera means to record the game or practice scene, and saves the recorded data in the terminal.

[1312] The device automatically uploads the recorded data to a cloud server, which collects professional match data from the internet and analyzes it using a video analysis engine.

[1313] It uses cameras and sensors to collect data on vehicle driving conditions, then uses AWS Lambda and Google Cloud Functions to analyze the data and generate safe and efficient driving tactics.

[1314] The emotion recognition engine recognizes the driver's emotional state from their facial expressions and tone of voice, and adjusts the tactical suggestions based on this information.

[1315] Specific examples

[1316] For example, if a driver is tired after a long drive, the emotion engine will recognize this state and suggest that the driver take a rest at the next rest area. This suggestion can also be experienced visually through the virtual reality system.

[1317] Prompt Sentence Examples

[1318] Please provide the user's camera footage and driving data to develop a system that performs driving analysis and emotion recognition.

[1319] Include the following elements:

[1320] 1. Recording devices and their data upload methods

[1321] 2. Professional driving data collection methods

[1322] 3. Data analysis approach

[1323] 4. How to propose operational tactics

[1324] 5. Emotion Recognition Algorithms

[1325] 6. How to run a VR simulation

[1326] This invention integrates driving and emotion recognition technology to improve driver safety and efficiency by providing improved driving instructions in real time.

[1327] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1328] Step 1:

[1329] A user uses a camera to capture video data of a match or practice session. The video data is input and temporarily saved.

[1330] Step 2:

[1331] The terminal automatically uploads the temporarily stored video data to a remote server. The input is the temporarily stored video data, and the output is the video data on the remote server.

[1332] Step 3:

[1333] The server collects professional match data from the Internet and uses a video analysis engine (e.g., TensorFlow or PyTorch) to analyze each scene in terms of positioning and course. The input is the professional match data, and the output is the analyzed data.

[1334] Step 4:

[1335] The server receives the user's play data, compares it with the professional match data, and proposes optimal tactics based on the machine learning model. The input is the user's play data and the analyzed professional match data, and the output is a tactical proposal.

[1336] Step 5:

[1337] The user uses the virtual reality system to actually simulate the tactics proposed by the server. The input is the tactical proposal from the server, and the output is the simulation data.

[1338] Step 6:

[1339] The user acquires driving situation data using the camera and sensor devices mounted on the vehicle. The input is the driving situation data, and the output is the temporarily saved data.

[1340] Step 7:

[1341] The terminal uploads the temporarily saved driving situation data to the cloud server. The input is the temporarily saved driving situation data, and the output is the data on the cloud server.

[1342] Step 8:

[1343] The server analyzes the driving situation data and the user's emotional state data and generates optimal driving tactics using a machine learning model (e.g., TensorFlow or PyTorch). The input is the driving situation data and the emotional state data, and the output is a proposed driving tactic.

[1344] Step 9:

[1345] The server uses an emotion recognition engine (e.g., OpenFace or Google Speech-to-Text API) to analyze the user's facial expressions and tone of voice to recognize their emotional state. The input is the user's facial expression data and voice data, and the output is emotional state data.

[1346] Step 10:

[1347] The server provides the generated driving tactics to the user and continues driving based on the proposed tactics. The input is the proposed driving tactics and emotional state data, and the output is safe and efficient driving.

[1348] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1349] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1350] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1351] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1352] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1353] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1354] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1355] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1356] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1357] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1358] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1359] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1360] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1361] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1362] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1363] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1364] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1365] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1366] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1367] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1368] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1369] The following is further disclosed regarding the above embodiment.

[1370] (Claim 1)

[1371] Camera means for recording matches and practice sessions;

[1372] A means for uploading recorded data to a cloud server;

[1373] We collect data from professional matches and analyze each scene from the perspective of positioning and course.

[1374] A method to receive user play data, compare it with professional match data, and propose optimal tactics based on machine learning models.

[1375] A means to actually simulate the proposed tactics using a VR system, and

[1376] A system including:

[1377] (Claim 2)

[1378] 10. The system of claim 1, wherein the cloud server comprises means for receiving data during the user's simulation and for further analysis.

[1379] (Claim 3)

[1380] 2. The system according to claim 1, further comprising means for automatically collecting professional match data and storing the analysis results in a database.

[1381] "Example 1"

[1382] (Claim 1)

[1383] A video recording device for recording matches and practice sessions,

[1384] A communication means for uploading the recording data to a cloud server;

[1385] A data analysis tool for collecting professional match data and analyzing each scene from the perspective of position information and progression path.

[1386] a tactic generation means for receiving the user's play data, comparing it with professional match data, and proposing optimal tactics based on a machine learning model;

[1387] a virtual reality display means for actually simulating the proposed tactic using a virtual reality system;

[1388] A system including:

[1389] (Claim 2)

[1390] 2. The system according to claim 1, wherein the cloud server comprises an analysis means for receiving data during the user's simulation and for further analysis.

[1391] (Claim 3)

[1392] 2. The system according to claim 1, further comprising a storage means for automatically collecting professional match data and storing the analysis results in a database.

[1393] "Application Example 1"

[1394] (Claim 1)

[1395] Camera means for recording matches and practice sessions;

[1396] A means for uploading recorded data to a cloud server;

[1397] We collect data from professional matches and analyze each scene from the perspective of positioning and course.

[1398] A method to receive user play data, compare it with professional match data, and propose optimal tactics based on machine learning models.

[1399] A means to actually simulate the proposed tactics using a VR system, and

[1400] It records factory work, analyzes the data, and learns and proposes efficient movements.

[1401] A means for checking and simulating the work content using smart glasses;

[1402] A system including:

[1403] (Claim 2)

[1404] 10. The system of claim 1, wherein the cloud server comprises means for receiving data during the user's simulation and for further analysis.

[1405] (Claim 3)

[1406] 2. The system according to claim 1, further comprising means for automatically collecting professional match data and storing the analysis results in a database.

[1407] "Example 2: Combining Emotion Engines"

[1408] (Claim 1)

[1409] a recording device for recording matches and practice scenes;

[1410] A means for uploading the recording data to a data server;

[1411] A method for collecting expert match data and analyzing each scene from the perspective of positional information and movement,

[1412] A means for receiving user action data, comparing it with expert match data, and proposing optimal tactics based on machine learning algorithms;

[1413] a means for actually simulating the proposed tactic using a virtual reality device;

[1414] an emotion analysis means for recognizing the user's emotional state and adjusting the suggestions;

[1415] A system including:

[1416] (Claim 2)

[1417] 10. The system of claim 1, wherein the data server comprises means for receiving data during a user's simulation and for further analysis.

[1418] (Claim 3)

[1419] 2. The system according to claim 1, further comprising means for automatically collecting expert match data and storing the analysis results in an information device.

[1420] "Application example 2 when combining emotion engines"

[1421] (Claim 1)

[1422] a camera means for recording the game or practice;

[1423] means for uploading the recording data to a remote server;

[1424] We collect data from professional matches and analyze each scene from the perspective of positioning and course.

[1425] A method for receiving user play data, comparing it with professional match data, and proposing optimal tactics based on machine learning models;

[1426] a means for actually simulating the proposed tactic using a virtual reality system; and

[1427] a means for collecting vehicle driving situation data using an imaging device and a sensor device;

[1428] A means for analyzing driving data and generating optimal driving tactics;

[1429] a means for recognizing the driver's emotional state and adjusting tactical suggestions;

[1430] A system including:

[1431] (Claim 2)

[1432] 10. The system of claim 1, wherein the remote server comprises means for receiving data during the user's simulation and for further analysis.

[1433] (Claim 3)

[1434] 2. The system according to claim 1, further comprising means for automatically collecting professional match data and storing the analysis results in a database. [Explanation of symbols]

[1435] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. Camera means for recording matches and practice sessions; A means for uploading recorded data to a cloud server; We collect data from professional matches and analyze each scene from the perspective of positioning and course. A method to receive user play data, compare it with professional match data, and propose optimal tactics based on machine learning models. A means to actually simulate the proposed tactics using a VR system, and A system including:

2. The system of claim 1 , wherein the cloud server comprises means for receiving data during the user's simulation and for further analysis.

3. 2. The system according to claim 1, further comprising means for automatically collecting professional match data and storing the analysis results in a database.

Images