System

Abstract

To provide a system capable of collecting a moving image from a camera installed at a beach, analyzing a wave condition by using a generated AI, and providing the wave condition in real time.SOLUTION: Means for using a generative AI for analyzing a height, a cycle, and a direction of a wave on the basis of the collected moving image data, means for providing the analyzed wave information in real time, means for collecting weather data and past wave condition data, and means for providing predicted wave information to a user; means for using a generative AI for analyzing a height, a cycle, and a direction of a wave on the basis of the collected moving image data, means for providing the analyzed wave information in real time, means for collecting weather data and past wave condition data, and means for providing predicted wave information to a user; The system includes a means using a generation AI for generating a riding form and technical feedback, and a means using a generation AI for recommending an optimal surfboard based on a user's body shape, skill, and wave conditions.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The surfing market is currently in its infancy, and many important elements have not yet been digitized. Wave conditions, in particular, are a crucial factor in surfing, yet information on them is provided primarily by local surf shops, lacking uniformity and accuracy. This makes it difficult for surfers to obtain accurate information and to surf effectively. Furthermore, there is little feedback for surfers to improve their own techniques, hindering individual skill development. Therefore, a system that provides comprehensive support for surfing information and skill development is needed. [Means for solving the problem]

[0005] To address these issues, this invention proposes a system that collects video data from cameras installed on the beach and uses a generative AI to analyze and provide real-time wave conditions. It also provides a generative AI that combines weather data and past wave condition data to predict waves one week in advance. It also includes a generative AI that analyzes surfing videos uploaded by users and generates riding form and technical feedback to help users improve their technique. It also builds a system that helps users select equipment that suits their individual characteristics by using a generative AI that recommends the optimal surfboard based on the user's body type, skill, and wave conditions.

[0006] "Surfing video" refers to video data captured during a surfing activity.

[0007] "Generative AI" refers to an artificial intelligence model that analyzes collected data and automatically generates wave conditions and technical feedback for users.

[0008] "Wave conditions" refers to information about the state of the waves, such as wave height, period, and direction.

[0009] "Weather data" is part of a weather forecast and refers to meteorological information such as wind speed, wind direction, temperature, and air pressure.

[0010] "Beach camera" refers to a camera device that is installed on a surfable beach and collects video data in real time.

[0011] "Real-time" refers to a situation in which information is provided immediately, without delay.

[0012] "Riding form" refers to the body movements and posture while surfing.

[0013] "Feedback" refers to evaluations and advice on a user's surfing skills.

[0014] A "surfboard" refers to a board specifically designed for surfing. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] The present invention relates to a system for providing wave information and supporting users in improving their surfing skills in the surfing market. Specific processes required to implement this system will now be described.

[0037] Overall overview

[0038] This system has the following main functions:

[0039] 1. Beach camera video data collection and analysis

[0040] 2. Providing wave information and wave forecasts for the next week

[0041] 3. User surfing video analysis and feedback

[0042] 4. Surfboard Recommendations

[0043] Beach camera video data collection and analysis

[0044] The server collects video data from cameras installed on each beach. The cameras stream the video in real time, and the video data is stored on the server for analysis.

[0045] Examples:

[0046] The server captures video data from the cameras at Shonan Beach at 30 frames per second and saves the data as a file every five minutes.

[0047] The saved video file is analyzed by the generation AI, and information such as wave height of 1.5m, period of 8 seconds, and direction to the southeast is obtained.

[0048] Providing wave information and one-week wave forecasts

[0049] The server uses AI to analyze wave conditions from the collected video data and provides this information to the user's device in real time. It also combines weather data and past wave condition data to use AI to predict wave conditions for the next week and provides this forecast data to the user.

[0050] Examples:

[0051] The server obtains data such as wind speed and temperature in real time from weather data providers, integrates it with wave data from the past year, and runs it through AI.

[0052] The generating AI analyzes the data and determines that "the predicted wave height at Shonan Beach next week will be between 1.2m and 1.7m," and notifies the user.

[0053] User surfing video analysis and feedback

[0054] When a user uploads a surfing video from their device, the server receives the video and sends it to the generation AI, which analyzes the user's riding form and technique and generates technical feedback, which is displayed on the user's device.

[0055] Examples:

[0056] Users upload their surfing videos to the server via the application.

[0057] The server runs the video through AI, which generates feedback such as "Be conscious of your forward-leaning posture," which is displayed on the user's device.

[0058] Surfboard Recommendations

[0059] Based on information such as the user's body type, skill level, and wave conditions, the server uses generative AI to analyze and recommend the most suitable surfboard. This recommendation is also sent to the user's device.

[0060] Examples:

[0061] Users enter their height, weight and intermediate surfing skills into the app.

[0062] Based on this information, the server recommends a "shortboard, 6.0 feet long" and displays it on the app.

[0063] The above is a concrete example of how to implement the present invention. By linking cameras installed on the beach with AI generation, this system provides detailed and accurate wave information and personalized technical feedback. It also recommends the optimal surfboard based on the user's unique information, allowing surfers to enjoy surfing more comfortably and efficiently.

[0064] The processing flow will be explained below.

[0065] Beach camera video data collection and analysis

[0066] Step 1:

[0067] The server receives video data in real time from cameras installed on the beach, connects to the IP addresses of the cameras, and streams the video using, for example, RTSP (Real Time Streaming Protocol).

[0068] Step 2:

[0069] The server saves the streamed video data as a file at regular intervals, typically as a new file every five minutes.

[0070] Step 3:

[0071] The server inputs the saved video file into the generative AI model and begins analysis, including wave height, period, and direction.

[0072] Step 4:

[0073] The server stores the analysis results of the generative AI model in a database and simultaneously provides them to the user's device in real time.

[0074] Providing wave information and one-week wave forecasts

[0075] Step 1:

[0076] The server uses the weather data provider API to retrieve the latest weather data, including wind speed, wind direction, temperature, and air pressure.

[0077] Step 2:

[0078] The server retrieves data on past wave conditions from a database, usually covering the past year.

[0079] Step 3:

[0080] The server inputs weather data and past wave condition data into the AI ​​generator, which then predicts waves for the next week.

[0081] Step 4:

[0082] The server stores the prediction results from the AI ​​in a database and provides them to the user's device, including the wave height, period, and direction for the next week.

[0083] User surfing video analysis and feedback

[0084] Step 1:

[0085] Users use their devices to upload surfing videos to the application, which then sends the videos to a cloud server.

[0086] Step 2:

[0087] The server receives the uploaded video data and stores it in temporary storage.

[0088] Step 3:

[0089] The server inputs the saved video data into a generation AI that analyzes the user's riding form and technique.

[0090] Step 4:

[0091] The server receives the analysis results from the AI ​​generator and provides them to the user as feedback, such as specific technical guidance such as "Be conscious of your forward-leaning posture."

[0092] Surfboard Recommendations

[0093] Step 1:

[0094] Users enter their body type, skill level, and wave conditions into the application on their device, including their height, weight, and surfing skill level.

[0095] Step 2:

[0096] The server receives the user's input data and sends it to the generation AI, which then uses this data to recommend the best surfboard for them.

[0097] Step 3:

[0098] The server stores the recommendations from the AI ​​in a database and provides them to the user's device. For example, a recommendation might be "shortboard, 6.0 feet long."

[0099] The above is the specific processing flow of the system that provides wave information and improves user skills in the surfing market. This series of steps enables users to obtain more accurate information and appropriate feedback, resulting in an improved surfing experience.

[0100] Example 1

[0101] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0102] In today's surfing world, it is difficult to obtain accurate wave information in real time or to efficiently support the improvement of individual surfing techniques. Furthermore, there are insufficient methods for users to select the most suitable surfing equipment. For this reason, there is a need for specific support to help users enjoy surfing more and improve their techniques.

[0103] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0104] In this invention, the server includes: means for collecting video data from a video collection device; means using a generation AI to analyze wave height, period, and direction based on the collected video data; means for providing the analyzed wave information to a user's device in real time; means for collecting weather data and past wave condition data; means using a generation AI to combine this data to predict waves one week in advance; means for providing the predicted wave information to the user's device; means using a generation AI to analyze surfing videos uploaded by users and generate riding form and technical feedback; means for providing the analysis results to the user's device; means using a generation AI to recommend optimal surfing equipment based on the user's body type, skill, and wave conditions; and means for providing the recommendation results to the user's device. This allows users to obtain detailed and accurate wave information in real time, improving their surfing skills and selecting optimal surfing equipment.

[0105] A "video collection device" is a device that is installed on a beach or other location and captures video data in real time.

[0106] "Video data" refers to video information acquired by a video collection device.

[0107] "Generative AI" is an artificial intelligence model that analyzes and generates specific patterns and information using large amounts of video and other data.

[0108] "Wave information" is information that indicates specific parameters and conditions related to surfing, such as wave height, period, and direction.

[0109] "User device" refers to an electronic device used by a user, such as a smartphone, tablet, or PC.

[0110] "Weather data" refers to information about weather conditions such as wind speed, temperature, and precipitation.

[0111] "Past wave condition data" refers to data such as past wave height, period, and direction.

[0112] "Surfing video" is video data taken by a user while surfing.

[0113] "Riding form" refers to the user's posture and movements while surfing.

[0114] "Technical feedback" is specific instruction or advice aimed at improving surfing technique.

[0115] "Surfing equipment" means equipment or tools used in surfing, such as a surfboard.

[0116] "Recommended results" are recommendations for optimal surfing equipment calculated by the generating AI based on the user's specific conditions.

[0117] This invention relates to a system for providing wave information to the surfing market and supporting users in improving their technique. To implement this system, the following specific hardware and software are used, and how data is processed and calculated will be described.

[0118] Hardware and software used

[0119] 1. Video collection device: Camera equipment installed on the beach. For example, a high-resolution IP camera is used. This allows for real-time streaming video.

[0120] 2. Server: A central system for collecting, storing, and analyzing data. It is equipped with a high-performance server processor and large-capacity storage.

[0121] 3. Generative AI model: Artificial intelligence for analyzing video data and other data, specifically using deep learning frameworks (e.g., TENSORFLOW (registered trademark) and PyTorch).

[0122] 4. User devices: smartphones, tablets, computers, etc., allowing users to receive information in real time and operate and input data through the interface.

[0123] Overview of program processing

[0124] Beach camera video data collection and analysis

[0125] The server collects video data from cameras installed on the beach. The cameras stream in real time, and the video data is stored on the server for analysis. The server acquires video data from the high-resolution IP cameras at 30 frames per second and saves it as an MP4 file every five minutes.

[0126] Providing wave information and one-week wave forecasts

[0127] The server uses a generative AI model to analyze wave conditions from the collected video data and provides this information to the user's device in real time. The server also obtains weather data such as wind speed and temperature from a weather data provider and combines it with past wave condition data to generate a wave forecast for the next week. This forecast data is also analyzed using generative AI and provided to the user.

[0128] User surfing video analysis and feedback

[0129] When a user uploads a surfing video from their device, the server receives the video and sends it to a generative AI model. The generative AI analyzes the user's riding form and technique and generates technical feedback. This feedback is displayed on the user's device. Users upload surfing videos using an application, and the server sends the video to the generative AI, which generates feedback such as "Be conscious of your forward leaning posture."

[0130] Surfboard Recommendations

[0131] Based on information such as the user's body type, skill level, and wave conditions, the server uses generative AI to analyze and recommend the most suitable surfboard. This recommendation result is also provided to the user's device. For example, if a user enters their height of 175 cm, weight of 70 kg, and intermediate skill level into the app, the server will recommend a "shortboard, 6.0 feet long" and display the result on the user's device.

[0132] Examples of specific examples and prompts

[0133] Examples:

[0134] The server collects video data from cameras at Shonan Beach and analyzes it using generative AI to obtain information such as wave height of 1.5m, wave period of 8 seconds, and direction to the southeast.

[0135] The server uses weather data to predict waves for the next week and notifies the user that "wave heights predicted for Shonan Beach next week are between 1.2m and 1.7m."

[0136] Users upload their surfing videos to a server and receive feedback such as "Improve your popping timing."

[0137] Based on the information entered by the user, the server recommends a "6.0 foot shortboard."

[0138] Example prompt sentence:

[0139] "Please provide current and forecast wave height data for Shonan Beach."

[0140] "Analyze my surfing video and give me feedback"

[0141] "Recommend a surfboard that suits your body type and skill level."

[0142] The above is a specific embodiment for carrying out the invention. This system allows users to obtain detailed and accurate wave information in real time, enabling them to improve their surfing skills and select the most suitable surfing equipment.

[0143] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0144] Step 1:

[0145] The server collects video data from video collection devices installed on the beach. The camera devices stream video in real time and send the data to the server. The server saves this data as MP4 files at regular intervals (e.g., every 5 minutes).

[0146] Specific behavior:

[0147] The server receives video data from the Shonan Beach camera in real time at 30 frames per second and saves it as one MP4 file every five minutes.

[0148] Input: Video data (real-time video stream)

[0149] Output: MP4 file (video data saved every 5 minutes)

[0150] Step 2:

[0151] The server inputs the stored video data into a generative AI model to analyze wave height, period, and direction, and the generative AI model uses a deep learning algorithm to extract wave characteristics.

[0152] Specific behavior:

[0153] The server sends the saved MP4 file to the generation AI, obtaining information such as wave height of 1.5m, period of 8 seconds, and direction southeast.

[0154] Input: Saved MP4 file

[0155] Output: Wave characteristics data (wave height, period, direction)

[0156] Step 3:

[0157] The server transmits the analyzed wave information in real time to the user's device, which displays this information.

[0158] Specific behavior:

[0159] The server sends wave characteristic data to the user's smartphone via push notification, displaying "Current wave height: 1.5m, direction: southeast."

[0160] Input: Wave property data

[0161] Output: Push notification (wave height, period, direction information)

[0162] Step 4:

[0163] The server retrieves weather data from a weather data provider. This data includes wind speed, temperature, etc. This is combined with historical wave condition data to generate a week-ahead wave forecast. The data is then analyzed using a generative AI model.

[0164] Specific behavior:

[0165] The server combines wind speed, temperature, and wave data from the past year, and uses generative AI to analyze the predicted wave height at Shonan Beach next week as 1.2m to 1.7m.

[0166] Input: Weather data, historical wave condition data

[0167] Output: Predicted wave information (future wave height, period, direction)

[0168] Step 5:

[0169] Users upload surfing videos from their devices to a server, which then inputs the videos into a generative AI model to analyze riding form and technique, and the generative AI then generates technical feedback.

[0170] Specific behavior:

[0171] Users use the application to upload surfing videos to a server, which then sends the videos to a generating AI, which then displays the videos on the user's device, providing feedback such as "be mindful of your forward leaning posture."

[0172] Input: surfing video

[0173] Output: Technical feedback (specific improvements)

[0174] Step 6:

[0175] The server uses the generative AI model to send the analysis results to the user's device, where technical feedback based on the analysis results is displayed.

[0176] Specific behavior:

[0177] The server sends the analysis results to the user's device as a push notification, displaying feedback such as "Improve your popping timing."

[0178] Input: Analysis results

[0179] Output: Push notification (technical feedback)

[0180] Step 7:

[0181] Based on the user's body type, skill level, and wave conditions, the server uses a generative AI model to recommend the optimal surfing equipment, which is then delivered to the user's device.

[0182] Specific behavior:

[0183] When a user inputs their height of 175 cm, weight of 70 kg, and intermediate skill level, the server uses generative AI to recommend a "shortboard, 6.0 feet long" and displays the results on the user's device.

[0184] Input: User's body type, skill, and wave conditions

[0185] Output: Recommended results (optimal surfing equipment)

[0186] (Application example 1)

[0187] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0188] Monitoring and optimization of manufacturing processes in conventional factories often relied on manual work or partial automation, making it difficult to improve efficiency in real time or provide flexible feedback. This resulted in problems such as the generation of defective products during the manufacturing process and delays in identifying and resolving bottlenecks throughout the line, resulting in a decline in overall productivity. Furthermore, line stoppages due to sudden equipment failures or lack of maintenance also had a negative impact on the production process. A system that could solve these issues and improve the efficiency of the entire manufacturing line was needed.

[0189] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0190] In this invention, the server includes means for collecting video data from cameras installed on the factory production line, means using a generation AI for analyzing the current state of the production process based on the collected video data, means for providing the analyzed process information to an operator's terminal in real time, means for collecting equipment operation data and past production data, means using a generation AI for combining this data to predict the production process one week ahead, means for providing the predicted process information to the operator's terminal, means using a generation AI for analyzing videos of specific processes uploaded by the operator and generating efficiency improvement feedback, means for providing the analysis results to the operator's terminal, means using a generation AI for recommending maintenance based on the equipment's operating status and process data, and means for providing the recommendation results to the operator's terminal.

[0191] This enables real-time monitoring of the operating status of the entire manufacturing line, enabling immediate process improvements and efficiency improvements, early identification of bottlenecks, and the provision of appropriate maintenance forecasts.

[0192] A "production line" refers to a series of processes and machinery that continuously processes and assembles products within a factory.

[0193] A "camera" is a device that captures video or images, and generally includes digital cameras and surveillance cameras.

[0194] "Video data" refers to data that digitally represents images that change continuously over time.

[0195] "Generative AI" refers to artificial intelligence that uses machine learning algorithms to analyze and make predictions from specific data.

[0196] "Process information" refers to data related to the manufacturing process, including production volume, quality, error rate, etc.

[0197] A "terminal" is a device that is connected to the system and exchanges information, including smartphones and computers.

[0198] "Operational data" refers to data related to the operating state of machines and equipment, including operating time, frequency of use, energy consumption, etc.

[0199] "Past manufacturing data" refers to the accumulation of data related to manufacturing processes that has been collected to date.

[0200] "Maintenance" refers to the maintenance and repair work required to keep machines and equipment operating normally.

[0201] A "bottleneck" is the factor that slows down progress in production or a process and reduces overall efficiency.

[0202] "Feedback" is information or instructions that are based on the results or status of a system or process to encourage appropriate improvements or modifications.

[0203] This invention provides a system for monitoring and optimizing a factory production line. The system is composed of the following hardware and software.

[0204] System hardware configuration

[0205] 1. Camera:

[0206] This is a video capture device installed at various points on the production line, allowing real-time video of the manufacturing process to be captured.

[0207] 2. Server:

[0208] It is a central processing unit for storing and analyzing collected video data. The server also provides functions for analyzing various data and using generative AI models.

[0209] 3. Terminal:

[0210] A device such as a smartphone or computer used by an operator to display feedback from the server and analysis results.

[0211] System software configuration

[0212] 1. Generative AI model:

[0213] It is an artificial intelligence algorithm that analyzes video data and various data related to manufacturing processes. Specifically, it uses machine learning models to optimize processes, detect errors, and recommend maintenance.

[0214] 2. Database:

[0215] The collected data is stored and managed using a DBMS such as SQL.

[0216] 3. Front-end application:

[0217] A user interface is provided to display analysis results and feedback to the operator, which is implemented as a smartphone application (ANDROID (registered trademark) / iOS) or a web application.

[0218] Example of operation

[0219] 1. Video Data Collection and Analysis:

[0220] A camera captures the production line and sends the video data to a server. The server inputs this video data into a generative AI model and analyzes the current state of the production process. For example, it can obtain specific analysis results such as "The component placement speed on assembly line 1 is declining."

[0221] 2. Providing process information:

[0222] The analysis results are provided to the operator's terminal in real time. For example, an alert such as "A bottleneck has occurred on the assembly line. Please consider countermeasures" is displayed on the terminal.

[0223] 3. Analysis of operational and historical data:

[0224] The server collects equipment operation data and past production data, and uses a generative AI model to predict the production process one week in advance. Specifically, it provides forecast information such as, "Supply shortages are predicted for next week. Please consider replenishing inventory."

[0225] 4. Feedback Generation:

[0226] When an operator uploads video data of a specific process to the server, the generative AI model analyzes it and provides feedback on how to improve efficiency, such as "Please be careful when installing parts."

[0227] 5. Maintenance Recommendations:

[0228] The server monitors the operating status of the equipment and recommends necessary maintenance. For example, a message such as "An abnormality has been detected in the robot arm on line 4. Inspection is recommended" is displayed on the operator's terminal.

[0229] Examples of prompt statements

[0230] "Analyze the video of the assembly process on production line 1 and identify areas for improvement."

[0231] This system enables real-time monitoring of the operating status of the entire manufacturing line, enabling immediate process improvements and efficiency gains, early identification of bottlenecks, and the provision of appropriate maintenance forecasts.

[0232] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0233] Step 1:

[0234] The server acquires video data in real time from cameras installed on the factory production line. The acquired video data is temporarily stored in the server's storage. The input is video data from the camera, and the output is video data saved in the server's storage. Specifically, the camera continuously captures video and sends the data to the server via an IP network.

[0235] Step 2:

[0236] The server inputs the stored video data into a generative AI model to analyze the current state of the manufacturing process. The generative AI model uses this data to identify problems and areas for improvement on the production line. The input is the temporarily stored video data, and the output is the process information that is the analysis result. Specifically, it uses an image analysis algorithm to detect specific errors and delays.

[0237] Step 3:

[0238] The server notifies the operator's device of the analyzed process information in real time. The input is the analysis results obtained from the generative AI model, and the output is feedback displayed on the operator's device. Specifically, the server uses a notification system to send push notifications to the device.

[0239] Step 4:

[0240] The server collects equipment operation data and past manufacturing data, inputs it into a generative AI model, and predicts future manufacturing processes. The input is equipment operation data and past manufacturing data, and the output is predicted manufacturing process information. Specifically, it retrieves the necessary data from the database via a query and analyzes it using the generative AI model.

[0241] Step 5:

[0242] The server provides predicted manufacturing process information to the operator's device. The input is predicted data from the generative AI model, and the output is predicted information displayed on the operator's device. Specifically, it generates a dashboard that displays the prediction results in a visually easy-to-understand manner.

[0243] Step 6:

[0244] The terminal provides a means for the operator to upload video data of a specific process to the server. The input is the operator's video data, and the output is the uploaded video stored on the server. Specifically, the terminal provides file selection and upload functions from the user interface.

[0245] Step 7:

[0246] The server inputs the uploaded video data of a specific process into a generative AI model to generate efficiency improvement feedback. The input is the uploaded video, and the output is the efficiency improvement feedback. Specifically, the video analysis algorithm is used to identify areas for efficiency improvement and generate feedback in text format.

[0247] Step 8:

[0248] The server provides feedback to the operator's device. The input is feedback data from the generative AI model, and the output is feedback displayed on the operator's device. Specifically, the feedback data is sent to the device in real time and displayed on the user interface.

[0249] Step 9:

[0250] The server monitors the operating status of the equipment and makes necessary maintenance recommendations based on the generative AI model. The input is the equipment's operating data and the output is the maintenance recommendation. Specifically, it collects operating data in real time and automatically generates a maintenance notification if an abnormality is detected.

[0251] Step 10:

[0252] The server provides maintenance recommendations to the operator's device. The input is maintenance recommendation data from the generative AI model, and the output is a maintenance notification displayed on the operator's device. Specifically, the server uses a notification system to push the recommendations to the device.

[0253] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0254] This invention relates to a system that provides wave information and helps users improve their surfing skills in the surfing market, as well as a system that provides more personalized and optimal feedback and recommendations by using user emotion recognition. The specific processing required to implement this system is described below.

[0255] Overall overview

[0256] This system has the following main functions:

[0257] 1. Beach camera video data collection and analysis

[0258] 2. Providing wave information and wave forecasts for the next week

[0259] 3. User surfing video analysis and feedback

[0260] 4. Surfboard Recommendations

[0261] 5. Emotion engine recognizes user emotions and optimizes feedback and recommendations

[0262] Beach camera video data collection and analysis

[0263] The server collects video data from cameras installed on each beach. The cameras stream the video in real time, and the video data is stored on the server for analysis.

[0264] Examples:

[0265] The server captures video data from the cameras at Shonan Beach at 30 frames per second and saves the data as a file every five minutes.

[0266] The saved video file is analyzed by the generation AI, and information such as wave height of 1.5m, period of 8 seconds, and direction to the southeast is obtained.

[0267] Providing wave information and one-week wave forecasts

[0268] The server uses AI to analyze wave conditions from the collected video data and provides this information to the user's device in real time. It also combines weather data and past wave condition data to use AI to predict wave conditions for the next week and provides this forecast data to the user.

[0269] Examples:

[0270] The server obtains data such as wind speed and temperature in real time from weather data providers, integrates it with wave data from the past year, and runs it through AI.

[0271] The generating AI analyzes the data and determines that "the predicted wave height at Shonan Beach next week will be between 1.2m and 1.7m," and notifies the user.

[0272] User surfing video analysis and feedback

[0273] When a user uploads a surfing video from their device, the server receives the video and sends it to the generation AI, which analyzes the user's riding form and technique and generates technical feedback, which is displayed on the user's device.

[0274] Examples:

[0275] Users upload their surfing videos to the server via the application.

[0276] The server runs the video through AI, which generates feedback such as "Be conscious of your forward-leaning posture," which is displayed on the user's device.

[0277] Surfboard Recommendations

[0278] Based on information such as the user's body type, skill level, and wave conditions, the server uses generative AI to analyze and recommend the most suitable surfboard. This recommendation is also sent to the user's device.

[0279] Examples:

[0280] Users enter their height, weight and intermediate surfing skills into the app.

[0281] Based on this information, the server recommends a "shortboard, 6.0 feet long" and displays it on the app.

[0282] Emotion recognition and feedback / recommendation optimization using an emotion engine

[0283] The system uses an emotion engine to specifically recognize the user's emotions and provide feedback and surfboard recommendations based on those emotions.

[0284] Examples:

[0285] When a user uploads a surfing video, the emotion engine analyzes their facial expressions and voice to recognize their emotional state. For example, if the user is dissatisfied, the feedback will include encouragement such as, "Next time, try to focus on your hip movement to improve your form."

[0286] Using an emotion engine, surfboard recommendations are also tailored based on the user's emotions, for example, recommending boards that require more skill if the user is feeling adventurous.

[0287] The above is a concrete example of how to implement the present invention. By linking cameras installed on the beach with generative AI and an emotion engine, this system provides detailed and accurate wave information, individual technical feedback, and recommendations for optimal surfboards based on emotions. This allows surfers to enjoy surfing more comfortably and efficiently.

[0288] The processing flow will be explained below.

[0289] Processing flow of a system that combines emotion engines

[0290] Beach camera video data collection and analysis

[0291] Step 1:

[0292] The server receives video data in real time from cameras installed on the beach, connects to the IP addresses of the cameras, and streams the video using RTSP (Real Time Streaming Protocol).

[0293] Step 2:

[0294] The server saves the streamed video data as a file at regular intervals, typically as a new file every five minutes.

[0295] Step 3:

[0296] The server inputs the saved video files into a generative AI model and analyzes wave conditions such as wave height, period, and direction.

[0297] Step 4:

[0298] The server stores the analysis results of the generative AI model in a database and simultaneously provides them to the user's device in real time.

[0299] Providing wave information and one-week wave forecasts

[0300] Step 1:

[0301] The server uses the weather data provider API to retrieve the latest weather data, including wind speed, wind direction, temperature, and air pressure.

[0302] Step 2:

[0303] The server retrieves historical wave condition data from a database, typically using data from the past year.

[0304] Step 3:

[0305] The server inputs weather data and past wave condition data into the AI ​​generator, which then predicts waves for the next week.

[0306] Step 4:

[0307] The server stores the prediction results from the AI ​​in a database and provides them to the user's device, including the wave height, period, and direction for the next week.

[0308] User surfing video analysis and feedback

[0309] Step 1:

[0310] Users use their devices to upload surfing videos to the application, which then sends the videos to a cloud server.

[0311] Step 2:

[0312] The server receives the uploaded video data and stores it in temporary storage.

[0313] Step 3:

[0314] The server inputs the saved video data into a generation AI that analyzes the user's riding form and technique.

[0315] Step 4:

[0316] The server receives the analysis results from the AI ​​generator and provides them to the user as feedback, such as specific technical guidance such as "Be conscious of your forward-leaning posture."

[0317] Surfboard Recommendations

[0318] Step 1:

[0319] Users enter their body type, skill level, and wave conditions into the application on their device, including their height, weight, and surfing skill level.

[0320] Step 2:

[0321] The server receives the user's input data and sends it to the generation AI, which then uses this data to recommend the best surfboard for them.

[0322] Step 3:

[0323] The server stores the recommendations from the AI ​​in a database and provides them to the user's device. For example, a recommendation might be "shortboard, 6.0 feet long."

[0324] Emotion recognition and feedback / recommendation optimization using an emotion engine

[0325] Step 1:

[0326] When a user uploads a surfing video, the server sends the facial and voice data from the video to the emotion engine.

[0327] Step 2:

[0328] The emotion engine uses facial and voice analysis to identify the user's emotional state (e.g., happy, unhappy, excited, relaxed, etc.).

[0329] Step 3:

[0330] The server inputs the analysis results of the emotion engine into the generative AI, optimizing feedback and surfboard recommendations based on the user's emotions.

[0331] Step 4:

[0332] The server then stores customized feedback and recommendations based on the user's emotions in a database and provides them to the user's device. For example, if the user is feeling adventurous, the server might suggest, "Try a more difficult move next time."

[0333] The above is the specific processing flow of the surfing information provision system that combines an emotion engine. Through this series of steps, users can receive more accurate wave information, appropriate technical feedback, and personalized surfboard recommendations based on their emotions.

[0334] Example 2

[0335] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0336] Conventional surfing support systems provide wave information and feedback to improve surfing technique, but they are unable to provide optimal feedback or recommendations that take into account the emotional state of each individual user. As a result, users only receive uniform information, making it difficult to receive advice tailored to their individual needs. This problem prevents users from receiving appropriate feedback to improve their technique, preventing them from maximizing the enjoyment and efficiency of surfing.

[0337] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting video data from a video collection device installed on the beach, means using a generation AI to analyze wave height, period, and direction based on the collected video data, means for providing the analyzed wave information to the user's device in real time, means for collecting weather data and past wave condition data, means using a generation AI to combine this data to predict waves one week in advance, means for providing the predicted wave information to the user's device, means using a generation AI to analyze surfing videos uploaded by the user and generate riding form and technical feedback, means for providing the analysis results to the user's device, means using a generation AI to recommend the optimal surfboard based on the user's body type, skill, and wave conditions, means for providing the recommendation results to the user's device, and means using an emotion recognition engine to analyze the user's emotional state and provide feedback and recommendations based on the emotions. This allows users to receive optimal feedback and recommendations based on their individual needs and emotional state.

[0338] "Beach-mounted video collection devices" refers to video capture devices such as cameras and drones placed around the beach for surfboard practice and wave monitoring.

[0339] "Video data" refers to video data that records and stores the movement of waves on the beach and surfing techniques in real time.

[0340] "Collection means" refers to the function of acquiring data from video collection devices installed on the beach and sending it to a server.

[0341] "Generative AI" is a type of artificial intelligence that uses collected data to analyze and predict wave information and surfing techniques.

[0342] "Analysis means" refers to the ability to use generative AI to analyze collected video data and other data to derive wave height, period, direction, technical advice, etc.

[0343] "Real-time provision means" refers to the function of displaying analyzed information on the user's device without delay.

[0344] "Weather data" refers to numerical data related to weather, such as wind speed, temperature, and air pressure.

[0345] "Wave condition data" refers to information that records past wave height, period, direction, and other data.

[0346] "Wave prediction means" refers to a function that uses AI generation based on weather data and wave condition data to predict future wave conditions.

[0347] "Surfing videos" refer to video files that users have filmed and saved of their surfing skills.

[0348] "Technical feedback" refers to advice and suggestions for technical improvement obtained by the generative AI analyzing surfing videos.

[0349] "Device" refers to a device such as a smartphone, tablet or computer that a surfer uses to receive information such as wave information, technical feedback and surfboard recommendations.

[0350] "Body type" refers to a user's physical characteristics such as height and weight.

[0351] "Skill" refers to the user's level of surfing skill and experience.

[0352] A "surfboard" refers to the board used for gliding when surfing.

[0353] "Recommendation method" refers to a function that uses generative AI to select and suggest the surfboard that is best suited to the user's body type, skill level, and wave conditions.

[0354] An "emotion recognition engine" refers to an algorithm or program that analyzes a user's facial expressions and voice to recognize their emotional state.

[0355] "Means for providing feedback and recommendations" refers to a feature that uses an emotion recognition engine to provide feedback and surfboard recommendations based on the user's emotional state.

[0356] The present invention is a system that collects video data from a video collection device installed on the beach, analyzes wave conditions and surfing techniques using a generative AI model, and provides optimal feedback and surfboard recommendations based on the user's emotional state. A specific embodiment of this system will be described.

[0357] Beach camera video data collection and analysis

[0358] The server collects video data in real time from video collection devices such as cameras and drones installed on the beach. This video data is saved on the server at regular intervals. For example, video data from a camera installed on Shonan Beach is captured at 30 frames per second and saved in MP4 format every five minutes. The server then inputs this saved video data into a generative AI model to analyze information such as wave height, period, and direction.

[0359] Example prompt sentence:

[0360] "Collect video data from Shonan Beach and analyze the wave height, period, and direction. Example: Wave height 1.5m, period 8 seconds, direction southeast."

[0361] Providing wave information and one-week wave forecasts

[0362] The server uses the generative AI model to analyze wave condition information and provides it to the user's device in real time. It also obtains real-time weather data, such as wind speed, temperature, and air pressure, from weather data providers, and combines it with past wave condition data before inputting it into the generative AI model. It then predicts waves one week in advance and notifies the user's device of the results. For example, it displays a forecast such as "Predicted wave heights at Shonan Beach next week are 1.2m to 1.7m."

[0363] Example prompt sentence:

[0364] "Based on weather data and wave data from the past year, please predict the wave height at Shonan Beach next week. Example: Wave height 1.2m - 1.7m."

[0365] Analysis and feedback of user surfing videos

[0366] Users upload their surfing videos from their devices to a server. The server then inputs the uploaded videos into a generative AI model to analyze the user's riding form and technique. For example, it generates feedback such as "Be conscious of leaning forward" and displays it on the user's device.

[0367] Example prompt sentence:

[0368] "Analyze user surfing videos and provide technical feedback. Example: Be mindful of forward leaning posture."

[0369] Surfboard Recommendations

[0370] Users input their body type (e.g., height 175 cm, weight 70 kg) and surfing skill level (e.g., intermediate) into their device. The server uses this information and wave condition data to recommend the optimal surfboard using a generative AI model. For example, a recommended result such as "shortboard, 6.0 feet long" is generated and displayed on the user's device.

[0371] Example prompt sentence:

[0372] "Recommend a surfboard that is suitable for a surfer who is 175cm tall, weighs 70kg, and has intermediate skill level. Example: shortboard, 6.0 feet long."

[0373] Optimization by Emotion Engine

[0374] The server uses an emotion recognition engine to analyze the surfing video and recognize the user's emotional state. For example, if the user's facial expressions and voice indicate dissatisfaction, the server provides encouraging feedback such as, "To improve your form, try to focus on your hip movement next time." The server also optimizes surfboard recommendations based on the user's emotional state. If the user is feeling challenged, the server recommends a surfboard that requires more advanced skills.

[0375] Example prompt sentence:

[0376] "Analyze emotions from users' surfing videos and provide encouraging feedback if they're not satisfied. For example, improve your form by focusing on your hip movement next time."

[0377] The above is a specific embodiment of the system of the present invention. The system of the present invention allows users to receive more detailed and accurate wave information, individual technical feedback, and even recommendations for the best surfboard based on their emotions, allowing users to maximize their surfing enjoyment.

[0378] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0379] Step 1:

[0380] The server collects video data from video collection devices installed on the beach. The cameras and drones capture video data in real time at 30 frames per second and send the data to the server. The server saves this video data in mp4 format every 5 minutes. The input is real-time video data from the cameras, and the output is a video file saved on the server.

[0381] Step 2:

[0382] The server inputs the saved video data into a generative AI model, which analyzes the wave height, period, and direction. The generative AI model analyzes the video frames and extracts wave characteristics. For example, it obtains data such as a wave height of 1.5m, a period of 8 seconds, and a direction southeast. The input is the saved video file, and the output is the wave information resulting from the analysis.

[0383] Step 3:

[0384] The server provides the analyzed wave information to the user's device in real time. The acquired wave height, period, and direction are displayed on the user's device. The input is the analysis result of the wave information by the generative AI model, and the output is the wave information displayed on the user's device.

[0385] Step 4:

[0386] The server obtains weather data in real time from a weather data provider. Weather data includes wind speed, temperature, and air pressure. This data is combined with past wave condition data and input into a generative AI model to predict waves one week in advance. For example, a prediction result such as "Predicted wave heights at Shonan Beach next week will be 1.2m to 1.7m" can be obtained. The input is real-time weather data and past wave condition data, and the output is a wave prediction result for one week in advance.

[0387] Step 5:

[0388] The server notifies the user's device of the wave prediction results, which are then displayed on the user's device for one week ahead. The input is the wave prediction results from the generative AI model, and the output is the prediction results displayed on the user's device.

[0389] Step 6:

[0390] The user uploads their surfing video from their device to the server. The user selects and submits the video using the application. The input is the user's surfing video file, and the output is the video file stored on the server.

[0391] Step 7:

[0392] The server inputs the uploaded surfing video into a generative AI model, which analyzes the user's riding form and technique. The generative AI model analyzes the video frames to identify the user's form and technical shortcomings. For example, it generates feedback such as, "Try to be more conscious of your forward leaning posture." The input is the user's surfing video file, and the output is technical feedback.

[0393] Step 8:

[0394] The server displays the generated feedback on the user's device, which displays the analysis result feedback. The input is the technical feedback from the generative AI model, and the output is the feedback displayed on the user's device.

[0395] Step 9:

[0396] The user enters their body type (e.g., height 175 cm, weight 70 kg) and surfing skill (e.g., intermediate) into the terminal. The terminal application has a form for entering this information. The input is the user's body type and skill information, and the output is the user information sent to the server.

[0397] Step 10:

[0398] The server uses a generative AI model to recommend the optimal surfboard based on the input user information and wave information. The generative AI model analyzes the user's body type, skill, and wave conditions to select the optimal surfboard. For example, a recommendation result of "shortboard, 6.0 feet long" may be obtained. The input is the user's body type, skill information, and wave information, and the output is the recommended surfboard information.

[0399] Step 11:

[0400] The server displays the recommendation results on the user's device, which then displays information about the recommended surfboards. The input is the surfboard recommendation results from the generative AI model, and the output is the recommendation results displayed on the user's device.

[0401] Step 12:

[0402] When the server analyzes the surfing video, it uses an emotion recognition engine to recognize the user's emotional state. It analyzes the user's facial expressions and voice to recognize the emotional state (e.g., dissatisfied). The input is the user's surfing video, and the output is the recognized emotional state.

[0403] Step 13:

[0404] The server adjusts the feedback and surfboard recommendations based on the recognized emotion. If the user is not satisfied, encouraging feedback such as "Next time, be mindful of your hip movement to improve your form" is generated. The input is the recognized emotional state, and the output is the adjusted feedback and surfboard recommendations.

[0405] Step 14:

[0406] The server displays the adjusted feedback and recommendation on the user's device. The user's device displays the optimal feedback and surfboard recommendation according to the emotion. The input is the adjusted feedback and surfboard recommendation, and the output is the feedback and recommendation displayed on the user's device.

[0407] (Application example 2)

[0408] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0409] While conventional systems provide information about beach waves and feedback to help users improve their surfing skills, they do not consider optimal in-store purchasing support using customer emotion recognition. This makes it difficult to provide individually optimized product recommendations and feedback, and poses the challenge of not improving the customer's purchasing experience.

[0410] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0411] In this invention, the server includes: means for collecting video data from cameras installed on the beach; means using a generation AI to analyze wave height, period, and direction based on the collected video data; means for providing the analyzed wave information to a user's device in real time; means for collecting weather data and past wave condition data; means using a generation AI to combine this data to predict waves one week in advance; means for providing the predicted wave information to a user's device; means using a generation AI to analyze surfing videos uploaded by a user and generate riding form and technical feedback; means for providing the analysis results to a user's device; means using a generation AI to recommend the optimal surfboard based on the user's body type, skill, and wave conditions; means for providing the recommendation results to a user's device; means using a generation AI to link video footage from cameras installed in a store with a customer's mobile app, analyze the customer's facial expressions and behavior in real time, and provide optimal product recommendations based on emotion recognition; means for providing product information and reviews to a customer's device; and means for analyzing the customer's facial expressions and behavior and providing feedback and product recommendations based on their emotional state. This allows customers to receive optimal purchasing support using emotion recognition technology in physical stores, and enables personalized feedback and recommendations.

[0412] A "camera" is a device that captures video in real time and records or transmits it as data.

[0413] "Video data" refers to a series of consecutive image information captured by a camera, and is data that can be viewed as a video.

[0414] "Generative AI" refers to algorithms and models that use artificial intelligence technology to analyze data and generate results.

[0415] "Analysis" is the process of processing and analyzing collected data to extract useful information.

[0416] "Wave height, period, and direction" are indicators of the physical characteristics of ocean waves and are important condition information for surfing.

[0417] "Real-time" refers to the property that data is processed synchronously and provided to users without any time delay.

[0418] "Weather data" refers to any data related to weather, such as weather, temperature, wind speed, etc.

[0419] "Past wave condition data" refers to previously recorded data such as wave height, period, and direction, and is information that indicates past wave conditions.

[0420] "Surfing video" is video data captured by a camera while surfing.

[0421] "Riding form" refers to a surfer's posture and movements while surfing.

[0422] "Technical feedback" is specific advice and instruction to improve surfing technique.

[0423] "User's body type, skill, and wave conditions" refers to information about the user's physical characteristics, surfing skill level, and current wave conditions.

[0424] A "surfboard" is a board used for surfing.

[0425] "Recommendation" refers to suggesting the best option based on a specific situation or condition.

[0426] "Images from cameras installed inside the store" refers to images captured by cameras installed inside the store.

[0427] "Customer mobile app" refers to application software that customers use on mobile devices such as smartphones and tablets.

[0428] "Analyzing facial expressions and behavior in real time" refers to capturing a customer's facial expressions and movements, and instantly processing and analyzing the data.

[0429] "Emotion recognition" is a technology that uses information such as facial expressions and voice to identify a person's emotional state.

[0430] "Product recommendation" refers to presenting appropriate products to customers.

[0431] "Product information" is detailed information about the product's price, specifications, functions, etc.

[0432] A "review" refers to a written opinion, such as an evaluation or impression, about a product or service.

[0433] "Feedback" refers to opinions or advice given in response to a particular action or result.

[0434] This invention provides a system for recognizing customer emotions and optimizing the purchasing experience in brick-and-mortar stores. The system is centered around a generative AI that analyzes customer facial expressions and behavior in real time using images from cameras installed in the store, and recommends products and provides information according to the customer's emotional state.

[0435] The specific means included in the system are as follows:

[0436] In-store cameras and video collection

[0437] The server collects video data in real time from IP cameras installed in the store, making it possible to monitor and record customer behavior and facial expressions.

[0438] Video analysis and emotion recognition

[0439] The server analyzes the collected video data using OpenCV and generative AI, and uses deep learning frameworks such as TensorFlow to analyze the customer's facial expressions and behavior and recognize their emotional state.

[0440] Product recommendations and information

[0441] The server then recommends the most suitable products to the customer based on the results of emotion recognition, and provides recommended product information, reviews, and detailed descriptions to the customer's mobile app, allowing the customer to receive product information tailored to their mood and interests.

[0442] Hardware and software used to process the program

[0443] Hardware:

[0444] IP camera (installed inside the store)

[0445] Server (equipped with a high-performance GPU)

[0446] Smartphone or tablet (customer's mobile device)

[0447] software:

[0448] TensorFlow / PyTorch (training and inferencing generative AI models)

[0449] OpenCV (video data preprocessing and analysis)

[0450] Mobile app (runs on smartphones)

[0451] Specific examples

[0452] 1. Customer emotion recognition and product recommendation:

[0453] The camera captures images of customers inside the store and transmits them to a server in real time.

[0454] The server uses a generative AI model to analyze the customer's facial expression and determine that "this customer is interested."

[0455] The server displays detailed information and reviews of Product A on the customer's smartphone.

[0456] example:

[0457] "You seem curious about this product. Let me offer you a sample or other relevant information."

[0458] "You seem unhappy with this product. Let me recommend a different product or have you check the reviews."

[0459] This allows customers to receive products and information that match their emotional state, improving their shopping experience.

[0460] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0461] Step 1:

[0462] Cameras inside the store capture video data and send it to a server.

[0463] Subject: Camera

[0464] Specific operation: Captured video data is acquired frame by frame and sent to the server using the RTSP protocol.

[0465] Input: Video of the store

[0466] Output: Frame-by-frame video data

[0467] Step 2:

[0468] The server captures the received video data in real time and pre-processes it for analysis.

[0469] Subject: Server

[0470] Specific operation: Video data is acquired frame by frame using OpenCV, and preprocessing such as face detection is performed.

[0471] Input: Frame-by-frame video data

[0472] Output: Preprocessed frame data

[0473] Step 3:

[0474] The server inputs the preprocessed frame data into a generative AI model, which analyzes facial expressions and behavior to recognize the customer's emotional state.

[0475] Subject: Server

[0476] How it works: Using TensorFlow, a generative AI model is run to analyze customer facial expression data and identify emotions.

[0477] Input: Preprocessed frame data

[0478] Output: Customer's emotional state (e.g., interested, satisfied, dissatisfied, etc.)

[0479] Step 4:

[0480] The server selects the most appropriate products and information based on the customer's emotional state, and generates recommended results using generative AI.

[0481] Subject: Server

[0482] How it works: Using a generative AI model, it matches the emotional state with a database of products in the store and selects the appropriate product.

[0483] Input: Customer emotional state, product database

[0484] Output: Recommended product information

[0485] Step 5:

[0486] The server notifies the customer's smartphone of the recommended product information.

[0487] Subject: Server

[0488] What it does: Recommendations are sent to the customer's mobile app via API and displayed within the app.

[0489] Input: Recommended product information

[0490] Output: Notified product information

[0491] Step 6:

[0492] Customers refer to the information displayed on their smartphones to check product details and reviews.

[0493] Subject: customer

[0494] Specific actions: Tap on information displayed in the smartphone app to see more details or read reviews.

[0495] Input: Notified product information

[0496] Output: Customer action (purchase, further enquiry, etc.)

[0497] These steps enable the system to analyze customer sentiment in real time and provide optimal products and information based on the results.

[0498] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0499] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0500] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0501] [Second embodiment]

[0502] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0503] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0504] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0505] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0506] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0507] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0508] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0509] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0510] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0511] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0512] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0513] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0514] The present invention relates to a system for providing wave information and supporting users in improving their surfing skills in the surfing market. Specific processes required to implement this system will now be described.

[0515] Overall overview

[0516] This system has the following main functions:

[0517] 1. Beach camera video data collection and analysis

[0518] 2. Providing wave information and wave forecasts for the next week

[0519] 3. User surfing video analysis and feedback

[0520] 4. Surfboard Recommendations

[0521] Beach camera video data collection and analysis

[0522] The server collects video data from cameras installed on each beach. The cameras stream the video in real time, and the video data is stored on the server for analysis.

[0523] Examples:

[0524] The server captures video data from the cameras at Shonan Beach at 30 frames per second and saves the data as a file every five minutes.

[0525] The saved video file is analyzed by the generation AI, and information such as wave height of 1.5m, period of 8 seconds, and direction to the southeast is obtained.

[0526] Providing wave information and one-week wave forecasts

[0527] The server uses AI to analyze wave conditions from the collected video data and provides this information to the user's device in real time. It also combines weather data and past wave condition data to use AI to predict wave conditions for the next week and provides this forecast data to the user.

[0528] Examples:

[0529] The server obtains data such as wind speed and temperature in real time from weather data providers, integrates it with wave data from the past year, and runs it through AI.

[0530] The generating AI analyzes the data and determines that "the predicted wave height at Shonan Beach next week will be between 1.2m and 1.7m," and notifies the user.

[0531] User surfing video analysis and feedback

[0532] When a user uploads a surfing video from their device, the server receives the video and sends it to the generation AI, which analyzes the user's riding form and technique and generates technical feedback, which is displayed on the user's device.

[0533] Examples:

[0534] Users upload their surfing videos to the server via the application.

[0535] The server runs the video through AI, which generates feedback such as "Be conscious of your forward-leaning posture," which is displayed on the user's device.

[0536] Surfboard Recommendations

[0537] Based on information such as the user's body type, skill level, and wave conditions, the server uses generative AI to analyze and recommend the most suitable surfboard. This recommendation is also sent to the user's device.

[0538] Examples:

[0539] Users enter their height, weight and intermediate surfing skills into the app.

[0540] Based on this information, the server recommends a "shortboard, 6.0 feet long" and displays it on the app.

[0541] The above is a concrete example of how to implement the present invention. By linking cameras installed on the beach with AI generation, this system provides detailed and accurate wave information and personalized technical feedback. It also recommends the optimal surfboard based on the user's unique information, allowing surfers to enjoy surfing more comfortably and efficiently.

[0542] The processing flow will be explained below.

[0543] Beach camera video data collection and analysis

[0544] Step 1:

[0545] The server receives video data in real time from cameras installed on the beach, connects to the IP addresses of the cameras, and streams the video using, for example, RTSP (Real Time Streaming Protocol).

[0546] Step 2:

[0547] The server saves the streamed video data as a file at regular intervals, typically as a new file every five minutes.

[0548] Step 3:

[0549] The server inputs the saved video file into the generative AI model and begins analysis, including wave height, period, and direction.

[0550] Step 4:

[0551] The server stores the analysis results of the generative AI model in a database and simultaneously provides them to the user's device in real time.

[0552] Providing wave information and one-week wave forecasts

[0553] Step 1:

[0554] The server uses the weather data provider API to retrieve the latest weather data, including wind speed, wind direction, temperature, and air pressure.

[0555] Step 2:

[0556] The server retrieves data on past wave conditions from a database, usually covering the past year.

[0557] Step 3:

[0558] The server inputs weather data and past wave condition data into the AI ​​generator, which then predicts waves for the next week.

[0559] Step 4:

[0560] The server stores the prediction results from the AI ​​in a database and provides them to the user's device, including the wave height, period, and direction for the next week.

[0561] User surfing video analysis and feedback

[0562] Step 1:

[0563] Users use their devices to upload surfing videos to the application, which then sends the videos to a cloud server.

[0564] Step 2:

[0565] The server receives the uploaded video data and stores it in temporary storage.

[0566] Step 3:

[0567] The server inputs the saved video data into a generation AI that analyzes the user's riding form and technique.

[0568] Step 4:

[0569] The server receives the analysis results from the AI ​​generator and provides them to the user as feedback, such as specific technical guidance such as "Be conscious of your forward-leaning posture."

[0570] Surfboard Recommendations

[0571] Step 1:

[0572] Users enter their body type, skill level, and wave conditions into the application on their device, including their height, weight, and surfing skill level.

[0573] Step 2:

[0574] The server receives the user's input data and sends it to the generation AI, which then uses this data to recommend the best surfboard for them.

[0575] Step 3:

[0576] The server stores the recommendations from the AI ​​in a database and provides them to the user's device. For example, a recommendation might be "shortboard, 6.0 feet long."

[0577] The above is the specific processing flow of the system that provides wave information and improves user skills in the surfing market. This series of steps enables users to obtain more accurate information and appropriate feedback, resulting in an improved surfing experience.

[0578] Example 1

[0579] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0580] In today's surfing world, it is difficult to obtain accurate wave information in real time or to efficiently support the improvement of individual surfing techniques. Furthermore, there are insufficient methods for users to select the most suitable surfing equipment. For this reason, there is a need for specific support to help users enjoy surfing more and improve their techniques.

[0581] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0582] In this invention, the server includes: means for collecting video data from a video collection device; means using a generation AI to analyze wave height, period, and direction based on the collected video data; means for providing the analyzed wave information to a user's device in real time; means for collecting weather data and past wave condition data; means using a generation AI to combine this data to predict waves one week in advance; means for providing the predicted wave information to the user's device; means using a generation AI to analyze surfing videos uploaded by users and generate riding form and technical feedback; means for providing the analysis results to the user's device; means using a generation AI to recommend optimal surfing equipment based on the user's body type, skill, and wave conditions; and means for providing the recommendation results to the user's device. This allows users to obtain detailed and accurate wave information in real time, improving their surfing skills and selecting optimal surfing equipment.

[0583] A "video collection device" is a device that is installed on a beach or other location and captures video data in real time.

[0584] "Video data" refers to video information acquired by a video collection device.

[0585] "Generative AI" is an artificial intelligence model that analyzes and generates specific patterns and information using large amounts of video and other data.

[0586] "Wave information" is information that indicates specific parameters and conditions related to surfing, such as wave height, period, and direction.

[0587] "User device" refers to an electronic device used by a user, such as a smartphone, tablet, or PC.

[0588] "Weather data" refers to information about weather conditions such as wind speed, temperature, and precipitation.

[0589] "Past wave condition data" refers to data such as past wave height, period, and direction.

[0590] "Surfing video" is video data taken by a user while surfing.

[0591] "Riding form" refers to the user's posture and movements while surfing.

[0592] "Technical feedback" is specific instruction or advice aimed at improving surfing technique.

[0593] "Surfing equipment" means equipment or tools used in surfing, such as a surfboard.

[0594] "Recommended results" are recommendations for optimal surfing equipment calculated by the generating AI based on the user's specific conditions.

[0595] This invention relates to a system for providing wave information to the surfing market and supporting users in improving their technique. To implement this system, the following specific hardware and software are used, and how data is processed and calculated will be described.

[0596] Hardware and software used

[0597] 1. Video collection device: Camera equipment installed on the beach. For example, a high-resolution IP camera is used. This allows for real-time streaming video.

[0598] 2. Server: A central system for collecting, storing, and analyzing data. It is equipped with a high-performance server processor and large-capacity storage.

[0599] 3. Generative AI models: Artificial intelligence for analyzing video and other data, specifically using deep learning frameworks (e.g., TensorFlow and PyTorch).

[0600] 4. User devices: smartphones, tablets, computers, etc., allowing users to receive information in real time and operate and input data through the interface.

[0601] Overview of program processing

[0602] Beach camera video data collection and analysis

[0603] The server collects video data from cameras installed on the beach. The cameras stream in real time, and the video data is stored on the server for analysis. The server acquires video data from the high-resolution IP cameras at 30 frames per second and saves it as an MP4 file every five minutes.

[0604] Providing wave information and one-week wave forecasts

[0605] The server uses a generative AI model to analyze wave conditions from the collected video data and provides this information to the user's device in real time. The server also obtains weather data such as wind speed and temperature from a weather data provider and combines it with past wave condition data to generate a wave forecast for the next week. This forecast data is also analyzed using generative AI and provided to the user.

[0606] User surfing video analysis and feedback

[0607] When a user uploads a surfing video from their device, the server receives the video and sends it to a generative AI model. The generative AI analyzes the user's riding form and technique and generates technical feedback. This feedback is displayed on the user's device. Users upload surfing videos using an application, and the server sends the video to the generative AI, which generates feedback such as "Be conscious of your forward leaning posture."

[0608] Surfboard Recommendations

[0609] Based on information such as the user's body type, skill level, and wave conditions, the server uses generative AI to analyze and recommend the most suitable surfboard. This recommendation result is also provided to the user's device. For example, if a user enters their height of 175 cm, weight of 70 kg, and intermediate skill level into the app, the server will recommend a "shortboard, 6.0 feet long" and display the result on the user's device.

[0610] Examples of specific examples and prompts

[0611] Examples:

[0612] The server collects video data from cameras at Shonan Beach and analyzes it using generative AI to obtain information such as wave height of 1.5m, wave period of 8 seconds, and direction to the southeast.

[0613] The server uses weather data to predict waves for the next week and notifies the user that "wave heights predicted for Shonan Beach next week are between 1.2m and 1.7m."

[0614] Users upload their surfing videos to a server and receive feedback such as "Improve your popping timing."

[0615] Based on the information entered by the user, the server recommends a "6.0 foot shortboard."

[0616] Example prompt sentence:

[0617] "Please provide current and forecast wave height data for Shonan Beach."

[0618] "Analyze my surfing video and give me feedback"

[0619] "Recommend a surfboard that suits your body type and skill level."

[0620] The above is a specific embodiment for carrying out the invention. This system allows users to obtain detailed and accurate wave information in real time, enabling them to improve their surfing skills and select the most suitable surfing equipment.

[0621] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0622] Step 1:

[0623] The server collects video data from video collection devices installed on the beach. The camera devices stream video in real time and send the data to the server. The server saves this data as MP4 files at regular intervals (e.g., every 5 minutes).

[0624] Specific behavior:

[0625] The server receives video data from the Shonan Beach camera in real time at 30 frames per second and saves it as one MP4 file every five minutes.

[0626] Input: Video data (real-time video stream)

[0627] Output: MP4 file (video data saved every 5 minutes)

[0628] Step 2:

[0629] The server inputs the stored video data into a generative AI model to analyze wave height, period, and direction, and the generative AI model uses a deep learning algorithm to extract wave characteristics.

[0630] Specific behavior:

[0631] The server sends the saved MP4 file to the generation AI, obtaining information such as wave height of 1.5m, period of 8 seconds, and direction southeast.

[0632] Input: Saved MP4 file

[0633] Output: Wave characteristics data (wave height, period, direction)

[0634] Step 3:

[0635] The server transmits the analyzed wave information in real time to the user's device, which displays this information.

[0636] Specific behavior:

[0637] The server sends wave characteristic data to the user's smartphone via push notification, displaying "Current wave height: 1.5m, direction: southeast."

[0638] Input: Wave property data

[0639] Output: Push notification (wave height, period, direction information)

[0640] Step 4:

[0641] The server retrieves weather data from a weather data provider. This data includes wind speed, temperature, etc. This is combined with historical wave condition data to generate a week-ahead wave forecast. The data is then analyzed using a generative AI model.

[0642] Specific behavior:

[0643] The server combines wind speed, temperature, and wave data from the past year, and uses generative AI to analyze the predicted wave height at Shonan Beach next week as 1.2m to 1.7m.

[0644] Input: Weather data, historical wave condition data

[0645] Output: Predicted wave information (future wave height, period, direction)

[0646] Step 5:

[0647] Users upload surfing videos from their devices to a server, which then inputs the videos into a generative AI model to analyze riding form and technique, and the generative AI then generates technical feedback.

[0648] Specific behavior:

[0649] Users use the application to upload surfing videos to a server, which then sends the videos to a generating AI, which then displays the videos on the user's device, providing feedback such as "be mindful of your forward leaning posture."

[0650] Input: surfing video

[0651] Output: Technical feedback (specific improvements)

[0652] Step 6:

[0653] The server uses the generative AI model to send the analysis results to the user's device, where technical feedback based on the analysis results is displayed.

[0654] Specific behavior:

[0655] The server sends the analysis results to the user's device as a push notification, displaying feedback such as "Improve your popping timing."

[0656] Input: Analysis results

[0657] Output: Push notification (technical feedback)

[0658] Step 7:

[0659] Based on the user's body type, skill level, and wave conditions, the server uses a generative AI model to recommend the optimal surfing equipment, which is then delivered to the user's device.

[0660] Specific behavior:

[0661] When a user inputs their height of 175 cm, weight of 70 kg, and intermediate skill level, the server uses generative AI to recommend a "shortboard, 6.0 feet long" and displays the results on the user's device.

[0662] Input: User's body type, skill, and wave conditions

[0663] Output: Recommended results (optimal surfing equipment)

[0664] (Application example 1)

[0665] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0666] Monitoring and optimization of manufacturing processes in conventional factories often relied on manual work or partial automation, making it difficult to improve efficiency in real time or provide flexible feedback. This resulted in problems such as the generation of defective products during the manufacturing process and delays in identifying and resolving bottlenecks throughout the line, resulting in a decline in overall productivity. Furthermore, line stoppages due to sudden equipment failures or lack of maintenance also had a negative impact on the production process. A system that could solve these issues and improve the efficiency of the entire manufacturing line was needed.

[0667] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0668] In this invention, the server includes means for collecting video data from cameras installed on the factory production line, means using a generation AI for analyzing the current state of the production process based on the collected video data, means for providing the analyzed process information to an operator's terminal in real time, means for collecting equipment operation data and past production data, means using a generation AI for combining this data to predict the production process one week ahead, means for providing the predicted process information to the operator's terminal, means using a generation AI for analyzing videos of specific processes uploaded by the operator and generating efficiency improvement feedback, means for providing the analysis results to the operator's terminal, means using a generation AI for recommending maintenance based on the equipment's operating status and process data, and means for providing the recommendation results to the operator's terminal.

[0669] This enables real-time monitoring of the operating status of the entire manufacturing line, enabling immediate process improvements and efficiency improvements, early identification of bottlenecks, and the provision of appropriate maintenance forecasts.

[0670] A "production line" refers to a series of processes and machinery that continuously processes and assembles products within a factory.

[0671] A "camera" is a device that captures video or images, and generally includes digital cameras and surveillance cameras.

[0672] "Video data" refers to data that digitally represents images that change continuously over time.

[0673] "Generative AI" refers to artificial intelligence that uses machine learning algorithms to analyze and make predictions from specific data.

[0674] "Process information" refers to data related to the manufacturing process, including production volume, quality, error rate, etc.

[0675] A "terminal" is a device that is connected to the system and exchanges information, including smartphones and computers.

[0676] "Operational data" refers to data related to the operating state of machines and equipment, including operating time, frequency of use, energy consumption, etc.

[0677] "Past manufacturing data" refers to the accumulation of data related to manufacturing processes that has been collected to date.

[0678] "Maintenance" refers to the maintenance and repair work required to keep machines and equipment operating normally.

[0679] A "bottleneck" is the factor that slows down progress in production or a process and reduces overall efficiency.

[0680] "Feedback" is information or instructions that are based on the results or status of a system or process to encourage appropriate improvements or modifications.

[0681] This invention provides a system for monitoring and optimizing a factory production line. The system is composed of the following hardware and software.

[0682] System hardware configuration

[0683] 1. Camera:

[0684] This is a video capture device installed at various points on the production line, allowing real-time video of the manufacturing process to be captured.

[0685] 2. Server:

[0686] It is a central processing unit for storing and analyzing collected video data. The server also provides functions for analyzing various data and using generative AI models.

[0687] 3. Terminal:

[0688] A device such as a smartphone or computer used by an operator to display feedback from the server and analysis results.

[0689] System software configuration

[0690] 1. Generative AI model:

[0691] It is an artificial intelligence algorithm that analyzes video data and various data related to manufacturing processes. Specifically, it uses machine learning models to optimize processes, detect errors, and recommend maintenance.

[0692] 2. Database:

[0693] The collected data is stored and managed using a DBMS such as SQL.

[0694] 3. Front-end application:

[0695] A user interface is provided to display analysis results and feedback to the operator, which is implemented as a smartphone application (Android / iOS) or a web application.

[0696] Example of operation

[0697] 1. Video Data Collection and Analysis:

[0698] A camera captures the production line and sends the video data to a server. The server inputs this video data into a generative AI model and analyzes the current state of the production process. For example, it can obtain specific analysis results such as "The component placement speed on assembly line 1 is declining."

[0699] 2. Providing process information:

[0700] The analysis results are provided to the operator's terminal in real time. For example, an alert such as "A bottleneck has occurred on the assembly line. Please consider countermeasures" is displayed on the terminal.

[0701] 3. Analysis of operational and historical data:

[0702] The server collects equipment operation data and past production data, and uses a generative AI model to predict the production process one week in advance. Specifically, it provides forecast information such as, "Supply shortages are predicted for next week. Please consider replenishing inventory."

[0703] 4. Feedback Generation:

[0704] When an operator uploads video data of a specific process to the server, the generative AI model analyzes it and provides feedback on how to improve efficiency, such as "Please be careful when installing parts."

[0705] 5. Maintenance Recommendations:

[0706] The server monitors the operating status of the equipment and recommends necessary maintenance. For example, a message such as "An abnormality has been detected in the robot arm on line 4. Inspection is recommended" is displayed on the operator's terminal.

[0707] Examples of prompt statements

[0708] "Analyze the video of the assembly process on production line 1 and identify areas for improvement."

[0709] This system enables real-time monitoring of the operating status of the entire manufacturing line, enabling immediate process improvements and efficiency gains, early identification of bottlenecks, and the provision of appropriate maintenance forecasts.

[0710] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0711] Step 1:

[0712] The server acquires video data in real time from cameras installed on the factory production line. The acquired video data is temporarily stored in the server's storage. The input is video data from the camera, and the output is video data saved in the server's storage. Specifically, the camera continuously captures video and sends the data to the server via an IP network.

[0713] Step 2:

[0714] The server inputs the stored video data into a generative AI model to analyze the current state of the manufacturing process. The generative AI model uses this data to identify problems and areas for improvement on the production line. The input is the temporarily stored video data, and the output is the process information that is the analysis result. Specifically, it uses an image analysis algorithm to detect specific errors and delays.

[0715] Step 3:

[0716] The server notifies the operator's device of the analyzed process information in real time. The input is the analysis results obtained from the generative AI model, and the output is feedback displayed on the operator's device. Specifically, the server uses a notification system to send push notifications to the device.

[0717] Step 4:

[0718] The server collects equipment operation data and past manufacturing data, inputs it into a generative AI model, and predicts future manufacturing processes. The input is equipment operation data and past manufacturing data, and the output is predicted manufacturing process information. Specifically, it retrieves the necessary data from the database via a query and analyzes it using the generative AI model.

[0719] Step 5:

[0720] The server provides predicted manufacturing process information to the operator's device. The input is predicted data from the generative AI model, and the output is predicted information displayed on the operator's device. Specifically, it generates a dashboard that displays the prediction results in a visually easy-to-understand manner.

[0721] Step 6:

[0722] The terminal provides a means for the operator to upload video data of a specific process to the server. The input is the operator's video data, and the output is the uploaded video stored on the server. Specifically, the terminal provides file selection and upload functions from the user interface.

[0723] Step 7:

[0724] The server inputs the uploaded video data of a specific process into a generative AI model to generate efficiency improvement feedback. The input is the uploaded video, and the output is the efficiency improvement feedback. Specifically, the video analysis algorithm is used to identify areas for efficiency improvement and generate feedback in text format.

[0725] Step 8:

[0726] The server provides feedback to the operator's device. The input is feedback data from the generative AI model, and the output is feedback displayed on the operator's device. Specifically, the feedback data is sent to the device in real time and displayed on the user interface.

[0727] Step 9:

[0728] The server monitors the operating status of the equipment and makes necessary maintenance recommendations based on the generative AI model. The input is the equipment's operating data and the output is the maintenance recommendation. Specifically, it collects operating data in real time and automatically generates a maintenance notification if an abnormality is detected.

[0729] Step 10:

[0730] The server provides maintenance recommendations to the operator's device. The input is maintenance recommendation data from the generative AI model, and the output is a maintenance notification displayed on the operator's device. Specifically, the server uses a notification system to push the recommendations to the device.

[0731] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0732] This invention relates to a system that provides wave information and helps users improve their surfing skills in the surfing market, as well as a system that provides more personalized and optimal feedback and recommendations by using user emotion recognition. The specific processing required to implement this system is described below.

[0733] Overall overview

[0734] This system has the following main functions:

[0735] 1. Beach camera video data collection and analysis

[0736] 2. Providing wave information and wave forecasts for the next week

[0737] 3. User surfing video analysis and feedback

[0738] 4. Surfboard Recommendations

[0739] 5. Emotion engine recognizes user emotions and optimizes feedback and recommendations

[0740] Beach camera video data collection and analysis

[0741] The server collects video data from cameras installed on each beach. The cameras stream the video in real time, and the video data is stored on the server for analysis.

[0742] Examples:

[0743] The server captures video data from the cameras at Shonan Beach at 30 frames per second and saves the data as a file every five minutes.

[0744] The saved video file is analyzed by the generation AI, and information such as wave height of 1.5m, period of 8 seconds, and direction to the southeast is obtained.

[0745] Providing wave information and one-week wave forecasts

[0746] The server uses AI to analyze wave conditions from the collected video data and provides this information to the user's device in real time. It also combines weather data and past wave condition data to use AI to predict wave conditions for the next week and provides this forecast data to the user.

[0747] Examples:

[0748] The server obtains data such as wind speed and temperature in real time from weather data providers, integrates it with wave data from the past year, and runs it through AI.

[0749] The generating AI analyzes the data and determines that "the predicted wave height at Shonan Beach next week will be between 1.2m and 1.7m," and notifies the user.

[0750] User surfing video analysis and feedback

[0751] When a user uploads a surfing video from their device, the server receives the video and sends it to the generation AI, which analyzes the user's riding form and technique and generates technical feedback, which is displayed on the user's device.

[0752] Examples:

[0753] Users upload their surfing videos to the server via the application.

[0754] The server runs the video through AI, which generates feedback such as "Be conscious of your forward-leaning posture," which is displayed on the user's device.

[0755] Surfboard Recommendations

[0756] Based on information such as the user's body type, skill level, and wave conditions, the server uses generative AI to analyze and recommend the most suitable surfboard. This recommendation is also sent to the user's device.

[0757] Examples:

[0758] Users enter their height, weight and intermediate surfing skills into the app.

[0759] Based on this information, the server recommends a "shortboard, 6.0 feet long" and displays it on the app.

[0760] Emotion recognition and feedback / recommendation optimization using an emotion engine

[0761] The system uses an emotion engine to specifically recognize the user's emotions and provide feedback and surfboard recommendations based on those emotions.

[0762] Examples:

[0763] When a user uploads a surfing video, the emotion engine analyzes their facial expressions and voice to recognize their emotional state. For example, if the user is dissatisfied, the feedback will include encouragement such as, "Next time, try to focus on your hip movement to improve your form."

[0764] Using an emotion engine, surfboard recommendations are also tailored based on the user's emotions, for example, recommending boards that require more skill if the user is feeling adventurous.

[0765] The above is a concrete example of how to implement the present invention. By linking cameras installed on the beach with generative AI and an emotion engine, this system provides detailed and accurate wave information, individual technical feedback, and recommendations for optimal surfboards based on emotions. This allows surfers to enjoy surfing more comfortably and efficiently.

[0766] The processing flow will be explained below.

[0767] Processing flow of a system that combines emotion engines

[0768] Beach camera video data collection and analysis

[0769] Step 1:

[0770] The server receives video data in real time from cameras installed on the beach, connects to the IP addresses of the cameras, and streams the video using RTSP (Real Time Streaming Protocol).

[0771] Step 2:

[0772] The server saves the streamed video data as a file at regular intervals, typically as a new file every five minutes.

[0773] Step 3:

[0774] The server inputs the saved video files into a generative AI model and analyzes wave conditions such as wave height, period, and direction.

[0775] Step 4:

[0776] The server stores the analysis results of the generative AI model in a database and simultaneously provides them to the user's device in real time.

[0777] Providing wave information and one-week wave forecasts

[0778] Step 1:

[0779] The server uses the weather data provider API to retrieve the latest weather data, including wind speed, wind direction, temperature, and air pressure.

[0780] Step 2:

[0781] The server retrieves historical wave condition data from a database, typically using data from the past year.

[0782] Step 3:

[0783] The server inputs weather data and past wave condition data into the AI ​​generator, which then predicts waves for the next week.

[0784] Step 4:

[0785] The server stores the prediction results from the AI ​​in a database and provides them to the user's device, including the wave height, period, and direction for the next week.

[0786] User surfing video analysis and feedback

[0787] Step 1:

[0788] Users use their devices to upload surfing videos to the application, which then sends the videos to a cloud server.

[0789] Step 2:

[0790] The server receives the uploaded video data and stores it in temporary storage.

[0791] Step 3:

[0792] The server inputs the saved video data into a generation AI that analyzes the user's riding form and technique.

[0793] Step 4:

[0794] The server receives the analysis results from the AI ​​generator and provides them to the user as feedback, such as specific technical guidance such as "Be conscious of your forward-leaning posture."

[0795] Surfboard Recommendations

[0796] Step 1:

[0797] Users enter their body type, skill level, and wave conditions into the application on their device, including their height, weight, and surfing skill level.

[0798] Step 2:

[0799] The server receives the user's input data and sends it to the generation AI, which then uses this data to recommend the best surfboard for them.

[0800] Step 3:

[0801] The server stores the recommendations from the AI ​​in a database and provides them to the user's device. For example, a recommendation might be "shortboard, 6.0 feet long."

[0802] Emotion recognition and feedback / recommendation optimization using an emotion engine

[0803] Step 1:

[0804] When a user uploads a surfing video, the server sends the facial and voice data from the video to the emotion engine.

[0805] Step 2:

[0806] The emotion engine uses facial and voice analysis to identify the user's emotional state (e.g., happy, unhappy, excited, relaxed, etc.).

[0807] Step 3:

[0808] The server inputs the analysis results of the emotion engine into the generative AI, optimizing feedback and surfboard recommendations based on the user's emotions.

[0809] Step 4:

[0810] The server then stores customized feedback and recommendations based on the user's emotions in a database and provides them to the user's device. For example, if the user is feeling adventurous, the server might suggest, "Try a more difficult move next time."

[0811] The above is the specific processing flow of the surfing information provision system that combines an emotion engine. Through this series of steps, users can receive more accurate wave information, appropriate technical feedback, and personalized surfboard recommendations based on their emotions.

[0812] Example 2

[0813] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0814] Conventional surfing support systems provide wave information and feedback to improve surfing technique, but they are unable to provide optimal feedback or recommendations that take into account the emotional state of each individual user. As a result, users only receive uniform information, making it difficult to receive advice tailored to their individual needs. This problem prevents users from receiving appropriate feedback to improve their technique, preventing them from maximizing the enjoyment and efficiency of surfing.

[0815] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting video data from a video collection device installed on the beach, means using a generation AI to analyze wave height, period, and direction based on the collected video data, means for providing the analyzed wave information to the user's device in real time, means for collecting weather data and past wave condition data, means using a generation AI to combine this data to predict waves one week in advance, means for providing the predicted wave information to the user's device, means using a generation AI to analyze surfing videos uploaded by the user and generate riding form and technical feedback, means for providing the analysis results to the user's device, means using a generation AI to recommend the optimal surfboard based on the user's body type, skill, and wave conditions, means for providing the recommendation results to the user's device, and means using an emotion recognition engine to analyze the user's emotional state and provide feedback and recommendations based on the emotions. This allows users to receive optimal feedback and recommendations based on their individual needs and emotional state.

[0816] "Beach-mounted video collection devices" refers to video capture devices such as cameras and drones placed around the beach for surfboard practice and wave monitoring.

[0817] "Video data" refers to video data that records and stores the movement of waves on the beach and surfing techniques in real time.

[0818] "Collection means" refers to the function of acquiring data from video collection devices installed on the beach and sending it to a server.

[0819] "Generative AI" is a type of artificial intelligence that uses collected data to analyze and predict wave information and surfing techniques.

[0820] "Analysis means" refers to the ability to use generative AI to analyze collected video data and other data to derive wave height, period, direction, technical advice, etc.

[0821] "Real-time provision means" refers to the function of displaying analyzed information on the user's device without delay.

[0822] "Weather data" refers to numerical data related to weather, such as wind speed, temperature, and air pressure.

[0823] "Wave condition data" refers to information that records past wave height, period, direction, and other data.

[0824] "Wave prediction means" refers to a function that uses AI generation based on weather data and wave condition data to predict future wave conditions.

[0825] "Surfing videos" refer to video files that users have filmed and saved of their surfing skills.

[0826] "Technical feedback" refers to advice and suggestions for technical improvement obtained by the generative AI analyzing surfing videos.

[0827] "Device" refers to a device such as a smartphone, tablet or computer that a surfer uses to receive information such as wave information, technical feedback and surfboard recommendations.

[0828] "Body type" refers to a user's physical characteristics such as height and weight.

[0829] "Skill" refers to the user's level of surfing skill and experience.

[0830] A "surfboard" refers to the board used for gliding when surfing.

[0831] "Recommendation method" refers to a function that uses generative AI to select and suggest the surfboard that is best suited to the user's body type, skill level, and wave conditions.

[0832] An "emotion recognition engine" refers to an algorithm or program that analyzes a user's facial expressions and voice to recognize their emotional state.

[0833] "Means for providing feedback and recommendations" refers to a feature that uses an emotion recognition engine to provide feedback and surfboard recommendations based on the user's emotional state.

[0834] The present invention is a system that collects video data from a video collection device installed on the beach, analyzes wave conditions and surfing techniques using a generative AI model, and provides optimal feedback and surfboard recommendations based on the user's emotional state. A specific embodiment of this system will be described.

[0835] Beach camera video data collection and analysis

[0836] The server collects video data in real time from video collection devices such as cameras and drones installed on the beach. This video data is saved on the server at regular intervals. For example, video data from a camera installed on Shonan Beach is captured at 30 frames per second and saved in MP4 format every five minutes. The server then inputs this saved video data into a generative AI model to analyze information such as wave height, period, and direction.

[0837] Example prompt sentence:

[0838] "Collect video data from Shonan Beach and analyze the wave height, period, and direction. Example: Wave height 1.5m, period 8 seconds, direction southeast."

[0839] Providing wave information and one-week wave forecasts

[0840] The server uses the generative AI model to analyze wave condition information and provides it to the user's device in real time. It also obtains real-time weather data, such as wind speed, temperature, and air pressure, from weather data providers, and combines it with past wave condition data before inputting it into the generative AI model. It then predicts waves one week in advance and notifies the user's device of the results. For example, it displays a forecast such as "Predicted wave heights at Shonan Beach next week are 1.2m to 1.7m."

[0841] Example prompt sentence:

[0842] "Based on weather data and wave data from the past year, please predict the wave height at Shonan Beach next week. Example: Wave height 1.2m - 1.7m."

[0843] Analysis and feedback of user surfing videos

[0844] Users upload their surfing videos from their devices to a server. The server then inputs the uploaded videos into a generative AI model to analyze the user's riding form and technique. For example, it generates feedback such as "Be conscious of leaning forward" and displays it on the user's device.

[0845] Example prompt sentence:

[0846] "Analyze user surfing videos and provide technical feedback. Example: Be mindful of forward leaning posture."

[0847] Surfboard Recommendations

[0848] Users input their body type (e.g., height 175 cm, weight 70 kg) and surfing skill level (e.g., intermediate) into their device. The server uses this information and wave condition data to recommend the optimal surfboard using a generative AI model. For example, a recommended result such as "shortboard, 6.0 feet long" is generated and displayed on the user's device.

[0849] Example prompt sentence:

[0850] "Recommend a surfboard that is suitable for a surfer who is 175cm tall, weighs 70kg, and has intermediate skill level. Example: shortboard, 6.0 feet long."

[0851] Optimization by Emotion Engine

[0852] The server uses an emotion recognition engine to analyze the surfing video and recognize the user's emotional state. For example, if the user's facial expressions and voice indicate dissatisfaction, the server provides encouraging feedback such as, "To improve your form, try to focus on your hip movement next time." The server also optimizes surfboard recommendations based on the user's emotional state. If the user is feeling challenged, the server recommends a surfboard that requires more advanced skills.

[0853] Example prompt sentence:

[0854] "Analyze emotions from users' surfing videos and provide encouraging feedback if they're not satisfied. For example, improve your form by focusing on your hip movement next time."

[0855] The above is a specific embodiment of the system of the present invention. The system of the present invention allows users to receive more detailed and accurate wave information, individual technical feedback, and even recommendations for the best surfboard based on their emotions, allowing users to maximize their surfing enjoyment.

[0856] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0857] Step 1:

[0858] The server collects video data from video collection devices installed on the beach. The cameras and drones capture video data in real time at 30 frames per second and send the data to the server. The server saves this video data in mp4 format every 5 minutes. The input is real-time video data from the cameras, and the output is a video file saved on the server.

[0859] Step 2:

[0860] The server inputs the saved video data into a generative AI model, which analyzes the wave height, period, and direction. The generative AI model analyzes the video frames and extracts wave characteristics. For example, it obtains data such as a wave height of 1.5m, a period of 8 seconds, and a direction southeast. The input is the saved video file, and the output is the wave information resulting from the analysis.

[0861] Step 3:

[0862] The server provides the analyzed wave information to the user's device in real time. The acquired wave height, period, and direction are displayed on the user's device. The input is the analysis result of the wave information by the generative AI model, and the output is the wave information displayed on the user's device.

[0863] Step 4:

[0864] The server obtains weather data in real time from a weather data provider. Weather data includes wind speed, temperature, and air pressure. This data is combined with past wave condition data and input into a generative AI model to predict waves one week in advance. For example, a prediction result such as "Predicted wave heights at Shonan Beach next week will be 1.2m to 1.7m" can be obtained. The input is real-time weather data and past wave condition data, and the output is a wave prediction result for one week in advance.

[0865] Step 5:

[0866] The server notifies the user's device of the wave prediction results, which are then displayed on the user's device for one week ahead. The input is the wave prediction results from the generative AI model, and the output is the prediction results displayed on the user's device.

[0867] Step 6:

[0868] The user uploads their surfing video from their device to the server. The user selects and submits the video using the application. The input is the user's surfing video file, and the output is the video file stored on the server.

[0869] Step 7:

[0870] The server inputs the uploaded surfing video into a generative AI model, which analyzes the user's riding form and technique. The generative AI model analyzes the video frames to identify the user's form and technical shortcomings. For example, it generates feedback such as, "Try to be more conscious of your forward leaning posture." The input is the user's surfing video file, and the output is technical feedback.

[0871] Step 8:

[0872] The server displays the generated feedback on the user's device, which displays the analysis result feedback. The input is the technical feedback from the generative AI model, and the output is the feedback displayed on the user's device.

[0873] Step 9:

[0874] The user enters their body type (e.g., height 175 cm, weight 70 kg) and surfing skill (e.g., intermediate) into the terminal. The terminal application has a form for entering this information. The input is the user's body type and skill information, and the output is the user information sent to the server.

[0875] Step 10:

[0876] The server uses a generative AI model to recommend the optimal surfboard based on the input user information and wave information. The generative AI model analyzes the user's body type, skill, and wave conditions to select the optimal surfboard. For example, a recommendation result of "shortboard, 6.0 feet long" may be obtained. The input is the user's body type, skill information, and wave information, and the output is the recommended surfboard information.

[0877] Step 11:

[0878] The server displays the recommendation results on the user's device, which then displays information about the recommended surfboards. The input is the surfboard recommendation results from the generative AI model, and the output is the recommendation results displayed on the user's device.

[0879] Step 12:

[0880] When the server analyzes the surfing video, it uses an emotion recognition engine to recognize the user's emotional state. It analyzes the user's facial expressions and voice to recognize the emotional state (e.g., dissatisfied). The input is the user's surfing video, and the output is the recognized emotional state.

[0881] Step 13:

[0882] The server adjusts the feedback and surfboard recommendations based on the recognized emotion. If the user is not satisfied, encouraging feedback such as "Next time, be mindful of your hip movement to improve your form" is generated. The input is the recognized emotional state, and the output is the adjusted feedback and surfboard recommendations.

[0883] Step 14:

[0884] The server displays the adjusted feedback and recommendation on the user's device. The user's device displays the optimal feedback and surfboard recommendation according to the emotion. The input is the adjusted feedback and surfboard recommendation, and the output is the feedback and recommendation displayed on the user's device.

[0885] (Application example 2)

[0886] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0887] While conventional systems provide information about beach waves and feedback to help users improve their surfing skills, they do not consider optimal in-store purchasing support using customer emotion recognition. This makes it difficult to provide individually optimized product recommendations and feedback, and poses the challenge of not improving the customer's purchasing experience.

[0888] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0889] In this invention, the server includes: means for collecting video data from cameras installed on the beach; means using a generation AI to analyze wave height, period, and direction based on the collected video data; means for providing the analyzed wave information to a user's device in real time; means for collecting weather data and past wave condition data; means using a generation AI to combine this data to predict waves one week in advance; means for providing the predicted wave information to a user's device; means using a generation AI to analyze surfing videos uploaded by a user and generate riding form and technical feedback; means for providing the analysis results to a user's device; means using a generation AI to recommend the optimal surfboard based on the user's body type, skill, and wave conditions; means for providing the recommendation results to a user's device; means using a generation AI to link video footage from cameras installed in a store with a customer's mobile app, analyze the customer's facial expressions and behavior in real time, and provide optimal product recommendations based on emotion recognition; means for providing product information and reviews to a customer's device; and means for analyzing the customer's facial expressions and behavior and providing feedback and product recommendations based on their emotional state. This allows customers to receive optimal purchasing support using emotion recognition technology in physical stores, and enables personalized feedback and recommendations.

[0890] A "camera" is a device that captures video in real time and records or transmits it as data.

[0891] "Video data" refers to a series of consecutive image information captured by a camera, and is data that can be viewed as a video.

[0892] "Generative AI" refers to algorithms and models that use artificial intelligence technology to analyze data and generate results.

[0893] "Analysis" is the process of processing and analyzing collected data to extract useful information.

[0894] "Wave height, period, and direction" are indicators of the physical characteristics of ocean waves and are important condition information for surfing.

[0895] "Real-time" refers to the property that data is processed synchronously and provided to users without any time delay.

[0896] "Weather data" refers to any data related to weather, such as weather, temperature, wind speed, etc.

[0897] "Past wave condition data" refers to previously recorded data such as wave height, period, and direction, and is information that indicates past wave conditions.

[0898] "Surfing video" is video data captured by a camera while surfing.

[0899] "Riding form" refers to a surfer's posture and movements while surfing.

[0900] "Technical feedback" is specific advice and instruction to improve surfing technique.

[0901] "User's body type, skill, and wave conditions" refers to information about the user's physical characteristics, surfing skill level, and current wave conditions.

[0902] A "surfboard" is a board used for surfing.

[0903] "Recommendation" refers to suggesting the best option based on a specific situation or condition.

[0904] "Images from cameras installed inside the store" refers to images captured by cameras installed inside the store.

[0905] "Customer mobile app" refers to application software that customers use on mobile devices such as smartphones and tablets.

[0906] "Analyzing facial expressions and behavior in real time" refers to capturing a customer's facial expressions and movements, and instantly processing and analyzing the data.

[0907] "Emotion recognition" is a technology that uses information such as facial expressions and voice to identify a person's emotional state.

[0908] "Product recommendation" refers to presenting appropriate products to customers.

[0909] "Product information" is detailed information about the product's price, specifications, functions, etc.

[0910] A "review" refers to a written opinion, such as an evaluation or impression, about a product or service.

[0911] "Feedback" refers to opinions or advice given in response to a particular action or result.

[0912] This invention provides a system for recognizing customer emotions and optimizing the purchasing experience in brick-and-mortar stores. The system is centered around a generative AI that analyzes customer facial expressions and behavior in real time using images from cameras installed in the store, and recommends products and provides information according to the customer's emotional state.

[0913] The specific means included in the system are as follows:

[0914] In-store cameras and video collection

[0915] The server collects video data in real time from IP cameras installed in the store, making it possible to monitor and record customer behavior and facial expressions.

[0916] Video analysis and emotion recognition

[0917] The server analyzes the collected video data using OpenCV and generative AI, and uses deep learning frameworks such as TensorFlow to analyze the customer's facial expressions and behavior and recognize their emotional state.

[0918] Product recommendations and information

[0919] The server then recommends the most suitable products to the customer based on the results of emotion recognition, and provides recommended product information, reviews, and detailed descriptions to the customer's mobile app, allowing the customer to receive product information tailored to their mood and interests.

[0920] Hardware and software used to process the program

[0921] Hardware:

[0922] IP camera (installed inside the store)

[0923] Server (equipped with a high-performance GPU)

[0924] Smartphone or tablet (customer's mobile device)

[0925] software:

[0926] TensorFlow / PyTorch (training and inferencing generative AI models)

[0927] OpenCV (video data preprocessing and analysis)

[0928] Mobile app (runs on smartphones)

[0929] Specific examples

[0930] 1. Customer emotion recognition and product recommendation:

[0931] The camera captures images of customers inside the store and transmits them to a server in real time.

[0932] The server uses a generative AI model to analyze the customer's facial expression and determine that "this customer is interested."

[0933] The server displays detailed information and reviews of Product A on the customer's smartphone.

[0934] example:

[0935] "You seem curious about this product. Let me offer you a sample or other relevant information."

[0936] "You seem unhappy with this product. Let me recommend a different product or have you check the reviews."

[0937] This allows customers to receive products and information that match their emotional state, improving their shopping experience.

[0938] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0939] Step 1:

[0940] Cameras inside the store capture video data and send it to a server.

[0941] Subject: Camera

[0942] Specific operation: Captured video data is acquired frame by frame and sent to the server using the RTSP protocol.

[0943] Input: Video of the store

[0944] Output: Frame-by-frame video data

[0945] Step 2:

[0946] The server captures the received video data in real time and pre-processes it for analysis.

[0947] Subject: Server

[0948] Specific operation: Video data is acquired frame by frame using OpenCV, and preprocessing such as face detection is performed.

[0949] Input: Frame-by-frame video data

[0950] Output: Preprocessed frame data

[0951] Step 3:

[0952] The server inputs the preprocessed frame data into a generative AI model, which analyzes facial expressions and behavior to recognize the customer's emotional state.

[0953] Subject: Server

[0954] How it works: Using TensorFlow, a generative AI model is run to analyze customer facial expression data and identify emotions.

[0955] Input: Preprocessed frame data

[0956] Output: Customer's emotional state (e.g., interested, satisfied, dissatisfied, etc.)

[0957] Step 4:

[0958] The server selects the most appropriate products and information based on the customer's emotional state, and generates recommended results using generative AI.

[0959] Subject: Server

[0960] How it works: Using a generative AI model, it matches the emotional state with a database of products in the store and selects the appropriate product.

[0961] Input: Customer emotional state, product database

[0962] Output: Recommended product information

[0963] Step 5:

[0964] The server notifies the customer's smartphone of the recommended product information.

[0965] Subject: Server

[0966] What it does: Recommendations are sent to the customer's mobile app via API and displayed within the app.

[0967] Input: Recommended product information

[0968] Output: Notified product information

[0969] Step 6:

[0970] Customers refer to the information displayed on their smartphones to check product details and reviews.

[0971] Subject: customer

[0972] Specific actions: Tap on information displayed in the smartphone app to see more details or read reviews.

[0973] Input: Notified product information

[0974] Output: Customer action (purchase, further enquiry, etc.)

[0975] These steps enable the system to analyze customer sentiment in real time and provide optimal products and information based on the results.

[0976] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0977] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0978] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0979] [Third embodiment]

[0980] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0981] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0982] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0983] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0984] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0985] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0986] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0987] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0988] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0989] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0990] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0991] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0992] The present invention relates to a system for providing wave information and supporting users in improving their surfing skills in the surfing market. Specific processes required to implement this system will now be described.

[0993] Overall overview

[0994] This system has the following main functions:

[0995] 1. Beach camera video data collection and analysis

[0996] 2. Providing wave information and wave forecasts for the next week

[0997] 3. User surfing video analysis and feedback

[0998] 4. Surfboard Recommendations

[0999] Beach camera video data collection and analysis

[1000] The server collects video data from cameras installed on each beach. The cameras stream the video in real time, and the video data is stored on the server for analysis.

[1001] Examples:

[1002] The server captures video data from the cameras at Shonan Beach at 30 frames per second and saves the data as a file every five minutes.

[1003] The saved video file is analyzed by the generation AI, and information such as wave height of 1.5m, period of 8 seconds, and direction to the southeast is obtained.

[1004] Providing wave information and one-week wave forecasts

[1005] The server uses AI to analyze wave conditions from the collected video data and provides this information to the user's device in real time. It also combines weather data and past wave condition data to use AI to predict wave conditions for the next week and provides this forecast data to the user.

[1006] Examples:

[1007] The server obtains data such as wind speed and temperature in real time from weather data providers, integrates it with wave data from the past year, and runs it through AI.

[1008] The generating AI analyzes the data and determines that "the predicted wave height at Shonan Beach next week will be between 1.2m and 1.7m," and notifies the user.

[1009] User surfing video analysis and feedback

[1010] When a user uploads a surfing video from their device, the server receives the video and sends it to the generation AI, which analyzes the user's riding form and technique and generates technical feedback, which is displayed on the user's device.

[1011] Examples:

[1012] Users upload their surfing videos to the server via the application.

[1013] The server runs the video through AI, which generates feedback such as "Be conscious of your forward-leaning posture," which is displayed on the user's device.

[1014] Surfboard Recommendations

[1015] Based on information such as the user's body type, skill level, and wave conditions, the server uses generative AI to analyze and recommend the most suitable surfboard. This recommendation is also sent to the user's device.

[1016] Examples:

[1017] Users enter their height, weight and intermediate surfing skills into the app.

[1018] Based on this information, the server recommends a "shortboard, 6.0 feet long" and displays it on the app.

[1019] The above is a concrete example of how to implement the present invention. By linking cameras installed on the beach with AI generation, this system provides detailed and accurate wave information and personalized technical feedback. It also recommends the optimal surfboard based on the user's unique information, allowing surfers to enjoy surfing more comfortably and efficiently.

[1020] The processing flow will be explained below.

[1021] Beach camera video data collection and analysis

[1022] Step 1:

[1023] The server receives video data in real time from cameras installed on the beach, connects to the IP addresses of the cameras, and streams the video using, for example, RTSP (Real Time Streaming Protocol).

[1024] Step 2:

[1025] The server saves the streamed video data as a file at regular intervals, typically as a new file every five minutes.

[1026] Step 3:

[1027] The server inputs the saved video file into the generative AI model and begins analysis, including wave height, period, and direction.

[1028] Step 4:

[1029] The server stores the analysis results of the generative AI model in a database and simultaneously provides them to the user's device in real time.

[1030] Providing wave information and one-week wave forecasts

[1031] Step 1:

[1032] The server uses the weather data provider API to retrieve the latest weather data, including wind speed, wind direction, temperature, and air pressure.

[1033] Step 2:

[1034] The server retrieves data on past wave conditions from a database, usually covering the past year.

[1035] Step 3:

[1036] The server inputs weather data and past wave condition data into the AI ​​generator, which then predicts waves for the next week.

[1037] Step 4:

[1038] The server stores the prediction results from the AI ​​in a database and provides them to the user's device, including the wave height, period, and direction for the next week.

[1039] User surfing video analysis and feedback

[1040] Step 1:

[1041] Users use their devices to upload surfing videos to the application, which then sends the videos to a cloud server.

[1042] Step 2:

[1043] The server receives the uploaded video data and stores it in temporary storage.

[1044] Step 3:

[1045] The server inputs the saved video data into a generation AI that analyzes the user's riding form and technique.

[1046] Step 4:

[1047] The server receives the analysis results from the AI ​​generator and provides them to the user as feedback, such as specific technical guidance such as "Be conscious of your forward-leaning posture."

[1048] Surfboard Recommendations

[1049] Step 1:

[1050] Users enter their body type, skill level, and wave conditions into the application on their device, including their height, weight, and surfing skill level.

[1051] Step 2:

[1052] The server receives the user's input data and sends it to the generation AI, which then uses this data to recommend the best surfboard for them.

[1053] Step 3:

[1054] The server stores the recommendations from the AI ​​in a database and provides them to the user's device. For example, a recommendation might be "shortboard, 6.0 feet long."

[1055] The above is the specific processing flow of the system that provides wave information and improves user skills in the surfing market. This series of steps enables users to obtain more accurate information and appropriate feedback, resulting in an improved surfing experience.

[1056] Example 1

[1057] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1058] In today's surfing world, it is difficult to obtain accurate wave information in real time or to efficiently support the improvement of individual surfing techniques. Furthermore, there are insufficient methods for users to select the most suitable surfing equipment. For this reason, there is a need for specific support to help users enjoy surfing more and improve their techniques.

[1059] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1060] In this invention, the server includes: means for collecting video data from a video collection device; means using a generation AI to analyze wave height, period, and direction based on the collected video data; means for providing the analyzed wave information to a user's device in real time; means for collecting weather data and past wave condition data; means using a generation AI to combine this data to predict waves one week in advance; means for providing the predicted wave information to the user's device; means using a generation AI to analyze surfing videos uploaded by users and generate riding form and technical feedback; means for providing the analysis results to the user's device; means using a generation AI to recommend optimal surfing equipment based on the user's body type, skill, and wave conditions; and means for providing the recommendation results to the user's device. This allows users to obtain detailed and accurate wave information in real time, improving their surfing skills and selecting optimal surfing equipment.

[1061] A "video collection device" is a device that is installed on a beach or other location and captures video data in real time.

[1062] "Video data" refers to video information acquired by a video collection device.

[1063] "Generative AI" is an artificial intelligence model that analyzes and generates specific patterns and information using large amounts of video and other data.

[1064] "Wave information" is information that indicates specific parameters and conditions related to surfing, such as wave height, period, and direction.

[1065] "User device" refers to an electronic device used by a user, such as a smartphone, tablet, or PC.

[1066] "Weather data" refers to information about weather conditions such as wind speed, temperature, and precipitation.

[1067] "Past wave condition data" refers to data such as past wave height, period, and direction.

[1068] "Surfing video" is video data taken by a user while surfing.

[1069] "Riding form" refers to the user's posture and movements while surfing.

[1070] "Technical feedback" is specific instruction or advice aimed at improving surfing technique.

[1071] "Surfing equipment" means equipment or tools used in surfing, such as a surfboard.

[1072] "Recommended results" are recommendations for optimal surfing equipment calculated by the generating AI based on the user's specific conditions.

[1073] This invention relates to a system for providing wave information to the surfing market and supporting users in improving their technique. To implement this system, the following specific hardware and software are used, and how data is processed and calculated will be described.

[1074] Hardware and software used

[1075] 1. Video collection device: Camera equipment installed on the beach. For example, a high-resolution IP camera is used. This allows for real-time streaming video.

[1076] 2. Server: A central system for collecting, storing, and analyzing data. It is equipped with a high-performance server processor and large-capacity storage.

[1077] 3. Generative AI models: Artificial intelligence for analyzing video and other data, specifically using deep learning frameworks (e.g., TensorFlow and PyTorch).

[1078] 4. User devices: smartphones, tablets, computers, etc., allowing users to receive information in real time and operate and input data through the interface.

[1079] Overview of program processing

[1080] Beach camera video data collection and analysis

[1081] The server collects video data from cameras installed on the beach. The cameras stream in real time, and the video data is stored on the server for analysis. The server acquires video data from the high-resolution IP cameras at 30 frames per second and saves it as an MP4 file every five minutes.

[1082] Providing wave information and one-week wave forecasts

[1083] The server uses a generative AI model to analyze wave conditions from the collected video data and provides this information to the user's device in real time. The server also obtains weather data such as wind speed and temperature from a weather data provider and combines it with past wave condition data to generate a wave forecast for the next week. This forecast data is also analyzed using generative AI and provided to the user.

[1084] User surfing video analysis and feedback

[1085] When a user uploads a surfing video from their device, the server receives the video and sends it to a generative AI model. The generative AI analyzes the user's riding form and technique and generates technical feedback. This feedback is displayed on the user's device. Users upload surfing videos using an application, and the server sends the video to the generative AI, which generates feedback such as "Be conscious of your forward leaning posture."

[1086] Surfboard Recommendations

[1087] Based on information such as the user's body type, skill level, and wave conditions, the server uses generative AI to analyze and recommend the most suitable surfboard. This recommendation result is also provided to the user's device. For example, if a user enters their height of 175 cm, weight of 70 kg, and intermediate skill level into the app, the server will recommend a "shortboard, 6.0 feet long" and display the result on the user's device.

[1088] Examples of specific examples and prompts

[1089] Examples:

[1090] The server collects video data from cameras at Shonan Beach and analyzes it using generative AI to obtain information such as wave height of 1.5m, wave period of 8 seconds, and direction to the southeast.

[1091] The server uses weather data to predict waves for the next week and notifies the user that "wave heights predicted for Shonan Beach next week are between 1.2m and 1.7m."

[1092] Users upload their surfing videos to a server and receive feedback such as "Improve your popping timing."

[1093] Based on the information entered by the user, the server recommends a "6.0 foot shortboard."

[1094] Example prompt sentence:

[1095] "Please provide current and forecast wave height data for Shonan Beach."

[1096] "Analyze my surfing video and give me feedback"

[1097] "Recommend a surfboard that suits your body type and skill level."

[1098] The above is a specific embodiment for carrying out the invention. This system allows users to obtain detailed and accurate wave information in real time, enabling them to improve their surfing skills and select the most suitable surfing equipment.

[1099] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1100] Step 1:

[1101] The server collects video data from video collection devices installed on the beach. The camera devices stream video in real time and send the data to the server. The server saves this data as MP4 files at regular intervals (e.g., every 5 minutes).

[1102] Specific behavior:

[1103] The server receives video data from the Shonan Beach camera in real time at 30 frames per second and saves it as one MP4 file every five minutes.

[1104] Input: Video data (real-time video stream)

[1105] Output: MP4 file (video data saved every 5 minutes)

[1106] Step 2:

[1107] The server inputs the stored video data into a generative AI model to analyze wave height, period, and direction, and the generative AI model uses a deep learning algorithm to extract wave characteristics.

[1108] Specific behavior:

[1109] The server sends the saved MP4 file to the generation AI, obtaining information such as wave height of 1.5m, period of 8 seconds, and direction southeast.

[1110] Input: Saved MP4 file

[1111] Output: Wave characteristics data (wave height, period, direction)

[1112] Step 3:

[1113] The server transmits the analyzed wave information in real time to the user's device, which displays this information.

[1114] Specific behavior:

[1115] The server sends wave characteristic data to the user's smartphone via push notification, displaying "Current wave height: 1.5m, direction: southeast."

[1116] Input: Wave property data

[1117] Output: Push notification (wave height, period, direction information)

[1118] Step 4:

[1119] The server retrieves weather data from a weather data provider. This data includes wind speed, temperature, etc. This is combined with historical wave condition data to generate a week-ahead wave forecast. The data is then analyzed using a generative AI model.

[1120] Specific behavior:

[1121] The server combines wind speed, temperature, and wave data from the past year, and uses generative AI to analyze the predicted wave height at Shonan Beach next week as 1.2m to 1.7m.

[1122] Input: Weather data, historical wave condition data

[1123] Output: Predicted wave information (future wave height, period, direction)

[1124] Step 5:

[1125] Users upload surfing videos from their devices to a server, which then inputs the videos into a generative AI model to analyze riding form and technique, and the generative AI then generates technical feedback.

[1126] Specific behavior:

[1127] Users use the application to upload surfing videos to a server, which then sends the videos to a generating AI, which then displays the videos on the user's device, providing feedback such as "be mindful of your forward leaning posture."

[1128] Input: surfing video

[1129] Output: Technical feedback (specific improvements)

[1130] Step 6:

[1131] The server uses the generative AI model to send the analysis results to the user's device, where technical feedback based on the analysis results is displayed.

[1132] Specific behavior:

[1133] The server sends the analysis results to the user's device as a push notification, displaying feedback such as "Improve your popping timing."

[1134] Input: Analysis results

[1135] Output: Push notification (technical feedback)

[1136] Step 7:

[1137] Based on the user's body type, skill level, and wave conditions, the server uses a generative AI model to recommend the optimal surfing equipment, which is then delivered to the user's device.

[1138] Specific behavior:

[1139] When a user inputs their height of 175 cm, weight of 70 kg, and intermediate skill level, the server uses generative AI to recommend a "shortboard, 6.0 feet long" and displays the results on the user's device.

[1140] Input: User's body type, skill, and wave conditions

[1141] Output: Recommended results (optimal surfing equipment)

[1142] (Application example 1)

[1143] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1144] Monitoring and optimization of manufacturing processes in conventional factories often relied on manual work or partial automation, making it difficult to improve efficiency in real time or provide flexible feedback. This resulted in problems such as the generation of defective products during the manufacturing process and delays in identifying and resolving bottlenecks throughout the line, resulting in a decline in overall productivity. Furthermore, line stoppages due to sudden equipment failures or lack of maintenance also had a negative impact on the production process. A system that could solve these issues and improve the efficiency of the entire manufacturing line was needed.

[1145] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1146] In this invention, the server includes means for collecting video data from cameras installed on the factory production line, means using a generation AI for analyzing the current state of the production process based on the collected video data, means for providing the analyzed process information to an operator's terminal in real time, means for collecting equipment operation data and past production data, means using a generation AI for combining this data to predict the production process one week ahead, means for providing the predicted process information to the operator's terminal, means using a generation AI for analyzing videos of specific processes uploaded by the operator and generating efficiency improvement feedback, means for providing the analysis results to the operator's terminal, means using a generation AI for recommending maintenance based on the equipment's operating status and process data, and means for providing the recommendation results to the operator's terminal.

[1147] This enables real-time monitoring of the operating status of the entire manufacturing line, enabling immediate process improvements and efficiency improvements, early identification of bottlenecks, and the provision of appropriate maintenance forecasts.

[1148] A "production line" refers to a series of processes and machinery that continuously processes and assembles products within a factory.

[1149] A "camera" is a device that captures video or images, and generally includes digital cameras and surveillance cameras.

[1150] "Video data" refers to data that digitally represents images that change continuously over time.

[1151] "Generative AI" refers to artificial intelligence that uses machine learning algorithms to analyze and make predictions from specific data.

[1152] "Process information" refers to data related to the manufacturing process, including production volume, quality, error rate, etc.

[1153] A "terminal" is a device that is connected to the system and exchanges information, including smartphones and computers.

[1154] "Operational data" refers to data related to the operating state of machines and equipment, including operating time, frequency of use, energy consumption, etc.

[1155] "Past manufacturing data" refers to the accumulation of data related to manufacturing processes that has been collected to date.

[1156] "Maintenance" refers to the maintenance and repair work required to keep machines and equipment operating normally.

[1157] A "bottleneck" is the factor that slows down progress in production or a process and reduces overall efficiency.

[1158] "Feedback" is information or instructions that are based on the results or status of a system or process to encourage appropriate improvements or modifications.

[1159] This invention provides a system for monitoring and optimizing a factory production line. The system is composed of the following hardware and software.

[1160] System hardware configuration

[1161] 1. Camera:

[1162] This is a video capture device installed at various points on the production line, allowing real-time video of the manufacturing process to be captured.

[1163] 2. Server:

[1164] It is a central processing unit for storing and analyzing collected video data. The server also provides functions for analyzing various data and using generative AI models.

[1165] 3. Terminal:

[1166] A device such as a smartphone or computer used by an operator to display feedback from the server and analysis results.

[1167] System software configuration

[1168] 1. Generative AI model:

[1169] It is an artificial intelligence algorithm that analyzes video data and various data related to manufacturing processes. Specifically, it uses machine learning models to optimize processes, detect errors, and recommend maintenance.

[1170] 2. Database:

[1171] The collected data is stored and managed using a DBMS such as SQL.

[1172] 3. Front-end application:

[1173] A user interface is provided to display analysis results and feedback to the operator, which is implemented as a smartphone application (Android / iOS) or a web application.

[1174] Example of operation

[1175] 1. Video Data Collection and Analysis:

[1176] A camera captures the production line and sends the video data to a server. The server inputs this video data into a generative AI model and analyzes the current state of the production process. For example, it can obtain specific analysis results such as "The component placement speed on assembly line 1 is declining."

[1177] 2. Providing process information:

[1178] The analysis results are provided to the operator's terminal in real time. For example, an alert such as "A bottleneck has occurred on the assembly line. Please consider countermeasures" is displayed on the terminal.

[1179] 3. Analysis of operational and historical data:

[1180] The server collects equipment operation data and past production data, and uses a generative AI model to predict the production process one week in advance. Specifically, it provides forecast information such as, "Supply shortages are predicted for next week. Please consider replenishing inventory."

[1181] 4. Feedback Generation:

[1182] When an operator uploads video data of a specific process to the server, the generative AI model analyzes it and provides feedback on how to improve efficiency, such as "Please be careful when installing parts."

[1183] 5. Maintenance Recommendations:

[1184] The server monitors the operating status of the equipment and recommends necessary maintenance. For example, a message such as "An abnormality has been detected in the robot arm on line 4. Inspection is recommended" is displayed on the operator's terminal.

[1185] Examples of prompt statements

[1186] "Analyze the video of the assembly process on production line 1 and identify areas for improvement."

[1187] This system enables real-time monitoring of the operating status of the entire manufacturing line, enabling immediate process improvements and efficiency gains, early identification of bottlenecks, and the provision of appropriate maintenance forecasts.

[1188] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1189] Step 1:

[1190] The server acquires video data in real time from cameras installed on the factory production line. The acquired video data is temporarily stored in the server's storage. The input is video data from the camera, and the output is video data saved in the server's storage. Specifically, the camera continuously captures video and sends the data to the server via an IP network.

[1191] Step 2:

[1192] The server inputs the stored video data into a generative AI model to analyze the current state of the manufacturing process. The generative AI model uses this data to identify problems and areas for improvement on the production line. The input is the temporarily stored video data, and the output is the process information that is the analysis result. Specifically, it uses an image analysis algorithm to detect specific errors and delays.

[1193] Step 3:

[1194] The server notifies the operator's device of the analyzed process information in real time. The input is the analysis results obtained from the generative AI model, and the output is feedback displayed on the operator's device. Specifically, the server uses a notification system to send push notifications to the device.

[1195] Step 4:

[1196] The server collects equipment operation data and past manufacturing data, inputs it into a generative AI model, and predicts future manufacturing processes. The input is equipment operation data and past manufacturing data, and the output is predicted manufacturing process information. Specifically, it retrieves the necessary data from the database via a query and analyzes it using the generative AI model.

[1197] Step 5:

[1198] The server provides predicted manufacturing process information to the operator's device. The input is predicted data from the generative AI model, and the output is predicted information displayed on the operator's device. Specifically, it generates a dashboard that displays the prediction results in a visually easy-to-understand manner.

[1199] Step 6:

[1200] The terminal provides a means for the operator to upload video data of a specific process to the server. The input is the operator's video data, and the output is the uploaded video stored on the server. Specifically, the terminal provides file selection and upload functions from the user interface.

[1201] Step 7:

[1202] The server inputs the uploaded video data of a specific process into a generative AI model to generate efficiency improvement feedback. The input is the uploaded video, and the output is the efficiency improvement feedback. Specifically, the video analysis algorithm is used to identify areas for efficiency improvement and generate feedback in text format.

[1203] Step 8:

[1204] The server provides feedback to the operator's device. The input is feedback data from the generative AI model, and the output is feedback displayed on the operator's device. Specifically, the feedback data is sent to the device in real time and displayed on the user interface.

[1205] Step 9:

[1206] The server monitors the operating status of the equipment and makes necessary maintenance recommendations based on the generative AI model. The input is the equipment's operating data and the output is the maintenance recommendation. Specifically, it collects operating data in real time and automatically generates a maintenance notification if an abnormality is detected.

[1207] Step 10:

[1208] The server provides maintenance recommendations to the operator's device. The input is maintenance recommendation data from the generative AI model, and the output is a maintenance notification displayed on the operator's device. Specifically, the server uses a notification system to push the recommendations to the device.

[1209] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1210] This invention relates to a system that provides wave information and helps users improve their surfing skills in the surfing market, as well as a system that provides more personalized and optimal feedback and recommendations by using user emotion recognition. The specific processing required to implement this system is described below.

[1211] Overall overview

[1212] This system has the following main functions:

[1213] 1. Beach camera video data collection and analysis

[1214] 2. Providing wave information and wave forecasts for the next week

[1215] 3. User surfing video analysis and feedback

[1216] 4. Surfboard Recommendations

[1217] 5. Emotion engine recognizes user emotions and optimizes feedback and recommendations

[1218] Beach camera video data collection and analysis

[1219] The server collects video data from cameras installed on each beach. The cameras stream the video in real time, and the video data is stored on the server for analysis.

[1220] Examples:

[1221] The server captures video data from the cameras at Shonan Beach at 30 frames per second and saves the data as a file every five minutes.

[1222] The saved video file is analyzed by the generation AI, and information such as wave height of 1.5m, period of 8 seconds, and direction to the southeast is obtained.

[1223] Providing wave information and one-week wave forecasts

[1224] The server uses AI to analyze wave conditions from the collected video data and provides this information to the user's device in real time. It also combines weather data and past wave condition data to use AI to predict wave conditions for the next week and provides this forecast data to the user.

[1225] Examples:

[1226] The server obtains data such as wind speed and temperature in real time from weather data providers, integrates it with wave data from the past year, and runs it through AI.

[1227] The generating AI analyzes the data and determines that "the predicted wave height at Shonan Beach next week will be between 1.2m and 1.7m," and notifies the user.

[1228] User surfing video analysis and feedback

[1229] When a user uploads a surfing video from their device, the server receives the video and sends it to the generation AI, which analyzes the user's riding form and technique and generates technical feedback, which is displayed on the user's device.

[1230] Examples:

[1231] Users upload their surfing videos to the server via the application.

[1232] The server runs the video through AI, which generates feedback such as "Be conscious of your forward-leaning posture," which is displayed on the user's device.

[1233] Surfboard Recommendations

[1234] Based on information such as the user's body type, skill level, and wave conditions, the server uses generative AI to analyze and recommend the most suitable surfboard. This recommendation is also sent to the user's device.

[1235] Examples:

[1236] Users enter their height, weight and intermediate surfing skills into the app.

[1237] Based on this information, the server recommends a "shortboard, 6.0 feet long" and displays it on the app.

[1238] Emotion recognition and feedback / recommendation optimization using an emotion engine

[1239] The system uses an emotion engine to specifically recognize the user's emotions and provide feedback and surfboard recommendations based on those emotions.

[1240] Examples:

[1241] When a user uploads a surfing video, the emotion engine analyzes their facial expressions and voice to recognize their emotional state. For example, if the user is dissatisfied, the feedback will include encouragement such as, "Next time, try to focus on your hip movement to improve your form."

[1242] Using an emotion engine, surfboard recommendations are also tailored based on the user's emotions, for example, recommending boards that require more skill if the user is feeling adventurous.

[1243] The above is a concrete example of how to implement the present invention. By linking cameras installed on the beach with generative AI and an emotion engine, this system provides detailed and accurate wave information, individual technical feedback, and recommendations for optimal surfboards based on emotions. This allows surfers to enjoy surfing more comfortably and efficiently.

[1244] The processing flow will be explained below.

[1245] Processing flow of a system that combines emotion engines

[1246] Beach camera video data collection and analysis

[1247] Step 1:

[1248] The server receives video data in real time from cameras installed on the beach, connects to the IP addresses of the cameras, and streams the video using RTSP (Real Time Streaming Protocol).

[1249] Step 2:

[1250] The server saves the streamed video data as a file at regular intervals, typically as a new file every five minutes.

[1251] Step 3:

[1252] The server inputs the saved video files into a generative AI model and analyzes wave conditions such as wave height, period, and direction.

[1253] Step 4:

[1254] The server stores the analysis results of the generative AI model in a database and simultaneously provides them to the user's device in real time.

[1255] Providing wave information and one-week wave forecasts

[1256] Step 1:

[1257] The server uses the weather data provider API to retrieve the latest weather data, including wind speed, wind direction, temperature, and air pressure.

[1258] Step 2:

[1259] The server retrieves historical wave condition data from a database, typically using data from the past year.

[1260] Step 3:

[1261] The server inputs weather data and past wave condition data into the AI ​​generator, which then predicts waves for the next week.

[1262] Step 4:

[1263] The server stores the prediction results from the AI ​​in a database and provides them to the user's device, including the wave height, period, and direction for the next week.

[1264] User surfing video analysis and feedback

[1265] Step 1:

[1266] Users use their devices to upload surfing videos to the application, which then sends the videos to a cloud server.

[1267] Step 2:

[1268] The server receives the uploaded video data and stores it in temporary storage.

[1269] Step 3:

[1270] The server inputs the saved video data into a generation AI that analyzes the user's riding form and technique.

[1271] Step 4:

[1272] The server receives the analysis results from the AI ​​generator and provides them to the user as feedback, such as specific technical guidance such as "Be conscious of your forward-leaning posture."

[1273] Surfboard Recommendations

[1274] Step 1:

[1275] Users enter their body type, skill level, and wave conditions into the application on their device, including their height, weight, and surfing skill level.

[1276] Step 2:

[1277] The server receives the user's input data and sends it to the generation AI, which then uses this data to recommend the best surfboard for them.

[1278] Step 3:

[1279] The server stores the recommendations from the AI ​​in a database and provides them to the user's device. For example, a recommendation might be "shortboard, 6.0 feet long."

[1280] Emotion recognition and feedback / recommendation optimization using an emotion engine

[1281] Step 1:

[1282] When a user uploads a surfing video, the server sends the facial and voice data from the video to the emotion engine.

[1283] Step 2:

[1284] The emotion engine uses facial and voice analysis to identify the user's emotional state (e.g., happy, unhappy, excited, relaxed, etc.).

[1285] Step 3:

[1286] The server inputs the analysis results of the emotion engine into the generative AI, optimizing feedback and surfboard recommendations based on the user's emotions.

[1287] Step 4:

[1288] The server then stores customized feedback and recommendations based on the user's emotions in a database and provides them to the user's device. For example, if the user is feeling adventurous, the server might suggest, "Try a more difficult move next time."

[1289] The above is the specific processing flow of the surfing information provision system that combines an emotion engine. Through this series of steps, users can receive more accurate wave information, appropriate technical feedback, and personalized surfboard recommendations based on their emotions.

[1290] Example 2

[1291] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1292] Conventional surfing support systems provide wave information and feedback to improve surfing technique, but they are unable to provide optimal feedback or recommendations that take into account the emotional state of each individual user. As a result, users only receive uniform information, making it difficult to receive advice tailored to their individual needs. This problem prevents users from receiving appropriate feedback to improve their technique, preventing them from maximizing the enjoyment and efficiency of surfing.

[1293] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting video data from a video collection device installed on the beach, means using a generation AI to analyze wave height, period, and direction based on the collected video data, means for providing the analyzed wave information to the user's device in real time, means for collecting weather data and past wave condition data, means using a generation AI to combine this data to predict waves one week in advance, means for providing the predicted wave information to the user's device, means using a generation AI to analyze surfing videos uploaded by the user and generate riding form and technical feedback, means for providing the analysis results to the user's device, means using a generation AI to recommend the optimal surfboard based on the user's body type, skill, and wave conditions, means for providing the recommendation results to the user's device, and means using an emotion recognition engine to analyze the user's emotional state and provide feedback and recommendations based on the emotions. This allows users to receive optimal feedback and recommendations based on their individual needs and emotional state.

[1294] "Beach-mounted video collection devices" refers to video capture devices such as cameras and drones placed around the beach for surfboard practice and wave monitoring.

[1295] "Video data" refers to video data that records and stores the movement of waves on the beach and surfing techniques in real time.

[1296] "Collection means" refers to the function of acquiring data from video collection devices installed on the beach and sending it to a server.

[1297] "Generative AI" is a type of artificial intelligence that uses collected data to analyze and predict wave information and surfing techniques.

[1298] "Analysis means" refers to the ability to use generative AI to analyze collected video data and other data to derive wave height, period, direction, technical advice, etc.

[1299] "Real-time provision means" refers to the function of displaying analyzed information on the user's device without delay.

[1300] "Weather data" refers to numerical data related to weather, such as wind speed, temperature, and air pressure.

[1301] "Wave condition data" refers to information that records past wave height, period, direction, and other data.

[1302] "Wave prediction means" refers to a function that uses AI generation based on weather data and wave condition data to predict future wave conditions.

[1303] "Surfing videos" refer to video files that users have filmed and saved of their surfing skills.

[1304] "Technical feedback" refers to advice and suggestions for technical improvement obtained by the generative AI analyzing surfing videos.

[1305] "Device" refers to a device such as a smartphone, tablet or computer that a surfer uses to receive information such as wave information, technical feedback and surfboard recommendations.

[1306] "Body type" refers to a user's physical characteristics such as height and weight.

[1307] "Skill" refers to the user's level of surfing skill and experience.

[1308] A "surfboard" refers to the board used for gliding when surfing.

[1309] "Recommendation method" refers to a function that uses generative AI to select and suggest the surfboard that is best suited to the user's body type, skill level, and wave conditions.

[1310] An "emotion recognition engine" refers to an algorithm or program that analyzes a user's facial expressions and voice to recognize their emotional state.

[1311] "Means for providing feedback and recommendations" refers to a feature that uses an emotion recognition engine to provide feedback and surfboard recommendations based on the user's emotional state.

[1312] The present invention is a system that collects video data from a video collection device installed on the beach, analyzes wave conditions and surfing techniques using a generative AI model, and provides optimal feedback and surfboard recommendations based on the user's emotional state. A specific embodiment of this system will be described.

[1313] Beach camera video data collection and analysis

[1314] The server collects video data in real time from video collection devices such as cameras and drones installed on the beach. This video data is saved on the server at regular intervals. For example, video data from a camera installed on Shonan Beach is captured at 30 frames per second and saved in MP4 format every five minutes. The server then inputs this saved video data into a generative AI model to analyze information such as wave height, period, and direction.

[1315] Example prompt sentence:

[1316] "Collect video data from Shonan Beach and analyze the wave height, period, and direction. Example: Wave height 1.5m, period 8 seconds, direction southeast."

[1317] Providing wave information and one-week wave forecasts

[1318] The server uses the generative AI model to analyze wave condition information and provides it to the user's device in real time. It also obtains real-time weather data, such as wind speed, temperature, and air pressure, from weather data providers, and combines it with past wave condition data before inputting it into the generative AI model. It then predicts waves one week in advance and notifies the user's device of the results. For example, it displays a forecast such as "Predicted wave heights at Shonan Beach next week are 1.2m to 1.7m."

[1319] Example prompt sentence:

[1320] "Based on weather data and wave data from the past year, please predict the wave height at Shonan Beach next week. Example: Wave height 1.2m - 1.7m."

[1321] Analysis and feedback of user surfing videos

[1322] Users upload their surfing videos from their devices to a server. The server then inputs the uploaded videos into a generative AI model to analyze the user's riding form and technique. For example, it generates feedback such as "Be conscious of leaning forward" and displays it on the user's device.

[1323] Example prompt sentence:

[1324] "Analyze user surfing videos and provide technical feedback. Example: Be mindful of forward leaning posture."

[1325] Surfboard Recommendations

[1326] Users input their body type (e.g., height 175 cm, weight 70 kg) and surfing skill level (e.g., intermediate) into their device. The server uses this information and wave condition data to recommend the optimal surfboard using a generative AI model. For example, a recommended result such as "shortboard, 6.0 feet long" is generated and displayed on the user's device.

[1327] Example prompt sentence:

[1328] "Recommend a surfboard that is suitable for a surfer who is 175cm tall, weighs 70kg, and has intermediate skill level. Example: shortboard, 6.0 feet long."

[1329] Optimization by Emotion Engine

[1330] The server uses an emotion recognition engine to analyze the surfing video and recognize the user's emotional state. For example, if the user's facial expressions and voice indicate dissatisfaction, the server provides encouraging feedback such as, "To improve your form, try to focus on your hip movement next time." The server also optimizes surfboard recommendations based on the user's emotional state. If the user is feeling challenged, the server recommends a surfboard that requires more advanced skills.

[1331] Example prompt sentence:

[1332] "Analyze emotions from users' surfing videos and provide encouraging feedback if they're not satisfied. For example, improve your form by focusing on your hip movement next time."

[1333] The above is a specific embodiment of the system of the present invention. The system of the present invention allows users to receive more detailed and accurate wave information, individual technical feedback, and even recommendations for the best surfboard based on their emotions, allowing users to maximize their surfing enjoyment.

[1334] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1335] Step 1:

[1336] The server collects video data from video collection devices installed on the beach. The cameras and drones capture video data in real time at 30 frames per second and send the data to the server. The server saves this video data in mp4 format every 5 minutes. The input is real-time video data from the cameras, and the output is a video file saved on the server.

[1337] Step 2:

[1338] The server inputs the saved video data into a generative AI model, which analyzes the wave height, period, and direction. The generative AI model analyzes the video frames and extracts wave characteristics. For example, it obtains data such as a wave height of 1.5m, a period of 8 seconds, and a direction southeast. The input is the saved video file, and the output is the wave information resulting from the analysis.

[1339] Step 3:

[1340] The server provides the analyzed wave information to the user's device in real time. The acquired wave height, period, and direction are displayed on the user's device. The input is the analysis result of the wave information by the generative AI model, and the output is the wave information displayed on the user's device.

[1341] Step 4:

[1342] The server obtains weather data in real time from a weather data provider. Weather data includes wind speed, temperature, and air pressure. This data is combined with past wave condition data and input into a generative AI model to predict waves one week in advance. For example, a prediction result such as "Predicted wave heights at Shonan Beach next week will be 1.2m to 1.7m" can be obtained. The input is real-time weather data and past wave condition data, and the output is a wave prediction result for one week in advance.

[1343] Step 5:

[1344] The server notifies the user's device of the wave prediction results, which are then displayed on the user's device for one week ahead. The input is the wave prediction results from the generative AI model, and the output is the prediction results displayed on the user's device.

[1345] Step 6:

[1346] The user uploads their surfing video from their device to the server. The user selects and submits the video using the application. The input is the user's surfing video file, and the output is the video file stored on the server.

[1347] Step 7:

[1348] The server inputs the uploaded surfing video into a generative AI model, which analyzes the user's riding form and technique. The generative AI model analyzes the video frames to identify the user's form and technical shortcomings. For example, it generates feedback such as, "Try to be more conscious of your forward leaning posture." The input is the user's surfing video file, and the output is technical feedback.

[1349] Step 8:

[1350] The server displays the generated feedback on the user's device, which displays the analysis result feedback. The input is the technical feedback from the generative AI model, and the output is the feedback displayed on the user's device.

[1351] Step 9:

[1352] The user enters their body type (e.g., height 175 cm, weight 70 kg) and surfing skill (e.g., intermediate) into the terminal. The terminal application has a form for entering this information. The input is the user's body type and skill information, and the output is the user information sent to the server.

[1353] Step 10:

[1354] The server uses a generative AI model to recommend the optimal surfboard based on the input user information and wave information. The generative AI model analyzes the user's body type, skill, and wave conditions to select the optimal surfboard. For example, a recommendation result of "shortboard, 6.0 feet long" may be obtained. The input is the user's body type, skill information, and wave information, and the output is the recommended surfboard information.

[1355] Step 11:

[1356] The server displays the recommendation results on the user's device, which then displays information about the recommended surfboards. The input is the surfboard recommendation results from the generative AI model, and the output is the recommendation results displayed on the user's device.

[1357] Step 12:

[1358] When the server analyzes the surfing video, it uses an emotion recognition engine to recognize the user's emotional state. It analyzes the user's facial expressions and voice to recognize the emotional state (e.g., dissatisfied). The input is the user's surfing video, and the output is the recognized emotional state.

[1359] Step 13:

[1360] The server adjusts the feedback and surfboard recommendations based on the recognized emotion. If the user is not satisfied, encouraging feedback such as "Next time, be mindful of your hip movement to improve your form" is generated. The input is the recognized emotional state, and the output is the adjusted feedback and surfboard recommendations.

[1361] Step 14:

[1362] The server displays the adjusted feedback and recommendation on the user's device. The user's device displays the optimal feedback and surfboard recommendation according to the emotion. The input is the adjusted feedback and surfboard recommendation, and the output is the feedback and recommendation displayed on the user's device.

[1363] (Application example 2)

[1364] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1365] While conventional systems provide information about beach waves and feedback to help users improve their surfing skills, they do not consider optimal in-store purchasing support using customer emotion recognition. This makes it difficult to provide individually optimized product recommendations and feedback, and poses the challenge of not improving the customer's purchasing experience.

[1366] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1367] In this invention, the server includes: means for collecting video data from cameras installed on the beach; means using a generation AI to analyze wave height, period, and direction based on the collected video data; means for providing the analyzed wave information to a user's device in real time; means for collecting weather data and past wave condition data; means using a generation AI to combine this data to predict waves one week in advance; means for providing the predicted wave information to a user's device; means using a generation AI to analyze surfing videos uploaded by a user and generate riding form and technical feedback; means for providing the analysis results to a user's device; means using a generation AI to recommend the optimal surfboard based on the user's body type, skill, and wave conditions; means for providing the recommendation results to a user's device; means using a generation AI to link video footage from cameras installed in a store with a customer's mobile app, analyze the customer's facial expressions and behavior in real time, and provide optimal product recommendations based on emotion recognition; means for providing product information and reviews to a customer's device; and means for analyzing the customer's facial expressions and behavior and providing feedback and product recommendations based on their emotional state. This allows customers to receive optimal purchasing support using emotion recognition technology in physical stores, and enables personalized feedback and recommendations.

[1368] A "camera" is a device that captures video in real time and records or transmits it as data.

[1369] "Video data" refers to a series of consecutive image information captured by a camera, and is data that can be viewed as a video.

[1370] "Generative AI" refers to algorithms and models that use artificial intelligence technology to analyze data and generate results.

[1371] "Analysis" is the process of processing and analyzing collected data to extract useful information.

[1372] "Wave height, period, and direction" are indicators of the physical characteristics of ocean waves and are important condition information for surfing.

[1373] "Real-time" refers to the property that data is processed synchronously and provided to users without any time delay.

[1374] "Weather data" refers to any data related to weather, such as weather, temperature, wind speed, etc.

[1375] "Past wave condition data" refers to previously recorded data such as wave height, period, and direction, and is information that indicates past wave conditions.

[1376] "Surfing video" is video data captured by a camera while surfing.

[1377] "Riding form" refers to a surfer's posture and movements while surfing.

[1378] "Technical feedback" is specific advice and instruction to improve surfing technique.

[1379] "User's body type, skill, and wave conditions" refers to information about the user's physical characteristics, surfing skill level, and current wave conditions.

[1380] A "surfboard" is a board used for surfing.

[1381] "Recommendation" refers to suggesting the best option based on a specific situation or condition.

[1382] "Images from cameras installed inside the store" refers to images captured by cameras installed inside the store.

[1383] "Customer mobile app" refers to application software that customers use on mobile devices such as smartphones and tablets.

[1384] "Analyzing facial expressions and behavior in real time" refers to capturing a customer's facial expressions and movements, and instantly processing and analyzing the data.

[1385] "Emotion recognition" is a technology that uses information such as facial expressions and voice to identify a person's emotional state.

[1386] "Product recommendation" refers to presenting appropriate products to customers.

[1387] "Product information" is detailed information about the product's price, specifications, functions, etc.

[1388] A "review" refers to a written opinion, such as an evaluation or impression, about a product or service.

[1389] "Feedback" refers to opinions or advice given in response to a particular action or result.

[1390] This invention provides a system for recognizing customer emotions and optimizing the purchasing experience in brick-and-mortar stores. The system is centered around a generative AI that analyzes customer facial expressions and behavior in real time using images from cameras installed in the store, and recommends products and provides information according to the customer's emotional state.

[1391] The specific means included in the system are as follows:

[1392] In-store cameras and video collection

[1393] The server collects video data in real time from IP cameras installed in the store, making it possible to monitor and record customer behavior and facial expressions.

[1394] Video analysis and emotion recognition

[1395] The server analyzes the collected video data using OpenCV and generative AI, and uses deep learning frameworks such as TensorFlow to analyze the customer's facial expressions and behavior and recognize their emotional state.

[1396] Product recommendations and information

[1397] The server then recommends the most suitable products to the customer based on the results of emotion recognition, and provides recommended product information, reviews, and detailed descriptions to the customer's mobile app, allowing the customer to receive product information tailored to their mood and interests.

[1398] Hardware and software used to process the program

[1399] Hardware:

[1400] IP camera (installed inside the store)

[1401] Server (equipped with a high-performance GPU)

[1402] Smartphone or tablet (customer's mobile device)

[1403] software:

[1404] TensorFlow / PyTorch (training and inferencing generative AI models)

[1405] OpenCV (video data preprocessing and analysis)

[1406] Mobile app (runs on smartphones)

[1407] Specific examples

[1408] 1. Customer emotion recognition and product recommendation:

[1409] The camera captures images of customers inside the store and transmits them to a server in real time.

[1410] The server uses a generative AI model to analyze the customer's facial expression and determine that "this customer is interested."

[1411] The server displays detailed information and reviews of Product A on the customer's smartphone.

[1412] example:

[1413] "You seem curious about this product. Let me offer you a sample or other relevant information."

[1414] "You seem unhappy with this product. Let me recommend a different product or have you check the reviews."

[1415] This allows customers to receive products and information that match their emotional state, improving their shopping experience.

[1416] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1417] Step 1:

[1418] Cameras inside the store capture video data and send it to a server.

[1419] Subject: Camera

[1420] Specific operation: Captured video data is acquired frame by frame and sent to the server using the RTSP protocol.

[1421] Input: Video of the store

[1422] Output: Frame-by-frame video data

[1423] Step 2:

[1424] The server captures the received video data in real time and pre-processes it for analysis.

[1425] Subject: Server

[1426] Specific operation: Video data is acquired frame by frame using OpenCV, and preprocessing such as face detection is performed.

[1427] Input: Frame-by-frame video data

[1428] Output: Preprocessed frame data

[1429] Step 3:

[1430] The server inputs the preprocessed frame data into a generative AI model, which analyzes facial expressions and behavior to recognize the customer's emotional state.

[1431] Subject: Server

[1432] How it works: Using TensorFlow, a generative AI model is run to analyze customer facial expression data and identify emotions.

[1433] Input: Preprocessed frame data

[1434] Output: Customer's emotional state (e.g., interested, satisfied, dissatisfied, etc.)

[1435] Step 4:

[1436] The server selects the most appropriate products and information based on the customer's emotional state, and generates recommended results using generative AI.

[1437] Subject: Server

[1438] How it works: Using a generative AI model, it matches the emotional state with a database of products in the store and selects the appropriate product.

[1439] Input: Customer emotional state, product database

[1440] Output: Recommended product information

[1441] Step 5:

[1442] The server notifies the customer's smartphone of the recommended product information.

[1443] Subject: Server

[1444] What it does: Recommendations are sent to the customer's mobile app via API and displayed within the app.

[1445] Input: Recommended product information

[1446] Output: Notified product information

[1447] Step 6:

[1448] Customers refer to the information displayed on their smartphones to check product details and reviews.

[1449] Subject: customer

[1450] Specific actions: Tap on information displayed in the smartphone app to see more details or read reviews.

[1451] Input: Notified product information

[1452] Output: Customer action (purchase, further enquiry, etc.)

[1453] These steps enable the system to analyze customer sentiment in real time and provide optimal products and information based on the results.

[1454] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1455] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1456] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1457] [Fourth embodiment]

[1458] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1459] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1460] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1461] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1462] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1463] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1464] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1465] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1466] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1467] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1468] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1469] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1470] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1471] The present invention relates to a system for providing wave information and supporting users in improving their surfing skills in the surfing market. Specific processes required to implement this system will now be described.

[1472] Overall overview

[1473] This system has the following main functions:

[1474] 1. Beach camera video data collection and analysis

[1475] 2. Providing wave information and wave forecasts for the next week

[1476] 3. User surfing video analysis and feedback

[1477] 4. Surfboard Recommendations

[1478] Beach camera video data collection and analysis

[1479] The server collects video data from cameras installed on each beach. The cameras stream the video in real time, and the video data is stored on the server for analysis.

[1480] Examples:

[1481] The server captures video data from the cameras at Shonan Beach at 30 frames per second and saves the data as a file every five minutes.

[1482] The saved video file is analyzed by the generation AI, and information such as wave height of 1.5m, period of 8 seconds, and direction to the southeast is obtained.

[1483] Providing wave information and one-week wave forecasts

[1484] The server uses AI to analyze wave conditions from the collected video data and provides this information to the user's device in real time. It also combines weather data and past wave condition data to use AI to predict wave conditions for the next week and provides this forecast data to the user.

[1485] Examples:

[1486] The server obtains data such as wind speed and temperature in real time from weather data providers, integrates it with wave data from the past year, and runs it through AI.

[1487] The generating AI analyzes the data and determines that "the predicted wave height at Shonan Beach next week will be between 1.2m and 1.7m," and notifies the user.

[1488] User surfing video analysis and feedback

[1489] When a user uploads a surfing video from their device, the server receives the video and sends it to the generation AI, which analyzes the user's riding form and technique and generates technical feedback, which is displayed on the user's device.

[1490] Examples:

[1491] Users upload their surfing videos to the server via the application.

[1492] The server runs the video through AI, which generates feedback such as "Be conscious of your forward-leaning posture," which is displayed on the user's device.

[1493] Surfboard Recommendations

[1494] Based on information such as the user's body type, skill level, and wave conditions, the server uses generative AI to analyze and recommend the most suitable surfboard. This recommendation is also sent to the user's device.

[1495] Examples:

[1496] Users enter their height, weight and intermediate surfing skills into the app.

[1497] Based on this information, the server recommends a "shortboard, 6.0 feet long" and displays it on the app.

[1498] The above is a concrete example of how to implement the present invention. By linking cameras installed on the beach with AI generation, this system provides detailed and accurate wave information and personalized technical feedback. It also recommends the optimal surfboard based on the user's unique information, allowing surfers to enjoy surfing more comfortably and efficiently.

[1499] The processing flow will be explained below.

[1500] Beach camera video data collection and analysis

[1501] Step 1:

[1502] The server receives video data in real time from cameras installed on the beach, connects to the IP addresses of the cameras, and streams the video using, for example, RTSP (Real Time Streaming Protocol).

[1503] Step 2:

[1504] The server saves the streamed video data as a file at regular intervals, typically as a new file every five minutes.

[1505] Step 3:

[1506] The server inputs the saved video file into the generative AI model and begins analysis, including wave height, period, and direction.

[1507] Step 4:

[1508] The server stores the analysis results of the generative AI model in a database and simultaneously provides them to the user's device in real time.

[1509] Providing wave information and one-week wave forecasts

[1510] Step 1:

[1511] The server uses the weather data provider API to retrieve the latest weather data, including wind speed, wind direction, temperature, and air pressure.

[1512] Step 2:

[1513] The server retrieves data on past wave conditions from a database, usually covering the past year.

[1514] Step 3:

[1515] The server inputs weather data and past wave condition data into the AI ​​generator, which then predicts waves for the next week.

[1516] Step 4:

[1517] The server stores the prediction results from the AI ​​in a database and provides them to the user's device, including the wave height, period, and direction for the next week.

[1518] User surfing video analysis and feedback

[1519] Step 1:

[1520] Users use their devices to upload surfing videos to the application, which then sends the videos to a cloud server.

[1521] Step 2:

[1522] The server receives the uploaded video data and stores it in temporary storage.

[1523] Step 3:

[1524] The server inputs the saved video data into a generation AI that analyzes the user's riding form and technique.

[1525] Step 4:

[1526] The server receives the analysis results from the AI ​​generator and provides them to the user as feedback, such as specific technical guidance such as "Be conscious of your forward-leaning posture."

[1527] Surfboard Recommendations

[1528] Step 1:

[1529] Users enter their body type, skill level, and wave conditions into the application on their device, including their height, weight, and surfing skill level.

[1530] Step 2:

[1531] The server receives the user's input data and sends it to the generation AI, which then uses this data to recommend the best surfboard for them.

[1532] Step 3:

[1533] The server stores the recommendations from the AI ​​in a database and provides them to the user's device. For example, a recommendation might be "shortboard, 6.0 feet long."

[1534] The above is the specific processing flow of the system that provides wave information and improves user skills in the surfing market. This series of steps enables users to obtain more accurate information and appropriate feedback, resulting in an improved surfing experience.

[1535] Example 1

[1536] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1537] In today's surfing world, it is difficult to obtain accurate wave information in real time or to efficiently support the improvement of individual surfing techniques. Furthermore, there are insufficient methods for users to select the most suitable surfing equipment. For this reason, there is a need for specific support to help users enjoy surfing more and improve their techniques.

[1538] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1539] In this invention, the server includes: means for collecting video data from a video collection device; means using a generation AI to analyze wave height, period, and direction based on the collected video data; means for providing the analyzed wave information to a user's device in real time; means for collecting weather data and past wave condition data; means using a generation AI to combine this data to predict waves one week in advance; means for providing the predicted wave information to the user's device; means using a generation AI to analyze surfing videos uploaded by users and generate riding form and technical feedback; means for providing the analysis results to the user's device; means using a generation AI to recommend optimal surfing equipment based on the user's body type, skill, and wave conditions; and means for providing the recommendation results to the user's device. This allows users to obtain detailed and accurate wave information in real time, improving their surfing skills and selecting optimal surfing equipment.

[1540] A "video collection device" is a device that is installed on a beach or other location and captures video data in real time.

[1541] "Video data" refers to video information acquired by a video collection device.

[1542] "Generative AI" is an artificial intelligence model that analyzes and generates specific patterns and information using large amounts of video and other data.

[1543] "Wave information" is information that indicates specific parameters and conditions related to surfing, such as wave height, period, and direction.

[1544] "User device" refers to an electronic device used by a user, such as a smartphone, tablet, or PC.

[1545] "Weather data" refers to information about weather conditions such as wind speed, temperature, and precipitation.

[1546] "Past wave condition data" refers to data such as past wave height, period, and direction.

[1547] "Surfing video" is video data taken by a user while surfing.

[1548] "Riding form" refers to the user's posture and movements while surfing.

[1549] "Technical feedback" is specific instruction or advice aimed at improving surfing technique.

[1550] "Surfing equipment" means equipment or tools used in surfing, such as a surfboard.

[1551] "Recommended results" are recommendations for optimal surfing equipment calculated by the generating AI based on the user's specific conditions.

[1552] This invention relates to a system for providing wave information to the surfing market and supporting users in improving their technique. To implement this system, the following specific hardware and software are used, and how data is processed and calculated will be described.

[1553] Hardware and software used

[1554] 1. Video collection device: Camera equipment installed on the beach. For example, a high-resolution IP camera is used. This allows for real-time streaming video.

[1555] 2. Server: A central system for collecting, storing, and analyzing data. It is equipped with a high-performance server processor and large-capacity storage.

[1556] 3. Generative AI models: Artificial intelligence for analyzing video and other data, specifically using deep learning frameworks (e.g., TensorFlow and PyTorch).

[1557] 4. User devices: smartphones, tablets, computers, etc., allowing users to receive information in real time and operate and input data through the interface.

[1558] Overview of program processing

[1559] Beach camera video data collection and analysis

[1560] The server collects video data from cameras installed on the beach. The cameras stream in real time, and the video data is stored on the server for analysis. The server acquires video data from the high-resolution IP cameras at 30 frames per second and saves it as an MP4 file every five minutes.

[1561] Providing wave information and one-week wave forecasts

[1562] The server uses a generative AI model to analyze wave conditions from the collected video data and provides this information to the user's device in real time. The server also obtains weather data such as wind speed and temperature from a weather data provider and combines it with past wave condition data to generate a wave forecast for the next week. This forecast data is also analyzed using generative AI and provided to the user.

[1563] User surfing video analysis and feedback

[1564] When a user uploads a surfing video from their device, the server receives the video and sends it to a generative AI model. The generative AI analyzes the user's riding form and technique and generates technical feedback. This feedback is displayed on the user's device. Users upload surfing videos using an application, and the server sends the video to the generative AI, which generates feedback such as "Be conscious of your forward leaning posture."

[1565] Surfboard Recommendations

[1566] Based on information such as the user's body type, skill level, and wave conditions, the server uses generative AI to analyze and recommend the most suitable surfboard. This recommendation result is also provided to the user's device. For example, if a user enters their height of 175 cm, weight of 70 kg, and intermediate skill level into the app, the server will recommend a "shortboard, 6.0 feet long" and display the result on the user's device.

[1567] Examples of specific examples and prompts

[1568] Examples:

[1569] The server collects video data from cameras at Shonan Beach and analyzes it using generative AI to obtain information such as wave height of 1.5m, wave period of 8 seconds, and direction to the southeast.

[1570] The server uses weather data to predict waves for the next week and notifies the user that "wave heights predicted for Shonan Beach next week are between 1.2m and 1.7m."

[1571] Users upload their surfing videos to a server and receive feedback such as "Improve your popping timing."

[1572] Based on the information entered by the user, the server recommends a "6.0 foot shortboard."

[1573] Example prompt sentence:

[1574] "Please provide current and forecast wave height data for Shonan Beach."

[1575] "Analyze my surfing video and give me feedback"

[1576] "Recommend a surfboard that suits your body type and skill level."

[1577] The above is a specific embodiment for carrying out the invention. This system allows users to obtain detailed and accurate wave information in real time, enabling them to improve their surfing skills and select the most suitable surfing equipment.

[1578] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1579] Step 1:

[1580] The server collects video data from video collection devices installed on the beach. The camera devices stream video in real time and send the data to the server. The server saves this data as MP4 files at regular intervals (e.g., every 5 minutes).

[1581] Specific behavior:

[1582] The server receives video data from the Shonan Beach camera in real time at 30 frames per second and saves it as one MP4 file every five minutes.

[1583] Input: Video data (real-time video stream)

[1584] Output: MP4 file (video data saved every 5 minutes)

[1585] Step 2:

[1586] The server inputs the stored video data into a generative AI model to analyze wave height, period, and direction, and the generative AI model uses a deep learning algorithm to extract wave characteristics.

[1587] Specific behavior:

[1588] The server sends the saved MP4 file to the generation AI, obtaining information such as wave height of 1.5m, period of 8 seconds, and direction southeast.

[1589] Input: Saved MP4 file

[1590] Output: Wave characteristics data (wave height, period, direction)

[1591] Step 3:

[1592] The server transmits the analyzed wave information in real time to the user's device, which displays this information.

[1593] Specific behavior:

[1594] The server sends wave characteristic data to the user's smartphone via push notification, displaying "Current wave height: 1.5m, direction: southeast."

[1595] Input: Wave property data

[1596] Output: Push notification (wave height, period, direction information)

[1597] Step 4:

[1598] The server retrieves weather data from a weather data provider. This data includes wind speed, temperature, etc. This is combined with historical wave condition data to generate a week-ahead wave forecast. The data is then analyzed using a generative AI model.

[1599] Specific behavior:

[1600] The server combines wind speed, temperature, and wave data from the past year, and uses generative AI to analyze the predicted wave height at Shonan Beach next week as 1.2m to 1.7m.

[1601] Input: Weather data, historical wave condition data

[1602] Output: Predicted wave information (future wave height, period, direction)

[1603] Step 5:

[1604] Users upload surfing videos from their devices to a server, which then inputs the videos into a generative AI model to analyze riding form and technique, and the generative AI then generates technical feedback.

[1605] Specific behavior:

[1606] Users use the application to upload surfing videos to a server, which then sends the videos to a generating AI, which then displays the videos on the user's device, providing feedback such as "be mindful of your forward leaning posture."

[1607] Input: surfing video

[1608] Output: Technical feedback (specific improvements)

[1609] Step 6:

[1610] The server uses the generative AI model to send the analysis results to the user's device, where technical feedback based on the analysis results is displayed.

[1611] Specific behavior:

[1612] The server sends the analysis results to the user's device as a push notification, displaying feedback such as "Improve your popping timing."

[1613] Input: Analysis results

[1614] Output: Push notification (technical feedback)

[1615] Step 7:

[1616] Based on the user's body type, skill level, and wave conditions, the server uses a generative AI model to recommend the optimal surfing equipment, which is then delivered to the user's device.

[1617] Specific behavior:

[1618] When a user inputs their height of 175 cm, weight of 70 kg, and intermediate skill level, the server uses generative AI to recommend a "shortboard, 6.0 feet long" and displays the results on the user's device.

[1619] Input: User's body type, skill, and wave conditions

[1620] Output: Recommended results (optimal surfing equipment)

[1621] (Application example 1)

[1622] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1623] Monitoring and optimization of manufacturing processes in conventional factories often relied on manual work or partial automation, making it difficult to improve efficiency in real time or provide flexible feedback. This resulted in problems such as the generation of defective products during the manufacturing process and delays in identifying and resolving bottlenecks throughout the line, resulting in a decline in overall productivity. Furthermore, line stoppages due to sudden equipment failures or lack of maintenance also had a negative impact on the production process. A system that could solve these issues and improve the efficiency of the entire manufacturing line was needed.

[1624] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1625] In this invention, the server includes means for collecting video data from cameras installed on the factory production line, means using a generation AI for analyzing the current state of the production process based on the collected video data, means for providing the analyzed process information to an operator's terminal in real time, means for collecting equipment operation data and past production data, means using a generation AI for combining this data to predict the production process one week ahead, means for providing the predicted process information to the operator's terminal, means using a generation AI for analyzing videos of specific processes uploaded by the operator and generating efficiency improvement feedback, means for providing the analysis results to the operator's terminal, means using a generation AI for recommending maintenance based on the equipment's operating status and process data, and means for providing the recommendation results to the operator's terminal.

[1626] This enables real-time monitoring of the operating status of the entire manufacturing line, enabling immediate process improvements and efficiency improvements, early identification of bottlenecks, and the provision of appropriate maintenance forecasts.

[1627] A "production line" refers to a series of processes and machinery that continuously processes and assembles products within a factory.

[1628] A "camera" is a device that captures video or images, and generally includes digital cameras and surveillance cameras.

[1629] "Video data" refers to data that digitally represents images that change continuously over time.

[1630] "Generative AI" refers to artificial intelligence that uses machine learning algorithms to analyze and make predictions from specific data.

[1631] "Process information" refers to data related to the manufacturing process, including production volume, quality, error rate, etc.

[1632] A "terminal" is a device that is connected to the system and exchanges information, including smartphones and computers.

[1633] "Operational data" refers to data related to the operating state of machines and equipment, including operating time, frequency of use, energy consumption, etc.

[1634] "Past manufacturing data" refers to the accumulation of data related to manufacturing processes that has been collected to date.

[1635] "Maintenance" refers to the maintenance and repair work required to keep machines and equipment operating normally.

[1636] A "bottleneck" is the factor that slows down progress in production or a process and reduces overall efficiency.

[1637] "Feedback" is information or instructions that are based on the results or status of a system or process to encourage appropriate improvements or modifications.

[1638] This invention provides a system for monitoring and optimizing a factory production line. The system is composed of the following hardware and software.

[1639] System hardware configuration

[1640] 1. Camera:

[1641] This is a video capture device installed at various points on the production line, allowing real-time video of the manufacturing process to be captured.

[1642] 2. Server:

[1643] It is a central processing unit for storing and analyzing collected video data. The server also provides functions for analyzing various data and using generative AI models.

[1644] 3. Terminal:

[1645] A device such as a smartphone or computer used by an operator to display feedback from the server and analysis results.

[1646] System software configuration

[1647] 1. Generative AI model:

[1648] It is an artificial intelligence algorithm that analyzes video data and various data related to manufacturing processes. Specifically, it uses machine learning models to optimize processes, detect errors, and recommend maintenance.

[1649] 2. Database:

[1650] The collected data is stored and managed using a DBMS such as SQL.

[1651] 3. Front-end application:

[1652] A user interface is provided to display analysis results and feedback to the operator, which is implemented as a smartphone application (Android / iOS) or a web application.

[1653] Example of operation

[1654] 1. Video Data Collection and Analysis:

[1655] A camera captures the production line and sends the video data to a server. The server inputs this video data into a generative AI model and analyzes the current state of the production process. For example, it can obtain specific analysis results such as "The component placement speed on assembly line 1 is declining."

[1656] 2. Providing process information:

[1657] The analysis results are provided to the operator's terminal in real time. For example, an alert such as "A bottleneck has occurred on the assembly line. Please consider countermeasures" is displayed on the terminal.

[1658] 3. Analysis of operational and historical data:

[1659] The server collects equipment operation data and past production data, and uses a generative AI model to predict the production process one week in advance. Specifically, it provides forecast information such as, "Supply shortages are predicted for next week. Please consider replenishing inventory."

[1660] 4. Feedback Generation:

[1661] When an operator uploads video data of a specific process to the server, the generative AI model analyzes it and provides feedback on how to improve efficiency, such as "Please be careful when installing parts."

[1662] 5. Maintenance Recommendations:

[1663] The server monitors the operating status of the equipment and recommends necessary maintenance. For example, a message such as "An abnormality has been detected in the robot arm on line 4. Inspection is recommended" is displayed on the operator's terminal.

[1664] Examples of prompt statements

[1665] "Analyze the video of the assembly process on production line 1 and identify areas for improvement."

[1666] This system enables real-time monitoring of the operating status of the entire manufacturing line, enabling immediate process improvements and efficiency gains, early identification of bottlenecks, and the provision of appropriate maintenance forecasts.

[1667] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1668] Step 1:

[1669] The server acquires video data in real time from cameras installed on the factory production line. The acquired video data is temporarily stored in the server's storage. The input is video data from the camera, and the output is video data saved in the server's storage. Specifically, the camera continuously captures video and sends the data to the server via an IP network.

[1670] Step 2:

[1671] The server inputs the stored video data into a generative AI model to analyze the current state of the manufacturing process. The generative AI model uses this data to identify problems and areas for improvement on the production line. The input is the temporarily stored video data, and the output is the process information that is the analysis result. Specifically, it uses an image analysis algorithm to detect specific errors and delays.

[1672] Step 3:

[1673] The server notifies the operator's device of the analyzed process information in real time. The input is the analysis results obtained from the generative AI model, and the output is feedback displayed on the operator's device. Specifically, the server uses a notification system to send push notifications to the device.

[1674] Step 4:

[1675] The server collects equipment operation data and past manufacturing data, inputs it into a generative AI model, and predicts future manufacturing processes. The input is equipment operation data and past manufacturing data, and the output is predicted manufacturing process information. Specifically, it retrieves the necessary data from the database via a query and analyzes it using the generative AI model.

[1676] Step 5:

[1677] The server provides predicted manufacturing process information to the operator's device. The input is predicted data from the generative AI model, and the output is predicted information displayed on the operator's device. Specifically, it generates a dashboard that displays the prediction results in a visually easy-to-understand manner.

[1678] Step 6:

[1679] The terminal provides a means for the operator to upload video data of a specific process to the server. The input is the operator's video data, and the output is the uploaded video stored on the server. Specifically, the terminal provides file selection and upload functions from the user interface.

[1680] Step 7:

[1681] The server inputs the uploaded video data of a specific process into a generative AI model to generate efficiency improvement feedback. The input is the uploaded video, and the output is the efficiency improvement feedback. Specifically, the video analysis algorithm is used to identify areas for efficiency improvement and generate feedback in text format.

[1682] Step 8:

[1683] The server provides feedback to the operator's device. The input is feedback data from the generative AI model, and the output is feedback displayed on the operator's device. Specifically, the feedback data is sent to the device in real time and displayed on the user interface.

[1684] Step 9:

[1685] The server monitors the operating status of the equipment and makes necessary maintenance recommendations based on the generative AI model. The input is the equipment's operating data and the output is the maintenance recommendation. Specifically, it collects operating data in real time and automatically generates a maintenance notification if an abnormality is detected.

[1686] Step 10:

[1687] The server provides maintenance recommendations to the operator's device. The input is maintenance recommendation data from the generative AI model, and the output is a maintenance notification displayed on the operator's device. Specifically, the server uses a notification system to push the recommendations to the device.

[1688] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1689] This invention relates to a system that provides wave information and helps users improve their surfing skills in the surfing market, as well as a system that provides more personalized and optimal feedback and recommendations by using user emotion recognition. The specific processing required to implement this system is described below.

[1690] Overall overview

[1691] This system has the following main functions:

[1692] 1. Beach camera video data collection and analysis

[1693] 2. Providing wave information and wave forecasts for the next week

[1694] 3. User surfing video analysis and feedback

[1695] 4. Surfboard Recommendations

[1696] 5. Emotion engine recognizes user emotions and optimizes feedback and recommendations

[1697] Beach camera video data collection and analysis

[1698] The server collects video data from cameras installed on each beach. The cameras stream the video in real time, and the video data is stored on the server for analysis.

[1699] Examples:

[1700] The server captures video data from the cameras at Shonan Beach at 30 frames per second and saves the data as a file every five minutes.

[1701] The saved video file is analyzed by the generation AI, and information such as wave height of 1.5m, period of 8 seconds, and direction to the southeast is obtained.

[1702] Providing wave information and one-week wave forecasts

[1703] The server uses AI to analyze wave conditions from the collected video data and provides this information to the user's device in real time. It also combines weather data and past wave condition data to use AI to predict wave conditions for the next week and provides this forecast data to the user.

[1704] Examples:

[1705] The server obtains data such as wind speed and temperature in real time from weather data providers, integrates it with wave data from the past year, and runs it through AI.

[1706] The generating AI analyzes the data and determines that "the predicted wave height at Shonan Beach next week will be between 1.2m and 1.7m," and notifies the user.

[1707] User surfing video analysis and feedback

[1708] When a user uploads a surfing video from their device, the server receives the video and sends it to the generation AI, which analyzes the user's riding form and technique and generates technical feedback, which is displayed on the user's device.

[1709] Examples:

[1710] Users upload their surfing videos to the server via the application.

[1711] The server runs the video through AI, which generates feedback such as "Be conscious of your forward-leaning posture," which is displayed on the user's device.

[1712] Surfboard Recommendations

[1713] Based on information such as the user's body type, skill level, and wave conditions, the server uses generative AI to analyze and recommend the most suitable surfboard. This recommendation is also sent to the user's device.

[1714] Examples:

[1715] Users enter their height, weight and intermediate surfing skills into the app.

[1716] Based on this information, the server recommends a "shortboard, 6.0 feet long" and displays it on the app.

[1717] Emotion recognition and feedback / recommendation optimization using an emotion engine

[1718] The system uses an emotion engine to specifically recognize the user's emotions and provide feedback and surfboard recommendations based on those emotions.

[1719] Examples:

[1720] When a user uploads a surfing video, the emotion engine analyzes their facial expressions and voice to recognize their emotional state. For example, if the user is dissatisfied, the feedback will include encouragement such as, "Next time, try to focus on your hip movement to improve your form."

[1721] Using an emotion engine, surfboard recommendations are also tailored based on the user's emotions, for example, recommending boards that require more skill if the user is feeling adventurous.

[1722] The above is a concrete example of how to implement the present invention. By linking cameras installed on the beach with generative AI and an emotion engine, this system provides detailed and accurate wave information, individual technical feedback, and recommendations for optimal surfboards based on emotions. This allows surfers to enjoy surfing more comfortably and efficiently.

[1723] The processing flow will be explained below.

[1724] Processing flow of a system that combines emotion engines

[1725] Beach camera video data collection and analysis

[1726] Step 1:

[1727] The server receives video data in real time from cameras installed on the beach, connects to the IP addresses of the cameras, and streams the video using RTSP (Real Time Streaming Protocol).

[1728] Step 2:

[1729] The server saves the streamed video data as a file at regular intervals, typically as a new file every five minutes.

[1730] Step 3:

[1731] The server inputs the saved video files into a generative AI model and analyzes wave conditions such as wave height, period, and direction.

[1732] Step 4:

[1733] The server stores the analysis results of the generative AI model in a database and simultaneously provides them to the user's device in real time.

[1734] Providing wave information and one-week wave forecasts

[1735] Step 1:

[1736] The server uses the weather data provider API to retrieve the latest weather data, including wind speed, wind direction, temperature, and air pressure.

[1737] Step 2:

[1738] The server retrieves historical wave condition data from a database, typically using data from the past year.

[1739] Step 3:

[1740] The server inputs weather data and past wave condition data into the AI ​​generator, which then predicts waves for the next week.

[1741] Step 4:

[1742] The server stores the prediction results from the AI ​​in a database and provides them to the user's device, including the wave height, period, and direction for the next week.

[1743] User surfing video analysis and feedback

[1744] Step 1:

[1745] Users use their devices to upload surfing videos to the application, which then sends the videos to a cloud server.

[1746] Step 2:

[1747] The server receives the uploaded video data and stores it in temporary storage.

[1748] Step 3:

[1749] The server inputs the saved video data into a generation AI that analyzes the user's riding form and technique.

[1750] Step 4:

[1751] The server receives the analysis results from the AI ​​generator and provides them to the user as feedback, such as specific technical guidance such as "Be conscious of your forward-leaning posture."

[1752] Surfboard Recommendations

[1753] Step 1:

[1754] Users enter their body type, skill level, and wave conditions into the application on their device, including their height, weight, and surfing skill level.

[1755] Step 2:

[1756] The server receives the user's input data and sends it to the generation AI, which then uses this data to recommend the best surfboard for them.

[1757] Step 3:

[1758] The server stores the recommendations from the AI ​​in a database and provides them to the user's device. For example, a recommendation might be "shortboard, 6.0 feet long."

[1759] Emotion recognition and feedback / recommendation optimization using an emotion engine

[1760] Step 1:

[1761] When a user uploads a surfing video, the server sends the facial and voice data from the video to the emotion engine.

[1762] Step 2:

[1763] The emotion engine uses facial and voice analysis to identify the user's emotional state (e.g., happy, unhappy, excited, relaxed, etc.).

[1764] Step 3:

[1765] The server inputs the analysis results of the emotion engine into the generative AI, optimizing feedback and surfboard recommendations based on the user's emotions.

[1766] Step 4:

[1767] The server then stores customized feedback and recommendations based on the user's emotions in a database and provides them to the user's device. For example, if the user is feeling adventurous, the server might suggest, "Try a more difficult move next time."

[1768] The above is the specific processing flow of the surfing information provision system that combines an emotion engine. Through this series of steps, users can receive more accurate wave information, appropriate technical feedback, and personalized surfboard recommendations based on their emotions.

[1769] Example 2

[1770] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1771] Conventional surfing support systems provide wave information and feedback to improve surfing technique, but they are unable to provide optimal feedback or recommendations that take into account the emotional state of each individual user. As a result, users only receive uniform information, making it difficult to receive advice tailored to their individual needs. This problem prevents users from receiving appropriate feedback to improve their technique, preventing them from maximizing the enjoyment and efficiency of surfing.

[1772] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting video data from a video collection device installed on the beach, means using a generation AI to analyze wave height, period, and direction based on the collected video data, means for providing the analyzed wave information to the user's device in real time, means for collecting weather data and past wave condition data, means using a generation AI to combine this data to predict waves one week in advance, means for providing the predicted wave information to the user's device, means using a generation AI to analyze surfing videos uploaded by the user and generate riding form and technical feedback, means for providing the analysis results to the user's device, means using a generation AI to recommend the optimal surfboard based on the user's body type, skill, and wave conditions, means for providing the recommendation results to the user's device, and means using an emotion recognition engine to analyze the user's emotional state and provide feedback and recommendations based on the emotions. This allows users to receive optimal feedback and recommendations based on their individual needs and emotional state.

[1773] "Beach-mounted video collection devices" refers to video capture devices such as cameras and drones placed around the beach for surfboard practice and wave monitoring.

[1774] "Video data" refers to video data that records and stores the movement of waves on the beach and surfing techniques in real time.

[1775] "Collection means" refers to the function of acquiring data from video collection devices installed on the beach and sending it to a server.

[1776] "Generative AI" is a type of artificial intelligence that uses collected data to analyze and predict wave information and surfing techniques.

[1777] "Analysis means" refers to the ability to use generative AI to analyze collected video data and other data to derive wave height, period, direction, technical advice, etc.

[1778] "Real-time provision means" refers to the function of displaying analyzed information on the user's device without delay.

[1779] "Weather data" refers to numerical data related to weather, such as wind speed, temperature, and air pressure.

[1780] "Wave condition data" refers to information that records past wave height, period, direction, and other data.

[1781] "Wave prediction means" refers to a function that uses AI generation based on weather data and wave condition data to predict future wave conditions.

[1782] "Surfing videos" refer to video files that users have filmed and saved of their surfing skills.

[1783] "Technical feedback" refers to advice and suggestions for technical improvement obtained by the generative AI analyzing surfing videos.

[1784] "Device" refers to a device such as a smartphone, tablet or computer that a surfer uses to receive information such as wave information, technical feedback and surfboard recommendations.

[1785] "Body type" refers to a user's physical characteristics such as height and weight.

[1786] "Skill" refers to the user's level of surfing skill and experience.

[1787] A "surfboard" refers to the board used for gliding when surfing.

[1788] "Recommendation method" refers to a function that uses generative AI to select and suggest the surfboard that is best suited to the user's body type, skill level, and wave conditions.

[1789] An "emotion recognition engine" refers to an algorithm or program that analyzes a user's facial expressions and voice to recognize their emotional state.

[1790] "Means for providing feedback and recommendations" refers to a feature that uses an emotion recognition engine to provide feedback and surfboard recommendations based on the user's emotional state.

[1791] The present invention is a system that collects video data from a video collection device installed on the beach, analyzes wave conditions and surfing techniques using a generative AI model, and provides optimal feedback and surfboard recommendations based on the user's emotional state. A specific embodiment of this system will be described.

[1792] Beach camera video data collection and analysis

[1793] The server collects video data in real time from video collection devices such as cameras and drones installed on the beach. This video data is saved on the server at regular intervals. For example, video data from a camera installed on Shonan Beach is captured at 30 frames per second and saved in MP4 format every five minutes. The server then inputs this saved video data into a generative AI model to analyze information such as wave height, period, and direction.

[1794] Example prompt sentence:

[1795] "Collect video data from Shonan Beach and analyze the wave height, period, and direction. Example: Wave height 1.5m, period 8 seconds, direction southeast."

[1796] Providing wave information and one-week wave forecasts

[1797] The server uses the generative AI model to analyze wave condition information and provides it to the user's device in real time. It also obtains real-time weather data, such as wind speed, temperature, and air pressure, from weather data providers, and combines it with past wave condition data before inputting it into the generative AI model. It then predicts waves one week in advance and notifies the user's device of the results. For example, it displays a forecast such as "Predicted wave heights at Shonan Beach next week are 1.2m to 1.7m."

[1798] Example prompt sentence:

[1799] "Based on weather data and wave data from the past year, please predict the wave height at Shonan Beach next week. Example: Wave height 1.2m - 1.7m."

[1800] Analysis and feedback of user surfing videos

[1801] Users upload their surfing videos from their devices to a server. The server then inputs the uploaded videos into a generative AI model to analyze the user's riding form and technique. For example, it generates feedback such as "Be conscious of leaning forward" and displays it on the user's device.

[1802] Example prompt sentence:

[1803] "Analyze user surfing videos and provide technical feedback. Example: Be mindful of forward leaning posture."

[1804] Surfboard Recommendations

[1805] Users input their body type (e.g., height 175 cm, weight 70 kg) and surfing skill level (e.g., intermediate) into their device. The server uses this information and wave condition data to recommend the optimal surfboard using a generative AI model. For example, a recommended result such as "shortboard, 6.0 feet long" is generated and displayed on the user's device.

[1806] Example prompt sentence:

[1807] "Recommend a surfboard that is suitable for a surfer who is 175cm tall, weighs 70kg, and has intermediate skill level. Example: shortboard, 6.0 feet long."

[1808] Optimization by Emotion Engine

[1809] The server uses an emotion recognition engine to analyze the surfing video and recognize the user's emotional state. For example, if the user's facial expressions and voice indicate dissatisfaction, the server provides encouraging feedback such as, "To improve your form, try to focus on your hip movement next time." The server also optimizes surfboard recommendations based on the user's emotional state. If the user is feeling challenged, the server recommends a surfboard that requires more advanced skills.

[1810] Example prompt sentence:

[1811] "Analyze emotions from users' surfing videos and provide encouraging feedback if they're not satisfied. For example, improve your form by focusing on your hip movement next time."

[1812] The above is a specific embodiment of the system of the present invention. The system of the present invention allows users to receive more detailed and accurate wave information, individual technical feedback, and even recommendations for the best surfboard based on their emotions, allowing users to maximize their surfing enjoyment.

[1813] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1814] Step 1:

[1815] The server collects video data from video collection devices installed on the beach. The cameras and drones capture video data in real time at 30 frames per second and send the data to the server. The server saves this video data in mp4 format every 5 minutes. The input is real-time video data from the cameras, and the output is a video file saved on the server.

[1816] Step 2:

[1817] The server inputs the saved video data into a generative AI model, which analyzes the wave height, period, and direction. The generative AI model analyzes the video frames and extracts wave characteristics. For example, it obtains data such as a wave height of 1.5m, a period of 8 seconds, and a direction southeast. The input is the saved video file, and the output is the wave information resulting from the analysis.

[1818] Step 3:

[1819] The server provides the analyzed wave information to the user's device in real time. The acquired wave height, period, and direction are displayed on the user's device. The input is the analysis result of the wave information by the generative AI model, and the output is the wave information displayed on the user's device.

[1820] Step 4:

[1821] The server obtains weather data in real time from a weather data provider. Weather data includes wind speed, temperature, and air pressure. This data is combined with past wave condition data and input into a generative AI model to predict waves one week in advance. For example, a prediction result such as "Predicted wave heights at Shonan Beach next week will be 1.2m to 1.7m" can be obtained. The input is real-time weather data and past wave condition data, and the output is a wave prediction result for one week in advance.

[1822] Step 5:

[1823] The server notifies the user's device of the wave prediction results, which are then displayed on the user's device for one week ahead. The input is the wave prediction results from the generative AI model, and the output is the prediction results displayed on the user's device.

[1824] Step 6:

[1825] The user uploads their surfing video from their device to the server. The user selects and submits the video using the application. The input is the user's surfing video file, and the output is the video file stored on the server.

[1826] Step 7:

[1827] The server inputs the uploaded surfing video into a generative AI model, which analyzes the user's riding form and technique. The generative AI model analyzes the video frames to identify the user's form and technical shortcomings. For example, it generates feedback such as, "Try to be more conscious of your forward leaning posture." The input is the user's surfing video file, and the output is technical feedback.

[1828] Step 8:

[1829] The server displays the generated feedback on the user's device, which displays the analysis result feedback. The input is the technical feedback from the generative AI model, and the output is the feedback displayed on the user's device.

[1830] Step 9:

[1831] The user enters their body type (e.g., height 175 cm, weight 70 kg) and surfing skill (e.g., intermediate) into the terminal. The terminal application has a form for entering this information. The input is the user's body type and skill information, and the output is the user information sent to the server.

[1832] Step 10:

[1833] The server uses a generative AI model to recommend the optimal surfboard based on the input user information and wave information. The generative AI model analyzes the user's body type, skill, and wave conditions to select the optimal surfboard. For example, a recommendation result of "shortboard, 6.0 feet long" may be obtained. The input is the user's body type, skill information, and wave information, and the output is the recommended surfboard information.

[1834] Step 11:

[1835] The server displays the recommendation results on the user's device, which then displays information about the recommended surfboards. The input is the surfboard recommendation results from the generative AI model, and the output is the recommendation results displayed on the user's device.

[1836] Step 12:

[1837] When the server analyzes the surfing video, it uses an emotion recognition engine to recognize the user's emotional state. It analyzes the user's facial expressions and voice to recognize the emotional state (e.g., dissatisfied). The input is the user's surfing video, and the output is the recognized emotional state.

[1838] Step 13:

[1839] The server adjusts the feedback and surfboard recommendations based on the recognized emotion. If the user is not satisfied, encouraging feedback such as "Next time, be mindful of your hip movement to improve your form" is generated. The input is the recognized emotional state, and the output is the adjusted feedback and surfboard recommendations.

[1840] Step 14:

[1841] The server displays the adjusted feedback and recommendation on the user's device. The user's device displays the optimal feedback and surfboard recommendation according to the emotion. The input is the adjusted feedback and surfboard recommendation, and the output is the feedback and recommendation displayed on the user's device.

[1842] (Application example 2)

[1843] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1844] While conventional systems provide information about beach waves and feedback to help users improve their surfing skills, they do not consider optimal in-store purchasing support using customer emotion recognition. This makes it difficult to provide individually optimized product recommendations and feedback, and poses the challenge of not improving the customer's purchasing experience.

[1845] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1846] In this invention, the server includes: means for collecting video data from cameras installed on the beach; means using a generation AI to analyze wave height, period, and direction based on the collected video data; means for providing the analyzed wave information to a user's device in real time; means for collecting weather data and past wave condition data; means using a generation AI to combine this data to predict waves one week in advance; means for providing the predicted wave information to a user's device; means using a generation AI to analyze surfing videos uploaded by a user and generate riding form and technical feedback; means for providing the analysis results to a user's device; means using a generation AI to recommend the optimal surfboard based on the user's body type, skill, and wave conditions; means for providing the recommendation results to a user's device; means using a generation AI to link video footage from cameras installed in a store with a customer's mobile app, analyze the customer's facial expressions and behavior in real time, and provide optimal product recommendations based on emotion recognition; means for providing product information and reviews to a customer's device; and means for analyzing the customer's facial expressions and behavior and providing feedback and product recommendations based on their emotional state. This allows customers to receive optimal purchasing support using emotion recognition technology in physical stores, and enables personalized feedback and recommendations.

[1847] A "camera" is a device that captures video in real time and records or transmits it as data.

[1848] "Video data" refers to a series of consecutive image information captured by a camera, and is data that can be viewed as a video.

[1849] "Generative AI" refers to algorithms and models that use artificial intelligence technology to analyze data and generate results.

[1850] "Analysis" is the process of processing and analyzing collected data to extract useful information.

[1851] "Wave height, period, and direction" are indicators of the physical characteristics of ocean waves and are important condition information for surfing.

[1852] "Real-time" refers to the property that data is processed synchronously and provided to users without any time delay.

[1853] "Weather data" refers to any data related to weather, such as weather, temperature, wind speed, etc.

[1854] "Past wave condition data" refers to previously recorded data such as wave height, period, and direction, and is information that indicates past wave conditions.

[1855] "Surfing video" is video data captured by a camera while surfing.

[1856] "Riding form" refers to a surfer's posture and movements while surfing.

[1857] "Technical feedback" is specific advice and instruction to improve surfing technique.

[1858] "User's body type, skill, and wave conditions" refers to information about the user's physical characteristics, surfing skill level, and current wave conditions.

[1859] A "surfboard" is a board used for surfing.

[1860] "Recommendation" refers to suggesting the best option based on a specific situation or condition.

[1861] "Images from cameras installed inside the store" refers to images captured by cameras installed inside the store.

[1862] "Customer mobile app" refers to application software that customers use on mobile devices such as smartphones and tablets.

[1863] "Analyzing facial expressions and behavior in real time" refers to capturing a customer's facial expressions and movements, and instantly processing and analyzing the data.

[1864] "Emotion recognition" is a technology that uses information such as facial expressions and voice to identify a person's emotional state.

[1865] "Product recommendation" refers to presenting appropriate products to customers.

[1866] "Product information" is detailed information about the product's price, specifications, functions, etc.

[1867] A "review" refers to a written opinion, such as an evaluation or impression, about a product or service.

[1868] "Feedback" refers to opinions or advice given in response to a particular action or result.

[1869] This invention provides a system for recognizing customer emotions and optimizing the purchasing experience in brick-and-mortar stores. The system is centered around a generative AI that analyzes customer facial expressions and behavior in real time using images from cameras installed in the store, and recommends products and provides information according to the customer's emotional state.

[1870] The specific means included in the system are as follows:

[1871] In-store cameras and video collection

[1872] The server collects video data in real time from IP cameras installed in the store, making it possible to monitor and record customer behavior and facial expressions.

[1873] Video analysis and emotion recognition

[1874] The server analyzes the collected video data using OpenCV and generative AI, and uses deep learning frameworks such as TensorFlow to analyze the customer's facial expressions and behavior and recognize their emotional state.

[1875] Product recommendations and information

[1876] The server then recommends the most suitable products to the customer based on the results of emotion recognition, and provides recommended product information, reviews, and detailed descriptions to the customer's mobile app, allowing the customer to receive product information tailored to their mood and interests.

[1877] Hardware and software used to process the program

[1878] Hardware:

[1879] IP camera (installed inside the store)

[1880] Server (equipped with a high-performance GPU)

[1881] Smartphone or tablet (customer's mobile device)

[1882] software:

[1883] TensorFlow / PyTorch (training and inferencing generative AI models)

[1884] OpenCV (video data preprocessing and analysis)

[1885] Mobile app (runs on smartphones)

[1886] Specific examples

[1887] 1. Customer emotion recognition and product recommendation:

[1888] The camera captures images of customers inside the store and transmits them to a server in real time.

[1889] The server uses a generative AI model to analyze the customer's facial expression and determine that "this customer is interested."

[1890] The server displays detailed information and reviews of Product A on the customer's smartphone.

[1891] example:

[1892] "You seem curious about this product. Let me offer you a sample or other relevant information."

[1893] "You seem unhappy with this product. Let me recommend a different product or have you check the reviews."

[1894] This allows customers to receive products and information that match their emotional state, improving their shopping experience.

[1895] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1896] Step 1:

[1897] Cameras inside the store capture video data and send it to a server.

[1898] Subject: Camera

[1899] Specific operation: Captured video data is acquired frame by frame and sent to the server using the RTSP protocol.

[1900] Input: Video of the store

[1901] Output: Frame-by-frame video data

[1902] Step 2:

[1903] The server captures the received video data in real time and pre-processes it for analysis.

[1904] Subject: Server

[1905] Specific operation: Video data is acquired frame by frame using OpenCV, and preprocessing such as face detection is performed.

[1906] Input: Frame-by-frame video data

[1907] Output: Preprocessed frame data

[1908] Step 3:

[1909] The server inputs the preprocessed frame data into a generative AI model, which analyzes facial expressions and behavior to recognize the customer's emotional state.

[1910] Subject: Server

[1911] How it works: Using TensorFlow, a generative AI model is run to analyze customer facial expression data and identify emotions.

[1912] Input: Preprocessed frame data

[1913] Output: Customer's emotional state (e.g., interested, satisfied, dissatisfied, etc.)

[1914] Step 4:

[1915] The server selects the most appropriate products and information based on the customer's emotional state, and generates recommended results using generative AI.

[1916] Subject: Server

[1917] How it works: Using a generative AI model, it matches the emotional state with a database of products in the store and selects the appropriate product.

[1918] Input: Customer emotional state, product database

[1919] Output: Recommended product information

[1920] Step 5:

[1921] The server notifies the customer's smartphone of the recommended product information.

[1922] Subject: Server

[1923] What it does: Recommendations are sent to the customer's mobile app via API and displayed within the app.

[1924] Input: Recommended product information

[1925] Output: Notified product information

[1926] Step 6:

[1927] Customers refer to the information displayed on their smartphones to check product details and reviews.

[1928] Subject: customer

[1929] Specific actions: Tap on information displayed in the smartphone app to see more details or read reviews.

[1930] Input: Notified product information

[1931] Output: Customer action (purchase, further enquiry, etc.)

[1932] These steps enable the system to analyze customer sentiment in real time and provide optimal products and information based on the results.

[1933] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1934] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1935] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1936] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1937] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1938] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1939] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1940] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1941] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1942] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1943] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1944] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1945] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1946] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1947] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1948] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1949] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1950] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1951] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1952] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described ...

Claims

1. a means of collecting video data from cameras installed on the beach; A method using generative AI to analyze wave height, period, and direction based on collected video data, A means for providing the analyzed wave information to a user's device in real time; a means for collecting weather data and historical wave condition data; A method using AI to combine these data and predict the waves one week in advance. means for providing predicted wave information to a user terminal; A method using generative AI that analyzes surfing videos uploaded by users and generates riding form and technical feedback. A means for providing the analysis results to the user's device; A method using generative AI to recommend the best surfboard based on the user's body type, skill level, and wave conditions. A means to provide recommendations to users' devices Including system.

2. The system of claim 1, further comprising means using a generation AI to combine weather data and past wave condition data to predict waves one week in advance.

3. The system of claim 1, further comprising means for using generative AI to analyze surfing videos uploaded by users and generate riding form and technical feedback.

Images