System

Abstract

A system is provided.SOLUTION: A system including means for receiving video data of a game in real time, means for analyzing the received video data and specifying positions of a player and a ball, means for determining a motion of the player and automatically determining a score and a foul, means for reflecting a determination result on a score board in real time, means for managing a game time, and means for generating statistical data and a moving image after the game ends.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Basketball game management requires skilled referees and scorers, which requires a great deal of manpower and technical effort. Furthermore, when recording a game, manual visual inspection is required, which can lead to inaccuracies. This poses a challenge to the fairness and efficiency of game management. Furthermore, compiling data and editing video after the game is also time-consuming and labor-intensive. [Means for solving the problem]

[0005] The present invention provides a system that includes a means for receiving video data of a game in real time, a means for analyzing the received video data and identifying the positions of players and the ball, a means for determining player movements and automatically determining scores and fouls, a means for updating the determination results on a scoreboard in real time, a means for managing the game time, and a means for generating statistical data and video after the game ends. This system efficiently and accurately performs all game management tasks, preventing manual misjudgments and scoring errors. Post-game data collection and video editing are also automated, reducing time and effort.

[0006] "Game video data" refers to real-time image information of a basketball game captured by a camera.

[0007] "Means for receiving in real time" refers to a device or system for immediately capturing captured video data without delay.

[0008] "Means of analysis" refers to technology or software for extracting and analyzing specific information based on received video data.

[0009] "Player" means a player participating in a basketball game.

[0010] "Means for identifying the ball's position" refers to technology that detects and tracks the current position of the ball from video data.

[0011] "Means for identifying player movements" refers to algorithms and technologies for recognizing player movements from video data and identifying their content.

[0012] "Means for automatically determining scores and fouls" refers to a system that determines scores and violations based on information obtained from players' actions and automatically takes the necessary measures.

[0013] "Means for reflecting the judgment results on the scoreboard in real time" refers to a technology or system for instantly reflecting the judgment results obtained on a display device.

[0014] "Means for managing game time" refers to technology that accurately measures and manages the start and end of a game, the time of each quarter, and timeouts.

[0015] "Means for generating statistical data" refers to a system that automatically creates various statistical information based on data collected during a match.

[0016] "Means for generating video" refers to the technology for editing video data taken during a match to create highlights or full match videos. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] This invention relates to a system that automates the management of basketball games. By using AI to perform tasks such as filming the game, refereeing, scoring, timekeeping, and creating video of the game, it prevents manual errors in judgment and scoring, and realizes accurate and efficient game management.

[0039] System configuration

[0040] The system includes the following main components:

[0041] 1. Camera equipment

[0042] Device: The device will be installed in a position that overlooks the entire court. The camera has a high resolution and a wide-angle lens, allowing for clear capture of the entire match.

[0043] 2. Server

[0044] Server: This is the central system that analyzes real-time video data received from the cameras. It uses AI algorithms and image recognition technology to identify the positions of players and the ball and make necessary decisions.

[0045] 3. Smart Device Applications

[0046] User: Uses an application to input and manage match and player information. Before a match, users can input information such as player names, uniform numbers, and positions, and can also check the situation during the match.

[0047] Embodiments of the invention

[0048] Filming the match

[0049] Device: Position the camera so that it can capture the entire court. Conduct test shots to ensure the camera captures all action during the match.

[0050] Entering match information

[0051] User: Launches the application on their smart device and inputs information such as the player's name, uniform number, and position. They also input information such as the start time of the game and the opposing team. The input data is sent to the server.

[0052] Real-time analytics

[0053] Server: Receives video data from the camera in real time and uses image recognition technology to identify the positions of players and the ball. AI algorithms distinguish between shots, passes, dribbles, and other plays, and automatically determines scores, fouls, and violations. The results are instantly updated on the scoreboard.

[0054] For example, when a player shoots from outside the three-point line, the system detects the shot and determines whether the ball went into the goal. If the goal is successful, the server immediately updates the score to three points.

[0055] Timekeeping

[0056] Server: Starts the timer at the start of the game and accurately manages the time for each quarter and timeout. Stops and restarts the timer at the appropriate time during timeouts and other interruptions in the game.

[0057] Post-match data generation

[0058] Server: After the match, it consolidates all data collected during the match to generate statistics and match records, extracts important moments from the match to create highlight videos, and edits and stores full match videos.

[0059] Users: Through their smart devices, they can access the generated statistics and videos and download them as needed.

[0060] Specific examples

[0061] For example, if Player B commits a foul during a match, the server analyzes the video data capturing that moment. The AI ​​determines Player B's movements and automatically determines whether it was a foul based on the rules. The determination result is immediately reflected on the scoreboard, and the foul count is updated. After the match ends, detailed statistical data, including a record of the fouls, is automatically generated and available for users to review.

[0062] In this way, the system of the present invention dramatically improves the efficiency of game management and enables fair and accurate refereeing and scoring.

[0063] The processing flow will be explained below.

[0064] Step 1:

[0065] The user sets the camera in a position that overlooks the entire court and adjusts it so that there are no blind spots. The camera is connected to the server via a network.

[0066] Step 2:

[0067] The user launches the application on their smart device and enters game information (start time, opposing teams, game format, etc.) and player information (name, uniform number, position, etc.). The entered data is sent to the server.

[0068] Step 3:

[0069] The server receives video data from the camera in real time and stores the received video data in a buffer for analysis.

[0070] Step 4:

[0071] The server analyzes the video data frame by frame and uses image recognition technology to identify the positions of players and the ball, tracking player movements.

[0072] Step 5:

[0073] The server uses AI algorithms to detect player actions (shooting, passing, dribbling, etc.), identify specific movement patterns, and determine events (goals, fouls, violations, etc.).

[0074] Step 6:

[0075] The server automatically determines the score based on detected events (e.g., goals, fouls), and updates the scoreboard in real time, updating individual performance data.

[0076] Step 7:

[0077] The server initiates timekeeping, accurately measures the start and end times of the game, the duration of each quarter, and timeouts, and stops and restarts game time as necessary.

[0078] Step 8:

[0079] The server consolidates all data collected during the game and automatically generates statistics and match records after the game ends, compiling detailed statistics for each player, such as points, rebounds, assists, and fouls.

[0080] Step 9:

[0081] The server analyzes video data during the match, extracts important scenes to create a highlight video, and edits and saves the full match video in the appropriate format.

[0082] Step 10:

[0083] Users can view the generated statistical data and game videos through their smart devices and download them as needed. The server then uploads the data and videos to the cloud and provides users with an access link.

[0084] Example 1

[0085] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0086] Traditional basketball game management relies on human intervention, which means that misjudgments and errors in scoring are likely to occur. Furthermore, refereeing, scoring, and timekeeping require a great deal of labor, making efficient management difficult. Furthermore, generating statistical data and video after the game is also time-consuming, making it necessary to provide fast and accurate results.

[0087] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0088] In this invention, the server includes means for receiving video data of a game in real time using a high-resolution camera, means for analyzing the received video data and identifying the positions of players and the ball, means for determining player movements using an artificial intelligence algorithm and automatically determining scores and fouls, means for updating the determination results on the scoreboard in real time, means for managing the game time, means for generating statistical data and edited game videos after the game ends, and means for inputting game information and player information using a smart device, thereby enabling efficient game management and accurate determination and scoring.

[0089] A "high-resolution camera" is a high-resolution camera that can capture clear footage of the match.

[0090] "Player" means a player participating in a basketball game.

[0091] "Ball" refers to the ball equipment used in the game of basketball.

[0092] An "artificial intelligence algorithm" is an algorithm that analyzes players' movements, recognizes plays such as shots, passes, and dribbles, and automatically determines scores and fouls.

[0093] A "scoreboard" is a device that displays the score and other information of a game.

[0094] "Timekeeping" refers to the act of accurately managing the time of a match.

[0095] "Statistical Data" means data about player and team performance collected during a match.

[0096] "Edited match video" is a video in which important scenes are extracted and edited after the match has ended.

[0097] A "smart device" is a mobile terminal with internet connectivity, such as a smartphone or tablet.

[0098] "Match information" refers to basic information about the match, such as the start time and opposing teams.

[0099] "Player information" refers to information about players participating in a match, such as their names, uniform numbers, and positions.

[0100] MODE FOR CARRYING OUT THE INVENTION

[0101] This invention is a system that automates the management of basketball games, enabling efficient and accurate game management. This is achieved by using AI to perform tasks such as filming the game, refereeing, scorekeeping, timekeeping, and creating game videos.

[0102] Filming the match

[0103] Device: A high-resolution camera is installed in a position that allows a bird's-eye view of the entire court. This camera has a wide-angle lens, allowing it to clearly capture all the action of the match. For example, during camera installation and test shooting, the camera angle and focus are manually adjusted.

[0104] Entering match information

[0105] User: Launches the application on a smart device and inputs information such as player names, uniform numbers, positions, match start time, and opposing teams. This information is sent to the server in real time, and the match is ready. Specifically, the user uses a smartphone or tablet to enter data into the application's input form and presses the "Submit" button.

[0106] Real-time video reception and analysis

[0107] Server: Uses a means to receive and analyze real-time video data from the camera. The received video data is input into an image analysis algorithm to identify the positions of players and the ball. Next, an artificial intelligence algorithm is used to identify player actions and analyze plays such as shooting, passing, and dribbling. The server uses libraries such as OpenCV and TENSORFLOW (registered trademark) to perform image analysis and action recognition.

[0108] For example, the server extracts the characteristics of a player's movements and determines the shooting action based on them. If a player shoots from outside the three-point line, the server detects the movement and determines whether the ball went into the goal. If the goal is successful, the server updates the score as three points.

[0109] Timekeeping

[0110] Server: Starts the timer as soon as the game starts, and accurately manages the time for each quarter and timeout. When the game is interrupted, stops and restarts the time at the appropriate time. Specifically, it calls timer management functions to start, stop, and restart the time.

[0111] Post-match data generation

[0112] Server: After the match, it consolidates all collected data and generates statistics and match records. It extracts important scenes to create highlight videos, and edits and saves full match videos. For example, it aggregates data from each play and generates statistical data in Excel or CSV format. It also uses the VideoEditor API to edit videos.

[0113] View and download results

[0114] Users can access the generated statistics and videos through their smart devices and download them as needed. Specifically, they can tap the "View Results" button in the app, select a match from the list, and view the data. If necessary, they can press the "Download" button to save the data to their local device.

[0115] Prompt Sentence Examples

[0116] "Please explain how to accurately determine the exact moment a player shoots during a basketball game and automatically update the score."

[0117] This system will enable more efficient match management and accurate judgment and scoring, thereby achieving fair, fast and accurate match management.

[0118] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0119] Step 1:

[0120] The device installs a high-resolution camera in a position that provides a bird's-eye view of the court. This camera has a wide-angle lens that captures the entire match clearly. Specific operations include adjusting the camera's angle and focus appropriately to cover the optimal field of view. The physical position and settings of the camera are specified as input, and high-resolution video is obtained as output.

[0121] Step 2:

[0122] The user starts up the smart device and launches the application. There, they enter game information such as player names, uniform numbers, positions, game start time, and opposing teams. The entered information is sent to the server in real time. Specifically, the user enters data into the input form on the smart device and presses the "Send" button. Player information and game information are given as input, and this information is sent to the server as output.

[0123] Step 3:

[0124] The server receives real-time video data from the camera. This video data is decoded and converted into an analyzable format. Specifically, the server decodes the video stream frame by frame and saves it as frame data. The video stream from the camera is given as input, and analyzable frame data is obtained as output.

[0125] Step 4:

[0126] The server inputs the received video data into an image analysis algorithm. This algorithm uses libraries such as OpenCV and TensorFlow to identify the positions of players and the ball. Specifically, it performs image analysis to detect player movements and the position of the ball. Frame data is given as input, and player and ball position data is obtained as output.

[0127] Step 5:

[0128] The server uses an artificial intelligence algorithm to determine player actions. This algorithm distinguishes between plays such as shooting, passing, and dribbling, and automatically determines scores and fouls. Specific actions are input into a motion recognition model, and a judgment result is obtained. Position data of the player and ball is given as input, and judgment data is obtained as output.

[0129] Step 6:

[0130] The server reflects the decision results on the scoreboard in real time. For example, if a player makes a successful shot from outside the three-point line, the score is updated to three points. Specifically, the server calls the score update API and updates the scoreboard. The decision data is given as input, and the updated score is displayed as output.

[0131] Step 7:

[0132] The server starts the timer as soon as the game starts, and accurately manages the time for each quarter and timeout. When the game is interrupted, it stops and restarts the time at the appropriate timing. Specifically, it calls timer management functions to start, stop, and restart the time. The game start time is given as input, and time management data is obtained as output.

[0133] Step 8:

[0134] After the match, the server consolidates all collected data and generates statistical information and match records. It extracts important scenes to create highlight videos, and edits and saves the full match video. Specific operations include aggregating data and editing the video using the VideoEditor API. The data collected during the match is given as input, and statistical data and edited video are obtained as output.

[0135] Step 9:

[0136] Users can access the generated statistical data and videos through their smart devices and download them as needed. Specifically, they tap the "View Results" button on the app, select a match from the list, and check the data. The user request is given as input, and the displayed data can be viewed as output. They can also save the data to their local device by pressing the "Download" button.

[0137] (Application example 1)

[0138] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0139] In traditional brick-and-mortar store operations, there were limited ways to understand customer behavior patterns and product popularity in real time, making it difficult to operate the store efficiently and provide appropriate customer service.In addition, store managers lacked a way to instantly grasp the congestion situation in the store or when it was necessary to replenish shelves, so they were required to respond quickly.

[0140] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0141] In this invention, the server includes means for receiving video data of the game in real time, means for analyzing the received video data and identifying the positions of objects and elements, means for determining the behavior of objects and automatically determining information and status, means for reflecting the determination results on a display device in real time, means for managing time, means for generating statistical data and videos after the game ends, means for analyzing the situation in the store in real time and identifying behavior patterns and product popularity, and means for notifying the user of the real-time analysis results. This allows store managers to grasp customer behavior patterns and product popularity in real time, and to immediately grasp the congestion situation in the store and when it is necessary to replenish shelves.

[0142] "Game video data" refers to video information received from cameras in real time during a game.

[0143] "Means for receiving" refers to the device or software function for acquiring the video data transmitted from the camera.

[0144] "Means for analyzing and locating objects and elements" refers to technologies and algorithms that recognize the location of specific objects or people based on video data.

[0145] "Means for identifying the behavior of a target and automatically determining its information and status" refers to technologies and algorithms that analyze the movements of a target object or person and automatically evaluate and judge its behavior and status.

[0146] "Means for reflecting the determination results on a display device in real time" refers to a function for instantly displaying the results of analysis and determination on a display device such as a monitor.

[0147] "Time management means" refers to a device or software function that measures and manages the time required for a match or operation.

[0148] "Means for generating post-match statistics and video" refers to the ability to consolidate collected data and create aggregated statistics and edited video after a match or event has concluded.

[0149] "Means of analyzing the situation within the store in real time and identifying behavioral patterns and product popularity" refers to technology that analyzes customer movement within the store and the level of interest in products in real time, and uses that data to identify behavioral patterns and popular products.

[0150] "Means for notifying real-time analysis results" refers to systems or functions that immediately notify relevant parties of the results obtained through analysis.

[0151] Overall system overview

[0152] This system automates customer behavior analysis and in-store management to improve the efficiency of brick-and-mortar store operations, and is primarily comprised of cameras, servers, and smart devices.

[0153] Hardware and Software

[0154] Hardware:

[0155] High-resolution wide-angle cameras capture customer movement and behavior within the store in real time.

[0156] The server is a central device that analyzes the received video data.

[0157] Smart devices (PDAs) are used by managers to view the situation in the store in real time.

[0158] software:

[0159] AI image recognition software (e.g., OpenCV and TensorFlow) analyzes video data to identify customer behavior and product popularity.

[0160] Data analysis algorithms identify customer behavior patterns and popular products.

[0161] The store management app provides analysis results and inventory information to managers.

[0162] System configuration

[0163] The system includes the following main components:

[0164] 1. Camera equipment

[0165] Device: High-resolution wide-angle cameras are installed in locations that overlook each area of ​​the store, capturing all of the customer's movements and actions.

[0166] 2. Server

[0167] Server: This is the central system that analyzes real-time video data received from in-store cameras. It uses AI image recognition software (e.g., OpenCV or TensorFlow) and data analysis algorithms to identify customer behavior and product popularity. It also notifies the store management app of the analysis results in real time.

[0168] 3. Store management application

[0169] User: This application is used by administrators to check customer behavior analysis data, inventory status, and sales information in real time. It is operated on smart devices such as PDAs.

[0170] Specific examples

[0171] For example, if analysis shows that a particular product shelf in a store in a shopping mall is extremely popular, this information will be notified to the manager, who can then take prompt action such as adding new products to that shelf.

[0172] Prompt Sentence Examples

[0173] Here are some example prompts for using generative AI models:

[0174] "Create an AI algorithm that analyzes customer behavior in stores and identifies popular areas and products."

[0175] "Analyze in-store congestion in real time and design a program to optimize staff allocation."

[0176] In this way, implementing this system is expected to improve the efficiency of physical store operations and customer service.

[0177] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0178] Step 1:

[0179] The device uses a high-resolution, wide-angle camera installed inside the store to capture customer movement and behavior in real time.

[0180] Input: Real-time video data captured by a high-resolution wide-angle camera.

[0181] Data processing / data calculation: Video data is acquired and sent directly to the server.

[0182] Output: Real-time video data is sent to the server.

[0183] Step 2:

[0184] The video data received by the server is analyzed using AI image recognition software (e.g., OpenCV or TensorFlow).

[0185] Input: Real-time video data received from a high-resolution wide-angle camera.

[0186] Data processing / data calculation: Apply object recognition and motion recognition algorithms to extract customer positions and movements from video data.

[0187] Output: Customer location and movement information.

[0188] Step 3:

[0189] The server uses data analysis algorithms to identify customer behavior patterns and product popularity.

[0190] Input: Customer location and movement information.

[0191] Data processing / data calculation: Analyze behavioral patterns based on location and movement information to calculate the popularity of specific areas and products.

[0192] Output: Behavioral pattern information and popular product information.

[0193] Step 4:

[0194] The server notifies the store management app of the analysis results in real time.

[0195] Input: behavioral pattern information and popular product information.

[0196] Data processing / data calculation: Formats the analysis results and generates notification messages.

[0197] Output: Real-time notification to the administrator's smart device.

[0198] Step 5:

[0199] The user (administrator) uses the store management app to check customer behavior analysis data, inventory status, and sales information in real time and take necessary action.

[0200] Input: Real-time notifications and analysis results displayed in the store management app.

[0201] Data processing / data calculation: Decide on product replenishment and staff allocation based on the information received.

[0202] Output: Improved store operations and faster response times.

[0203] Step 6:

[0204] After a match or event ends, the server consolidates the collected data and automatically generates statistical data and edited videos.

[0205] Input: Customer behavioral pattern information and sales data.

[0206] Data processing / data calculation: Statistical processing of all data and generation of statistical reports and videos.

[0207] Output: Statistical report and highlight video.

[0208] The above steps will realize a system that is expected to make physical store operations more efficient and improve customer service.

[0209] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0210] This invention relates to a system that automates the management of basketball games, in which AI performs the tasks of filming the game, refereeing, recording the score, timekeeping, and creating game videos, and by combining it with an emotion engine, it recognizes the user's emotions in real time and adjusts the system's behavior.

[0211] System configuration

[0212] The system includes the following main components:

[0213] 1. Camera equipment

[0214] Device: Installed in a position overlooking the entire court, it is equipped with a high-resolution, wide-angle lens to capture clear images of the entire game.

[0215] 2. Server

[0216] Server: Analyzes real-time video data received from the camera to identify the positions of players and the ball, and uses AI algorithms and image recognition technology to make decisions and provide scoring.

[0217] 3. Smart Device Applications

[0218] User: Uses an application to input and manage match and player information. Before a match, the user can input player names, uniform numbers, positions, etc., and check the situation during the match.

[0219] 4. Emotion Engine

[0220] Server: Equipped with an engine that analyzes the user's facial expressions and voice to recognize their emotional state, and adjusts the system's behavior based on the recognized emotional data.

[0221] Embodiments of the invention

[0222] Filming the match

[0223] Device: Position the camera so that it can capture the entire court. Conduct test shots to ensure the camera captures all action during the match.

[0224] Entering match information

[0225] User: Launches the application on their smart device and inputs information such as the player's name, uniform number, and position. They also input information such as the start time of the game and the opposing team. The input data is sent to the server.

[0226] Real-time analytics

[0227] Server: Receives video data from the camera in real time and uses image recognition technology to identify the positions of players and the ball. AI algorithms distinguish between shots, passes, dribbles, and other plays, and automatically determines scores, fouls, and violations. The results are instantly updated on the scoreboard.

[0228] For example, when a player shoots from outside the three-point line, the system detects the shot and determines whether the ball went into the goal. If the goal is successful, the server immediately updates the score to three points.

[0229] Timekeeping

[0230] Server: Starts the timer at the start of the game and accurately manages the time for each quarter and timeout. Stops and restarts the timer at the appropriate time during timeouts and other interruptions in the game.

[0231] Recognizing and reflecting emotions

[0232] Server: Using the emotion engine, analyzes the user's facial expressions and voice data in real time to recognize their emotional state. Based on the recognized emotional data, the game progress and notification content are adaptively changed.

[0233] For example, if the system detects that the user is excited during a tense match, it will adjust the frequency of screen displays and notifications to prevent the user from feeling stressed. Conversely, if the system determines that the user is bored, it can send notifications highlighting the highlights of the match.

[0234] Post-match data generation

[0235] Server: After the match, it consolidates all data collected during the match to generate statistics and match records. It extracts key moments from the match to create highlight videos and provide emotional highlights that reflect the emotional data used. It also edits and saves the full match video in the appropriate format.

[0236] Users: Access the generated statistical data and videos through their smart devices and download them as needed. The server uploads the data and videos to the cloud and provides users with an access link.

[0237] Specific examples

[0238] For example, if Player B commits a foul during a match, the server analyzes the video data capturing that moment. The AI ​​distinguishes Player B's movements and automatically determines whether it was a foul based on the rules. At the same time, the emotion engine analyzes the user's reaction, and if it detects excitement or surprise, it can instantly provide a replay video that reflects that emotion.

[0239] By combining an emotion engine with video analysis, the system of the present invention can not only improve the efficiency of match management, but also the user experience.

[0240] The processing flow will be explained below.

[0241] Step 1:

[0242] The user sets the camera in a position that overlooks the entire court and adjusts it so that there are no blind spots. The camera is connected to the server via a network.

[0243] Step 2:

[0244] The user launches the application on their smart device and enters game information (start time, opposing teams, game format, etc.) and player information (name, uniform number, position, etc.). The entered data is sent to the server.

[0245] Step 3:

[0246] The server receives video data from the camera in real time and stores the received video data in a buffer for analysis.

[0247] Step 4:

[0248] The server analyzes the video data frame by frame and uses image recognition technology to identify the positions of players and the ball, tracking player movements.

[0249] Step 5:

[0250] The server uses AI algorithms to detect player actions (shooting, passing, dribbling, etc.), identify specific movement patterns, and determine events (goals, fouls, violations, etc.).

[0251] Step 6:

[0252] The server automatically determines the score based on detected events (e.g., goals, fouls), and updates the scoreboard in real time, updating individual performance data.

[0253] Step 7:

[0254] The server initiates timekeeping, accurately measures the start and end times of the game, the duration of each quarter, and timeouts, and stops and restarts game time as necessary.

[0255] Step 8:

[0256] The server uses an emotion engine to analyze the user's facial expressions and voice data in real time, recognizes the user's emotional state based on the analysis results, and adjusts the system's behavior based on that data.

[0257] Step 9:

[0258] For example, if the user is excited, the server will minimize the information displayed to reduce the user's stress. Conversely, if the user is perceived as bored, the server will send highlight notifications more frequently.

[0259] Step 10:

[0260] The server consolidates all data collected during the game and automatically generates statistics and match records after the game ends, compiling detailed statistics for each player, such as points, rebounds, assists, and fouls.

[0261] Step 11:

[0262] The server analyzes video data during the match, extracts important scenes, and creates highlight videos. It also provides emotional highlight videos that reflect the emotional data.

[0263] Step 12:

[0264] Users can access the generated statistical data and emotional highlight videos through their smart devices and download them as needed. The server uploads the data and videos to the cloud and provides users with an access link.

[0265] In this way, the system streamlines match management, provides fair and accurate refereeing and scoring, and also provides an interactive experience that responds to the user's emotions.

[0266] Example 2

[0267] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0268] Basketball game management requires a lot of manual work, and is prone to human error in refereeing, scoring, timekeeping, and game video production. Furthermore, there is a lack of ways to properly reflect user emotions during the game to provide a better experience.

[0269] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for receiving video data of the game in real time, means for analyzing the received video data and identifying the positions of the players and the ball, and means for analyzing the emotional state of the user in real time and adaptively adjusting the behavior of the system. This enables the automation of game management, reduces human error, and enables flexible responses according to the user's emotions.

[0270] "Means for receiving game video data in real time" refers to the cameras and communication infrastructure that capture video of the basketball game and transmit it to a server in real time.

[0271] "Means for analyzing received video data and identifying the positions of players and the ball" refers to algorithms or software that use image recognition technology to analyze and identify the positions of players and the ball from received video data.

[0272] "Means of determining player movements and automatically determining scores and fouls" refers to a system that uses AI algorithms to analyze player movements and automatically determine scores, fouls, and violations.

[0273] "Means for reflecting the judgement results on the scoreboard in real time" refers to a communication system and a display device for instantly displaying the judgement results on the scoreboard.

[0274] "Means for controlling the time of a match" means a digital timer or control system for accurately measuring the start, finish and stoppage times of a match.

[0275] "Means for analyzing the user's emotional state in real time and adaptively adjusting the system's behavior" refers to an emotion recognition engine that analyzes the user's facial expressions and voice data and dynamically changes the system's behavior and notification content based on that emotional state.

[0276] "Means for generating post-match statistics and video" means software and servers that aggregate data collected during a match and generate statistics, match records, and edited match video.

[0277] "Using a camera to capture a bird's-eye view of the entire court" refers to a method of capturing the entire court with a wide-angle lens to monitor the movement of all players and the ball at once.

[0278] "Entering match and player information using a smart device" refers to the process of entering player and match information before a match using a digital device such as a smartphone or tablet.

[0279] Overall system overview

[0280] The present invention is a system for automating basketball game management that integrates game filming, refereeing, scorekeeping, timekeeping, game video production, and an emotion engine that recognizes user emotions in real time and adjusts the system's behavior accordingly.

[0281] System Components

[0282] 1. Terminal (camera device)

[0283] A device equipped with a high-resolution, wide-angle lens is placed in a position that allows it to capture a bird's-eye view of the entire court. The camera is used to capture clear images of the entire game. For example, the camera can take a panoramic image of the entire court before the start of a game and send the image to a server.

[0284] 2. Server

[0285] The server has the following main functions:

[0286] 1. Receiving and analyzing video data: Video data is received in real time and the positions of players and the ball are analyzed using image recognition technology (e.g., OpenCV or TensorFlow).

[0287] 2. Judgment by AI algorithm: Based on the AI ​​algorithm, plays such as shots, passes, and dribbles are judged, and scores, fouls, and violations are automatically determined.

[0288] 3. Reflection of judgement results: The judgement results will be reflected immediately on the scoreboard.

[0289] 4. Timekeeping: Manage the game time, start the timer at the start of the game, and accurately manage the times of each quarter and timeout.

[0290] 5. Emotion Engine: Analyzes the user's facial expressions and voice data in real time to recognize their emotional state and adaptively adjust the system's behavior. For example, if the emotion engine detects that the user is excited, it will reduce the frequency of notifications.

[0291] 6. Data generation: Generate statistics and videos after the game, extract key moments and create highlight videos.

[0292] 3. User (Smart Device Application)

[0293] The user does the following:

[0294] 1. Entering match and player information: Using the smart device application, users enter information such as player names, uniform numbers, positions, match start times, and opposing teams. This information is then sent to the server.

[0295] 2. Data and video access: Access the generated statistical data and videos through smart devices and download them as needed. The server uploads the data and videos to the cloud and provides an access link.

[0296] Specific examples

[0297] For example, if player B commits a foul during a game, the server analyzes the video data capturing that moment. The AI ​​distinguishes player B's movements and automatically determines whether it was a foul based on the rules. At the same time, the emotion engine analyzes the user's reaction, and if it detects excitement or surprise, it instantly provides a replay video that reflects that emotion.

[0298] Example prompts to input to the generative AI model

[0299] Design a system that automatically identifies specific plays in an ongoing basketball game and adaptively adjusts notifications based on the user's emotional state. This system includes a camera that records the game, a server that analyzes real-time footage, a smart device application that manages game information, and an emotion engine that recognizes the user's emotions.

[0300] The above is an embodiment of the present invention that makes it easy to implement. This system can automate the management of matches and improve the user experience.

[0301] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0302] Step 1: Filming the match

[0303] Terminal

[0304] Input: Notification of the start of the match

[0305] Operation:

[0306] 1. A camera with a high resolution and wide-angle lens will be installed in a position that allows for a bird's-eye view of the entire court.

[0307] 2. The camera prepares to capture the entire match in real time.

[0308] 3. Conduct test shooting to check and adjust the shooting range and image quality.

[0309] Output: High-resolution real-time video data

[0310] Step 2: Enter match information

[0311] User

[0312] Input: Pre-match player and match information

[0313] Operation:

[0314] 1. Launch the smart device application.

[0315] 2. Enter the player's name, uniform number, position, game start time, and opposing team information.

[0316] 3. The entered information is sent to the server.

[0317] Output: Match and player information sent to the server

[0318] Step 3: Receiving video data

[0319] server

[0320] Input: Real-time video data from the camera

[0321] Operation:

[0322] 1. Receive video data sent from the camera in real time.

[0323] 2. The received video data is saved and prepared for the next analysis step.

[0324] Output: Received and stored video data

[0325] Step 4: Analyzing the video data

[0326] server

[0327] Input: Received video data

[0328] Operation:

[0329] 1. Analyze the video data using image recognition technology (e.g., OpenCV, TensorFlow).

[0330] 2. Identify the location of the players and the ball.

[0331] Output: Player and ball position data

[0332] Step 5: Motion detection and automatic judgment

[0333] server

[0334] Input: Player and ball position data

[0335] Operation:

[0336] 1. Using AI algorithms to identify player actions (shooting, passing, dribbling, etc.).

[0337] 2. Automatically judge scores, fouls, and violations based on the game rules.

[0338] Output: Scores, fouls, and violations

[0339] Step 6: Reflecting the results

[0340] server

[0341] Input: Judgment result

[0342] Operation:

[0343] 1. The judgement results will be reflected immediately on the scoreboard.

[0344] 2. Send notifications to users as needed.

[0345] Output: Updated leaderboard

[0346] Step 7: Timekeeping

[0347] server

[0348] Input: Notification of the start of the match

[0349] Operation:

[0350] 1. Start the digital timer at the same time as the match begins.

[0351] 2. Accurately manage the time of each quarter and timeout.

[0352] 3. Stop and restart the clock at the appropriate time when the game is interrupted.

[0353] Output: Precisely controlled game time

[0354] Step 8: Recognize and reflect on your emotions

[0355] server

[0356] Input: User facial and voice data

[0357] Operation:

[0358] 1. Analyze the user's facial expressions and voice data in real time using an emotion engine.

[0359] 2. Recognize the user's emotional state and adaptively adjust the system's behavior (notification frequency, display content, etc.).

[0360] Output: Changing the system's behavior based on the recognized emotion data.

[0361] Step 9: Post-match data generation

[0362] server

[0363] Input: Data collected during the match

[0364] Operation:

[0365] 1. After the match, all data is aggregated to generate statistics and match records.

[0366] 2. Extract important scenes and create a highlight video.

[0367] 3. Edit and save the full match video in the appropriate format.

[0368] Output: Statistics, match records, highlight videos, full match videos

[0369] Step 10: Accessing Data and Videos

[0370] User

[0371] Enter: Post-match notification

[0372] Operation:

[0373] 1. Access the generated statistics and videos using your smart device.

[0374] 2. Download data and videos as needed.

[0375] Output: Downloadable statistics and a link to the video

[0376] This is the specific processing flow of this system. This system automates the management of matches and enables flexible responses according to the user's emotions.

[0377] (Application example 2)

[0378] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0379] The problem that this invention aims to solve is to reduce the stress of managers in large spaces such as public buildings and plazas by detecting abnormal behavior and analyzing emotions, which conventional monitoring systems cannot handle. There is a need to detect abnormalities in real time and respond efficiently while reducing the burden on managers.

[0380] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving video data of the game in real time, means for analyzing the received video data and identifying the positions of players and the ball, means for determining player actions and automatically determining scores and fouls, means for reflecting the determination results on the scoreboard in real time, means for managing the game time, means for generating statistical data and videos after the game ends, means for detecting and notifying abnormal behavior, and means for analyzing the user's emotional state and adjusting the system's behavior. This enables efficient monitoring of large spaces, rapid detection of abnormal behavior, and adjustment of the system's behavior in accordance with the administrator's emotional state.

[0381] "Video data" refers to digital data of video footage captured by a camera.

[0382] "Real-time" refers to processing, sending, and receiving information instantly, without delay.

[0383] "Analysis" is the act of extracting specific patterns or information from digital data.

[0384] "Player" refers to a person participating in a sport or action.

[0385] "Ball" refers to a spherical object used for sports and games.

[0386] "Locate" means to determine the current coordinates or location of an object or person.

[0387] "Distinguishing behavior" refers to recognizing and classifying the behavior of a target person or object.

[0388] A "score" is a numerical value that indicates the score in a sporting event.

[0389] A "foul" is an act that violates the rules in a sporting event.

[0390] "Decision result" refers to a judgment or conclusion drawn based on an analysis.

[0391] A "scoreboard" is a device or screen that displays scores and other information from a sports event.

[0392] "Time management" means monitoring and regulating the progress of a competition or task.

[0393] "Statistical data" is quantitative information obtained from the results of analysis.

[0394] "Generating a moving image" means editing still images and video data to create a moving image.

[0395] "Abnormal behavior" refers to unusual or suspicious behavior that differs from normal behavior.

[0396] A "notification" is a message or signal that conveys specific information.

[0397] "Emotional state" indicates the type and strength of the emotion a person is feeling at the time.

[0398] "Analyzing and adjusting the system's behavior" refers to analyzing the emotional state and changing the system's behavior accordingly.

[0399] This invention is a surveillance system for large spaces such as public buildings and squares, which detects abnormal behavior in real time and adjusts the system's behavior by analyzing the user's emotional state. The system consists of a camera device, a server, a terminal application, and an emotion engine.

[0400] System configuration

[0401] 1. Camera equipment

[0402] The cameras have high-resolution, wide-angle lenses that can cover a wide area. They are recommended to be installed in strategic locations inside public buildings or plazas. The cameras transmit video data to a server in real time.

[0403] 2. Server

[0404] The server is equipped with an AI algorithm that analyzes the received video data and detects abnormal behavior. The server includes the following main components:

[0405] Real-time video analysis method: OpenCV and TensorFlow are used to analyze video data and detect abnormal behavior.

[0406] Notification method: Uses the Flask framework to send notifications to the user's device in real time.

[0407] Emotional state analysis means: Azure® Emotion API is used to analyze the user's emotional state and adjust the system's behavior.

[0408] Data storage method: MongoDB is used to store abnormal behavior data and emotion data, and data analysis is performed using Pandas.

[0409] 3. Terminal Applications

[0410] Users can use the terminal application to manage monitoring areas and notification settings, such as the area they want to monitor, the time period, and the frequency of notifications.

[0411] 4. Emotion Engine

[0412] The emotion engine analyzes the user's facial expressions and voice data in real time and sends the results to the server, which adjusts the content and frequency of notifications based on the user's emotional state, supporting efficient monitoring.

[0413] Specific examples

[0414] For example, when implementing it as a surveillance system for a shopping mall, multiple cameras are installed on each floor and at the entrance. The video data captured by the cameras is sent to a server in real time. The server uses OpenCV to analyze the video and detects abnormal behavior, such as suspiciously left luggage or suspicious movements in a crowd. Any abnormal behavior detected is immediately reported, and a warning is sent to the user's device.

[0415] At the same time, the emotion engine analyzes the user's facial expressions and voice, and if the administrator is feeling stressed, it adjusts the frequency and content of notifications to reduce the burden. For example, even if multiple anomalies are detected, the system automatically adjusts to prioritize notifications of only the most important detection results, and reports other information together later.

[0416] Below is an example of a prompt sentence to input to the generative AI model.

[0417] import cv2

[0418] import tensorflow as tf

[0419] from azure.ai.textanalytics import TextAnalyticsClient

[0420] from azure.core.credentials import AzureKeyCredential

[0421] import pymongo

[0422] from pandas import DataFrame

[0423] Importing camera footage

[0424] cap = cv2.VideoCapture(0)

[0425] Loading an AI model

[0426] model = tf.keras.models.load_model('anomaly_detection_model.h5')

[0427] Azure Emotion Engine Client Configuration

[0428] credential = AzureKeyCredential("API_KEY")

[0429] client = TextAnalyticsClient(endpoint="https: / / YOUR_ENDPOINT.cognitiveservices.azure.com / ", credential=credential)

[0430] MongoDB Configuration

[0431] client = pymongo.MongoClient("mongodb: / / localhost:27017 / ")

[0432] db = client["security_data"]

[0433] collection = db["anomaly_records"]

[0434] Video analysis processing

[0435] while True:

[0436] ret, frame = cap.read()

[0437] if not ret:

[0438] break

[0439] Input the frame into the AI ​​model

[0440] predictions = model.predict(frame)

[0441] if is_anomaly_detected(predictions):

[0442] What to do if an abnormality is detected

[0443] emotion_state = client.analyze_sentiment(documents=["Get the administrator's emotional state"])

[0444] collection.insert_one({"frame": frame.tolist(), "emotion": emotion_state[0].sentiment.name})

[0445] if emotion_state[0].sentiment.name != 'negative':

[0446] Notification Processing

[0447] send_alert_to_admin(frame, emotion_state[0].sentiment.name)

[0448] System shutdown processing

[0449] cap.release()

[0450] cv2.destroyAllWindows()

[0451] This embodiment makes it possible to realize efficient and effective wide-area monitoring while reducing the burden on the user.

[0452] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0453] Step 1:

[0454] The server receives video data in real time from the camera devices, which monitor a designated area and provide high-resolution video, and transmits the video data as successive frames to the server.

[0455] Step 2:

[0456] The server processes the received video data using OpenCV to split the individual frames, converting the video frames into an analyzable format.

[0457] Step 3:

[0458] The server uses TensorFlow to input the analyzed frame data into an anomalous behavior detection model. The model analyzes the movement patterns of people and objects in each frame and detects anomalous behavior (e.g., suspicious movements or abandoned luggage). The input is the analyzed frame data, and the output is the anomalous behavior detection results.

[0459] Step 4:

[0460] When abnormal behavior is detected, the server generates an anomaly detection notification using the Flask framework. This notification includes image data for the frame in which the anomaly was identified and the type of anomaly detected. The notification is sent to the user's device in real time.

[0461] Step 5:

[0462] The user's device receives anomaly detection notifications sent from the server in real time. The received notification includes an image of the frame showing the abnormal behavior and details of the anomaly. The user can check the content of the notification and take the necessary action immediately.

[0463] Step 6:

[0464] The server uses the Azure Emotion API to analyze the user's emotional state. The analysis uses the user's facial expression and voice data acquired in real time. The input is the user's facial expression and voice data, and the output is the emotion analysis results.

[0465] Step 7:

[0466] The server adjusts the system's behavior based on the results of emotion analysis. Specifically, if the user is feeling stressed, the server adjusts the frequency and content of notifications to reduce the burden on the user. This allows the user to perform monitoring tasks more efficiently.

[0467] Step 8:

[0468] The server stores anomaly detection data and emotion analysis data in MongoDB. The stored data is later used for analysis and report creation. The stored data includes video data, types of abnormal behavior, emotion analysis results, etc.

[0469] Step 9:

[0470] The server analyzes the stored data using Pandas and generates weekly or monthly reports, including the frequency and trends of abnormal behavior and fluctuations in the emotional state of administrators. The reports are provided to administrators to help them improve their monitoring measures.

[0471] Through the above processing steps, the system can efficiently and effectively realize wide-area monitoring and reduce the burden on the user.

[0472] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0473] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0474] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0475] [Second embodiment]

[0476] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0477] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0478] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0479] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0480] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0481] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0482] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0483] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0484] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0485] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0486] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0487] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0488] This invention relates to a system that automates the management of basketball games. By using AI to perform tasks such as filming the game, refereeing, scoring, timekeeping, and creating video of the game, it prevents manual errors in judgment and scoring, and realizes accurate and efficient game management.

[0489] System configuration

[0490] The system includes the following main components:

[0491] 1. Camera equipment

[0492] Device: The device will be installed in a position that overlooks the entire court. The camera has a high resolution and a wide-angle lens, allowing for clear capture of the entire match.

[0493] 2. Server

[0494] Server: This is the central system that analyzes real-time video data received from the cameras. It uses AI algorithms and image recognition technology to identify the positions of players and the ball and make necessary decisions.

[0495] 3. Smart Device Applications

[0496] User: Uses an application to input and manage match and player information. Before a match, users can input information such as player names, uniform numbers, and positions, and can also check the situation during the match.

[0497] Embodiments of the invention

[0498] Filming the match

[0499] Device: Position the camera so that it can capture the entire court. Conduct test shots to ensure the camera captures all action during the match.

[0500] Entering match information

[0501] User: Launches the application on their smart device and inputs information such as the player's name, uniform number, and position. They also input information such as the start time of the game and the opposing team. The input data is sent to the server.

[0502] Real-time analytics

[0503] Server: Receives video data from the camera in real time and uses image recognition technology to identify the positions of players and the ball. AI algorithms distinguish between shots, passes, dribbles, and other plays, and automatically determines scores, fouls, and violations. The results are instantly updated on the scoreboard.

[0504] For example, when a player shoots from outside the three-point line, the system detects the shot and determines whether the ball went into the goal. If the goal is successful, the server immediately updates the score to three points.

[0505] Timekeeping

[0506] Server: Starts the timer at the start of the game and accurately manages the time for each quarter and timeout. Stops and restarts the timer at the appropriate time during timeouts and other interruptions in the game.

[0507] Post-match data generation

[0508] Server: After the match, it consolidates all data collected during the match to generate statistics and match records, extracts important moments from the match to create highlight videos, and edits and stores full match videos.

[0509] Users: Through their smart devices, they can access the generated statistics and videos and download them as needed.

[0510] Specific examples

[0511] For example, if Player B commits a foul during a match, the server analyzes the video data capturing that moment. The AI ​​determines Player B's movements and automatically determines whether it was a foul based on the rules. The determination result is immediately reflected on the scoreboard, and the foul count is updated. After the match ends, detailed statistical data, including a record of the fouls, is automatically generated and available for users to review.

[0512] In this way, the system of the present invention dramatically improves the efficiency of game management and enables fair and accurate refereeing and scoring.

[0513] The processing flow will be explained below.

[0514] Step 1:

[0515] The user sets the camera in a position that overlooks the entire court and adjusts it so that there are no blind spots. The camera is connected to the server via a network.

[0516] Step 2:

[0517] The user launches the application on their smart device and enters game information (start time, opposing teams, game format, etc.) and player information (name, uniform number, position, etc.). The entered data is sent to the server.

[0518] Step 3:

[0519] The server receives video data from the camera in real time and stores the received video data in a buffer for analysis.

[0520] Step 4:

[0521] The server analyzes the video data frame by frame and uses image recognition technology to identify the positions of players and the ball, tracking player movements.

[0522] Step 5:

[0523] The server uses AI algorithms to detect player actions (shooting, passing, dribbling, etc.), identify specific movement patterns, and determine events (goals, fouls, violations, etc.).

[0524] Step 6:

[0525] The server automatically determines the score based on detected events (e.g., goals, fouls), and updates the scoreboard in real time, updating individual performance data.

[0526] Step 7:

[0527] The server initiates timekeeping, accurately measures the start and end times of the game, the duration of each quarter, and timeouts, and stops and restarts game time as necessary.

[0528] Step 8:

[0529] The server consolidates all data collected during the game and automatically generates statistics and match records after the game ends, compiling detailed statistics for each player, such as points, rebounds, assists, and fouls.

[0530] Step 9:

[0531] The server analyzes video data during the match, extracts important scenes to create a highlight video, and edits and saves the full match video in the appropriate format.

[0532] Step 10:

[0533] Users can view the generated statistical data and game videos through their smart devices and download them as needed. The server then uploads the data and videos to the cloud and provides users with an access link.

[0534] Example 1

[0535] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0536] Traditional basketball game management relies on human intervention, which means that misjudgments and errors in scoring are likely to occur. Furthermore, refereeing, scoring, and timekeeping require a great deal of labor, making efficient management difficult. Furthermore, generating statistical data and video after the game is also time-consuming, making it necessary to provide fast and accurate results.

[0537] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0538] In this invention, the server includes means for receiving video data of a game in real time using a high-resolution camera, means for analyzing the received video data and identifying the positions of players and the ball, means for determining player movements using an artificial intelligence algorithm and automatically determining scores and fouls, means for updating the determination results on the scoreboard in real time, means for managing the game time, means for generating statistical data and edited game videos after the game ends, and means for inputting game information and player information using a smart device, thereby enabling efficient game management and accurate determination and scoring.

[0539] A "high-resolution camera" is a high-resolution camera that can capture clear footage of the match.

[0540] "Player" means a player participating in a basketball game.

[0541] "Ball" refers to the ball equipment used in the game of basketball.

[0542] An "artificial intelligence algorithm" is an algorithm that analyzes players' movements, recognizes plays such as shots, passes, and dribbles, and automatically determines scores and fouls.

[0543] A "scoreboard" is a device that displays the score and other information of a game.

[0544] "Timekeeping" refers to the act of accurately managing the time of a match.

[0545] "Statistical Data" means data about player and team performance collected during a match.

[0546] "Edited match video" is a video in which important scenes are extracted and edited after the match has ended.

[0547] A "smart device" is a mobile terminal with internet connectivity, such as a smartphone or tablet.

[0548] "Match information" refers to basic information about the match, such as the start time and opposing teams.

[0549] "Player information" refers to information about players participating in a match, such as their names, uniform numbers, and positions.

[0550] MODE FOR CARRYING OUT THE INVENTION

[0551] This invention is a system that automates the management of basketball games, enabling efficient and accurate game management. This is achieved by using AI to perform tasks such as filming the game, refereeing, scorekeeping, timekeeping, and creating game videos.

[0552] Filming the match

[0553] Device: A high-resolution camera is installed in a position that allows a bird's-eye view of the entire court. This camera has a wide-angle lens, allowing it to clearly capture all the action of the match. For example, during camera installation and test shooting, the camera angle and focus are manually adjusted.

[0554] Entering match information

[0555] User: Launches the application on a smart device and inputs information such as player names, uniform numbers, positions, match start time, and opposing teams. This information is sent to the server in real time, and the match is ready. Specifically, the user uses a smartphone or tablet to enter data into the application's input form and presses the "Submit" button.

[0556] Real-time video reception and analysis

[0557] Server: Uses a means to receive and analyze real-time video data from the camera. The received video data is input into an image analysis algorithm to identify the positions of players and the ball. Next, an artificial intelligence algorithm is used to identify player movements and analyze plays such as shots, passes, and dribbles. The server uses libraries such as OpenCV and TensorFlow to perform image analysis and action recognition.

[0558] For example, the server extracts the characteristics of a player's movements and determines the shooting action based on them. If a player shoots from outside the three-point line, the server detects the movement and determines whether the ball went into the goal. If the goal is successful, the server updates the score as three points.

[0559] Timekeeping

[0560] Server: Starts the timer as soon as the game starts, and accurately manages the time for each quarter and timeout. When the game is interrupted, stops and restarts the time at the appropriate time. Specifically, it calls timer management functions to start, stop, and restart the time.

[0561] Post-match data generation

[0562] Server: After the match, it consolidates all collected data and generates statistics and match records. It extracts important scenes to create highlight videos, and edits and saves full match videos. For example, it aggregates data from each play and generates statistical data in Excel or CSV format. It also uses the VideoEditor API to edit videos.

[0563] View and download results

[0564] Users can access the generated statistics and videos through their smart devices and download them as needed. Specifically, they can tap the "View Results" button in the app, select a match from the list, and view the data. If necessary, they can press the "Download" button to save the data to their local device.

[0565] Prompt Sentence Examples

[0566] "Please explain how to accurately determine the exact moment a player shoots during a basketball game and automatically update the score."

[0567] This system will enable more efficient match management and accurate judgment and scoring, thereby achieving fair, fast and accurate match management.

[0568] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0569] Step 1:

[0570] The device installs a high-resolution camera in a position that provides a bird's-eye view of the court. This camera has a wide-angle lens that captures the entire match clearly. Specific operations include adjusting the camera's angle and focus appropriately to cover the optimal field of view. The physical position and settings of the camera are specified as input, and high-resolution video is obtained as output.

[0571] Step 2:

[0572] The user starts up the smart device and launches the application. There, they enter game information such as player names, uniform numbers, positions, game start time, and opposing teams. The entered information is sent to the server in real time. Specifically, the user enters data into the input form on the smart device and presses the "Send" button. Player information and game information are given as input, and this information is sent to the server as output.

[0573] Step 3:

[0574] The server receives real-time video data from the camera. This video data is decoded and converted into an analyzable format. Specifically, the server decodes the video stream frame by frame and saves it as frame data. The video stream from the camera is given as input, and analyzable frame data is obtained as output.

[0575] Step 4:

[0576] The server inputs the received video data into an image analysis algorithm. This algorithm uses libraries such as OpenCV and TensorFlow to identify the positions of players and the ball. Specifically, it performs image analysis to detect player movements and the position of the ball. Frame data is given as input, and player and ball position data is obtained as output.

[0577] Step 5:

[0578] The server uses an artificial intelligence algorithm to determine player actions. This algorithm distinguishes between plays such as shooting, passing, and dribbling, and automatically determines scores and fouls. Specific actions are input into a motion recognition model, and a judgment result is obtained. Position data of the player and ball is given as input, and judgment data is obtained as output.

[0579] Step 6:

[0580] The server reflects the decision results on the scoreboard in real time. For example, if a player makes a successful shot from outside the three-point line, the score is updated to three points. Specifically, the server calls the score update API and updates the scoreboard. The decision data is given as input, and the updated score is displayed as output.

[0581] Step 7:

[0582] The server starts the timer as soon as the game starts, and accurately manages the time for each quarter and timeout. When the game is interrupted, it stops and restarts the time at the appropriate timing. Specifically, it calls timer management functions to start, stop, and restart the time. The game start time is given as input, and time management data is obtained as output.

[0583] Step 8:

[0584] After the match, the server consolidates all collected data and generates statistical information and match records. It extracts important scenes to create highlight videos, and edits and saves the full match video. Specific operations include aggregating data and editing the video using the VideoEditor API. The data collected during the match is given as input, and statistical data and edited video are obtained as output.

[0585] Step 9:

[0586] Users can access the generated statistical data and videos through their smart devices and download them as needed. Specifically, they tap the "View Results" button on the app, select a match from the list, and check the data. The user request is given as input, and the displayed data can be viewed as output. They can also save the data to their local device by pressing the "Download" button.

[0587] (Application example 1)

[0588] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0589] In traditional brick-and-mortar store operations, there were limited ways to understand customer behavior patterns and product popularity in real time, making it difficult to operate the store efficiently and provide appropriate customer service.In addition, store managers lacked a way to instantly grasp the congestion situation in the store or when it was necessary to replenish shelves, so they were required to respond quickly.

[0590] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0591] In this invention, the server includes means for receiving video data of the game in real time, means for analyzing the received video data and identifying the positions of objects and elements, means for determining the behavior of objects and automatically determining information and status, means for reflecting the determination results on a display device in real time, means for managing time, means for generating statistical data and videos after the game ends, means for analyzing the situation in the store in real time and identifying behavior patterns and product popularity, and means for notifying the user of the real-time analysis results. This allows store managers to grasp customer behavior patterns and product popularity in real time, and to immediately grasp the congestion situation in the store and when it is necessary to replenish shelves.

[0592] "Game video data" refers to video information received from cameras in real time during a game.

[0593] "Means for receiving" refers to the device or software function for acquiring the video data transmitted from the camera.

[0594] "Means for analyzing and locating objects and elements" refers to technologies and algorithms that recognize the location of specific objects or people based on video data.

[0595] "Means for identifying the behavior of a target and automatically determining its information and status" refers to technologies and algorithms that analyze the movements of a target object or person and automatically evaluate and judge its behavior and status.

[0596] "Means for reflecting the determination results on a display device in real time" refers to a function for instantly displaying the results of analysis and determination on a display device such as a monitor.

[0597] "Time management means" refers to a device or software function that measures and manages the time required for a match or operation.

[0598] "Means for generating post-match statistics and video" refers to the ability to consolidate collected data and create aggregated statistics and edited video after a match or event has concluded.

[0599] "Means of analyzing the situation within the store in real time and identifying behavioral patterns and product popularity" refers to technology that analyzes customer movement within the store and the level of interest in products in real time, and uses that data to identify behavioral patterns and popular products.

[0600] "Means for notifying real-time analysis results" refers to systems or functions that immediately notify relevant parties of the results obtained through analysis.

[0601] Overall system overview

[0602] This system automates customer behavior analysis and in-store management to improve the efficiency of brick-and-mortar store operations, and is primarily comprised of cameras, servers, and smart devices.

[0603] Hardware and Software

[0604] Hardware:

[0605] High-resolution wide-angle cameras capture customer movement and behavior within the store in real time.

[0606] The server is a central device that analyzes the received video data.

[0607] Smart devices (PDAs) are used by managers to view the situation in the store in real time.

[0608] software:

[0609] AI image recognition software (e.g., OpenCV and TensorFlow) analyzes video data to identify customer behavior and product popularity.

[0610] Data analysis algorithms identify customer behavior patterns and popular products.

[0611] The store management app provides analysis results and inventory information to managers.

[0612] System configuration

[0613] The system includes the following main components:

[0614] 1. Camera equipment

[0615] Device: High-resolution wide-angle cameras are installed in locations that overlook each area of ​​the store, capturing all of the customer's movements and actions.

[0616] 2. Server

[0617] Server: This is the central system that analyzes real-time video data received from in-store cameras. It uses AI image recognition software (e.g., OpenCV or TensorFlow) and data analysis algorithms to identify customer behavior and product popularity. It also notifies the store management app of the analysis results in real time.

[0618] 3. Store management application

[0619] User: This application is used by administrators to check customer behavior analysis data, inventory status, and sales information in real time. It is operated on smart devices such as PDAs.

[0620] Specific examples

[0621] For example, if analysis shows that a particular product shelf in a store in a shopping mall is extremely popular, this information will be notified to the manager, who can then take prompt action such as adding new products to that shelf.

[0622] Prompt Sentence Examples

[0623] Here are some example prompts for using generative AI models:

[0624] "Create an AI algorithm that analyzes customer behavior in stores and identifies popular areas and products."

[0625] "Analyze in-store congestion in real time and design a program to optimize staff allocation."

[0626] In this way, implementing this system is expected to improve the efficiency of physical store operations and customer service.

[0627] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0628] Step 1:

[0629] The device uses a high-resolution, wide-angle camera installed inside the store to capture customer movement and behavior in real time.

[0630] Input: Real-time video data captured by a high-resolution wide-angle camera.

[0631] Data processing / data calculation: Video data is acquired and sent directly to the server.

[0632] Output: Real-time video data is sent to the server.

[0633] Step 2:

[0634] The video data received by the server is analyzed using AI image recognition software (e.g., OpenCV or TensorFlow).

[0635] Input: Real-time video data received from a high-resolution wide-angle camera.

[0636] Data processing / data calculation: Apply object recognition and motion recognition algorithms to extract customer positions and movements from video data.

[0637] Output: Customer location and movement information.

[0638] Step 3:

[0639] The server uses data analysis algorithms to identify customer behavior patterns and product popularity.

[0640] Input: Customer location and movement information.

[0641] Data processing / data calculation: Analyze behavioral patterns based on location and movement information to calculate the popularity of specific areas and products.

[0642] Output: Behavioral pattern information and popular product information.

[0643] Step 4:

[0644] The server notifies the store management app of the analysis results in real time.

[0645] Input: behavioral pattern information and popular product information.

[0646] Data processing / data calculation: Formats the analysis results and generates notification messages.

[0647] Output: Real-time notification to the administrator's smart device.

[0648] Step 5:

[0649] The user (administrator) uses the store management app to check customer behavior analysis data, inventory status, and sales information in real time and take necessary action.

[0650] Input: Real-time notifications and analysis results displayed in the store management app.

[0651] Data processing / data calculation: Decide on product replenishment and staff allocation based on the information received.

[0652] Output: Improved store operations and faster response times.

[0653] Step 6:

[0654] After a match or event ends, the server consolidates the collected data and automatically generates statistical data and edited videos.

[0655] Input: Customer behavioral pattern information and sales data.

[0656] Data processing / data calculation: Statistical processing of all data and generation of statistical reports and videos.

[0657] Output: Statistical report and highlight video.

[0658] The above steps will realize a system that is expected to make physical store operations more efficient and improve customer service.

[0659] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0660] This invention relates to a system that automates the management of basketball games, in which AI performs the tasks of filming the game, refereeing, recording the score, timekeeping, and creating game videos, and by combining it with an emotion engine, it recognizes the user's emotions in real time and adjusts the system's behavior.

[0661] System configuration

[0662] The system includes the following main components:

[0663] 1. Camera equipment

[0664] Device: Installed in a position overlooking the entire court, it is equipped with a high-resolution, wide-angle lens to capture clear images of the entire game.

[0665] 2. Server

[0666] Server: Analyzes real-time video data received from the camera to identify the positions of players and the ball, and uses AI algorithms and image recognition technology to make decisions and provide scoring.

[0667] 3. Smart Device Applications

[0668] User: Uses an application to input and manage match and player information. Before a match, the user can input player names, uniform numbers, positions, etc., and check the situation during the match.

[0669] 4. Emotion Engine

[0670] Server: Equipped with an engine that analyzes the user's facial expressions and voice to recognize their emotional state, and adjusts the system's behavior based on the recognized emotional data.

[0671] Embodiments of the invention

[0672] Filming the match

[0673] Device: Position the camera so that it can capture the entire court. Conduct test shots to ensure the camera captures all action during the match.

[0674] Entering match information

[0675] User: Launches the application on their smart device and inputs information such as the player's name, uniform number, and position. They also input information such as the start time of the game and the opposing team. The input data is sent to the server.

[0676] Real-time analytics

[0677] Server: Receives video data from the camera in real time and uses image recognition technology to identify the positions of players and the ball. AI algorithms distinguish between shots, passes, dribbles, and other plays, and automatically determines scores, fouls, and violations. The results are instantly updated on the scoreboard.

[0678] For example, when a player shoots from outside the three-point line, the system detects the shot and determines whether the ball went into the goal. If the goal is successful, the server immediately updates the score to three points.

[0679] Timekeeping

[0680] Server: Starts the timer at the start of the game and accurately manages the time for each quarter and timeout. Stops and restarts the timer at the appropriate time during timeouts and other interruptions in the game.

[0681] Recognizing and reflecting emotions

[0682] Server: Using the emotion engine, analyzes the user's facial expressions and voice data in real time to recognize their emotional state. Based on the recognized emotional data, the game progress and notification content are adaptively changed.

[0683] For example, if the system detects that the user is excited during a tense match, it will adjust the frequency of screen displays and notifications to prevent the user from feeling stressed. Conversely, if the system determines that the user is bored, it can send notifications highlighting the highlights of the match.

[0684] Post-match data generation

[0685] Server: After the match, it consolidates all data collected during the match to generate statistics and match records. It extracts key moments from the match to create highlight videos and provide emotional highlights that reflect the emotional data used. It also edits and saves the full match video in the appropriate format.

[0686] Users: Access the generated statistical data and videos through their smart devices and download them as needed. The server uploads the data and videos to the cloud and provides users with an access link.

[0687] Specific examples

[0688] For example, if Player B commits a foul during a match, the server analyzes the video data capturing that moment. The AI ​​distinguishes Player B's movements and automatically determines whether it was a foul based on the rules. At the same time, the emotion engine analyzes the user's reaction, and if it detects excitement or surprise, it can instantly provide a replay video that reflects that emotion.

[0689] By combining an emotion engine with video analysis, the system of the present invention can not only improve the efficiency of match management, but also the user experience.

[0690] The processing flow will be explained below.

[0691] Step 1:

[0692] The user sets the camera in a position that overlooks the entire court and adjusts it so that there are no blind spots. The camera is connected to the server via a network.

[0693] Step 2:

[0694] The user launches the application on their smart device and enters game information (start time, opposing teams, game format, etc.) and player information (name, uniform number, position, etc.). The entered data is sent to the server.

[0695] Step 3:

[0696] The server receives video data from the camera in real time and stores the received video data in a buffer for analysis.

[0697] Step 4:

[0698] The server analyzes the video data frame by frame and uses image recognition technology to identify the positions of players and the ball, tracking player movements.

[0699] Step 5:

[0700] The server uses AI algorithms to detect player actions (shooting, passing, dribbling, etc.), identify specific movement patterns, and determine events (goals, fouls, violations, etc.).

[0701] Step 6:

[0702] The server automatically determines the score based on detected events (e.g., goals, fouls), and updates the scoreboard in real time, updating individual performance data.

[0703] Step 7:

[0704] The server initiates timekeeping, accurately measures the start and end times of the game, the duration of each quarter, and timeouts, and stops and restarts game time as necessary.

[0705] Step 8:

[0706] The server uses an emotion engine to analyze the user's facial expressions and voice data in real time, recognizes the user's emotional state based on the analysis results, and adjusts the system's behavior based on that data.

[0707] Step 9:

[0708] For example, if the user is excited, the server will minimize the information displayed to reduce the user's stress. Conversely, if the user is perceived as bored, the server will send highlight notifications more frequently.

[0709] Step 10:

[0710] The server consolidates all data collected during the game and automatically generates statistics and match records after the game ends, compiling detailed statistics for each player, such as points, rebounds, assists, and fouls.

[0711] Step 11:

[0712] The server analyzes video data during the match, extracts important scenes, and creates highlight videos. It also provides emotional highlight videos that reflect the emotional data.

[0713] Step 12:

[0714] Users can access the generated statistical data and emotional highlight videos through their smart devices and download them as needed. The server uploads the data and videos to the cloud and provides users with an access link.

[0715] In this way, the system streamlines match management, provides fair and accurate refereeing and scoring, and also provides an interactive experience that responds to the user's emotions.

[0716] Example 2

[0717] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0718] Basketball game management requires a lot of manual work, and is prone to human error in refereeing, scoring, timekeeping, and game video production. Furthermore, there is a lack of ways to properly reflect user emotions during the game to provide a better experience.

[0719] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for receiving video data of the game in real time, means for analyzing the received video data and identifying the positions of the players and the ball, and means for analyzing the emotional state of the user in real time and adaptively adjusting the behavior of the system. This enables the automation of game management, reduces human error, and enables flexible responses according to the user's emotions.

[0720] "Means for receiving game video data in real time" refers to the cameras and communication infrastructure that capture video of the basketball game and transmit it to a server in real time.

[0721] "Means for analyzing received video data and identifying the positions of players and the ball" refers to algorithms or software that use image recognition technology to analyze and identify the positions of players and the ball from received video data.

[0722] "Means of determining player movements and automatically determining scores and fouls" refers to a system that uses AI algorithms to analyze player movements and automatically determine scores, fouls, and violations.

[0723] "Means for reflecting the judgement results on the scoreboard in real time" refers to a communication system and a display device for instantly displaying the judgement results on the scoreboard.

[0724] "Means for controlling the time of a match" means a digital timer or control system for accurately measuring the start, finish and stoppage times of a match.

[0725] "Means for analyzing the user's emotional state in real time and adaptively adjusting the system's behavior" refers to an emotion recognition engine that analyzes the user's facial expressions and voice data and dynamically changes the system's behavior and notification content based on that emotional state.

[0726] "Means for generating post-match statistics and video" means software and servers that aggregate data collected during a match and generate statistics, match records, and edited match video.

[0727] "Using a camera to capture a bird's-eye view of the entire court" refers to a method of capturing the entire court with a wide-angle lens to monitor the movement of all players and the ball at once.

[0728] "Entering match and player information using a smart device" refers to the process of entering player and match information before a match using a digital device such as a smartphone or tablet.

[0729] Overall system overview

[0730] The present invention is a system for automating basketball game management that integrates game filming, refereeing, scorekeeping, timekeeping, game video production, and an emotion engine that recognizes user emotions in real time and adjusts the system's behavior accordingly.

[0731] System Components

[0732] 1. Terminal (camera device)

[0733] A device equipped with a high-resolution, wide-angle lens is placed in a position that allows it to capture a bird's-eye view of the entire court. The camera is used to capture clear images of the entire game. For example, the camera can take a panoramic image of the entire court before the start of a game and send the image to a server.

[0734] 2. Server

[0735] The server has the following main functions:

[0736] 1. Receiving and analyzing video data: Video data is received in real time and the positions of players and the ball are analyzed using image recognition technology (e.g., OpenCV or TensorFlow).

[0737] 2. Judgment by AI algorithm: Based on the AI ​​algorithm, plays such as shots, passes, and dribbles are judged, and scores, fouls, and violations are automatically determined.

[0738] 3. Reflection of judgement results: The judgement results will be reflected immediately on the scoreboard.

[0739] 4. Timekeeping: Manage the game time, start the timer at the start of the game, and accurately manage the times of each quarter and timeout.

[0740] 5. Emotion Engine: Analyzes the user's facial expressions and voice data in real time to recognize their emotional state and adaptively adjust the system's behavior. For example, if the emotion engine detects that the user is excited, it will reduce the frequency of notifications.

[0741] 6. Data generation: Generate statistics and videos after the game, extract key moments and create highlight videos.

[0742] 3. User (Smart Device Application)

[0743] The user does the following:

[0744] 1. Entering match and player information: Using the smart device application, users enter information such as player names, uniform numbers, positions, match start times, and opposing teams. This information is then sent to the server.

[0745] 2. Data and video access: Access the generated statistical data and videos through smart devices and download them as needed. The server uploads the data and videos to the cloud and provides an access link.

[0746] Specific examples

[0747] For example, if player B commits a foul during a game, the server analyzes the video data capturing that moment. The AI ​​distinguishes player B's movements and automatically determines whether it was a foul based on the rules. At the same time, the emotion engine analyzes the user's reaction, and if it detects excitement or surprise, it instantly provides a replay video that reflects that emotion.

[0748] Example prompts to input to the generative AI model

[0749] Design a system that automatically identifies specific plays in an ongoing basketball game and adaptively adjusts notifications based on the user's emotional state. This system includes a camera that records the game, a server that analyzes real-time footage, a smart device application that manages game information, and an emotion engine that recognizes the user's emotions.

[0750] The above is an embodiment of the present invention that makes it easy to implement. This system can automate the management of matches and improve the user experience.

[0751] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0752] Step 1: Filming the match

[0753] Terminal

[0754] Input: Notification of the start of the match

[0755] Operation:

[0756] 1. A camera with a high resolution and wide-angle lens will be installed in a position that allows for a bird's-eye view of the entire court.

[0757] 2. The camera prepares to capture the entire match in real time.

[0758] 3. Conduct test shooting to check and adjust the shooting range and image quality.

[0759] Output: High-resolution real-time video data

[0760] Step 2: Enter match information

[0761] User

[0762] Input: Pre-match player and match information

[0763] Operation:

[0764] 1. Launch the smart device application.

[0765] 2. Enter the player's name, uniform number, position, game start time, and opposing team information.

[0766] 3. The entered information is sent to the server.

[0767] Output: Match and player information sent to the server

[0768] Step 3: Receiving video data

[0769] server

[0770] Input: Real-time video data from the camera

[0771] Operation:

[0772] 1. Receive video data sent from the camera in real time.

[0773] 2. The received video data is saved and prepared for the next analysis step.

[0774] Output: Received and stored video data

[0775] Step 4: Analyzing the video data

[0776] server

[0777] Input: Received video data

[0778] Operation:

[0779] 1. Analyze the video data using image recognition technology (e.g., OpenCV, TensorFlow).

[0780] 2. Identify the location of the players and the ball.

[0781] Output: Player and ball position data

[0782] Step 5: Motion detection and automatic judgment

[0783] server

[0784] Input: Player and ball position data

[0785] Operation:

[0786] 1. Using AI algorithms to identify player actions (shooting, passing, dribbling, etc.).

[0787] 2. Automatically judge scores, fouls, and violations based on the game rules.

[0788] Output: Scores, fouls, and violations

[0789] Step 6: Reflecting the results

[0790] server

[0791] Input: Judgment result

[0792] Operation:

[0793] 1. The judgement results will be reflected immediately on the scoreboard.

[0794] 2. Send notifications to users as needed.

[0795] Output: Updated leaderboard

[0796] Step 7: Timekeeping

[0797] server

[0798] Input: Notification of the start of the match

[0799] Operation:

[0800] 1. Start the digital timer at the same time as the match begins.

[0801] 2. Accurately manage the time of each quarter and timeout.

[0802] 3. Stop and restart the clock at the appropriate time when the game is interrupted.

[0803] Output: Precisely controlled game time

[0804] Step 8: Recognize and reflect on your emotions

[0805] server

[0806] Input: User facial and voice data

[0807] Operation:

[0808] 1. Analyze the user's facial expressions and voice data in real time using an emotion engine.

[0809] 2. Recognize the user's emotional state and adaptively adjust the system's behavior (notification frequency, display content, etc.).

[0810] Output: Changing the system's behavior based on the recognized emotion data.

[0811] Step 9: Post-match data generation

[0812] server

[0813] Input: Data collected during the match

[0814] Operation:

[0815] 1. After the match, all data is aggregated to generate statistics and match records.

[0816] 2. Extract important scenes and create a highlight video.

[0817] 3. Edit and save the full match video in the appropriate format.

[0818] Output: Statistics, match records, highlight videos, full match videos

[0819] Step 10: Accessing Data and Videos

[0820] User

[0821] Enter: Post-match notification

[0822] Operation:

[0823] 1. Access the generated statistics and videos using your smart device.

[0824] 2. Download data and videos as needed.

[0825] Output: Downloadable statistics and a link to the video

[0826] This is the specific processing flow of this system. This system automates the management of matches and enables flexible responses according to the user's emotions.

[0827] (Application example 2)

[0828] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0829] The problem that this invention aims to solve is to reduce the stress of managers in large spaces such as public buildings and plazas by detecting abnormal behavior and analyzing emotions, which conventional monitoring systems cannot handle. There is a need to detect abnormalities in real time and respond efficiently while reducing the burden on managers.

[0830] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving video data of the game in real time, means for analyzing the received video data and identifying the positions of players and the ball, means for determining player actions and automatically determining scores and fouls, means for reflecting the determination results on the scoreboard in real time, means for managing the game time, means for generating statistical data and videos after the game ends, means for detecting and notifying abnormal behavior, and means for analyzing the user's emotional state and adjusting the system's behavior. This enables efficient monitoring of large spaces, rapid detection of abnormal behavior, and adjustment of the system's behavior in accordance with the administrator's emotional state.

[0831] "Video data" refers to digital data of video footage captured by a camera.

[0832] "Real-time" refers to processing, sending, and receiving information instantly, without delay.

[0833] "Analysis" is the act of extracting specific patterns or information from digital data.

[0834] "Player" refers to a person participating in a sport or action.

[0835] "Ball" refers to a spherical object used for sports and games.

[0836] "Locate" means to determine the current coordinates or location of an object or person.

[0837] "Distinguishing behavior" refers to recognizing and classifying the behavior of a target person or object.

[0838] A "score" is a numerical value that indicates the score in a sporting event.

[0839] A "foul" is an act that violates the rules in a sporting event.

[0840] "Decision result" refers to a judgment or conclusion drawn based on an analysis.

[0841] A "scoreboard" is a device or screen that displays scores and other information from a sports event.

[0842] "Time management" means monitoring and regulating the progress of a competition or task.

[0843] "Statistical data" is quantitative information obtained from the results of analysis.

[0844] "Generating a moving image" means editing still images and video data to create a moving image.

[0845] "Abnormal behavior" refers to unusual or suspicious behavior that differs from normal behavior.

[0846] A "notification" is a message or signal that conveys specific information.

[0847] "Emotional state" indicates the type and strength of the emotion a person is feeling at the time.

[0848] "Analyzing and adjusting the system's behavior" refers to analyzing the emotional state and changing the system's behavior accordingly.

[0849] This invention is a surveillance system for large spaces such as public buildings and squares, which detects abnormal behavior in real time and adjusts the system's behavior by analyzing the user's emotional state. The system consists of a camera device, a server, a terminal application, and an emotion engine.

[0850] System configuration

[0851] 1. Camera equipment

[0852] The cameras have high-resolution, wide-angle lenses that can cover a wide area. They are recommended to be installed in strategic locations inside public buildings or plazas. The cameras transmit video data to a server in real time.

[0853] 2. Server

[0854] The server is equipped with an AI algorithm that analyzes the received video data and detects abnormal behavior. The server includes the following main components:

[0855] Real-time video analysis method: OpenCV and TensorFlow are used to analyze video data and detect abnormal behavior.

[0856] Notification method: Uses the Flask framework to send notifications to the user's device in real time.

[0857] Emotional state analysis: Azure Emotion API is used to analyze the user's emotional state and adjust the system's behavior.

[0858] Data storage method: MongoDB is used to store abnormal behavior data and emotion data, and data analysis is performed using Pandas.

[0859] 3. Terminal Applications

[0860] Users can use the terminal application to manage monitoring areas and notification settings, such as the area they want to monitor, the time period, and the frequency of notifications.

[0861] 4. Emotion Engine

[0862] The emotion engine analyzes the user's facial expressions and voice data in real time and sends the results to the server, which adjusts the content and frequency of notifications based on the user's emotional state, supporting efficient monitoring.

[0863] Specific examples

[0864] For example, when implementing it as a surveillance system for a shopping mall, multiple cameras are installed on each floor and at the entrance. The video data captured by the cameras is sent to a server in real time. The server uses OpenCV to analyze the video and detects abnormal behavior, such as suspiciously left luggage or suspicious movements in a crowd. Any abnormal behavior detected is immediately reported, and a warning is sent to the user's device.

[0865] At the same time, the emotion engine analyzes the user's facial expressions and voice, and if the administrator is feeling stressed, it adjusts the frequency and content of notifications to reduce the burden. For example, even if multiple anomalies are detected, the system automatically adjusts to prioritize notifications of only the most important detection results, and reports other information together later.

[0866] Below is an example of a prompt sentence to input to the generative AI model.

[0867] import cv2

[0868] import tensorflow as tf

[0869] from azure.ai.textanalytics import TextAnalyticsClient

[0870] from azure.core.credentials import AzureKeyCredential

[0871] import pymongo

[0872] from pandas import DataFrame

[0873] Importing camera footage

[0874] cap = cv2.VideoCapture(0)

[0875] Loading an AI model

[0876] model = tf.keras.models.load_model('anomaly_detection_model.h5')

[0877] Azure Emotion Engine Client Configuration

[0878] credential = AzureKeyCredential("API_KEY")

[0879] client = TextAnalyticsClient(endpoint="https: / / YOUR_ENDPOINT.cognitiveservices.azure.com / ", credential=credential)

[0880] MongoDB Configuration

[0881] client = pymongo.MongoClient("mongodb: / / localhost:27017 / ")

[0882] db = client["security_data"]

[0883] collection = db["anomaly_records"]

[0884] Video analysis processing

[0885] while True:

[0886] ret, frame = cap.read()

[0887] if not ret:

[0888] break

[0889] Input the frame into the AI ​​model

[0890] predictions = model.predict(frame)

[0891] if is_anomaly_detected(predictions):

[0892] What to do if an abnormality is detected

[0893] emotion_state = client.analyze_sentiment(documents=["Get the administrator's emotional state"])

[0894] collection.insert_one({"frame": frame.tolist(), "emotion": emotion_state[0].sentiment.name})

[0895] if emotion_state[0].sentiment.name != 'negative':

[0896] Notification Processing

[0897] send_alert_to_admin(frame, emotion_state[0].sentiment.name)

[0898] System shutdown processing

[0899] cap.release()

[0900] cv2.destroyAllWindows()

[0901] This embodiment makes it possible to realize efficient and effective wide-area monitoring while reducing the burden on the user.

[0902] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0903] Step 1:

[0904] The server receives video data in real time from the camera devices, which monitor a designated area and provide high-resolution video, and transmits the video data as successive frames to the server.

[0905] Step 2:

[0906] The server processes the received video data using OpenCV to split the individual frames, converting the video frames into an analyzable format.

[0907] Step 3:

[0908] The server uses TensorFlow to input the analyzed frame data into an anomalous behavior detection model. The model analyzes the movement patterns of people and objects in each frame and detects anomalous behavior (e.g., suspicious movements or abandoned luggage). The input is the analyzed frame data, and the output is the anomalous behavior detection results.

[0909] Step 4:

[0910] When abnormal behavior is detected, the server generates an anomaly detection notification using the Flask framework. This notification includes image data for the frame in which the anomaly was identified and the type of anomaly detected. The notification is sent to the user's device in real time.

[0911] Step 5:

[0912] The user's device receives anomaly detection notifications sent from the server in real time. The received notification includes an image of the frame showing the abnormal behavior and details of the anomaly. The user can check the content of the notification and take the necessary action immediately.

[0913] Step 6:

[0914] The server uses the Azure Emotion API to analyze the user's emotional state. The analysis uses the user's facial expression and voice data acquired in real time. The input is the user's facial expression and voice data, and the output is the emotion analysis results.

[0915] Step 7:

[0916] The server adjusts the system's behavior based on the results of emotion analysis. Specifically, if the user is feeling stressed, the server adjusts the frequency and content of notifications to reduce the burden on the user. This allows the user to perform monitoring tasks more efficiently.

[0917] Step 8:

[0918] The server stores anomaly detection data and emotion analysis data in MongoDB. The stored data is later used for analysis and report creation. The stored data includes video data, types of abnormal behavior, emotion analysis results, etc.

[0919] Step 9:

[0920] The server analyzes the stored data using Pandas and generates weekly or monthly reports, including the frequency and trends of abnormal behavior and fluctuations in the emotional state of administrators. The reports are provided to administrators to help them improve their monitoring measures.

[0921] Through the above processing steps, the system can efficiently and effectively realize wide-area monitoring and reduce the burden on the user.

[0922] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0923] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0924] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0925] [Third embodiment]

[0926] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0927] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0928] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0929] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0930] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0931] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0932] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0933] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0934] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0935] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0936] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0937] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0938] This invention relates to a system that automates the management of basketball games. By using AI to perform tasks such as filming the game, refereeing, scoring, timekeeping, and creating video of the game, it prevents manual errors in judgment and scoring, and realizes accurate and efficient game management.

[0939] System configuration

[0940] The system includes the following main components:

[0941] 1. Camera equipment

[0942] Device: The device will be installed in a position that overlooks the entire court. The camera has a high resolution and a wide-angle lens, allowing for clear capture of the entire match.

[0943] 2. Server

[0944] Server: This is the central system that analyzes real-time video data received from the cameras. It uses AI algorithms and image recognition technology to identify the positions of players and the ball and make necessary decisions.

[0945] 3. Smart Device Applications

[0946] User: Uses an application to input and manage match and player information. Before a match, users can input information such as player names, uniform numbers, and positions, and can also check the situation during the match.

[0947] Embodiments of the invention

[0948] Filming the match

[0949] Device: Position the camera so that it can capture the entire court. Conduct test shots to ensure the camera captures all action during the match.

[0950] Entering match information

[0951] User: Launches the application on their smart device and inputs information such as the player's name, uniform number, and position. They also input information such as the start time of the game and the opposing team. The input data is sent to the server.

[0952] Real-time analytics

[0953] Server: Receives video data from the camera in real time and uses image recognition technology to identify the positions of players and the ball. AI algorithms distinguish between shots, passes, dribbles, and other plays, and automatically determines scores, fouls, and violations. The results are instantly updated on the scoreboard.

[0954] For example, when a player shoots from outside the three-point line, the system detects the shot and determines whether the ball went into the goal. If the goal is successful, the server immediately updates the score to three points.

[0955] Timekeeping

[0956] Server: Starts the timer at the start of the game and accurately manages the time for each quarter and timeout. Stops and restarts the timer at the appropriate time during timeouts and other interruptions in the game.

[0957] Post-match data generation

[0958] Server: After the match, it consolidates all data collected during the match to generate statistics and match records, extracts important moments from the match to create highlight videos, and edits and stores full match videos.

[0959] Users: Through their smart devices, they can access the generated statistics and videos and download them as needed.

[0960] Specific examples

[0961] For example, if Player B commits a foul during a match, the server analyzes the video data capturing that moment. The AI ​​determines Player B's movements and automatically determines whether it was a foul based on the rules. The determination result is immediately reflected on the scoreboard, and the foul count is updated. After the match ends, detailed statistical data, including a record of the fouls, is automatically generated and available for users to review.

[0962] In this way, the system of the present invention dramatically improves the efficiency of game management and enables fair and accurate refereeing and scoring.

[0963] The processing flow will be explained below.

[0964] Step 1:

[0965] The user sets the camera in a position that overlooks the entire court and adjusts it so that there are no blind spots. The camera is connected to the server via a network.

[0966] Step 2:

[0967] The user launches the application on their smart device and enters game information (start time, opposing teams, game format, etc.) and player information (name, uniform number, position, etc.). The entered data is sent to the server.

[0968] Step 3:

[0969] The server receives video data from the camera in real time and stores the received video data in a buffer for analysis.

[0970] Step 4:

[0971] The server analyzes the video data frame by frame and uses image recognition technology to identify the positions of players and the ball, tracking player movements.

[0972] Step 5:

[0973] The server uses AI algorithms to detect player actions (shooting, passing, dribbling, etc.), identify specific movement patterns, and determine events (goals, fouls, violations, etc.).

[0974] Step 6:

[0975] The server automatically determines the score based on detected events (e.g., goals, fouls), and updates the scoreboard in real time, updating individual performance data.

[0976] Step 7:

[0977] The server initiates timekeeping, accurately measures the start and end times of the game, the duration of each quarter, and timeouts, and stops and restarts game time as necessary.

[0978] Step 8:

[0979] The server consolidates all data collected during the game and automatically generates statistics and match records after the game ends, compiling detailed statistics for each player, such as points, rebounds, assists, and fouls.

[0980] Step 9:

[0981] The server analyzes video data during the match, extracts important scenes to create a highlight video, and edits and saves the full match video in the appropriate format.

[0982] Step 10:

[0983] Users can view the generated statistical data and game videos through their smart devices and download them as needed. The server then uploads the data and videos to the cloud and provides users with an access link.

[0984] Example 1

[0985] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0986] Traditional basketball game management relies on human intervention, which means that misjudgments and errors in scoring are likely to occur. Furthermore, refereeing, scoring, and timekeeping require a great deal of labor, making efficient management difficult. Furthermore, generating statistical data and video after the game is also time-consuming, making it necessary to provide fast and accurate results.

[0987] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0988] In this invention, the server includes means for receiving video data of a game in real time using a high-resolution camera, means for analyzing the received video data and identifying the positions of players and the ball, means for determining player movements using an artificial intelligence algorithm and automatically determining scores and fouls, means for updating the determination results on the scoreboard in real time, means for managing the game time, means for generating statistical data and edited game videos after the game ends, and means for inputting game information and player information using a smart device, thereby enabling efficient game management and accurate determination and scoring.

[0989] A "high-resolution camera" is a high-resolution camera that can capture clear footage of the match.

[0990] "Player" means a player participating in a basketball game.

[0991] "Ball" refers to the ball equipment used in the game of basketball.

[0992] An "artificial intelligence algorithm" is an algorithm that analyzes players' movements, recognizes plays such as shots, passes, and dribbles, and automatically determines scores and fouls.

[0993] A "scoreboard" is a device that displays the score and other information of a game.

[0994] "Timekeeping" refers to the act of accurately managing the time of a match.

[0995] "Statistical Data" means data about player and team performance collected during a match.

[0996] "Edited match video" is a video in which important scenes are extracted and edited after the match has ended.

[0997] A "smart device" is a mobile terminal with internet connectivity, such as a smartphone or tablet.

[0998] "Match information" refers to basic information about the match, such as the start time and opposing teams.

[0999] "Player information" refers to information about players participating in a match, such as their names, uniform numbers, and positions.

[1000] MODE FOR CARRYING OUT THE INVENTION

[1001] This invention is a system that automates the management of basketball games, enabling efficient and accurate game management. This is achieved by using AI to perform tasks such as filming the game, refereeing, scorekeeping, timekeeping, and creating game videos.

[1002] Filming the match

[1003] Device: A high-resolution camera is installed in a position that allows a bird's-eye view of the entire court. This camera has a wide-angle lens, allowing it to clearly capture all the action of the match. For example, during camera installation and test shooting, the camera angle and focus are manually adjusted.

[1004] Entering match information

[1005] User: Launches the application on a smart device and inputs information such as player names, uniform numbers, positions, match start time, and opposing teams. This information is sent to the server in real time, and the match is ready. Specifically, the user uses a smartphone or tablet to enter data into the application's input form and presses the "Submit" button.

[1006] Real-time video reception and analysis

[1007] Server: Uses a means to receive and analyze real-time video data from the camera. The received video data is input into an image analysis algorithm to identify the positions of players and the ball. Next, an artificial intelligence algorithm is used to identify player movements and analyze plays such as shots, passes, and dribbles. The server uses libraries such as OpenCV and TensorFlow to perform image analysis and action recognition.

[1008] For example, the server extracts the characteristics of a player's movements and determines the shooting action based on them. If a player shoots from outside the three-point line, the server detects the movement and determines whether the ball went into the goal. If the goal is successful, the server updates the score as three points.

[1009] Timekeeping

[1010] Server: Starts the timer as soon as the game starts, and accurately manages the time for each quarter and timeout. When the game is interrupted, stops and restarts the time at the appropriate time. Specifically, it calls timer management functions to start, stop, and restart the time.

[1011] Post-match data generation

[1012] Server: After the match, it consolidates all collected data and generates statistics and match records. It extracts important scenes to create highlight videos, and edits and saves full match videos. For example, it aggregates data from each play and generates statistical data in Excel or CSV format. It also uses the VideoEditor API to edit videos.

[1013] View and download results

[1014] Users can access the generated statistics and videos through their smart devices and download them as needed. Specifically, they can tap the "View Results" button in the app, select a match from the list, and view the data. If necessary, they can press the "Download" button to save the data to their local device.

[1015] Prompt Sentence Examples

[1016] "Please explain how to accurately determine the exact moment a player shoots during a basketball game and automatically update the score."

[1017] This system will enable more efficient match management and accurate judgment and scoring, thereby achieving fair, fast and accurate match management.

[1018] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1019] Step 1:

[1020] The device installs a high-resolution camera in a position that provides a bird's-eye view of the court. This camera has a wide-angle lens that captures the entire match clearly. Specific operations include adjusting the camera's angle and focus appropriately to cover the optimal field of view. The physical position and settings of the camera are specified as input, and high-resolution video is obtained as output.

[1021] Step 2:

[1022] The user starts up the smart device and launches the application. There, they enter game information such as player names, uniform numbers, positions, game start time, and opposing teams. The entered information is sent to the server in real time. Specifically, the user enters data into the input form on the smart device and presses the "Send" button. Player information and game information are given as input, and this information is sent to the server as output.

[1023] Step 3:

[1024] The server receives real-time video data from the camera. This video data is decoded and converted into an analyzable format. Specifically, the server decodes the video stream frame by frame and saves it as frame data. The video stream from the camera is given as input, and analyzable frame data is obtained as output.

[1025] Step 4:

[1026] The server inputs the received video data into an image analysis algorithm. This algorithm uses libraries such as OpenCV and TensorFlow to identify the positions of players and the ball. Specifically, it performs image analysis to detect player movements and the position of the ball. Frame data is given as input, and player and ball position data is obtained as output.

[1027] Step 5:

[1028] The server uses an artificial intelligence algorithm to determine player actions. This algorithm distinguishes between plays such as shooting, passing, and dribbling, and automatically determines scores and fouls. Specific actions are input into a motion recognition model, and a judgment result is obtained. Position data of the player and ball is given as input, and judgment data is obtained as output.

[1029] Step 6:

[1030] The server reflects the decision results on the scoreboard in real time. For example, if a player makes a successful shot from outside the three-point line, the score is updated to three points. Specifically, the server calls the score update API and updates the scoreboard. The decision data is given as input, and the updated score is displayed as output.

[1031] Step 7:

[1032] The server starts the timer as soon as the game starts, and accurately manages the time for each quarter and timeout. When the game is interrupted, it stops and restarts the time at the appropriate timing. Specifically, it calls timer management functions to start, stop, and restart the time. The game start time is given as input, and time management data is obtained as output.

[1033] Step 8:

[1034] After the match, the server consolidates all collected data and generates statistical information and match records. It extracts important scenes to create highlight videos, and edits and saves the full match video. Specific operations include aggregating data and editing the video using the VideoEditor API. The data collected during the match is given as input, and statistical data and edited video are obtained as output.

[1035] Step 9:

[1036] Users can access the generated statistical data and videos through their smart devices and download them as needed. Specifically, they tap the "View Results" button on the app, select a match from the list, and check the data. The user request is given as input, and the displayed data can be viewed as output. They can also save the data to their local device by pressing the "Download" button.

[1037] (Application example 1)

[1038] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1039] In traditional brick-and-mortar store operations, there were limited ways to understand customer behavior patterns and product popularity in real time, making it difficult to operate the store efficiently and provide appropriate customer service.In addition, store managers lacked a way to instantly grasp the congestion situation in the store or when it was necessary to replenish shelves, so they were required to respond quickly.

[1040] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1041] In this invention, the server includes means for receiving video data of the game in real time, means for analyzing the received video data and identifying the positions of objects and elements, means for determining the behavior of objects and automatically determining information and status, means for reflecting the determination results on a display device in real time, means for managing time, means for generating statistical data and videos after the game ends, means for analyzing the situation in the store in real time and identifying behavior patterns and product popularity, and means for notifying the user of the real-time analysis results. This allows store managers to grasp customer behavior patterns and product popularity in real time, and to immediately grasp the congestion situation in the store and when it is necessary to replenish shelves.

[1042] "Game video data" refers to video information received from cameras in real time during a game.

[1043] "Means for receiving" refers to the device or software function for acquiring the video data transmitted from the camera.

[1044] "Means for analyzing and locating objects and elements" refers to technologies and algorithms that recognize the location of specific objects or people based on video data.

[1045] "Means for identifying the behavior of a target and automatically determining its information and status" refers to technologies and algorithms that analyze the movements of a target object or person and automatically evaluate and judge its behavior and status.

[1046] "Means for reflecting the determination results on a display device in real time" refers to a function for instantly displaying the results of analysis and determination on a display device such as a monitor.

[1047] "Time management means" refers to a device or software function that measures and manages the time required for a match or operation.

[1048] "Means for generating post-match statistics and video" refers to the ability to consolidate collected data and create aggregated statistics and edited video after a match or event has concluded.

[1049] "Means of analyzing the situation within the store in real time and identifying behavioral patterns and product popularity" refers to technology that analyzes customer movement within the store and the level of interest in products in real time, and uses that data to identify behavioral patterns and popular products.

[1050] "Means for notifying real-time analysis results" refers to systems or functions that immediately notify relevant parties of the results obtained through analysis.

[1051] Overall system overview

[1052] This system automates customer behavior analysis and in-store management to improve the efficiency of brick-and-mortar store operations, and is primarily comprised of cameras, servers, and smart devices.

[1053] Hardware and Software

[1054] Hardware:

[1055] High-resolution wide-angle cameras capture customer movement and behavior within the store in real time.

[1056] The server is a central device that analyzes the received video data.

[1057] Smart devices (PDAs) are used by managers to view the situation in the store in real time.

[1058] software:

[1059] AI image recognition software (e.g., OpenCV and TensorFlow) analyzes video data to identify customer behavior and product popularity.

[1060] Data analysis algorithms identify customer behavior patterns and popular products.

[1061] The store management app provides analysis results and inventory information to managers.

[1062] System configuration

[1063] The system includes the following main components:

[1064] 1. Camera equipment

[1065] Device: High-resolution wide-angle cameras are installed in locations that overlook each area of ​​the store, capturing all of the customer's movements and actions.

[1066] 2. Server

[1067] Server: This is the central system that analyzes real-time video data received from in-store cameras. It uses AI image recognition software (e.g., OpenCV or TensorFlow) and data analysis algorithms to identify customer behavior and product popularity. It also notifies the store management app of the analysis results in real time.

[1068] 3. Store management application

[1069] User: This application is used by administrators to check customer behavior analysis data, inventory status, and sales information in real time. It is operated on smart devices such as PDAs.

[1070] Specific examples

[1071] For example, if analysis shows that a particular product shelf in a store in a shopping mall is extremely popular, this information will be notified to the manager, who can then take prompt action such as adding new products to that shelf.

[1072] Prompt Sentence Examples

[1073] Here are some example prompts for using generative AI models:

[1074] "Create an AI algorithm that analyzes customer behavior in stores and identifies popular areas and products."

[1075] "Analyze in-store congestion in real time and design a program to optimize staff allocation."

[1076] In this way, implementing this system is expected to improve the efficiency of physical store operations and customer service.

[1077] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1078] Step 1:

[1079] The device uses a high-resolution, wide-angle camera installed inside the store to capture customer movement and behavior in real time.

[1080] Input: Real-time video data captured by a high-resolution wide-angle camera.

[1081] Data processing / data calculation: Video data is acquired and sent directly to the server.

[1082] Output: Real-time video data is sent to the server.

[1083] Step 2:

[1084] The video data received by the server is analyzed using AI image recognition software (e.g., OpenCV or TensorFlow).

[1085] Input: Real-time video data received from a high-resolution wide-angle camera.

[1086] Data processing / data calculation: Apply object recognition and motion recognition algorithms to extract customer positions and movements from video data.

[1087] Output: Customer location and movement information.

[1088] Step 3:

[1089] The server uses data analysis algorithms to identify customer behavior patterns and product popularity.

[1090] Input: Customer location and movement information.

[1091] Data processing / data calculation: Analyze behavioral patterns based on location and movement information to calculate the popularity of specific areas and products.

[1092] Output: Behavioral pattern information and popular product information.

[1093] Step 4:

[1094] The server notifies the store management app of the analysis results in real time.

[1095] Input: behavioral pattern information and popular product information.

[1096] Data processing / data calculation: Formats the analysis results and generates notification messages.

[1097] Output: Real-time notification to the administrator's smart device.

[1098] Step 5:

[1099] The user (administrator) uses the store management app to check customer behavior analysis data, inventory status, and sales information in real time and take necessary action.

[1100] Input: Real-time notifications and analysis results displayed in the store management app.

[1101] Data processing / data calculation: Decide on product replenishment and staff allocation based on the information received.

[1102] Output: Improved store operations and faster response times.

[1103] Step 6:

[1104] After a match or event ends, the server consolidates the collected data and automatically generates statistical data and edited videos.

[1105] Input: Customer behavioral pattern information and sales data.

[1106] Data processing / data calculation: Statistical processing of all data and generation of statistical reports and videos.

[1107] Output: Statistical report and highlight video.

[1108] The above steps will realize a system that is expected to make physical store operations more efficient and improve customer service.

[1109] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1110] This invention relates to a system that automates the management of basketball games, in which AI performs the tasks of filming the game, refereeing, recording the score, timekeeping, and creating game videos, and by combining it with an emotion engine, it recognizes the user's emotions in real time and adjusts the system's behavior.

[1111] System configuration

[1112] The system includes the following main components:

[1113] 1. Camera equipment

[1114] Device: Installed in a position overlooking the entire court, it is equipped with a high-resolution, wide-angle lens to capture clear images of the entire game.

[1115] 2. Server

[1116] Server: Analyzes real-time video data received from the camera to identify the positions of players and the ball, and uses AI algorithms and image recognition technology to make decisions and provide scoring.

[1117] 3. Smart Device Applications

[1118] User: Uses an application to input and manage match and player information. Before a match, the user can input player names, uniform numbers, positions, etc., and check the situation during the match.

[1119] 4. Emotion Engine

[1120] Server: Equipped with an engine that analyzes the user's facial expressions and voice to recognize their emotional state, and adjusts the system's behavior based on the recognized emotional data.

[1121] Embodiments of the invention

[1122] Filming the match

[1123] Device: Position the camera so that it can capture the entire court. Conduct test shots to ensure the camera captures all action during the match.

[1124] Entering match information

[1125] User: Launches the application on their smart device and inputs information such as the player's name, uniform number, and position. They also input information such as the start time of the game and the opposing team. The input data is sent to the server.

[1126] Real-time analytics

[1127] Server: Receives video data from the camera in real time and uses image recognition technology to identify the positions of players and the ball. AI algorithms distinguish between shots, passes, dribbles, and other plays, and automatically determines scores, fouls, and violations. The results are instantly updated on the scoreboard.

[1128] For example, when a player shoots from outside the three-point line, the system detects the shot and determines whether the ball went into the goal. If the goal is successful, the server immediately updates the score to three points.

[1129] Timekeeping

[1130] Server: Starts the timer at the start of the game and accurately manages the time for each quarter and timeout. Stops and restarts the timer at the appropriate time during timeouts and other interruptions in the game.

[1131] Recognizing and reflecting emotions

[1132] Server: Using the emotion engine, analyzes the user's facial expressions and voice data in real time to recognize their emotional state. Based on the recognized emotional data, the game progress and notification content are adaptively changed.

[1133] For example, if the system detects that the user is excited during a tense match, it will adjust the frequency of screen displays and notifications to prevent the user from feeling stressed. Conversely, if the system determines that the user is bored, it can send notifications highlighting the highlights of the match.

[1134] Post-match data generation

[1135] Server: After the match, it consolidates all data collected during the match to generate statistics and match records. It extracts key moments from the match to create highlight videos and provide emotional highlights that reflect the emotional data used. It also edits and saves the full match video in the appropriate format.

[1136] Users: Access the generated statistical data and videos through their smart devices and download them as needed. The server uploads the data and videos to the cloud and provides users with an access link.

[1137] Specific examples

[1138] For example, if Player B commits a foul during a match, the server analyzes the video data capturing that moment. The AI ​​distinguishes Player B's movements and automatically determines whether it was a foul based on the rules. At the same time, the emotion engine analyzes the user's reaction, and if it detects excitement or surprise, it can instantly provide a replay video that reflects that emotion.

[1139] By combining an emotion engine with video analysis, the system of the present invention can not only improve the efficiency of match management, but also the user experience.

[1140] The processing flow will be explained below.

[1141] Step 1:

[1142] The user sets the camera in a position that overlooks the entire court and adjusts it so that there are no blind spots. The camera is connected to the server via a network.

[1143] Step 2:

[1144] The user launches the application on their smart device and enters game information (start time, opposing teams, game format, etc.) and player information (name, uniform number, position, etc.). The entered data is sent to the server.

[1145] Step 3:

[1146] The server receives video data from the camera in real time and stores the received video data in a buffer for analysis.

[1147] Step 4:

[1148] The server analyzes the video data frame by frame and uses image recognition technology to identify the positions of players and the ball, tracking player movements.

[1149] Step 5:

[1150] The server uses AI algorithms to detect player actions (shooting, passing, dribbling, etc.), identify specific movement patterns, and determine events (goals, fouls, violations, etc.).

[1151] Step 6:

[1152] The server automatically determines the score based on detected events (e.g., goals, fouls), and updates the scoreboard in real time, updating individual performance data.

[1153] Step 7:

[1154] The server initiates timekeeping, accurately measures the start and end times of the game, the duration of each quarter, and timeouts, and stops and restarts game time as necessary.

[1155] Step 8:

[1156] The server uses an emotion engine to analyze the user's facial expressions and voice data in real time, recognizes the user's emotional state based on the analysis results, and adjusts the system's behavior based on that data.

[1157] Step 9:

[1158] For example, if the user is excited, the server will minimize the information displayed to reduce the user's stress. Conversely, if the user is perceived as bored, the server will send highlight notifications more frequently.

[1159] Step 10:

[1160] The server consolidates all data collected during the game and automatically generates statistics and match records after the game ends, compiling detailed statistics for each player, such as points, rebounds, assists, and fouls.

[1161] Step 11:

[1162] The server analyzes video data during the match, extracts important scenes, and creates highlight videos. It also provides emotional highlight videos that reflect the emotional data.

[1163] Step 12:

[1164] Users can access the generated statistical data and emotional highlight videos through their smart devices and download them as needed. The server uploads the data and videos to the cloud and provides users with an access link.

[1165] In this way, the system streamlines match management, provides fair and accurate refereeing and scoring, and also provides an interactive experience that responds to the user's emotions.

[1166] Example 2

[1167] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1168] Basketball game management requires a lot of manual work, and is prone to human error in refereeing, scoring, timekeeping, and game video production. Furthermore, there is a lack of ways to properly reflect user emotions during the game to provide a better experience.

[1169] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for receiving video data of the game in real time, means for analyzing the received video data and identifying the positions of the players and the ball, and means for analyzing the emotional state of the user in real time and adaptively adjusting the behavior of the system. This enables the automation of game management, reduces human error, and enables flexible responses according to the user's emotions.

[1170] "Means for receiving game video data in real time" refers to the cameras and communication infrastructure that capture video of the basketball game and transmit it to a server in real time.

[1171] "Means for analyzing received video data and identifying the positions of players and the ball" refers to algorithms or software that use image recognition technology to analyze and identify the positions of players and the ball from received video data.

[1172] "Means of determining player movements and automatically determining scores and fouls" refers to a system that uses AI algorithms to analyze player movements and automatically determine scores, fouls, and violations.

[1173] "Means for reflecting the judgement results on the scoreboard in real time" refers to a communication system and a display device for instantly displaying the judgement results on the scoreboard.

[1174] "Means for controlling the time of a match" means a digital timer or control system for accurately measuring the start, finish and stoppage times of a match.

[1175] "Means for analyzing the user's emotional state in real time and adaptively adjusting the system's behavior" refers to an emotion recognition engine that analyzes the user's facial expressions and voice data and dynamically changes the system's behavior and notification content based on that emotional state.

[1176] "Means for generating post-match statistics and video" means software and servers that aggregate data collected during a match and generate statistics, match records, and edited match video.

[1177] "Using a camera to capture a bird's-eye view of the entire court" refers to a method of capturing the entire court with a wide-angle lens to monitor the movement of all players and the ball at once.

[1178] "Entering match and player information using a smart device" refers to the process of entering player and match information before a match using a digital device such as a smartphone or tablet.

[1179] Overall system overview

[1180] The present invention is a system for automating basketball game management that integrates game filming, refereeing, scorekeeping, timekeeping, game video production, and an emotion engine that recognizes user emotions in real time and adjusts the system's behavior accordingly.

[1181] System Components

[1182] 1. Terminal (camera device)

[1183] A device equipped with a high-resolution, wide-angle lens is placed in a position that allows it to capture a bird's-eye view of the entire court. The camera is used to capture clear images of the entire game. For example, the camera can take a panoramic image of the entire court before the start of a game and send the image to a server.

[1184] 2. Server

[1185] The server has the following main functions:

[1186] 1. Receiving and analyzing video data: Video data is received in real time and the positions of players and the ball are analyzed using image recognition technology (e.g., OpenCV or TensorFlow).

[1187] 2. Judgment by AI algorithm: Based on the AI ​​algorithm, plays such as shots, passes, and dribbles are judged, and scores, fouls, and violations are automatically determined.

[1188] 3. Reflection of judgement results: The judgement results will be reflected immediately on the scoreboard.

[1189] 4. Timekeeping: Manage the game time, start the timer at the start of the game, and accurately manage the times of each quarter and timeout.

[1190] 5. Emotion Engine: Analyzes the user's facial expressions and voice data in real time to recognize their emotional state and adaptively adjust the system's behavior. For example, if the emotion engine detects that the user is excited, it will reduce the frequency of notifications.

[1191] 6. Data generation: Generate statistics and videos after the game, extract key moments and create highlight videos.

[1192] 3. User (Smart Device Application)

[1193] The user does the following:

[1194] 1. Entering match and player information: Using the smart device application, users enter information such as player names, uniform numbers, positions, match start times, and opposing teams. This information is then sent to the server.

[1195] 2. Data and video access: Access the generated statistical data and videos through smart devices and download them as needed. The server uploads the data and videos to the cloud and provides an access link.

[1196] Specific examples

[1197] For example, if player B commits a foul during a game, the server analyzes the video data capturing that moment. The AI ​​distinguishes player B's movements and automatically determines whether it was a foul based on the rules. At the same time, the emotion engine analyzes the user's reaction, and if it detects excitement or surprise, it instantly provides a replay video that reflects that emotion.

[1198] Example prompts to input to the generative AI model

[1199] Design a system that automatically identifies specific plays in an ongoing basketball game and adaptively adjusts notifications based on the user's emotional state. This system includes a camera that records the game, a server that analyzes real-time footage, a smart device application that manages game information, and an emotion engine that recognizes the user's emotions.

[1200] The above is an embodiment of the present invention that makes it easy to implement. This system can automate the management of matches and improve the user experience.

[1201] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1202] Step 1: Filming the match

[1203] Terminal

[1204] Input: Notification of the start of the match

[1205] Operation:

[1206] 1. A camera with a high resolution and wide-angle lens will be installed in a position that allows for a bird's-eye view of the entire court.

[1207] 2. The camera prepares to capture the entire match in real time.

[1208] 3. Conduct test shooting to check and adjust the shooting range and image quality.

[1209] Output: High-resolution real-time video data

[1210] Step 2: Enter match information

[1211] User

[1212] Input: Pre-match player and match information

[1213] Operation:

[1214] 1. Launch the smart device application.

[1215] 2. Enter the player's name, uniform number, position, game start time, and opposing team information.

[1216] 3. The entered information is sent to the server.

[1217] Output: Match and player information sent to the server

[1218] Step 3: Receiving video data

[1219] server

[1220] Input: Real-time video data from the camera

[1221] Operation:

[1222] 1. Receive video data sent from the camera in real time.

[1223] 2. The received video data is saved and prepared for the next analysis step.

[1224] Output: Received and stored video data

[1225] Step 4: Analyzing the video data

[1226] server

[1227] Input: Received video data

[1228] Operation:

[1229] 1. Analyze the video data using image recognition technology (e.g., OpenCV, TensorFlow).

[1230] 2. Identify the location of the players and the ball.

[1231] Output: Player and ball position data

[1232] Step 5: Motion detection and automatic judgment

[1233] server

[1234] Input: Player and ball position data

[1235] Operation:

[1236] 1. Using AI algorithms to identify player actions (shooting, passing, dribbling, etc.).

[1237] 2. Automatically judge scores, fouls, and violations based on the game rules.

[1238] Output: Scores, fouls, and violations

[1239] Step 6: Reflecting the results

[1240] server

[1241] Input: Judgment result

[1242] Operation:

[1243] 1. The judgement results will be reflected immediately on the scoreboard.

[1244] 2. Send notifications to users as needed.

[1245] Output: Updated leaderboard

[1246] Step 7: Timekeeping

[1247] server

[1248] Input: Notification of the start of the match

[1249] Operation:

[1250] 1. Start the digital timer at the same time as the match begins.

[1251] 2. Accurately manage the time of each quarter and timeout.

[1252] 3. Stop and restart the clock at the appropriate time when the game is interrupted.

[1253] Output: Precisely controlled game time

[1254] Step 8: Recognize and reflect on your emotions

[1255] server

[1256] Input: User facial and voice data

[1257] Operation:

[1258] 1. Analyze the user's facial expressions and voice data in real time using an emotion engine.

[1259] 2. Recognize the user's emotional state and adaptively adjust the system's behavior (notification frequency, display content, etc.).

[1260] Output: Changing the system's behavior based on the recognized emotion data.

[1261] Step 9: Post-match data generation

[1262] server

[1263] Input: Data collected during the match

[1264] Operation:

[1265] 1. After the match, all data is aggregated to generate statistics and match records.

[1266] 2. Extract important scenes and create a highlight video.

[1267] 3. Edit and save the full match video in the appropriate format.

[1268] Output: Statistics, match records, highlight videos, full match videos

[1269] Step 10: Accessing Data and Videos

[1270] User

[1271] Enter: Post-match notification

[1272] Operation:

[1273] 1. Access the generated statistics and videos using your smart device.

[1274] 2. Download data and videos as needed.

[1275] Output: Downloadable statistics and a link to the video

[1276] This is the specific processing flow of this system. This system automates the management of matches and enables flexible responses according to the user's emotions.

[1277] (Application example 2)

[1278] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1279] The problem that this invention aims to solve is to reduce the stress of managers in large spaces such as public buildings and plazas by detecting abnormal behavior and analyzing emotions, which conventional monitoring systems cannot handle. There is a need to detect abnormalities in real time and respond efficiently while reducing the burden on managers.

[1280] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving video data of the game in real time, means for analyzing the received video data and identifying the positions of players and the ball, means for determining player actions and automatically determining scores and fouls, means for reflecting the determination results on the scoreboard in real time, means for managing the game time, means for generating statistical data and videos after the game ends, means for detecting and notifying abnormal behavior, and means for analyzing the user's emotional state and adjusting the system's behavior. This enables efficient monitoring of large spaces, rapid detection of abnormal behavior, and adjustment of the system's behavior in accordance with the administrator's emotional state.

[1281] "Video data" refers to digital data of video footage captured by a camera.

[1282] "Real-time" refers to processing, sending, and receiving information instantly, without delay.

[1283] "Analysis" is the act of extracting specific patterns or information from digital data.

[1284] "Player" refers to a person participating in a sport or action.

[1285] "Ball" refers to a spherical object used for sports and games.

[1286] "Locate" means to determine the current coordinates or location of an object or person.

[1287] "Distinguishing behavior" refers to recognizing and classifying the behavior of a target person or object.

[1288] A "score" is a numerical value that indicates the score in a sporting event.

[1289] A "foul" is an act that violates the rules in a sporting event.

[1290] "Decision result" refers to a judgment or conclusion drawn based on an analysis.

[1291] A "scoreboard" is a device or screen that displays scores and other information from a sports event.

[1292] "Time management" means monitoring and regulating the progress of a competition or task.

[1293] "Statistical data" is quantitative information obtained from the results of analysis.

[1294] "Generating a moving image" means editing still images and video data to create a moving image.

[1295] "Abnormal behavior" refers to unusual or suspicious behavior that differs from normal behavior.

[1296] A "notification" is a message or signal that conveys specific information.

[1297] "Emotional state" indicates the type and strength of the emotion a person is feeling at the time.

[1298] "Analyzing and adjusting the system's behavior" refers to analyzing the emotional state and changing the system's behavior accordingly.

[1299] This invention is a surveillance system for large spaces such as public buildings and squares, which detects abnormal behavior in real time and adjusts the system's behavior by analyzing the user's emotional state. The system consists of a camera device, a server, a terminal application, and an emotion engine.

[1300] System configuration

[1301] 1. Camera equipment

[1302] The cameras have high-resolution, wide-angle lenses that can cover a wide area. They are recommended to be installed in strategic locations inside public buildings or plazas. The cameras transmit video data to a server in real time.

[1303] 2. Server

[1304] The server is equipped with an AI algorithm that analyzes the received video data and detects abnormal behavior. The server includes the following main components:

[1305] Real-time video analysis method: OpenCV and TensorFlow are used to analyze video data and detect abnormal behavior.

[1306] Notification method: Uses the Flask framework to send notifications to the user's device in real time.

[1307] Emotional state analysis: Azure Emotion API is used to analyze the user's emotional state and adjust the system's behavior.

[1308] Data storage method: MongoDB is used to store abnormal behavior data and emotion data, and data analysis is performed using Pandas.

[1309] 3. Terminal Applications

[1310] Users can use the terminal application to manage monitoring areas and notification settings, such as the area they want to monitor, the time period, and the frequency of notifications.

[1311] 4. Emotion Engine

[1312] The emotion engine analyzes the user's facial expressions and voice data in real time and sends the results to the server, which adjusts the content and frequency of notifications based on the user's emotional state, supporting efficient monitoring.

[1313] Specific examples

[1314] For example, when implementing it as a surveillance system for a shopping mall, multiple cameras are installed on each floor and at the entrance. The video data captured by the cameras is sent to a server in real time. The server uses OpenCV to analyze the video and detects abnormal behavior, such as suspiciously left luggage or suspicious movements in a crowd. Any abnormal behavior detected is immediately reported, and a warning is sent to the user's device.

[1315] At the same time, the emotion engine analyzes the user's facial expressions and voice, and if the administrator is feeling stressed, it adjusts the frequency and content of notifications to reduce the burden. For example, even if multiple anomalies are detected, the system automatically adjusts to prioritize notifications of only the most important detection results, and reports other information together later.

[1316] Below is an example of a prompt sentence to input to the generative AI model.

[1317] import cv2

[1318] import tensorflow as tf

[1319] from azure.ai.textanalytics import TextAnalyticsClient

[1320] from azure.core.credentials import AzureKeyCredential

[1321] import pymongo

[1322] from pandas import DataFrame

[1323] Importing camera footage

[1324] cap = cv2.VideoCapture(0)

[1325] Loading an AI model

[1326] model = tf.keras.models.load_model('anomaly_detection_model.h5')

[1327] Azure Emotion Engine Client Configuration

[1328] credential = AzureKeyCredential("API_KEY")

[1329] client = TextAnalyticsClient(endpoint="https: / / YOUR_ENDPOINT.cognitiveservices.azure.com / ", credential=credential)

[1330] MongoDB Configuration

[1331] client = pymongo.MongoClient("mongodb: / / localhost:27017 / ")

[1332] db = client["security_data"]

[1333] collection = db["anomaly_records"]

[1334] Video analysis processing

[1335] while True:

[1336] ret, frame = cap.read()

[1337] if not ret:

[1338] break

[1339] Input the frame into the AI ​​model

[1340] predictions = model.predict(frame)

[1341] if is_anomaly_detected(predictions):

[1342] What to do if an abnormality is detected

[1343] emotion_state = client.analyze_sentiment(documents=["Get the administrator's emotional state"])

[1344] collection.insert_one({"frame": frame.tolist(), "emotion": emotion_state[0].sentiment.name})

[1345] if emotion_state[0].sentiment.name != 'negative':

[1346] Notification Processing

[1347] send_alert_to_admin(frame, emotion_state[0].sentiment.name)

[1348] System shutdown processing

[1349] cap.release()

[1350] cv2.destroyAllWindows()

[1351] This embodiment makes it possible to realize efficient and effective wide-area monitoring while reducing the burden on the user.

[1352] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1353] Step 1:

[1354] The server receives video data in real time from the camera devices, which monitor a designated area and provide high-resolution video, and transmits the video data as successive frames to the server.

[1355] Step 2:

[1356] The server processes the received video data using OpenCV to split the individual frames, converting the video frames into an analyzable format.

[1357] Step 3:

[1358] The server uses TensorFlow to input the analyzed frame data into an anomalous behavior detection model. The model analyzes the movement patterns of people and objects in each frame and detects anomalous behavior (e.g., suspicious movements or abandoned luggage). The input is the analyzed frame data, and the output is the anomalous behavior detection results.

[1359] Step 4:

[1360] When abnormal behavior is detected, the server generates an anomaly detection notification using the Flask framework. This notification includes image data for the frame in which the anomaly was identified and the type of anomaly detected. The notification is sent to the user's device in real time.

[1361] Step 5:

[1362] The user's device receives anomaly detection notifications sent from the server in real time. The received notification includes an image of the frame showing the abnormal behavior and details of the anomaly. The user can check the content of the notification and take the necessary action immediately.

[1363] Step 6:

[1364] The server uses the Azure Emotion API to analyze the user's emotional state. The analysis uses the user's facial expression and voice data acquired in real time. The input is the user's facial expression and voice data, and the output is the emotion analysis results.

[1365] Step 7:

[1366] The server adjusts the system's behavior based on the results of emotion analysis. Specifically, if the user is feeling stressed, the server adjusts the frequency and content of notifications to reduce the burden on the user. This allows the user to perform monitoring tasks more efficiently.

[1367] Step 8:

[1368] The server stores anomaly detection data and emotion analysis data in MongoDB. The stored data is later used for analysis and report creation. The stored data includes video data, types of abnormal behavior, emotion analysis results, etc.

[1369] Step 9:

[1370] The server analyzes the stored data using Pandas and generates weekly or monthly reports, including the frequency and trends of abnormal behavior and fluctuations in the emotional state of administrators. The reports are provided to administrators to help them improve their monitoring measures.

[1371] Through the above processing steps, the system can efficiently and effectively realize wide-area monitoring and reduce the burden on the user.

[1372] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1373] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1374] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1375] [Fourth embodiment]

[1376] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1377] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1378] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1379] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1380] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1381] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1382] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1383] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1384] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1385] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1386] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1387] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1388] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1389] This invention relates to a system that automates the management of basketball games. By using AI to perform tasks such as filming the game, refereeing, scoring, timekeeping, and creating video of the game, it prevents manual errors in judgment and scoring, and realizes accurate and efficient game management.

[1390] System configuration

[1391] The system includes the following main components:

[1392] 1. Camera equipment

[1393] Device: The device will be installed in a position that overlooks the entire court. The camera has a high resolution and a wide-angle lens, allowing for clear capture of the entire match.

[1394] 2. Server

[1395] Server: This is the central system that analyzes real-time video data received from the cameras. It uses AI algorithms and image recognition technology to identify the positions of players and the ball and make necessary decisions.

[1396] 3. Smart Device Applications

[1397] User: Uses an application to input and manage match and player information. Before a match, users can input information such as player names, uniform numbers, and positions, and can also check the situation during the match.

[1398] Embodiments of the invention

[1399] Filming the match

[1400] Device: Position the camera so that it can capture the entire court. Conduct test shots to ensure the camera captures all action during the match.

[1401] Entering match information

[1402] User: Launches the application on their smart device and inputs information such as the player's name, uniform number, and position. They also input information such as the start time of the game and the opposing team. The input data is sent to the server.

[1403] Real-time analytics

[1404] Server: Receives video data from the camera in real time and uses image recognition technology to identify the positions of players and the ball. AI algorithms distinguish between shots, passes, dribbles, and other plays, and automatically determines scores, fouls, and violations. The results are instantly updated on the scoreboard.

[1405] For example, when a player shoots from outside the three-point line, the system detects the shot and determines whether the ball went into the goal. If the goal is successful, the server immediately updates the score to three points.

[1406] Timekeeping

[1407] Server: Starts the timer at the start of the game and accurately manages the time for each quarter and timeout. Stops and restarts the timer at the appropriate time during timeouts and other interruptions in the game.

[1408] Post-match data generation

[1409] Server: After the match, it consolidates all data collected during the match to generate statistics and match records, extracts important moments from the match to create highlight videos, and edits and stores full match videos.

[1410] Users: Through their smart devices, they can access the generated statistics and videos and download them as needed.

[1411] Specific examples

[1412] For example, if Player B commits a foul during a match, the server analyzes the video data capturing that moment. The AI ​​determines Player B's movements and automatically determines whether it was a foul based on the rules. The determination result is immediately reflected on the scoreboard, and the foul count is updated. After the match ends, detailed statistical data, including a record of the fouls, is automatically generated and available for users to review.

[1413] In this way, the system of the present invention dramatically improves the efficiency of game management and enables fair and accurate refereeing and scoring.

[1414] The processing flow will be explained below.

[1415] Step 1:

[1416] The user sets the camera in a position that overlooks the entire court and adjusts it so that there are no blind spots. The camera is connected to the server via a network.

[1417] Step 2:

[1418] The user launches the application on their smart device and enters game information (start time, opposing teams, game format, etc.) and player information (name, uniform number, position, etc.). The entered data is sent to the server.

[1419] Step 3:

[1420] The server receives video data from the camera in real time and stores the received video data in a buffer for analysis.

[1421] Step 4:

[1422] The server analyzes the video data frame by frame and uses image recognition technology to identify the positions of players and the ball, tracking player movements.

[1423] Step 5:

[1424] The server uses AI algorithms to detect player actions (shooting, passing, dribbling, etc.), identify specific movement patterns, and determine events (goals, fouls, violations, etc.).

[1425] Step 6:

[1426] The server automatically determines the score based on detected events (e.g., goals, fouls), and updates the scoreboard in real time, updating individual performance data.

[1427] Step 7:

[1428] The server initiates timekeeping, accurately measures the start and end times of the game, the duration of each quarter, and timeouts, and stops and restarts game time as necessary.

[1429] Step 8:

[1430] The server consolidates all data collected during the game and automatically generates statistics and match records after the game ends, compiling detailed statistics for each player, such as points, rebounds, assists, and fouls.

[1431] Step 9:

[1432] The server analyzes video data during the match, extracts important scenes to create a highlight video, and edits and saves the full match video in the appropriate format.

[1433] Step 10:

[1434] Users can view the generated statistical data and game videos through their smart devices and download them as needed. The server then uploads the data and videos to the cloud and provides users with an access link.

[1435] Example 1

[1436] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1437] Traditional basketball game management relies on human intervention, which means that misjudgments and errors in scoring are likely to occur. Furthermore, refereeing, scoring, and timekeeping require a great deal of labor, making efficient management difficult. Furthermore, generating statistical data and video after the game is also time-consuming, making it necessary to provide fast and accurate results.

[1438] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1439] In this invention, the server includes means for receiving video data of a game in real time using a high-resolution camera, means for analyzing the received video data and identifying the positions of players and the ball, means for determining player movements using an artificial intelligence algorithm and automatically determining scores and fouls, means for updating the determination results on the scoreboard in real time, means for managing the game time, means for generating statistical data and edited game videos after the game ends, and means for inputting game information and player information using a smart device, thereby enabling efficient game management and accurate determination and scoring.

[1440] A "high-resolution camera" is a high-resolution camera that can capture clear footage of the match.

[1441] "Player" means a player participating in a basketball game.

[1442] "Ball" refers to the ball equipment used in the game of basketball.

[1443] An "artificial intelligence algorithm" is an algorithm that analyzes players' movements, recognizes plays such as shots, passes, and dribbles, and automatically determines scores and fouls.

[1444] A "scoreboard" is a device that displays the score and other information of a game.

[1445] "Timekeeping" refers to the act of accurately managing the time of a match.

[1446] "Statistical Data" means data about player and team performance collected during a match.

[1447] "Edited match video" is a video in which important scenes are extracted and edited after the match has ended.

[1448] A "smart device" is a mobile terminal with internet connectivity, such as a smartphone or tablet.

[1449] "Match information" refers to basic information about the match, such as the start time and opposing teams.

[1450] "Player information" refers to information about players participating in a match, such as their names, uniform numbers, and positions.

[1451] MODE FOR CARRYING OUT THE INVENTION

[1452] This invention is a system that automates the management of basketball games, enabling efficient and accurate game management. This is achieved by using AI to perform tasks such as filming the game, refereeing, scorekeeping, timekeeping, and creating game videos.

[1453] Filming the match

[1454] Device: A high-resolution camera is installed in a position that allows a bird's-eye view of the entire court. This camera has a wide-angle lens, allowing it to clearly capture all the action of the match. For example, during camera installation and test shooting, the camera angle and focus are manually adjusted.

[1455] Entering match information

[1456] User: Launches the application on a smart device and inputs information such as player names, uniform numbers, positions, match start time, and opposing teams. This information is sent to the server in real time, and the match is ready. Specifically, the user uses a smartphone or tablet to enter data into the application's input form and presses the "Submit" button.

[1457] Real-time video reception and analysis

[1458] Server: Uses a means to receive and analyze real-time video data from the camera. The received video data is input into an image analysis algorithm to identify the positions of players and the ball. Next, an artificial intelligence algorithm is used to identify player movements and analyze plays such as shots, passes, and dribbles. The server uses libraries such as OpenCV and TensorFlow to perform image analysis and action recognition.

[1459] For example, the server extracts the characteristics of a player's movements and determines the shooting action based on them. If a player shoots from outside the three-point line, the server detects the movement and determines whether the ball went into the goal. If the goal is successful, the server updates the score as three points.

[1460] Timekeeping

[1461] Server: Starts the timer as soon as the game starts, and accurately manages the time for each quarter and timeout. When the game is interrupted, stops and restarts the time at the appropriate time. Specifically, it calls timer management functions to start, stop, and restart the time.

[1462] Post-match data generation

[1463] Server: After the match, it consolidates all collected data and generates statistics and match records. It extracts important scenes to create highlight videos, and edits and saves full match videos. For example, it aggregates data from each play and generates statistical data in Excel or CSV format. It also uses the VideoEditor API to edit videos.

[1464] View and download results

[1465] Users can access the generated statistics and videos through their smart devices and download them as needed. Specifically, they can tap the "View Results" button in the app, select a match from the list, and view the data. If necessary, they can press the "Download" button to save the data to their local device.

[1466] Prompt Sentence Examples

[1467] "Please explain how to accurately determine the exact moment a player shoots during a basketball game and automatically update the score."

[1468] This system will enable more efficient match management and accurate judgment and scoring, thereby achieving fair, fast and accurate match management.

[1469] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1470] Step 1:

[1471] The device installs a high-resolution camera in a position that provides a bird's-eye view of the court. This camera has a wide-angle lens that captures the entire match clearly. Specific operations include adjusting the camera's angle and focus appropriately to cover the optimal field of view. The physical position and settings of the camera are specified as input, and high-resolution video is obtained as output.

[1472] Step 2:

[1473] The user starts up the smart device and launches the application. There, they enter game information such as player names, uniform numbers, positions, game start time, and opposing teams. The entered information is sent to the server in real time. Specifically, the user enters data into the input form on the smart device and presses the "Send" button. Player information and game information are given as input, and this information is sent to the server as output.

[1474] Step 3:

[1475] The server receives real-time video data from the camera. This video data is decoded and converted into an analyzable format. Specifically, the server decodes the video stream frame by frame and saves it as frame data. The video stream from the camera is given as input, and analyzable frame data is obtained as output.

[1476] Step 4:

[1477] The server inputs the received video data into an image analysis algorithm. This algorithm uses libraries such as OpenCV and TensorFlow to identify the positions of players and the ball. Specifically, it performs image analysis to detect player movements and the position of the ball. Frame data is given as input, and player and ball position data is obtained as output.

[1478] Step 5:

[1479] The server uses an artificial intelligence algorithm to determine player actions. This algorithm distinguishes between plays such as shooting, passing, and dribbling, and automatically determines scores and fouls. Specific actions are input into a motion recognition model, and a judgment result is obtained. Position data of the player and ball is given as input, and judgment data is obtained as output.

[1480] Step 6:

[1481] The server reflects the decision results on the scoreboard in real time. For example, if a player makes a successful shot from outside the three-point line, the score is updated to three points. Specifically, the server calls the score update API and updates the scoreboard. The decision data is given as input, and the updated score is displayed as output.

[1482] Step 7:

[1483] The server starts the timer as soon as the game starts, and accurately manages the time for each quarter and timeout. When the game is interrupted, it stops and restarts the time at the appropriate timing. Specifically, it calls timer management functions to start, stop, and restart the time. The game start time is given as input, and time management data is obtained as output.

[1484] Step 8:

[1485] After the match, the server consolidates all collected data and generates statistical information and match records. It extracts important scenes to create highlight videos, and edits and saves the full match video. Specific operations include aggregating data and editing the video using the VideoEditor API. The data collected during the match is given as input, and statistical data and edited video are obtained as output.

[1486] Step 9:

[1487] Users can access the generated statistical data and videos through their smart devices and download them as needed. Specifically, they tap the "View Results" button on the app, select a match from the list, and check the data. The user request is given as input, and the displayed data can be viewed as output. They can also save the data to their local device by pressing the "Download" button.

[1488] (Application example 1)

[1489] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1490] In traditional brick-and-mortar store operations, there were limited ways to understand customer behavior patterns and product popularity in real time, making it difficult to operate the store efficiently and provide appropriate customer service.In addition, store managers lacked a way to instantly grasp the congestion situation in the store or when it was necessary to replenish shelves, so they were required to respond quickly.

[1491] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1492] In this invention, the server includes means for receiving video data of the game in real time, means for analyzing the received video data and identifying the positions of objects and elements, means for determining the behavior of objects and automatically determining information and status, means for reflecting the determination results on a display device in real time, means for managing time, means for generating statistical data and videos after the game ends, means for analyzing the situation in the store in real time and identifying behavior patterns and product popularity, and means for notifying the user of the real-time analysis results. This allows store managers to grasp customer behavior patterns and product popularity in real time, and to immediately grasp the congestion situation in the store and when it is necessary to replenish shelves.

[1493] "Game video data" refers to video information received from cameras in real time during a game.

[1494] "Means for receiving" refers to the device or software function for acquiring the video data transmitted from the camera.

[1495] "Means for analyzing and locating objects and elements" refers to technologies and algorithms that recognize the location of specific objects or people based on video data.

[1496] "Means for identifying the behavior of a target and automatically determining its information and status" refers to technologies and algorithms that analyze the movements of a target object or person and automatically evaluate and judge its behavior and status.

[1497] "Means for reflecting the determination results on a display device in real time" refers to a function for instantly displaying the results of analysis and determination on a display device such as a monitor.

[1498] "Time management means" refers to a device or software function that measures and manages the time required for a match or operation.

[1499] "Means for generating post-match statistics and video" refers to the ability to consolidate collected data and create aggregated statistics and edited video after a match or event has concluded.

[1500] "Means of analyzing the situation within the store in real time and identifying behavioral patterns and product popularity" refers to technology that analyzes customer movement within the store and the level of interest in products in real time, and uses that data to identify behavioral patterns and popular products.

[1501] "Means for notifying real-time analysis results" refers to systems or functions that immediately notify relevant parties of the results obtained through analysis.

[1502] Overall system overview

[1503] This system automates customer behavior analysis and in-store management to improve the efficiency of brick-and-mortar store operations, and is primarily comprised of cameras, servers, and smart devices.

[1504] Hardware and Software

[1505] Hardware:

[1506] High-resolution wide-angle cameras capture customer movement and behavior within the store in real time.

[1507] The server is a central device that analyzes the received video data.

[1508] Smart devices (PDAs) are used by managers to view the situation in the store in real time.

[1509] software:

[1510] AI image recognition software (e.g., OpenCV and TensorFlow) analyzes video data to identify customer behavior and product popularity.

[1511] Data analysis algorithms identify customer behavior patterns and popular products.

[1512] The store management app provides analysis results and inventory information to managers.

[1513] System configuration

[1514] The system includes the following main components:

[1515] 1. Camera equipment

[1516] Device: High-resolution wide-angle cameras are installed in locations that overlook each area of ​​the store, capturing all of the customer's movements and actions.

[1517] 2. Server

[1518] Server: This is the central system that analyzes real-time video data received from in-store cameras. It uses AI image recognition software (e.g., OpenCV or TensorFlow) and data analysis algorithms to identify customer behavior and product popularity. It also notifies the store management app of the analysis results in real time.

[1519] 3. Store management application

[1520] User: This application is used by administrators to check customer behavior analysis data, inventory status, and sales information in real time. It is operated on smart devices such as PDAs.

[1521] Specific examples

[1522] For example, if analysis shows that a particular product shelf in a store in a shopping mall is extremely popular, this information will be notified to the manager, who can then take prompt action such as adding new products to that shelf.

[1523] Prompt Sentence Examples

[1524] Here are some example prompts for using generative AI models:

[1525] "Create an AI algorithm that analyzes customer behavior in stores and identifies popular areas and products."

[1526] "Analyze in-store congestion in real time and design a program to optimize staff allocation."

[1527] In this way, implementing this system is expected to improve the efficiency of physical store operations and customer service.

[1528] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1529] Step 1:

[1530] The device uses a high-resolution, wide-angle camera installed inside the store to capture customer movement and behavior in real time.

[1531] Input: Real-time video data captured by a high-resolution wide-angle camera.

[1532] Data processing / data calculation: Video data is acquired and sent directly to the server.

[1533] Output: Real-time video data is sent to the server.

[1534] Step 2:

[1535] The video data received by the server is analyzed using AI image recognition software (e.g., OpenCV or TensorFlow).

[1536] Input: Real-time video data received from a high-resolution wide-angle camera.

[1537] Data processing / data calculation: Apply object recognition and motion recognition algorithms to extract customer positions and movements from video data.

[1538] Output: Customer location and movement information.

[1539] Step 3:

[1540] The server uses data analysis algorithms to identify customer behavior patterns and product popularity.

[1541] Input: Customer location and movement information.

[1542] Data processing / data calculation: Analyze behavioral patterns based on location and movement information to calculate the popularity of specific areas and products.

[1543] Output: Behavioral pattern information and popular product information.

[1544] Step 4:

[1545] The server notifies the store management app of the analysis results in real time.

[1546] Input: behavioral pattern information and popular product information.

[1547] Data processing / data calculation: Formats the analysis results and generates notification messages.

[1548] Output: Real-time notification to the administrator's smart device.

[1549] Step 5:

[1550] The user (administrator) uses the store management app to check customer behavior analysis data, inventory status, and sales information in real time and take necessary action.

[1551] Input: Real-time notifications and analysis results displayed in the store management app.

[1552] Data processing / data calculation: Decide on product replenishment and staff allocation based on the information received.

[1553] Output: Improved store operations and faster response times.

[1554] Step 6:

[1555] After a match or event ends, the server consolidates the collected data and automatically generates statistical data and edited videos.

[1556] Input: Customer behavioral pattern information and sales data.

[1557] Data processing / data calculation: Statistical processing of all data and generation of statistical reports and videos.

[1558] Output: Statistical report and highlight video.

[1559] The above steps will realize a system that is expected to make physical store operations more efficient and improve customer service.

[1560] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1561] This invention relates to a system that automates the management of basketball games, in which AI performs the tasks of filming the game, refereeing, recording the score, timekeeping, and creating game videos, and by combining it with an emotion engine, it recognizes the user's emotions in real time and adjusts the system's behavior.

[1562] System configuration

[1563] The system includes the following main components:

[1564] 1. Camera equipment

[1565] Device: Installed in a position overlooking the entire court, it is equipped with a high-resolution, wide-angle lens to capture clear images of the entire game.

[1566] 2. Server

[1567] Server: Analyzes real-time video data received from the camera to identify the positions of players and the ball, and uses AI algorithms and image recognition technology to make decisions and provide scoring.

[1568] 3. Smart Device Applications

[1569] User: Uses an application to input and manage match and player information. Before a match, the user can input player names, uniform numbers, positions, etc., and check the situation during the match.

[1570] 4. Emotion Engine

[1571] Server: Equipped with an engine that analyzes the user's facial expressions and voice to recognize their emotional state, and adjusts the system's behavior based on the recognized emotional data.

[1572] Embodiments of the invention

[1573] Filming the match

[1574] Device: Position the camera so that it can capture the entire court. Conduct test shots to ensure the camera captures all action during the match.

[1575] Entering match information

[1576] User: Launches the application on their smart device and inputs information such as the player's name, uniform number, and position. They also input information such as the start time of the game and the opposing team. The input data is sent to the server.

[1577] Real-time analytics

[1578] Server: Receives video data from the camera in real time and uses image recognition technology to identify the positions of players and the ball. AI algorithms distinguish between shots, passes, dribbles, and other plays, and automatically determines scores, fouls, and violations. The results are instantly updated on the scoreboard.

[1579] For example, when a player shoots from outside the three-point line, the system detects the shot and determines whether the ball went into the goal. If the goal is successful, the server immediately updates the score to three points.

[1580] Timekeeping

[1581] Server: Starts the timer at the start of the game and accurately manages the time for each quarter and timeout. Stops and restarts the timer at the appropriate time during timeouts and other interruptions in the game.

[1582] Recognizing and reflecting emotions

[1583] Server: Using the emotion engine, analyzes the user's facial expressions and voice data in real time to recognize their emotional state. Based on the recognized emotional data, the game progress and notification content are adaptively changed.

[1584] For example, if the system detects that the user is excited during a tense match, it will adjust the frequency of screen displays and notifications to prevent the user from feeling stressed. Conversely, if the system determines that the user is bored, it can send notifications highlighting the highlights of the match.

[1585] Post-match data generation

[1586] Server: After the match, it consolidates all data collected during the match to generate statistics and match records. It extracts key moments from the match to create highlight videos and provide emotional highlights that reflect the emotional data used. It also edits and saves the full match video in the appropriate format.

[1587] Users: Access the generated statistical data and videos through their smart devices and download them as needed. The server uploads the data and videos to the cloud and provides users with an access link.

[1588] Specific examples

[1589] For example, if Player B commits a foul during a match, the server analyzes the video data capturing that moment. The AI ​​distinguishes Player B's movements and automatically determines whether it was a foul based on the rules. At the same time, the emotion engine analyzes the user's reaction, and if it detects excitement or surprise, it can instantly provide a replay video that reflects that emotion.

[1590] By combining an emotion engine with video analysis, the system of the present invention can not only improve the efficiency of match management, but also the user experience.

[1591] The processing flow will be explained below.

[1592] Step 1:

[1593] The user sets the camera in a position that overlooks the entire court and adjusts it so that there are no blind spots. The camera is connected to the server via a network.

[1594] Step 2:

[1595] The user launches the application on their smart device and enters game information (start time, opposing teams, game format, etc.) and player information (name, uniform number, position, etc.). The entered data is sent to the server.

[1596] Step 3:

[1597] The server receives video data from the camera in real time and stores the received video data in a buffer for analysis.

[1598] Step 4:

[1599] The server analyzes the video data frame by frame and uses image recognition technology to identify the positions of players and the ball, tracking player movements.

[1600] Step 5:

[1601] The server uses AI algorithms to detect player actions (shooting, passing, dribbling, etc.), identify specific movement patterns, and determine events (goals, fouls, violations, etc.).

[1602] Step 6:

[1603] The server automatically determines the score based on detected events (e.g., goals, fouls), and updates the scoreboard in real time, updating individual performance data.

[1604] Step 7:

[1605] The server initiates timekeeping, accurately measures the start and end times of the game, the duration of each quarter, and timeouts, and stops and restarts game time as necessary.

[1606] Step 8:

[1607] The server uses an emotion engine to analyze the user's facial expressions and voice data in real time, recognizes the user's emotional state based on the analysis results, and adjusts the system's behavior based on that data.

[1608] Step 9:

[1609] For example, if the user is excited, the server will minimize the information displayed to reduce the user's stress. Conversely, if the user is perceived as bored, the server will send highlight notifications more frequently.

[1610] Step 10:

[1611] The server consolidates all data collected during the game and automatically generates statistics and match records after the game ends, compiling detailed statistics for each player, such as points, rebounds, assists, and fouls.

[1612] Step 11:

[1613] The server analyzes video data during the match, extracts important scenes, and creates highlight videos. It also provides emotional highlight videos that reflect the emotional data.

[1614] Step 12:

[1615] Users can access the generated statistical data and emotional highlight videos through their smart devices and download them as needed. The server uploads the data and videos to the cloud and provides users with an access link.

[1616] In this way, the system streamlines match management, provides fair and accurate refereeing and scoring, and also provides an interactive experience that responds to the user's emotions.

[1617] Example 2

[1618] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1619] Basketball game management requires a lot of manual work, and is prone to human error in refereeing, scoring, timekeeping, and game video production. Furthermore, there is a lack of ways to properly reflect user emotions during the game to provide a better experience.

[1620] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for receiving video data of the game in real time, means for analyzing the received video data and identifying the positions of the players and the ball, and means for analyzing the emotional state of the user in real time and adaptively adjusting the behavior of the system. This enables the automation of game management, reduces human error, and enables flexible responses according to the user's emotions.

[1621] "Means for receiving game video data in real time" refers to the cameras and communication infrastructure that capture video of the basketball game and transmit it to a server in real time.

[1622] "Means for analyzing received video data and identifying the positions of players and the ball" refers to algorithms or software that use image recognition technology to analyze and identify the positions of players and the ball from received video data.

[1623] "Means of determining player movements and automatically determining scores and fouls" refers to a system that uses AI algorithms to analyze player movements and automatically determine scores, fouls, and violations.

[1624] "Means for reflecting the judgement results on the scoreboard in real time" refers to a communication system and a display device for instantly displaying the judgement results on the scoreboard.

[1625] "Means for controlling the time of a match" means a digital timer or control system for accurately measuring the start, finish and stoppage times of a match.

[1626] "Means for analyzing the user's emotional state in real time and adaptively adjusting the system's behavior" refers to an emotion recognition engine that analyzes the user's facial expressions and voice data and dynamically changes the system's behavior and notification content based on that emotional state.

[1627] "Means for generating post-match statistics and video" means software and servers that aggregate data collected during a match and generate statistics, match records, and edited match video.

[1628] "Using a camera to capture a bird's-eye view of the entire court" refers to a method of capturing the entire court with a wide-angle lens to monitor the movement of all players and the ball at once.

[1629] "Entering match and player information using a smart device" refers to the process of entering player and match information before a match using a digital device such as a smartphone or tablet.

[1630] Overall system overview

[1631] The present invention is a system for automating basketball game management that integrates game filming, refereeing, scorekeeping, timekeeping, game video production, and an emotion engine that recognizes user emotions in real time and adjusts the system's behavior accordingly.

[1632] System Components

[1633] 1. Terminal (camera device)

[1634] A device equipped with a high-resolution, wide-angle lens is placed in a position that allows it to capture a bird's-eye view of the entire court. The camera is used to capture clear images of the entire game. For example, the camera can take a panoramic image of the entire court before the start of a game and send the image to a server.

[1635] 2. Server

[1636] The server has the following main functions:

[1637] 1. Receiving and analyzing video data: Video data is received in real time and the positions of players and the ball are analyzed using image recognition technology (e.g., OpenCV or TensorFlow).

[1638] 2. Judgment by AI algorithm: Based on the AI ​​algorithm, plays such as shots, passes, and dribbles are judged, and scores, fouls, and violations are automatically determined.

[1639] 3. Reflection of judgement results: The judgement results will be reflected immediately on the scoreboard.

[1640] 4. Timekeeping: Manage the game time, start the timer at the start of the game, and accurately manage the times of each quarter and timeout.

[1641] 5. Emotion Engine: Analyzes the user's facial expressions and voice data in real time to recognize their emotional state and adaptively adjust the system's behavior. For example, if the emotion engine detects that the user is excited, it will reduce the frequency of notifications.

[1642] 6. Data generation: Generate statistics and videos after the game, extract key moments and create highlight videos.

[1643] 3. User (Smart Device Application)

[1644] The user does the following:

[1645] 1. Entering match and player information: Using the smart device application, users enter information such as player names, uniform numbers, positions, match start times, and opposing teams. This information is then sent to the server.

[1646] 2. Data and video access: Access the generated statistical data and videos through smart devices and download them as needed. The server uploads the data and videos to the cloud and provides an access link.

[1647] Specific examples

[1648] For example, if player B commits a foul during a game, the server analyzes the video data capturing that moment. The AI ​​distinguishes player B's movements and automatically determines whether it was a foul based on the rules. At the same time, the emotion engine analyzes the user's reaction, and if it detects excitement or surprise, it instantly provides a replay video that reflects that emotion.

[1649] Example prompts to input to the generative AI model

[1650] Design a system that automatically identifies specific plays in an ongoing basketball game and adaptively adjusts notifications based on the user's emotional state. This system includes a camera that records the game, a server that analyzes real-time footage, a smart device application that manages game information, and an emotion engine that recognizes the user's emotions.

[1651] The above is an embodiment of the present invention that makes it easy to implement. This system can automate the management of matches and improve the user experience.

[1652] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1653] Step 1: Filming the match

[1654] Terminal

[1655] Input: Notification of the start of the match

[1656] Operation:

[1657] 1. A camera with a high resolution and wide-angle lens will be installed in a position that allows for a bird's-eye view of the entire court.

[1658] 2. The camera prepares to capture the entire match in real time.

[1659] 3. Conduct test shooting to check and adjust the shooting range and image quality.

[1660] Output: High-resolution real-time video data

[1661] Step 2: Enter match information

[1662] User

[1663] Input: Pre-match player and match information

[1664] Operation:

[1665] 1. Launch the smart device application.

[1666] 2. Enter the player's name, uniform number, position, game start time, and opposing team information.

[1667] 3. The entered information is sent to the server.

[1668] Output: Match and player information sent to the server

[1669] Step 3: Receiving video data

[1670] server

[1671] Input: Real-time video data from the camera

[1672] Operation:

[1673] 1. Receive video data sent from the camera in real time.

[1674] 2. The received video data is saved and prepared for the next analysis step.

[1675] Output: Received and stored video data

[1676] Step 4: Analyzing the video data

[1677] server

[1678] Input: Received video data

[1679] Operation:

[1680] 1. Analyze the video data using image recognition technology (e.g., OpenCV, TensorFlow).

[1681] 2. Identify the location of the players and the ball.

[1682] Output: Player and ball position data

[1683] Step 5: Motion detection and automatic judgment

[1684] server

[1685] Input: Player and ball position data

[1686] Operation:

[1687] 1. Using AI algorithms to identify player actions (shooting, passing, dribbling, etc.).

[1688] 2. Automatically judge scores, fouls, and violations based on the game rules.

[1689] Output: Scores, fouls, and violations

[1690] Step 6: Reflecting the results

[1691] server

[1692] Input: Judgment result

[1693] Operation:

[1694] 1. The judgement results will be reflected immediately on the scoreboard.

[1695] 2. Send notifications to users as needed.

[1696] Output: Updated leaderboard

[1697] Step 7: Timekeeping

[1698] server

[1699] Input: Notification of the start of the match

[1700] Operation:

[1701] 1. Start the digital timer at the same time as the match begins.

[1702] 2. Accurately manage the time of each quarter and timeout.

[1703] 3. Stop and restart the clock at the appropriate time when the game is interrupted.

[1704] Output: Precisely controlled game time

[1705] Step 8: Recognize and reflect on your emotions

[1706] server

[1707] Input: User facial and voice data

[1708] Operation:

[1709] 1. Analyze the user's facial expressions and voice data in real time using an emotion engine.

[1710] 2. Recognize the user's emotional state and adaptively adjust the system's behavior (notification frequency, display content, etc.).

[1711] Output: Changing the system's behavior based on the recognized emotion data.

[1712] Step 9: Post-match data generation

[1713] server

[1714] Input: Data collected during the match

[1715] Operation:

[1716] 1. After the match, all data is aggregated to generate statistics and match records.

[1717] 2. Extract important scenes and create a highlight video.

[1718] 3. Edit and save the full match video in the appropriate format.

[1719] Output: Statistics, match records, highlight videos, full match videos

[1720] Step 10: Accessing Data and Videos

[1721] User

[1722] Enter: Post-match notification

[1723] Operation:

[1724] 1. Access the generated statistics and videos using your smart device.

[1725] 2. Download data and videos as needed.

[1726] Output: Downloadable statistics and a link to the video

[1727] This is the specific processing flow of this system. This system automates the management of matches and enables flexible responses according to the user's emotions.

[1728] (Application example 2)

[1729] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1730] The problem that this invention aims to solve is to reduce the stress of managers in large spaces such as public buildings and plazas by detecting abnormal behavior and analyzing emotions, which conventional monitoring systems cannot handle. There is a need to detect abnormalities in real time and respond efficiently while reducing the burden on managers.

[1731] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving video data of the game in real time, means for analyzing the received video data and identifying the positions of players and the ball, means for determining player actions and automatically determining scores and fouls, means for reflecting the determination results on the scoreboard in real time, means for managing the game time, means for generating statistical data and videos after the game ends, means for detecting and notifying abnormal behavior, and means for analyzing the user's emotional state and adjusting the system's behavior. This enables efficient monitoring of large spaces, rapid detection of abnormal behavior, and adjustment of the system's behavior in accordance with the administrator's emotional state.

[1732] "Video data" refers to digital data of video footage captured by a camera.

[1733] "Real-time" refers to processing, sending, and receiving information instantly, without delay.

[1734] "Analysis" is the act of extracting specific patterns or information from digital data.

[1735] "Player" refers to a person participating in a sport or action.

[1736] "Ball" refers to a spherical object used for sports and games.

[1737] "Locate" means to determine the current coordinates or location of an object or person.

[1738] "Distinguishing behavior" refers to recognizing and classifying the behavior of a target person or object.

[1739] A "score" is a numerical value that indicates the score in a sporting event.

[1740] A "foul" is an act that violates the rules in a sporting event.

[1741] "Decision result" refers to a judgment or conclusion drawn based on an analysis.

[1742] A "scoreboard" is a device or screen that displays scores and other information from a sports event.

[1743] "Time management" means monitoring and regulating the progress of a competition or task.

[1744] "Statistical data" is quantitative information obtained from the results of analysis.

[1745] "Generating a moving image" means editing still images and video data to create a moving image.

[1746] "Abnormal behavior" refers to unusual or suspicious behavior that differs from normal behavior.

[1747] A "notification" is a message or signal that conveys specific information.

[1748] "Emotional state" indicates the type and strength of the emotion a person is feeling at the time.

[1749] "Analyzing and adjusting the system's behavior" refers to analyzing the emotional state and changing the system's behavior accordingly.

[1750] This invention is a surveillance system for large spaces such as public buildings and squares, which detects abnormal behavior in real time and adjusts the system's behavior by analyzing the user's emotional state. The system consists of a camera device, a server, a terminal application, and an emotion engine.

[1751] System configuration

[1752] 1. Camera equipment

[1753] The cameras have high-resolution, wide-angle lenses that can cover a wide area. They are recommended to be installed in strategic locations inside public buildings or plazas. The cameras transmit video data to a server in real time.

[1754] 2. Server

[1755] The server is equipped with an AI algorithm that analyzes the received video data and detects abnormal behavior. The server includes the following main components:

[1756] Real-time video analysis method: OpenCV and TensorFlow are used to analyze video data and detect abnormal behavior.

[1757] Notification method: Uses the Flask framework to send notifications to the user's device in real time.

[1758] Emotional state analysis: Azure Emotion API is used to analyze the user's emotional state and adjust the system's behavior.

[1759] Data storage method: MongoDB is used to store abnormal behavior data and emotion data, and data analysis is performed using Pandas.

[1760] 3. Terminal Applications

[1761] Users can use the terminal application to manage monitoring areas and notification settings, such as the area they want to monitor, the time period, and the frequency of notifications.

[1762] 4. Emotion Engine

[1763] The emotion engine analyzes the user's facial expressions and voice data in real time and sends the results to the server, which adjusts the content and frequency of notifications based on the user's emotional state, supporting efficient monitoring.

[1764] Specific examples

[1765] For example, when implementing it as a surveillance system for a shopping mall, multiple cameras are installed on each floor and at the entrance. The video data captured by the cameras is sent to a server in real time. The server uses OpenCV to analyze the video and detects abnormal behavior, such as suspiciously left luggage or suspicious movements in a crowd. Any abnormal behavior detected is immediately reported, and a warning is sent to the user's device.

[1766] At the same time, the emotion engine analyzes the user's facial expressions and voice, and if the administrator is feeling stressed, it adjusts the frequency and content of notifications to reduce the burden. For example, even if multiple anomalies are detected, the system automatically adjusts to prioritize notifications of only the most important detection results, and reports other information together later.

[1767] Below is an example of a prompt sentence to input to the generative AI model.

[1768] import cv2

[1769] import tensorflow as tf

[1770] from azure.ai.textanalytics import TextAnalyticsClient

[1771] from azure.core.credentials import AzureKeyCredential

[1772] import pymongo

[1773] from pandas import DataFrame

[1774] Importing camera footage

[1775] cap = cv2.VideoCapture(0)

[1776] Loading an AI model

[1777] model = tf.keras.models.load_model('anomaly_detection_model.h5')

[1778] Azure Emotion Engine Client Configuration

[1779] credential = AzureKeyCredential("API_KEY")

[1780] client = TextAnalyticsClient(endpoint="https: / / YOUR_ENDPOINT.cognitiveservices.azure.com / ", credential=credential)

[1781] MongoDB Configuration

[1782] client = pymongo.MongoClient("mongodb: / / localhost:27017 / ")

[1783] db = client["security_data"]

[1784] collection = db["anomaly_records"]

[1785] Video analysis processing

[1786] while True:

[1787] ret, frame = cap.read()

[1788] if not ret:

[1789] break

[1790] Input the frame into the AI ​​model

[1791] predictions = model.predict(frame)

[1792] if is_anomaly_detected(predictions):

[1793] What to do if an abnormality is detected

[1794] emotion_state = client.analyze_sentiment(documents=["Get the administrator's emotional state"])

[1795] collection.insert_one({"frame": frame.tolist(), "emotion": emotion_state[0].sentiment.name})

[1796] if emotion_state[0].sentiment.name != 'negative':

[1797] Notification Processing

[1798] send_alert_to_admin(frame, emotion_state[0].sentiment.name)

[1799] System shutdown processing

[1800] cap.release()

[1801] cv2.destroyAllWindows()

[1802] This embodiment makes it possible to realize efficient and effective wide-area monitoring while reducing the burden on the user.

[1803] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1804] Step 1:

[1805] The server receives video data in real time from the camera devices, which monitor a designated area and provide high-resolution video, and transmits the video data as successive frames to the server.

[1806] Step 2:

[1807] The server processes the received video data using OpenCV to split the individual frames, converting the video frames into an analyzable format.

[1808] Step 3:

[1809] The server uses TensorFlow to input the analyzed frame data into an anomalous behavior detection model. The model analyzes the movement patterns of people and objects in each frame and detects anomalous behavior (e.g., suspicious movements or abandoned luggage). The input is the analyzed frame data, and the output is the anomalous behavior detection results.

[1810] Step 4:

[1811] When abnormal behavior is detected, the server generates an anomaly detection notification using the Flask framework. This notification includes image data for the frame in which the anomaly was identified and the type of anomaly detected. The notification is sent to the user's device in real time.

[1812] Step 5:

[1813] The user's device receives anomaly detection notifications sent from the server in real time. The received notification includes an image of the frame showing the abnormal behavior and details of the anomaly. The user can check the content of the notification and take the necessary action immediately.

[1814] Step 6:

[1815] The server uses the Azure Emotion API to analyze the user's emotional state. The analysis uses the user's facial expression and voice data acquired in real time. The input is the user's facial expression and voice data, and the output is the emotion analysis results.

[1816] Step 7:

[1817] The server adjusts the system's behavior based on the results of emotion analysis. Specifically, if the user is feeling stressed, the server adjusts the frequency and content of notifications to reduce the burden on the user. This allows the user to perform monitoring tasks more efficiently.

[1818] Step 8:

[1819] The server stores anomaly detection data and emotion analysis data in MongoDB. The stored data is later used for analysis and report creation. The stored data includes video data, types of abnormal behavior, emotion analysis results, etc.

[1820] Step 9:

[1821] The server analyzes the stored data using Pandas and generates weekly or monthly reports, including the frequency and trends of abnormal behavior and fluctuations in the emotional state of administrators. The reports are provided to administrators to help them improve their monitoring measures.

[1822] Through the above processing steps, the system can efficiently and effectively realize wide-area monitoring and reduce the burden on the user.

[1823] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1824] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1825] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1826] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1827] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1828] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1829] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1830] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1831] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1832] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1833] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1834] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1835] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1836] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1837] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1838] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1839] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1840] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1841] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1842] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1843] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1844] The following is further disclosed regarding the above embodiment.

[1845] (Claim 1)

[1846] a means for receiving video data of the match in real time;

[1847] means for analyzing the received video data and determining the positions of the player and the ball;

[1848] A means of determining player movements and automatically determining scores and fouls,

[1849] A means to reflect the judgment results on the scoreboard in real time,

[1850] A means of managing the game time;

[1851] A means to generate statistics and videos after the match has finished,

[1852] A system including:

[1853] (Claim 2)

[1854] The system of claim 1, which uses a camera to take a bird's-eye view of the entire court.

[1855] (Claim 3)

[1856] 10. The system of claim 1, wherein match information and player information are input using a smart device.

[1857] "Example 1"

[1858] (Claim 1)

[1859] A means of receiving real-time video data of the match using high-resolution cameras;

[1860] means for analyzing the received video data and determining the positions of the player and the ball;

[1861] A means of automatically determining scores and fouls by using artificial intelligence algorithms to distinguish player movements, and

[1862] A means to reflect the judgement results on the scoreboard in real time,

[1863] A means of managing the game time;

[1864] a means for generating statistics and edited match video after the match is over;

[1865] A means for inputting match information and player information using a smart device;

[1866] A system including:

[1867] (Claim 2)

[1868] The system according to claim 1, wherein a camera is installed to take a bird's-eye view of the entire match court.

[1869] (Claim 3)

[1870] The system according to claim 1, which receives and analyzes video data in real time to distinguish between plays such as shooting, passing, and dribbling, and automatically performs scoring and foul counting.

[1871] "Application Example 1"

[1872] (Claim 1)

[1873] a means for receiving video data of the match in real time;

[1874] means for analyzing the received video data and determining the location of objects and elements;

[1875] A means for automatically determining the target's behavior and its information and status;

[1876] means for reflecting the determination result on a display device in real time;

[1877] A way to manage time,

[1878] A means to generate statistics and videos after completion;

[1879] Real-time analysis of in-store conditions to identify behavioral patterns and product popularity,

[1880] A means of notifying real-time analysis results;

[1881] A system including:

[1882] (Claim 2)

[1883] The system according to claim 1, wherein the system uses a camera to take an overhead view of the entire area.

[1884] (Claim 3)

[1885] 10. The system of claim 1, wherein the smart device is used to input information and data.

[1886] "Example 2: Combining Emotion Engines"

[1887] (Claim 1)

[1888] a means for receiving video data of the match in real time;

[1889] means for analyzing the received video data and determining the positions of the player and the ball;

[1890] A means of determining player movements and automatically determining scores and fouls,

[1891] A means to reflect the judgment results on the scoreboard in real time,

[1892] A means of managing the game time;

[1893] A means for analyzing the user's emotional state in real time and adaptively adjusting the system's behavior;

[1894] A means to generate statistics and videos after the match has finished,

[1895] A system including:

[1896] (Claim 2)

[1897] The system of claim 1, which uses a camera to take a bird's-eye view of the entire court.

[1898] (Claim 3)

[1899] 10. The system of claim 1, wherein match information and player information are input using a smart device.

[1900] "Application example 2 when combining emotion engines"

[1901] (Claim 1)

[1902] a means for receiving video data of the match in real time;

[1903] means for analyzing the received video data and determining the positions of the player and the ball;

[1904] A means of determining player movements and automatically determining scores and fouls,

[1905] A means to reflect the judgment results on the scoreboard in real time,

[1906] A means of managing the game time;

[1907] A means to generate statistics and videos after the match has finished,

[1908] means for detecting and notifying anomalous behavior;

[1909] means for analyzing the user's emotional state and adjusting the system's behavior;

[1910] A system including:

[1911] (Claim 2)

[1912] 2. The system of claim 1, wherein the camera is used to capture an overhead view of the entire area.

[1913] (Claim 3)

[1914] 10. The system of claim 1, wherein the monitoring information and the target information are input using a terminal. [Explanation of symbols]

[1915] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for receiving video data of the match in real time; means for analyzing the received video data and determining the positions of the player and the ball; A means of determining player movements and automatically determining scores and fouls, A means to reflect the judgment results on the scoreboard in real time, A means of managing the game time; A means to generate statistics and videos after the match has finished, A system including:

2. The system according to claim 1, wherein the camera is used to take a bird's-eye view of the entire court.

3. The system of claim 1, wherein game information and player information are input using a smart device.

Images