Processing apparatus, processing method, processing program, and processing system

Abstract

The present invention provides a processing device, processing method, processing program, and processing system that enable the effective use of coaching images. [Solution] A processing device comprising a display control unit that causes one or more displays to display a coaching image for coaching in a sport conducted in a predetermined field, the first area displaying an overhead image showing an overhead view of a virtual field corresponding to the predetermined field in a predetermined situation, and a second area displaying a field view image showing the view from at least one of the one or more player objects located in the virtual field, wherein the processing device displays a coaching image on one or more displays, The display control unit receives an input for selecting a selected player object, which is one or more player objects located in the virtual field, and switches the view image of the second region to the view image from the selected player object and displays it.

Description

Technical Field

[0001] The present disclosure relates to a processing device, a processing method, a processing program, and a processing system for controlling display of coaching images.

Background Art

[0002] Conventionally, there has been proposed a system using "an information terminal for supporting tactical coaching and tactical training in curling competitions, the information terminal comprising: a display unit configured by a touch panel and displaying various types of information; an input unit configured by the touch panel and allowing input to be performed on the displayed screen; a storage unit storing information input from the input unit and various pre-input icon images related to curling competitions; and a control unit controlling the display unit, the input unit, and the storage unit, wherein when inputting match information of a curling competition, the control unit selectively reads out match-related information including at least icon images of a curling sheet and stones from the storage unit to display the same on the display unit, and is configured to store position information of the stones in the storage unit when the icon image of the stone displayed via the input unit is dragged and moved" (Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

[0004] However, the above system is only capable of displaying images of a sheet and stones used in a specific sport and moving the icon image of the stones by dragging, and cannot display images in a form such as a player's field-of-view image, for example.

Summary of Invention

Problems to be Solved by the Invention

[0005] Therefore, based on the technologies described above, this disclosure aims to provide a processing device, processing method, processing program, and processing system that are capable of displaying a wider variety of information. [Means for solving the problem]

[0006] According to one aspect of the present disclosure, a processing device is provided which includes a display control unit that causes one or more displays to display a coaching image, which includes a first region that displays an overhead image of a virtual field corresponding to a predetermined field in a predetermined situation for coaching in a sport played on a predetermined field, and a second region that displays a field of view image showing the view from at least one of one or more player objects located in the virtual field, wherein the display control unit receives an operation input for selecting a selected player object, which is one of one or more player objects located in the virtual field, and switches the field of view image in the second region to display the field of view image from the selected player object. [Effects of the Invention]

[0007] According to this disclosure, it is possible to provide a processing device, processing method, processing program, and processing system that can display a variety of images, such as a coaching image that includes an overhead view of a virtual field suitable for coaching in a sport played on a predetermined field, and a view image of a player object.

[0008] The effects described above are merely illustrative for the sake of explanation and are not limiting. In addition to, or in lieu of, any other effects described herein or that would be obvious to those skilled in the art may be achieved. [Brief explanation of the drawing]

[0009] [Figure 1]Figure 1 is a block diagram showing the configuration of a processing system 1 according to one embodiment of the present disclosure. [Figure 2A] Figure 2A is a block diagram showing the configuration of a server device 100 according to one embodiment of the present disclosure. [Figure 2B] Figure 2B is a block diagram showing the configuration of a terminal device 200 according to one embodiment of the present disclosure. [Figure 3] Figure 3 is a functional block diagram showing the functions performed in a server device 100 according to one embodiment of this disclosure. [Figure 4] Figure 4 is a conceptual diagram showing a player management table stored in a server device 100 according to one embodiment of this disclosure. [Figure 5] Figure 5 is a conceptual diagram showing a scene management table stored in a server device 100 according to one embodiment of the present disclosure. [Figure 6] Figure 6 shows a processing sequence executed by a processing system 1 according to one embodiment of the present disclosure. [Figure 7] Figure 7 shows an example of a field 10 on which a sport according to one embodiment of this disclosure is played. [Figure 8] Figure 8 shows an example of a coaching image 20 that includes a first region displaying an overhead view image according to one embodiment of the present disclosure. [Figure 9] Figure 9 shows an example of a player object 24 according to one embodiment of the present disclosure. [Figure 10] Figure 10 shows an example of a coaching image 40 that includes a second region displaying a field of view image according to one embodiment of the present disclosure. [Figure 11] Figure 11 is a diagram conceptually illustrating the relationship between a field of view image and the ability information and height information of a player object according to one embodiment of this disclosure. [Figure 12A] Figure 12A shows an example of a coaching image 40 that includes a second region displaying a field of view image according to one embodiment of the present disclosure. [Figure 12B] Figure 12B shows an example of a coaching image 40 that includes a second region displaying a field of view image according to one embodiment of the present disclosure. [Figure 12C] FIG. 12C is a diagram illustrating an example of a coaching image 40 including a second region for displaying a field-of-view image according to an embodiment of the present disclosure. [Figure 12D] FIG. 12D is a diagram illustrating an example of a coaching image 40 including a second region for displaying a field-of-view image according to an embodiment of the present disclosure. [Figure 13] FIG. 13 is a diagram illustrating an example of a coaching image 20 including a first region for displaying a bird's-eye view image according to an embodiment of the present disclosure. [Mode for Carrying Out the Invention]

[0010] 1. Overview of the Processing System A processing system according to the present disclosure is used to control display of a coaching image used for coaching in a sport performed on a predetermined field. Specifically, the processing system generates a coaching image including a first region that displays a bird's-eye view image overlooking a virtual field corresponding to the field in a predetermined situation, and a second region that displays a field-of-view image indicating a field of view from at least one player object among one or more player objects located in the virtual field. The processing system performs control to display the coaching image on a first terminal device usable by a coach, and performs control to display the coaching image on a second terminal device usable by a player.

[0011] Further, the processing system accepts an operation input for selecting a selected player object that is any one of the one or more player objects located in the virtual field, on at least one of the first terminal device and the second terminal device. Then, the processing system 1 performs control to switch and display the field-of-view image in the second region to a field-of-view image from the selected player object based on the operation input.

[0012] As described above, by using the processing system, it is possible to perform various displays such as a coaching image including a bird's-eye view image and a field-of-view image. Further, by switching the field-of-view image displayed in the second area to the field-of-view image of another player object (selected player object) based on an operation input accepted by at least one of the first terminal device and the second terminal device, it is possible to effectively provide the coaching image.

[0013] Note that in the present disclosure, "sport" may be any sport as long as it is a sport played by players on a field. Examples of such sports include various types such as soccer, futsal, baseball, softball, rugby, American football, volleyball, basketball, hockey, ice hockey, handball, water polo, cricket, kabaddi, table tennis, badminton, tennis, bowling, cycling, motor sports, track and field, rowing, skiing, yachting, or golf. Among these, the processing system 1 can be suitably applied to sports performed by a team composed of a plurality of teammates, for example, soccer, futsal, baseball, softball, rugby, American football, volleyball, basketball, hockey, ice hockey, handball, water polo, cricket, kabaddi, table tennis, badminton, or tennis.

[0014] In the present disclosure, "coach" and "player" do not mean persons having specific titles such as a coach, a manager, an athlete, a player, or a competitor. That is, "coach" broadly means a person who conveys guidance, play improvement suggestions, or opinions on play to another person (for example, a player) in a specific scene. In addition, "player" broadly means a person who plays a sport on a field. Therefore, in some cases, the player themselves may act as a coach, or the coach themselves may act as a player.

[0015] In this disclosure, “coaching image” means an image displayed during coaching. Here, as stated above, in this disclosure, “coach” broadly means a person who provides guidance, suggestions for improving play, or opinions on play to another person (e.g., a player) in a specific situation. Therefore, coaching images can similarly include a variety of images as long as they are images used in a situation where guidance, suggestions for improving play, or opinions on play are provided to another person in a specific situation.

[0016] Specifically, the coaching image includes a first area that displays an overhead image showing an overview of a virtual field corresponding to a predetermined field, and a second area that displays a view image showing the view from player objects placed in the virtual field corresponding to one or more players playing on the predetermined field. Typically, the coaching image is output to a first terminal device usable by the coach and a second terminal device usable by the player, and is displayed on each terminal device. At this time, the coaching image displayed on the first terminal device and the coaching image displayed on the second terminal device may be exactly the same or may be different. Typically, the following coaching image may be displayed as an example. A: The same image is displayed on both the first terminal device and the second terminal, which includes both a first area displaying an overhead view image and a second area displaying a field of view image. B: The first terminal device displays a first area showing an overhead view image, and the second terminal device displays a second area showing a field of view image. • C: The first terminal device displays a first area showing an overhead view image, while the second terminal device displays both the first area showing the overhead view image and the second area showing the field of view image. ·D: The first terminal device displays both the first area showing the overhead view and the second area showing the field of view, while the second terminal device displays the second area showing the field of view. ·E: The first terminal device displays both the first area showing the overhead image and the second area showing the reduced field of view image, while the second terminal device displays the first area showing the reduced overhead image and the field of view image of the second area showing the field of view image.

[0017] While this disclosure primarily describes case E above, coaching images are not limited to this example.

[0018] 2. Configuration of Processing System 1 Figure 1 is a block diagram showing the configuration of a processing system 1 according to one embodiment of the present disclosure. According to Figure 1, the processing system 1 includes at least a server device 100 and a terminal device 200, and each device is connected to communicate via a wired or wireless communication network. The server device 100 mainly performs display control processing to display a coaching image on at least one of the displays of the first terminal device and the second terminal device. The first terminal device 200-1 mainly performs processing to display a coaching image 214 including a first region that displays an overhead view image and to accept operation input for selecting a player object. The second terminal device 200-2 mainly performs processing to display a coaching image 214 including a second region that displays a field of view image.

[0019] In this disclosure, the processing unit means the server device 100, the first terminal device 200-1, the second terminal device 200-2, or a combination thereof. That is, although the following description will focus on the case where the server device 100 functions as a processing unit, the first terminal device 200-1 or the second terminal device 200-2 can also function as a processing unit. Furthermore, in this disclosure, the storage and processing performed by the processing unit may be distributed to other terminal devices or other server devices. In other words, the processing unit is not limited to those consisting of a single enclosure, but includes the server device 100, the first terminal device 200-1, the second terminal device 200-2, other server devices, other terminal devices, or a combination thereof.

[0020] Furthermore, although only one server device 100 is shown in Figure 1, as described above, the server device 100 can be composed of multiple server devices and multiple database devices, and processing and storage can be distributed among these devices.

[0021] Furthermore, although only one instance each of the first terminal device 200-1 and the second terminal device 200-2 is shown in Figure 1, each device may include multiple devices depending on the number of users. Also, the first terminal device 200-1 can be any device that is at least usable by the coach, and similarly, the second terminal device 200-2 can be any device that is at least usable by the player. Therefore, the first terminal device 200-1 does not always need to function as the first terminal device 200-1, and can function as the second terminal device 200-2 when used by a player. Similarly, the second terminal device 200-2 does not always need to function as the second terminal device 200-2, and can function as the first terminal device 200-1 when used by a coach.

[0022] 3. Configuration of Server Device 100 Figure 2A is a block diagram showing the configuration of a server device 100 according to one embodiment of the present disclosure. According to Figure 2A, the server device 100 includes a processor 111, a memory 112, and a communication interface 113. Each of these components is electrically connected to the others via control lines and data lines. The server device 100 does not need to have all of the components shown in Figure 2A; it is possible to omit some components or add other components. For example, it is possible to use a communicationally connected external memory, a database device, or another server device as memory. It is also possible to distribute some processing to other server devices or other processing units. In other words, the server device 100 is not limited to a single device, but also includes cases where it is distributed across multiple devices depending on the handling of information and the processing load.

[0023] The processor 111 functions as a control unit (for example, the control unit 101b in Figure 3) that controls other components of the processing system 1 based on a processing program stored in the memory 112. The processor 111 is mainly composed of one or more CPUs, but a GPU, FPGA, etc. may be combined as appropriate. The processor 111 performs the function of the control unit 101b shown in Figure 3, for example, by processing the processing program stored in the memory 112. Details of the control unit 101b will be explained in Figure 3.

[0024] Memory 112 is composed of RAM, ROM, non-volatile memory, HDD, SSD, etc., and functions as a storage unit (for example, storage unit 101a in Figure 3). Memory 112 stores instruction commands for various controls of the processing system 1 according to this embodiment as processing programs. Specifically, memory 112 stores processing programs and information for the processor 111 to execute functions as a control unit 101b as shown in Figure 3. In addition to the processing programs, memory 112 also stores various information stored in the player management table, scene management table, etc. Note that this information does not need to be constantly stored in memory 112 within the server device 100, but may be stored in a database device installed remotely. In that case, the database device is also included in memory 112.

[0025] The communication interface 113 functions as a communication unit for sending and receiving various information with terminal devices 200 connected via a wired or wireless communication network. Examples of the communication interface 113 include wired communication connectors such as USB and SCSI, wireless communication transceivers such as wireless LAN, Bluetooth®, LTE, 5G, and infrared, as well as various connection terminals for printed circuit boards and flexible circuit boards.

[0026] Figure 3 is a functional block diagram showing the functions performed in a server device 100 according to one embodiment of the present disclosure. Specifically, Figure 3 is a diagram showing each function performed by the processor 111 executing a processing program stored in memory 112 in the server device 100 shown in Figure 2A. According to Figure 3, the server device 100 performs at least the functions of the storage unit 101a and the control unit 101b.

[0027] The memory unit 101a stores at least a processing program for the processor 111 to execute the functions of the control unit 101b, and various information stored in the player management table, scene management table, etc. Details of this information will be explained in Figures 4 and 5.

[0028] The control unit 101b includes at least a display control unit that performs processing to display a coaching image, which includes a first area for displaying an overhead view image and a second area for displaying a field of view image, on at least one of the displays of the first terminal device 200-1 and the second terminal device 200-2.

[0029] In addition to the above processing, the display control unit switches the view image of the second region to the view image from the selected player object when at least one selected player object is selected by an operation input received in at least one of the first terminal device 200-1 and the second terminal device 200-2. By switching and displaying the view image in this way, it becomes possible to effectively utilize the coaching image.

[0030] Furthermore, the display control unit receives an input for moving the position of the selected player object in the first area, and changes the view image displayed in the second area to, for example, the view image after the movement. By changing the view image to the view image after the movement in this way, it becomes possible to effectively utilize the coaching image.

[0031] Furthermore, the display control unit displays an image in the first region that indicates the direction of the player object's view, corresponding to the view image displayed in the second region, and associates it with the player object. By displaying an image that indicates the direction of the view in this way, it becomes possible to efficiently determine the direction the player object is facing in the overhead view image.

[0032] Furthermore, the display control unit generates a field of view image according to at least one of the height information and ability information stored in the memory unit 101a for each player object, and displays the field of view image. By displaying a field of view image according to at least one of the height information and ability information in this way, it becomes possible to reproduce an image that is closer to a real field of view.

[0033] Furthermore, when the display control unit receives an input from at least one of the first terminal device 200-1 and the second terminal device 200-2 to select one of a plurality of scenes, it generates and displays at least one of an overhead view image and a field of view image based on the position information of the player object associated with the scene and the field of view information of each player object. In this way, by displaying an image according to the scene selection, it becomes possible to efficiently reproduce the desired scene.

[0034] Furthermore, the display control unit receives an operation input from at least one of the first terminal device 200-1 and the second terminal device 200-2 to indicate a player object displayed in the second area, and displays an identification indicator in the first area corresponding to the player object to indicate that such an instruction has been given. In this way, by identifying the player object indicated in the second area in the first area, it becomes possible to efficiently grasp the position of the player object in the view image in the overhead image.

[0035] Furthermore, the display control unit receives an input from at least one of the first terminal device 200-1 and the second terminal device 200-2 to input an input object (e.g., a mark or text) at an arbitrary position on the overhead image, and displays the input object at a position corresponding to that arbitrary position in the field of view image displayed in the second region. In this way, by displaying the input object input in the overhead image in the field of view image as well, it becomes possible to make coaching smoother.

[0036] Furthermore, the display control unit displays a series of coaching images over a specified period. By displaying a series of coaching images, such as a video, it becomes possible to effectively grasp a sequence of movements.

[0037] 4. Configuration of terminal device 200 Figure 2B is a block diagram showing the configuration of a terminal device 200 according to one embodiment of the present disclosure. According to Figure 2B, the terminal device 200 includes a processor 211, memory 212, input interface 213, output interface 214, and communication interface 215. In this disclosure, the first terminal device 200-1 and the second terminal device 200-2 are collectively referred to as terminal device 200.

[0038] The terminal device 200 can be any device capable of communicating with the server device 100 via a wired or wireless network, and can be a whiteboard device, smartphone device, tablet device, laptop PC device, or desktop PC device.

[0039] Furthermore, as described above, if there are multiple coaches or multiple players, each can use their own terminal device, but the first terminal device 200-1 and the second terminal device 200-2 may be of different types. Also, it is not necessary for each user to have their own terminal device; multiple users may use one terminal device, or one user may use multiple terminal devices. In addition, the first terminal device 200-1 and the second terminal device 200-2 may be of the same type, or they may be of different types. Specifically, the first terminal device 200-1 may use a whiteboard device, and the second terminal device 200-2 may use a tablet device.

[0040] As shown in Figure 2B, each component of the terminal device 200 is electrically connected to one another via control lines and data lines. The terminal device 200 does not need to have all of the components shown in Figure 2B; it is possible to omit some components or add other components.

[0041] The processor 211 functions as a control unit that controls other components of the processing system 1 based on processing programs stored in memory 212. The processor 211 is mainly composed of one or more CPUs, but may be combined with a GPU, FPGA, etc. as appropriate. Based on the processing programs stored in memory 212, the processor 211 performs the necessary functions as at least one of the first terminal device 200-1 and the second terminal device 200-2.

[0042] Specifically, when the terminal device 200 functions as the first terminal device 200-1, the processor 211 performs the following processing: "to display at least the first region of a coaching image on the display via the output interface 214, which includes a first region that displays an overhead image showing an overhead view of a virtual field corresponding to a predetermined field in a predetermined situation for coaching in a sport performed on a predetermined field, and a second region that displays a view image showing the view from at least one of the one or more player objects located in the virtual field," and "to display the view image of the second region to a selected player object which is one of the one or more player objects located in the virtual field." The system performs operations such as: "a process to accept operation input from the coach via the input interface 213 to select a player object in order to switch to a view image from the first area"; "a process to accept operation input from the coach via the input interface 213 to move the position of the player object in the first area in order to change the view image displayed in the second area"; "a process to accept operation input from the coach via the input interface 213 to select a desired situation from among a plurality of predetermined situations"; and "a process to accept operation input from the coach via the input interface 213 to input an input object to an arbitrary position on the overhead image displayed in the first area".

[0043] Furthermore, when terminal device 200 functions as a second terminal device 200-2, processor 211 performs the following processes: "to display at least the second region of a coaching image on a display via output interface 214, which includes a first region that displays an overhead image showing an overhead view of a virtual field corresponding to a predetermined field in a predetermined situation for coaching in a sport played on a predetermined field, and a second region that displays a view image showing the view from at least one player object among one or more player objects located in the virtual field" and "to receive operation input from a player via input interface 213 to indicate a player object different from the selected player object displayed in the second region."

[0044] Memory 212 consists of RAM, ROM, non-volatile memory, HDD, SSD, etc., and functions as a storage unit. Memory 212 stores instruction commands for various control of the processing system 1 according to this embodiment as programs. Specifically, memory 212 stores processing programs for the processor 211 to execute each of the processes described above.

[0045] The input interface 213 functions as an input unit that receives user (e.g., coach or player) input to the terminal device 200. Examples of the input interface 213 include physical key buttons and a touch panel having an input coordinate system corresponding to the display coordinate system of the display. In the case of a touch panel, icons are displayed on the display, and the user makes a selection for each icon by making an input via the touch panel. The method for detecting user input via the touch panel can be any method, such as capacitive or resistive. The input interface 213 does not always need to be physically provided on the terminal device 200 and may be connected as needed via a wired or wireless network. Therefore, in addition to the above, a mouse or keyboard can also be used as the input interface 213.

[0046] The output interface 214 functions as an output unit for outputting various notifications and other information received from the server device 100. An example of the output interface 214 is a display composed of an LCD panel, an organic EL display, or a plasma display. However, the terminal device 200 itself does not necessarily need to be equipped with a display. For example, an interface for connecting to a display that can be connected to the terminal device 200 via a wired or wireless communication network can also function as the output interface 214 for outputting display data to the display.

[0047] The communication interface 215 functions as a communication unit for sending and receiving various types of information between the server device 100 and other terminal devices connected via a wired or wireless communication network. Examples of the communication interface 215 include wired communication connectors such as USB and SCSI, wireless communication transceivers such as wireless LAN, Bluetooth®, LTE, 5G, and infrared, as well as various connection terminals for printed circuit boards and flexible circuit boards.

[0048] 5. Information stored in server device 100 (1) Player management table Figure 4 is a conceptual diagram showing a player management table stored in a server device 100 according to one embodiment of the present disclosure. According to Figure 4, the player management table stores player object information, height information, ability information, and attribute information, associated with player ID information.

[0049] "Player ID information" is information unique to players who play sports on a designated field, and is used to identify each player. Player ID information is newly generated by the server device 100 when it receives a registration request from the first terminal device 200-1, for example, from a coach in processing system 1, requesting that the player be newly registered.

[0050] "Player object information" refers to objects (e.g., image data) placed on a virtual field in a virtual space, corresponding to the position of each player on the field. Such player objects can take various forms, such as circles or other shapes, avatars modeled after each player, characters, monsters, text, or combinations thereof. Preferably, player objects may be arranged so that player objects of players on the friendly team and player objects of players on the opposing team are mutually identifiable among multiple players located on the field. Alternatively, player objects may be arranged so that the player currently holding the ball on the field can be identified by placing a ball object adjacent to a player object.

[0051] In this embodiment, the player object is displayed in both the overhead view image displayed in the first region and the field of view image displayed in the second region. Therefore, at least two different forms of player objects may be generated for the same player. For example, the player object placed on the overhead view image may be a two-dimensional object in the form of a symbol. The player object placed on the field of view image may be a three-dimensional object that mimics the appearance of each player. In this way, different player objects may be stored for the overhead view image and the field of view image.

[0052] "Height information" is information provided corresponding to each player's height, and is registered as attribute information indicating the height of each player's player object. The height information does not need to correspond to the player's current height; for example, it may correspond to the height measured before the season, or to the height updated at any time.

[0053] "Ability information" is information provided in accordance with each player's ability when playing a sport, and indicates the ability of each player's player object. Examples of such ability information include a wide field of vision, running speed, kicking power, endurance, defensive ability, jumping ability, tactical understanding, playing accuracy, or combinations thereof. Preferably, the field of vision is used as ability information. Ability information is typically composed of numerical values, classifications, or combinations thereof that indicate the degree of each ability. Numerical values ​​indicating each ability may be calculated based on sensing information obtained by sensing each player's movements during play, and these may be stored as ability information for each player. Alternatively, numerical values ​​entered by a user such as a coach or player may be stored as ability information for each player.

[0054] "Attribute information" refers to information that indicates the attributes of each player. Examples of such attribute information include name, nickname, team affiliation, position, past performance, training history, or a combination of these. Attribute information may also be stored based on information entered by users such as coaches or players for each player.

[0055] (2) Scene management table Figure 5 is a conceptual diagram showing a scene management table stored in a server device 100 according to one embodiment of the present disclosure. According to Figure 5, the scene management table stores attribute information, time information, sensing information, location information, and field of view information, etc., associated with scene ID information.

[0056] "Scene ID information" is information unique to each scene and used to identify each scene. In this embodiment, the field in a predetermined situation during the play of a sport is reproduced using an overhead image that provides an overview of the corresponding virtual field, or a view image from a player object placed in the corresponding virtual field. Therefore, Scene ID information is generated in accordance with the situation (scene) on the field at each time point during the play of the sport.

[0057] "Attribute information" is information that indicates the attributes of each scene. Examples of such attribute information include a scene description, scene title, type of sport, match name, tournament name, name of the team, name of the opposing team, names of the players on the team, names of the players on the opposing team, date of implementation, or a combination of these. Attribute information may be stored based on information entered by the coach or player via the first terminal device 200-1 or the second terminal device 200-2.

[0058] "Time information" refers to information indicating the time at which each situation (scene) took place. Such time information can include various types of time, such as the time at which the event took place, the elapsed time since the start of the sport, the remaining time until the end of the sport, or a combination of these. Time information may be generated and stored based on time detected by a timer.

[0059] "Sensing information" is information obtained by sensing the movements of each player during the play of a sport being conducted on a field. Such sensing information includes GPS coordinate information, field coordinate information, angle information indicating the direction of the field of view, or a combination thereof. Of these, GPS coordinate information and field coordinate information are information detected by wearable sensors (including GPS sensors, accelerometers, angular velocity sensors, image sensors that capture the field, or a combination thereof) that are attached to each player in advance. Angle information indicating the direction of the field of view is information obtained by detecting each player from an image of the field captured by an image sensor using arbitrary object detection technology, and similarly detecting the direction of each player using arbitrary object detection technology.

[0060] "Location information" refers to information indicating the position of a player object on a virtual field placed in the virtual space. This information is obtained by converting GPS coordinate information, field coordinate information, or a combination thereof obtained from sensing information into location information on a virtual field constructed in the virtual space corresponding to the field. For example, such location information is obtained by setting the origin of the virtual three-dimensional coordinate space to correspond to the origin of the three-dimensional coordinate space that represents coordinate information obtained in the real world, and setting a position in the virtual three-dimensional coordinate space such that the distance from the origin in the real world matches the distance from the origin in the virtual three-dimensional coordinate space. This is how location information is obtained.

[0061] "Field of view information" is information indicating the direction of the player object's line of view. Field of view information is angle information obtained by converting the angle information indicating the direction of the line of view, obtained from sensing information, into the corresponding direction within the virtual three-dimensional coordinate space described above. In other words, the angle indicated by the field of view information is set to correspond to the direction the player is actually facing on the field in the real world.

[0062] Here, Figure 7 shows an example of a field 10 in which a sport is played according to one embodiment of the present disclosure. Specifically, Figure 7 shows an example of a field 10 in which a sport is played in a real three-dimensional coordinate space. According to Figure 7, the field 10 has a total of 11 players, including player 11 playing the sport and players on the team of player 11, and a total of 11 players on the opposing team, and a game of soccer is being played. Each player, including player 11, is fitted with a wearable sensor as described above, and sensing information is acquired from the wearable sensor along with information indicating the time at which it was acquired. The acquired sensing information is then stored in association with scene ID information having time information corresponding to that time. Subsequently, the sensing information is converted into position information in a virtual space.

[0063] Furthermore, field 10 is acquired by an image sensor along with information indicating the time the field image was captured, allowing for an overall overview of the information. From the captured image, the direction of each player's field of view is detected as described above, and acquired as angle information (sensing information) indicating the direction of each player's field of view. The acquired angle information (sensing information) is then stored in association with scene ID information that has time information corresponding to that time. Subsequently, the angle information (sensing information) is converted into field of view information indicating the direction of the player object's field of view in the virtual space.

[0064] Returning to Figure 5, the sensing information shown in Figure 7 is acquired for each player and converted into location information and field of view information. Therefore, as shown in Figure 5, the sensing information, location information, and field of view information are stored in association with the player ID information of each player included in the scene, which is identified by the scene ID information.

[0065] The information stored in each table in Figures 4 and 5 is merely an example; it is not necessary to store all of this information, and other information may be stored as well.

[0066] 6. Processing sequence executed by processing system 1 Figure 6 is a diagram showing a processing sequence executed by a processing system 1 according to one embodiment of the present disclosure. Specifically, Figure 6 shows a processing sequence for displaying a coaching image when a scene is selected in S11 to S18, a processing sequence for displaying a coaching image when an operation input is made to the overhead image by the first terminal device 200-1 in S21 to S24, and a processing sequence for displaying a coaching image when an operation input is made to the field of view image by the second terminal device 200-2 in S31 to S34.

[0067] (1) Processing sequence for displaying a coaching image when a scene is selected As shown in Figure 6, the processor 211 of the first terminal device 200-1 receives the coach's operation input via the input interface 213 and generates a coaching image request to request the generation of a coaching image (S11). Although not specifically shown in Figure 6, the first terminal device 200-1 and the second terminal device 200-2 have an application program for displaying coaching images installed in advance, and this application program is running. The first terminal device 200-1 then displays a home screen with icons for requesting the generation of a coaching image, and the coaching image request is generated when the operation input is received on this home screen.

[0068] The processor 211 of the first terminal device 200-1 transmits the generated coaching image request (T11) to the server device 100 via the communication interface 215. At this time, the processor 211 of the first terminal device 200-1 also transmits its own user ID information or information identifying the team to which it belongs along with the coaching image request.

[0069] In Figure 6, the case where the coaching image generation request is sent from the first terminal device 200-1 is explained, but it may also be sent from the second terminal device 200-2, which is available to the player, using a similar method.

[0070] Next, the processor 111 of the server device 100, acting as a display control unit, receives a coaching image request from the first terminal device 200-1 via the communication interface 113. Based on the coach's user ID information or team identification information received along with the request, it identifies scene ID information corresponding to scenes accessible to the coach. Then, the processor 111 of the server device 100, acting as a display control unit, reads the attribute information (e.g., scene description and scene title) of the identified scene ID information and generates scene list information (S12). The processor 111 of the server device 100, acting as a display control unit, transmits the scene list information to the first terminal device 200-1 via the communication interface 113 and controls the display of the first terminal device 200-1 to display the scene list information.

[0071] When the processor 211 of the first terminal device 200-1 receives scene list information from the server device 100 via the communication interface 215, it displays the scene list information via the output interface 214. The processor 211 of the first terminal device 200-1 then accepts the coach's operation input via the input interface 213 and selects a desired scene from among the multiple scenes included in the scene list information (S13). The processor 211 of the first terminal device 200-1 generates an image generation request that includes scene ID information indicating the selected scene, and sends the image generation request (T12) to the server device 100 via the communication interface 215.

[0072] The processor 111 of the server device 100, acting as a display control unit, receives an image generation request from the first terminal device 200-1 via the communication interface 113 and reads scene information in association with the scene ID information included in the image generation request (S14). Specifically, the processor 111 of the server device 100, acting as a display control unit, refers to the scene management table and reads the position information and field of view information of each player based on the scene ID information. The processor 111 of the server device 100, acting as a display control unit, also refers to the player management table and reads height information and ability information based on the player ID information of each player from which the position information, etc., has been read.

[0073] Next, the processor 111 of the server device 100, acting as a display control unit, places the player objects of each player on a virtual field corresponding to the field where the sport is being played, based on the read position information and field of view information, and generates an overhead view image (S15). The processor 111 of the server device 100, acting as a display control unit, also generates a field of view image showing the view from a specific player's player object, based on the read position information, field of view information, height information, and ability information (S16). Note that in this case, only the scene selection was performed in S13, and the player object of the player whose field of view image is to be displayed was not selected. In such cases, for example, a player may be selected according to a predetermined rule, and a field of view image showing the view from that player's player object may be generated. Examples of such rules include the player holding the ball, the player closest to the ball, the player closest to the opponent's goal, the player closest to their own goal, or a pre-specified player. Here, a view image is generated that shows the view from the player object of the player holding the ball (for example, player 11 in Figure 7).

[0074] The processor 111 of the server device 100, acting as a display control unit, generates a coaching image that includes a first region for displaying the generated overhead image and a second region for displaying the generated field of view image. The processor 111 of the server device 100, acting as a display control unit, then transmits the coaching image (T13) to the first terminal device 200-1 via the communication interface 113 and controls the display of the first terminal device 200-1 to display the coaching image. The processor 111 of the server device 100, acting as a display control unit, also transmits the coaching image (T14) to the second terminal device 200-2 via the communication interface 113 and controls the display of the second terminal device 200-2 to display the coaching image.

[0075] When the processor 211 of the first terminal device 200-1 receives a coaching image from the server device via the communication interface 215, it displays the coaching image on the display via the output interface 214 (S17). Similarly, when the processor 211 of the second terminal device 200-2 receives a coaching image from the server device via the communication interface 215, it displays the coaching image on the display via the output interface 214 (S18).

[0076] <Example of the first region displaying an overhead view> Here, Figure 8 shows an example of a coaching image 20 including a first region that displays an overhead view image according to one embodiment of the present disclosure. Specifically, it shows an example of a coaching image 20 which is the first region displayed on the display of the first terminal device 200-1 in S17 of Figure 6. According to Figure 8, the coaching image 20 (first region) includes a scene information display region 21, an overhead view image 22, and a reduced view image 23.

[0077] The scene information display area 21 includes attribute information such as the scene title from the retrieved scene information. This allows users such as coaches who are viewing the first area to easily understand the displayed scene.

[0078] The reduced-size field of view image 23 includes a reduced-size version of the field of view image currently displayed on the second terminal device 200-2. By displaying the reduced-size field of view image 23 in this way, it becomes possible to properly grasp the field of view image displayed on the second terminal device 200-2.

[0079] The overhead view image 22 includes an overhead view of the virtual field corresponding to field 10. The overhead view image 22 displays an image that mimics a soccer field, corresponding to the shape of field 10 (for example, the shape of a soccer field). In addition, the overhead view image 22 has player objects positioned to correspond to the 11 players of the player's own team and the 11 players of the opposing team, respectively, located on field 10 in Figure 7. In other words, the overhead view image 22 is an image virtually captured by a virtual camera from an arbitrary position moved approximately vertically (upward) from the center of the plane that constitutes the virtual field, so as to include the entire virtual field. By using such an overhead view image 22, it becomes possible to accurately grasp the position of each player on the field.

[0080] In Figure 8, the overhead view 22 is configured to include the entire virtual field, but it may also be configured to include only a part of the virtual field, such as half of it, or to include multiple virtual fields. Furthermore, although the overhead view 22 is configured to include only the virtual field, it may also be configured to include other structures in addition to the virtual field, such as surrounding stands or benches.

[0081] Figure 8 illustrates the case where a field of view image is generated for player object 24, one of several player objects included in the overhead image 22. Here, Figure 9 shows an example of player object 24 according to one embodiment of the present disclosure. Specifically, Figure 9 shows an example of an image taken from Figure 8, where only player object 24 and its surroundings are extracted and enlarged. According to Figure 9, along with player object 24, a field of view direction image 24a indicating the direction of the field of view and a target image 24b indicating the player object for which the field of view image was generated are displayed, corresponding to player object 24.

[0082] The field of view direction image 24a is an image that shows the direction of the player object's field of view, corresponding to the field of view image displayed in the second region. The field of view direction image 24a is formed in a roughly fan-shaped form, but it is displayed so that the direction extending perpendicularly from the center of the arc forming the fan-shaped trapezoid coincides with the direction indicated by the field of view information. In addition, the shorter arc of the fan-shaped trapezoid is formed close to the shape of the player object 24, but the length W1 of the longer arc is formed according to the capability information of the player object 24. That is, a numerical value indicating the field of view width is stored as capability information associated with the player of the player object 24. A larger numerical value indicates a wider field of view, and a smaller numerical value indicates a narrower field of view, but a longer length W1 indicates a wider field of view, and a shorter length W1 indicates a narrower field of view.

[0083] In this way, by displaying the field of view direction image 24a on the overhead view image 22 shown in Figure 8, the field of view width and direction of the player object 24 corresponding to the field of view direction image 24a are shown simultaneously, making it possible to more efficiently grasp the characteristics of the player represented by the player object 24.

[0084] Note that while the field of view image 24a is formed in a fan-shaped form in the example of Figure 9, it is not limited to this shape. As described above, any shape is acceptable as long as it indicates the field of view width and direction of the player object. In other words, instead of, or in combination with, the field of view image 24a may be used to display, for example, a numerical value or classification indicating the field of view width, or a symbol or arrow indicating the direction of the view.

[0085] Furthermore, the target image 24b is an image that shows the player object that was the target of the view image generation. For example, the target image 24b is displayed superimposed on the player object 24, making it possible to distinguish the player object 24 that was the target of the view image generation from other player objects.

[0086] In the example in Figure 9, the target image 24b is formed in a circular shape, but it is not limited to this shape. As long as the player object that was the target of the view image generation can be identified, as described above, text or symbols for identification may be displayed instead of, or in addition to, the target image 24b.

[0087] Furthermore, although the ball image representing the ball is not specifically shown in Figure 9 for the sake of explanation, if the player object 24 is a player holding the ball, the ball image may be displayed so as to be adjacent to the player object 24, as shown in Figure 8.

[0088] <Example of a second region displaying a field of view image> Figure 10 shows an example of a coaching image 40 including a second region that displays a field of view image according to one embodiment of the present disclosure. Specifically, it shows an example of a coaching image 40 which is the second region displayed on the display of the second terminal device 200-2 in S18 of Figure 6. According to Figure 10, the coaching image 40 (second region) includes a scene information display region 41, a field of view image 42, and a reduced overhead view image 43.

[0089] The scene information display area 41 includes attribute information such as the scene title from the retrieved scene information. This allows users such as players who view the second area to easily understand the displayed scene.

[0090] The reduced overhead image 43 includes a reduced version of the overhead image currently displayed on the first terminal device 200-1. As shown in Figures 8 and 9, the overhead image includes a target image 24b that indicates the player object from which the field of view image is generated. By displaying the reduced overhead image 43, it becomes possible to properly determine the position of the player object from which the field of view image 42 was generated.

[0091] The field of view image 42 has a width set based on capability information indicating the width of the field of view of the player object (e.g., player object 24) from which the field of view image was generated, field of view information indicating the direction of the player object's (e.g., player object 24) field of view, position information indicating the position of the player object (e.g., player object 24), and height information indicating the height of the player object (e.g., player object 24), and includes other player objects positioned within the field of view of the player object (e.g., player object 24).

[0092] Specifically, first, a player object (e.g., player object 24) is placed at a location identified based on positional information, and the orientation of the player object (e.g., player object 24) is set to align with the direction of the view indicated by the view information. The view image 42 is an image taken by a virtual camera in this state, aligned with the direction of the view from the position of the player object's (e.g., player object 24) eyes.

[0093] At this time, the horizontal length W2 of the field of view image 42 is set according to the capability information of the player object (for example, player object 24). That is, a numerical value indicating the field of view width is stored as capability information associated with the player of the player object (for example, player object 24). A larger numerical value indicates a wider field of view, and a smaller numerical value indicates a narrower field of view. The virtual camera captures images at a wider angle when the numerical value is larger so that the length W2 is longer, and at a narrower angle when the numerical value is smaller so that the length W2 is shorter.

[0094] Furthermore, the vertical length W3 of the field of view image 42 is set according to the height information of the player object (for example, player object 24). That is, height information is stored in association with the player object (for example, player object 24), and the higher the height indicated by the height information, the higher the viewpoint, and the lower the height, the lower the viewpoint. In other words, the longer the height, the longer the length W3 is set, and the shorter the height, the shorter the length W3 is set.

[0095] Therefore, in the example shown in Figure 10, the field of view image 42 includes two friendly player objects and four opposing player objects, but player objects 44 and 45, which are located outside the field of view of the virtual camera indicated by lengths W2 and W3, are not included in the field of view image 42.

[0096] Here, Figure 11 is a diagram conceptually illustrating the relationship between a field of view image and the ability information and height information of a player object according to one embodiment of the present disclosure. Specifically, in Figure 11, (a) shows an example of a field of view image 52a of a player object 51a having a tall height and a wide field of view, and (b) shows an example of a field of view image 52b of a player object 51b having a short height and a narrow field of view. As is clear from comparing the field of view image 52a in (a) and the field of view image 52b in (b), the horizontal length W2 is set to be longer as the numerical value of the ability information indicating the width of the field of view increases. Also, the vertical length W3 is set to be longer as the height indicated by the height information increases. Therefore, the field of view image 52a in (a) is virtually captured with a wider angle of view than the field of view image 52b in (b).

[0097] Figure 11 shows examples of the field of view image 52a of a player object 51a with a tall stature and wide field of view (Figure 11(a)) and an example of the field of view image 52b of a player object 51b with a short stature and narrow field of view (Figure 11(b)). Naturally, there are also field of view images of player objects with a tall stature and narrow field of view, and field of view images of player objects with a short stature and wide field of view, and lengths W2 and W3 will be set based on the height and field of view, similar to the example in Figure 11.

[0098] Returning to Figure 6, the first region, which includes the overhead view image as shown in Figures 8 to 11 above, and the second region, which includes the field of view image, are displayed as coaching images on the first terminal device 200-1 and the second terminal device 200-2, respectively.

[0099] The processing sequence is now terminated.

[0100] (2) Processing sequence for displaying a coaching image when an operation input is made on the overhead image at the first terminal device 200-1 When the coaching image is displayed on the first terminal device 200-1 (S17), the processor 211 of the terminal device 200-1 can perform various processes, such as switching the field of view image, by receiving operation input for the overhead view image. An example of such a process is as follows: A: A process that switches to a view image from the selected player object by selecting at least one player object from among the one or more player objects included in the overhead view image as the selected player object. • B: A process that changes the currently displayed view image to the view image from the moved position by moving the position of the player object from which the view image is currently being generated among the one or more player objects included in the overhead view image. • C: A process that selects at least two player objects from among multiple player objects included in the overhead view image, and moves the position of each player object simultaneously or separately, thereby changing the overhead view image and field of view image to reflect the new positions of each player object. • D: A process that changes the field of view image to one in which an input object (e.g., a mark or text) is placed at an arbitrary position on the overhead image.

[0101] <Example A above> The processor 211 of the first terminal device 200-1 receives user input, such as a coach's, via the input interface 213, and selects a desired player object (for example, player object 25 in Figure 8) from among one or more player objects included in the overhead image as the selected player object (S21). Once the selected player object is selected, the processor 211 of the first terminal device 200-1 generates an image generation request that includes player ID information associated with the selected player object, and transmits the image generation request (T21) to the server device 100 via the communication interface 215.

[0102] The processor 111 of the server device 100, acting as a display control unit, receives an image generation request from the first terminal device 200-1 via the communication interface 113 and reads the location information, field of view information, height information, and ability information stored in association with the player ID information included in the image generation request. Then, the processor 111 of the server device 100, acting as a display control unit, generates a field of view image showing the field of view from the selected player object (for example, player object 25 in Figure 8) based on the information it has read (S22).

[0103] The processor 111 of the server device 100, acting as a display control unit, generates a coaching image that includes a first region for displaying the overhead image generated in S15 of Figure 6, and a second region for displaying the newly generated field of view image in S22. The processor 111 of the server device 100, acting as a display control unit, then transmits the coaching image (T22) to the first terminal device 200-1 via the communication interface 113 and controls the display of the first terminal device 200-1 to display the coaching image. The processor 111 of the server device 100, acting as a display control unit, also transmits the coaching image (T23) to the second terminal device 200-2 via the communication interface 113 and controls the display of the second terminal device 200-2 to display the coaching image.

[0104] When the processor 211 of the first terminal device 200-1 receives a coaching image from the server device via the communication interface 215, it displays the coaching image on the display via the output interface 214 (S23). As a result, the coaching image displayed on the first terminal device 200-1 includes a reduced field of view image, and this reduced field of view image is replaced in S22 with a reduced version of the newly generated field of view image.

[0105] Furthermore, when the processor 211 of the second terminal device 200-2 receives a coaching image from the server device via the communication interface 215, it displays the coaching image on the display via the output interface 214 (S24). As a result, the coaching image displayed on the second terminal device 200-2 includes a field of view image, and this field of view image is switched to a newly generated field of view image in S22 and displayed.

[0106] Here, Figure 12A shows an example of a coaching image 40 including a second region that displays a field of view image according to one embodiment of the present disclosure. Specifically, Figure 12A shows an example of a second region (coaching image) that displays a field of view image 42a showing the field of view from a selected player object (for example, player object 25 in Figure 8). According to Figure 12A, the field of view image 42a is displayed by switching to a field of view image showing the field of view from player object 25 in Figure 8. Specifically, the field of view image displayed in S18 of Figure 6 includes two friendly player objects and four opponent player objects, as shown in Figure 10, but in the field of view image 42a of Figure 12A, it is changed to the four opponent player objects included in the field of view from player object 25. Furthermore, in the field of view image displayed at S18 in Figure 6, lengths W2 and W3 are set based on the ability information and height information associated with the player object 24, whereas in the field of view image 42a in Figure 12A, lengths W2a and W3a are set based on the ability information and height information associated with the player object 25.

[0107] <Example B above> In case B above, the processor 211 of the first terminal device 200-1 receives user input, such as a coach's, via the input interface 213, and moves the position of the player object on which the view image is currently displayed (for example, moving the player object 24 in Figure 8 to the position indicated by the arrow 26). The processor 211 of the first terminal device 200-1 generates an image generation request that includes the position information of the player object after the move and the player ID information, and transmits the image generation request to the server device 100 via the communication interface 215.

[0108] The processor 111 of the server device 100, acting as a display control unit, receives an image generation request from the first terminal device 200-1 via the communication interface 113. It then reads the field of view information, height information, and ability information stored in association with the player ID information included in the image generation request. The processor 111 of the server device 100, acting as a display control unit, then generates a field of view image showing the field of view from the selected player object based on the read information and the position information included in the image generation request.

[0109] The processor 111 of the server device 100, acting as a display control unit, generates a coaching image that includes a first area for displaying the overhead image generated in S15 of Figure 6, and a second area for displaying the newly generated field of view image. The following steps are processed in the same manner as in S22 to S24 of Figure 6. As a result, the coaching image displayed on the first terminal device 200-1 includes a reduced field of view image, but this reduced field of view image is replaced with a reduced version of the newly generated field of view image (the field of view image from the player object's position after movement) and displayed accordingly.

[0110] Furthermore, the coaching image displayed on the second terminal device 200-2 includes a field of view image, and this field of view image will be changed to a newly generated field of view image (a field of view image from the position of the player object after it has moved) and displayed accordingly.

[0111] Here, Figure 12B shows an example of a coaching image 40 including a second region that displays a field of view image according to one embodiment of the present disclosure. Specifically, Figure 12B shows an example of a second region (coaching image) that displays a field of view image 42b from the position after a player object (e.g., player object 24) has moved. According to Figure 12B, the field of view image 42b is displayed after the player object 24 in Figure 8 has been moved to the position indicated by arrow 26. Specifically, the field of view image displayed in S18 of Figure 6 includes two friendly player objects and four opponent player objects, as shown in Figure 10, but in the field of view image 42b of Figure 12B, the field of view image has been changed to include three friendly player objects and one opponent player object due to the movement of player object 24.

[0112] <Example C above> In case C above, the processor 211 of the first terminal device 200-1 receives user input, such as a coach's, via the input interface 213, and selects at least two player objects (for example, player object 28 and player object 30 in Figure 8) from among multiple player objects included in the overhead image as selected player objects. The processor 211 of the first terminal device 200-1 also receives user input, such as a coach's, via the input interface 213, and moves the positions of at least two selected player objects simultaneously or separately (for example, in Figure 8, move player object 28 to the position indicated by arrow 29, and move player object 30 to the position indicated by arrow 31). The processor 211 of the first terminal device 200-1 generates an image generation request that includes the position information of each selected player object after movement and the player ID information of each, and transmits the image generation request to the server device 100 via the communication interface 215.

[0113] The processor 111 of the server device 100, acting as a display control unit, receives an image generation request from the first terminal device 200-1 via the communication interface 113. It then modifies the location information stored in association with each player ID information included in the image generation request, and generates an overhead image and a field-of-view image based on the modified location information.

[0114] The processor 111 of the server device 100, acting as a display control unit, generates a coaching image that includes a first area for displaying a newly generated overhead image and a second area for displaying a newly generated field of view image. The process is carried out in the same manner as in S22 to S24 of Figure 6. As a result, the coaching image displayed on the first terminal device 200-1 includes an overhead image and a reduced field of view image, but the overhead image is changed to a newly generated overhead image, and the reduced field of view image is changed to a reduced version of the newly generated field of view image.

[0115] Furthermore, the coaching image displayed on the second terminal device 200-2 includes a field-of-view image and a reduced overhead view image. The field-of-view image is replaced with a newly generated field-of-view image, and the reduced overhead view image is replaced with a reduced version of the newly generated overhead view image.

[0116] Here, Figure 12C is a diagram showing an example of a coaching image 40 including a second region that displays a field of view image according to one embodiment of the present disclosure. Specifically, Figure 12C is a diagram showing an example of a second region (coaching image) that displays a field of view image 42c after the movement of two player objects. According to Figure 12C, the field of view image 42c is displayed after the player object 28 in Figure 8 has been moved to the position indicated by arrow 29, and the player object 30 has been moved to the position indicated by arrow 31. Specifically, the field of view image displayed in S18 of Figure 6 includes two friendly player objects and four opponent player objects, as shown in Figure 10, but as the positions of player objects 28 and 30 have moved, the field of view image 42c in Figure 12C has been changed to a field of view image showing the positions of player objects 28 and 30.

[0117] <Example of the above D> In the case of D described above, the processor 211 of the first terminal device 200-1 receives user input, such as a coach's, via the input interface 213 and inputs an input object (for example, the input object 32 in Figure 8) at an arbitrary position on the overhead image. The processor 211 of the first terminal device 200-1 stores the drawing information and position information of the input object 32 input on the overhead image, generates an image generation request including the drawing information and position information, and transmits the image generation request to the server device 100 via the communication interface 215.

[0118] The processor 111 of the server device 100, acting as a display control unit, receives an image generation request from the first terminal device 200-1 via the communication interface 113, and generates a field-of-view image that includes the input image based on the drawing information and position information included in the image generation request.

[0119] The processor 111 of the server device 100, acting as a display control unit, generates a coaching image that includes a first area for displaying the overhead image generated in S15 of Figure 6, and a second area for displaying the newly generated field of view image. The following processing is the same as in S22 to S24 of Figure 6. As a result, the coaching image displayed on the first terminal device 200-1 includes the overhead image and the reduced field of view image, and the reduced field of view image is changed to a reduced version of the newly generated field of view image for display.

[0120] Furthermore, the coaching image displayed on the second terminal device 200-2 includes a field-of-view image and a reduced overhead view image, and the field-of-view image is replaced with a newly generated field-of-view image for display.

[0121] Here, Figure 12D shows an example of a coaching image 40 including a second region that displays a field of view image according to one embodiment of the present disclosure. Specifically, Figure 12D shows an example of a second region (coaching image) that displays a field of view image 42d after an input object 32 has been input in the overhead view image. According to Figure 12D, the field of view image 42d includes an input object 46 at a position corresponding to the input object 32 in Figure 8, as the input object 32 in Figure 8 has been input into the overhead view image. In this way, by reflecting the input object input by the coach on the overhead view image (for example, the input object 32 in Figure 8 that shows the space behind the defense) in the field of view image, it becomes possible to effectively grasp the position indicated by the input object in the field of view image as well.

[0122] Returning to Figure 6, the field of view image is switched to the one shown in Figures 12A to 12D above, or the modified second region is displayed as a coaching image on the first terminal device 200-1 and the second terminal device 200-2, respectively.

[0123] The processing sequence is now terminated.

[0124] (3) Processing sequence for displaying a coaching image when an operation input is made to the visual field image on the second terminal device 200-2 When the coaching image is displayed on the second terminal device 200-2 (S18), the processor 211 of the terminal device 200-2 can perform various processes, such as changing the overhead view image, by receiving operation input for the field of view image. An example of such a process is as follows: A: By receiving an operation input to indicate a player object included in the field of view image of the second region, the overhead image of the first region is changed to an overhead image that includes an identification display for identifying the player object.

[0125] Specifically, the processor 211 of the second terminal device 200-2 receives user input, such as a player input, via the input interface 213, and selects a desired player object (for example, player object 47 in Figure 10) from among one or more player objects included in the field of view image as the selected player object (S31). Once the selected player object is selected, the processor 211 of the second terminal device 200-2 generates an image generation request that includes player ID information associated with the selected player object, and transmits the image generation request (T31) to the server device 100 via the communication interface 215.

[0126] The processor 111 of the server device 100, acting as a display control unit, receives an image generation request from the second terminal device 200-2 via the communication interface 113 and reads the location information stored in association with the player ID information included in the image generation request. Then, the processor 111 of the server device 100, acting as a display control unit, generates an identification display image superimposed on the read location information and generates an overhead image including the identification display image (S32).

[0127] The processor 111 of the server device 100, acting as a display control unit, generates a coaching image that includes a first region for displaying the overhead image newly generated in S32 and a second region for displaying the field of view image generated in S16 of Figure 6. The processor 111 of the server device 100, acting as a display control unit, then transmits the coaching image (T32) to the first terminal device 200-1 via the communication interface 113 and controls the display of the first terminal device 200-1 to display the coaching image. The processor 111 of the server device 100, acting as a display control unit, also transmits the coaching image (T33) to the second terminal device 200-2 via the communication interface 113 and controls the display of the second terminal device 200-2 to display the coaching image.

[0128] When the processor 211 of the first terminal device 200-1 receives a coaching image from the server device via the communication interface 215, it displays the coaching image on the display via the output interface 214 (S33). As a result, the coaching image displayed on the first terminal device 200-1 includes a first area that displays an overhead view image, and this overhead view image is changed to a newly generated overhead view image in S32 and displayed.

[0129] Furthermore, when the processor 211 of the second terminal device 200-2 receives a coaching image from the server device via the communication interface 215, it displays the coaching image on the display via the output interface 214 (S34). As a result, the coaching image displayed on the second terminal device 200-2 includes a reduced overhead image, and this reduced overhead image is changed to a reduced version of the overhead image newly generated in S32 and displayed.

[0130] Here, Figure 13 shows an example of a coaching image 20 including a first region that displays an overhead image according to one embodiment of the present disclosure. Specifically, Figure 13 shows an example of a first region (coaching image) that displays an overhead image 22a including an identification indicator 33 for identifying a selected player object (e.g., player object 47 in Figure 10) selected in the second terminal device 200-2. According to Figure 13, the overhead image 22a includes the identification indicator 33 superimposed on the position of the player object 47, making it possible to effectively understand that the player object 47 has been selected in the second terminal device 200-2.

[0131] Returning to Figure 6, the first region, which has been changed to an overhead view as shown in Figure 13 above, is displayed as a coaching image on the first terminal device 200-1 and the second terminal device 200-2, respectively.

[0132] The processing sequence is now terminated.

[0133] 7. Other (1) Example of a visual field image The view images shown in Figure 10 and other images represent the view from the player object, but the player object itself is not included in the view images. In other words, the view images shown in Figure 10 and other images are generated as so-called first-person view images, but instead, they may be generated as so-called third-person view images in which at least a part of the player object is displayed within the view image.

[0134] (2) Example of input interface 213 While a whiteboard device was given as an example of the first terminal device 200-1, in such cases, a combination of a touch panel and a stylus pen or magnet for operation input can be used as the input interface 213. For example, the input interface 213 of the first terminal device 200-1 accepts input of an input object as shown in input object 32 of Figure 8, but by using a touch panel and a stylus pen as the input interface 213, it becomes possible to input the input object more intuitively. Also, the input interface 213 of the first terminal device 200-1 accepts operation input for selecting and moving player objects on an overhead image, but by using a touch panel and a magnetic input device whose position can be detected by the touch panel as the input interface 213, the user, such as a coach, can grasp the input device with their hand to select and move it, enabling more intuitive operation input.

[0135] (3) Examples of a series of coaching images such as videos Figures 8 to 13 illustrate the process of generating overhead and field-of-view images for each predetermined situation (scene), displaying coaching images that include these, and performing switching and modification processes. However, as shown in Figure 5, the scene management table stores various information such as position information and field-of-view information for each player object, associated with time information. Therefore, the display control unit acquires various information from the scene management table at predetermined intervals during a predetermined period of time, for example, during a sports game, and generates coaching images that include overhead and field-of-view images based on this information. The display control unit then controls the display to continuously display the series of coaching images generated over the predetermined period on the first terminal device 200-1 and the second terminal device 200-2, respectively, making it possible to reproduce the coaching images as a video.

[0136] (4) Use of pre-trained predictive models As shown in Figures 4 and 5, the scene management table stores location information and field of view information associated with scene ID information for each predetermined situation (scene). By acquiring and storing this information at predetermined intervals over a certain period, it is possible to grasp changes in the situation over that period. Therefore, it is also possible to store the location information and field of view information acquired over a certain period in association with time information, and use a trained predictive model that has been trained by inputting the combination of said location information, field of view information and said time information into a learning device.

[0137] Specifically, the memory 112 of the server device 100 stores location information and field of view information and time information acquired over a certain period of time in association with each other. The processor of the model generation device provides the pair of location information and field of view information and the time information to a learner and generates a trained predictive model by machine learning. Examples of such learners include neural networks, convolutional neural networks, multilayer herceptons (MLP), LSTM (Long Short Term Memory), GRU (Gated Recurrent Unit), GNN (Graph Neural Network), and Transformer; gradient boosting decision trees (GBDT) such as LightGBM (Light Gradient Boosting Machine), XGBoost, and CatBoost; and learners using ridge regression, logistic regression, support vector regression (SVR), nearest neighbor, decision trees, regression trees, and random forests.

[0138] Then, when a trained predictive model is generated, the processor 111 of the server device 100 retrieves the trained predictive model from the model generation device and stores it in memory 112.

[0139] Next, the processor 111 of the server device 100, acting as a display control unit, inputs a set of newly acquired location information and field of view information for a predetermined situation (scene) and time information specifying a predetermined future time into a trained prediction model. As a result, it obtains predicted location information and field of view information for the future situation indicated by the time specified by the time information as output. The processor 111 of the server device 100, acting as a display control unit, then generates an overhead image and a field of view image based on the acquired predicted location information and field of view information, and controls them to display on each terminal device. This makes it possible to more effectively advance coaching by predicting scenes that may occur in the future and reproducing them in the overhead image and field of view image.

[0140] (5) Changes to visual information Figures 6S21, 12A, and 12B describe the case where the view image is switched or changed and displayed by changing the player object that displays the view image or by moving the position of the player object. However, instead of this, or in addition to this, the view information with a changed view direction may be switched or changed and displayed by changing the view information stored in association with the player ID information of the player object.

[0141] Specifically, the processor 211 of the first terminal device 200-1 receives user input, such as a coach's, via the input interface 213, and modifies the field of view information of the player object whose field of view image is currently displayed (for example, by changing the angle stored as field of view information). The processor 211 of the first terminal device 200-1 generates an image generation request that includes the modified field of view information of the player object and player ID information, and transmits the image generation request to the server device 100 via the communication interface 215.

[0142] The processor 111 of the server device 100, acting as a display control unit, receives an image generation request from the first terminal device 200-1 via the communication interface 113. It then reads the location information, height information, and ability information stored in association with the player ID information included in the image generation request. The processor 111 of the server device 100, acting as a display control unit, then generates a field of view image showing the field of view from the selected player object based on the read information and the modified field of view information included in the image generation request.

[0143] The processor 111 of the server device 100, acting as a display control unit, generates a coaching image that includes a first area for displaying the overhead image generated in S15 of Figure 6, and a second area for displaying the newly generated field of view image. The following steps are processed in the same manner as in S22 to S24 of Figure 6. As a result, the coaching image displayed on the first terminal device 200-1 includes a reduced field of view image, but this reduced field of view image is replaced with a reduced version of the newly generated field of view image (the field of view image after the player object's field of view information has been changed) and displayed.

[0144] Furthermore, the coaching image displayed on the second terminal device 200-2 includes a field of view image, and this field of view image will be changed to a newly generated field of view image (a field of view image after the player object's field of view information has been changed) and displayed accordingly.

[0145] In this way, by displaying a modified view image, it becomes possible to simulate a view that is different from the view the player object actually saw during gameplay.

[0146] 8. Summary In this embodiment, we can provide a processing device, processing method, processing program, and processing system capable of displaying a variety of images, including a coaching image that includes an overhead view of a virtual field suitable for coaching in sports conducted on a predetermined field, and a view image of a player object. In particular, when at least one selected player object is selected by an operation input received in at least one of the first terminal device 200-1 and the second terminal device 200-2, the view image of the second region is switched to the view image from the selected player object and displayed. By switching and displaying the view images in this way, it becomes possible to effectively utilize the coaching image.

[0147] This disclosure aims to provide a processing device, processing method, processing program, and processing system capable of displaying a variety of information.

[0148] [Note 1] One of the purposes of this disclosure is to effectively utilize coaching images. Accordingly, the apparatus of the present disclosure is an apparatus that causes one or more displays to display a coaching image, which includes a first region that displays an overhead image showing an overhead view of a virtual field corresponding to a predetermined field in a predetermined situation, and a second region that displays a field view image showing the view from at least one of one or more player objects located in the virtual field, for coaching in a sport performed in a predetermined field, The display control unit receives an operation input to select a selected player object, which is one or more player objects located in the virtual field, and switches the view image of the second region to the view image from the selected player object and displays it. In this way, by switching and displaying the field of view images, it becomes possible to effectively utilize coaching images.

[0149] [Note 2] One of the purposes of this disclosure is to effectively utilize coaching images. Therefore, in the processing apparatus described in Appendix 1, the display control unit changes the field of view image displayed in the second area by receiving an operation input for moving the position of the one player object in the first area. In this way, by changing the field of view image to the field of view image after movement and displaying it, it becomes possible to effectively utilize coaching images.

[0150] [Note 3] One of the purposes of this disclosure is to provide a wider variety of coaching images. Therefore, in the processing device described in Appendix 1 or 2, when two or more player objects are located in the virtual field under the predetermined circumstances, at least two player objects are selected, and the positions of the two player objects are made movable. This makes it possible to reproduce how the player's position changes in a given situation, and to provide a wider variety of coaching images.

[0151] [Note 4] One of the purposes of this disclosure is to efficiently determine the direction that the player object is facing in an overhead view image. Therefore, in the processing apparatus described in any of the appendices 1 to 3, the display control unit displays an image in the first area that indicates the direction of the view of the player object corresponding to the view image displayed in the second area, in association with the player object. In this way, by displaying an image that indicates the direction of the view, it becomes possible to efficiently determine the direction the player object is facing in the overhead view.

[0152] [Note 5] One of the purposes of this disclosure is to reproduce images that are closer to the real view. Therefore, the processing device described in any of the appendices 1 to 4 further includes a storage unit that stores at least one of the height information and ability information indicating the height of each of the one or more player objects, associated with each player object. The display control unit displays the field of view image corresponding to at least one of the height information and the ability information associated with the player object. In this way, by displaying a field of view image corresponding to at least one of the height information and ability information, it becomes possible to reproduce an image that is closer to the real field of view.

[0153] [Note 6] One of the purposes of this disclosure is to efficiently reproduce a desired scene. Therefore, the processing device described in any of the appendices 1 to 5 further comprises a storage unit that stores, for each predetermined situation, position information indicating the position of each of the one or more player objects and field of view information indicating the direction of the view, The display control unit receives an operation input to select one of the predetermined situations, and then displays at least one of the overhead view image and the field of view image based on the position information and field of view information associated with the selected predetermined situation. In this way, by displaying images corresponding to the selected scene, it becomes possible to efficiently recreate the desired scene.

[0154] [Note 7] One of the purposes of this disclosure is to faithfully reproduce actual sports play. Therefore, in the processing device described in Appendix 6, the position information is position information based on sensing information acquired by sensors attached to each of the players corresponding to the one or more player objects. This makes it possible to faithfully reproduce actual sports gameplay.

[0155] [Note 8] One of the purposes of this disclosure is to efficiently determine the position of player objects in the field of view image in the overhead view image. Therefore, in the processing apparatus described in any of appendices 1 to 7, the display control unit receives an operation input for instructing another player object different from the one player object displayed in the second area, and displays an identification display in the first area in association with the other player object indicating that the instruction has been given. In this way, by identifying the player object specified in the second region in the first region, it becomes possible to efficiently grasp the position of the player object in the field of view image in the overhead view image.

[0156] [Note 9] One of the purposes of this disclosure is to make coaching smoother. Therefore, in the processing apparatus described in any of appendices 1 to 8, the display control unit receives an operation input to input an input object to an arbitrary position on the overhead image displayed in the first area, thereby causing the input object to be displayed in the field of view image displayed in the second area at a position corresponding to the arbitrary position. In this way, by displaying the input objects entered in the overhead view image in the field of view image as well, it becomes possible to make coaching smoother.

[0157] [Note 10] One of the purposes of this disclosure is to effectively understand a series of player movements. Therefore, the processing device described in any of the appendices 1 to 9 further comprises a storage unit that stores, for each of the one or more player objects, position information indicating the position for each predetermined situation and field of view information indicating the direction of the view for each predetermined situation. The display control unit displays a series of coaching images for any given period based on the position information and field of view information stored in the storage unit. In this way, by displaying a series of coaching images, such as a video, it becomes possible to effectively grasp a player's movements in sequence.

[0158] [Note 11] One of the purposes of this disclosure is to effectively predict the future movements of each player. Therefore, in the processing device described in Appendix 10, by inputting the location information and field of view information of any of the one or more player objects into a trained prediction model obtained by machine learning the location information and field of view information for each predetermined situation stored in the storage unit, location prediction information and field of view prediction information are obtained that predict the location information and field of view information in the future after the predetermined situation. This makes it possible to effectively predict the future moves of each player.

[0159] [Note 12] One of the purposes of this disclosure is to enable the efficient implementation of coaching. Therefore, in the processing apparatus described in any of appendices 1 to 11, the display control unit causes the first area to be displayed on a display provided in a first terminal device usable by the coach providing the coaching, and the second area to be displayed on a display provided in a second terminal device usable by the player receiving the coaching. This makes it possible to conduct coaching more efficiently.

[0160] [Note 13] One of the purposes of this disclosure is to effectively utilize coaching images. Accordingly, the processing method of the present disclosure causes a processor to display a coaching image on one or more displays, which includes a first region that displays an overhead image showing an overhead view of a virtual field corresponding to a predetermined field in a predetermined situation, and a second region that displays a view image showing the view from at least one player object among one or more player objects located in the virtual field, for coaching in a sport performed in a predetermined field. The processor receives an input for selecting a selected player object, which is one or more player objects located in the virtual field, and switches the view image of the second region to the view image from the selected player object and displays it. In this way, by switching and displaying the field of view images, it becomes possible to effectively utilize coaching images.

[0161] [Note 14] One of the purposes of this disclosure is to effectively utilize coaching images. Accordingly, the processing program of the present disclosure causes the processor to display at least the first region of a coaching image which includes a first region that displays an overhead image showing an overhead view of a virtual field corresponding to a predetermined field in a predetermined situation, and a second region that displays a view image showing the view from at least one player object among one or more player objects located in the virtual field, for coaching in a sport performed in a predetermined field. The processor is instructed to accept an input for selecting a selected player object in order to switch the view image of the second region from the view image of a selected player object, which is one of the one or more player objects located in the virtual field. Processing program. In this way, by switching and displaying the field of view images, it becomes possible to effectively utilize coaching images.

[0162] [Note 15] One of the purposes of this disclosure is to effectively utilize coaching images. Accordingly, the processing system of the present disclosure is a processing system that includes a processing device used for coaching in a sport conducted in a predetermined field, a first terminal device that can be used by at least a coach providing the coaching, and a second terminal device that can be used by at least a player receiving the coaching, The processing device causes at least one of the displays of the first terminal device and the second terminal device to display a coaching image which includes a first region that displays an overhead image showing an overhead view of a virtual field corresponding to a predetermined field in a predetermined situation, and a second region that displays a view image showing the view from at least one player object among one or more player objects located in the virtual field. The first terminal device receives an operation input for selecting a selected player object, which is one or more player objects located in the virtual field. Based on the operation input, the processing unit switches the view image of the second region to the view image from the selected player object and displays it. In this way, by switching and displaying the field of view images, it becomes possible to effectively utilize coaching images.

[0163] The embodiments and variations of this disclosure are presented as examples and are not intended to limit the scope of this disclosure. These embodiments and variations can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of this disclosure. Furthermore, these embodiments and variations can be combined as appropriate without departing from the spirit of this disclosure. These embodiments and their variations are included in the scope and spirit of the invention and are included in the claims and their equivalents.

[0164] The processes and procedures described in this disclosure can be implemented not only by those expressly described in the embodiments, but also by software, hardware, or a combination thereof. Specifically, the processes and procedures described in this disclosure can be implemented by implementing logic corresponding to the processes on media such as integrated circuits, volatile memory, non-volatile memory, magnetic disks, and optical storage. Furthermore, the processes and procedures described in this disclosure can be implemented as computer programs and executed by various computers, including processing units and server devices.

[0165] Even if the processes and procedures described in this disclosure are described as being performed by a single device, software, component, or module, such processes or procedures may be performed by multiple devices, multiple software, multiple components, and / or multiple modules. Similarly, even if the various types of information described in this disclosure are described as being stored in a single memory or storage unit, such information may be distributed and stored in multiple memories on a single device or in multiple memories distributed across multiple devices. Furthermore, the software and hardware elements described in this disclosure may be implemented by integrating them into fewer components or by decomposing them into more components. [Explanation of Symbols]

[0166] 1. Processing System 100 Server Devices 200-1 First Terminal Device 200-2 Second Terminal Device

Claims

1. A processing device comprising: a display control unit that causes one or more displays to display a coaching image for coaching in a sport conducted on a predetermined field, the coaching image including a first region that displays an overhead image of a virtual field corresponding to the predetermined field in a predetermined situation, and a second region that displays a field of view image showing the view from at least one player object among one or more player objects located in the virtual field, wherein the processing device causes one or more displays to display a coaching image, The display control unit receives an operation input to select a selected player object, which is one or more player objects located in the virtual field, and switches the view image of the second region to the view image from the selected player object and displays it. Processing device.

2. The processing apparatus according to claim 1, wherein the display control unit changes the field of view image displayed in the second region by receiving an operation input for moving the position of the one player object in the first region.

3. The processing apparatus according to claim 1, wherein, in the predetermined circumstances, when two or more player objects are located in the virtual field, at least two player objects can be selected and the positions of the two player objects can be moved.

4. The processing apparatus according to claim 1, wherein the display control unit causes an image indicating the direction of the view of the player object corresponding to the view image displayed in the second area to be displayed in the first area in association with the player object.

5. The system further includes a storage unit that stores, in association with each player object, at least one of height information indicating the height of each of the one or more player objects and ability information indicating their abilities. The display control unit causes the field of view image to display according to at least one of the height information and the ability information associated with the one player object. The apparatus according to claim 1.

6. The system further includes a storage unit that stores positional information indicating the position of each of the one or more player objects and field of view information indicating the direction of the view, for each of the predetermined situations. The display control unit, upon receiving an operation input to select one of the predetermined situations, displays at least one of the overhead view image and the field of view image based on the position information and field of view information associated with the selected predetermined situation. The apparatus according to claim 1.

7. The processing apparatus according to claim 6, wherein the position information is position information based on sensing information acquired by sensors attached to each of the players corresponding to the one or more player objects.

8. The processing apparatus according to claim 1, wherein the display control unit receives an operation input for instructing another player object different from the one player object displayed in the second area, and displays an identification display in the first area in association with the other player object indicating that the instruction has been given.

9. The processing apparatus according to claim 1, wherein the display control unit receives an operation input for inputting an input object to an arbitrary position on the overhead image displayed in the first region, thereby displaying the input object at a position corresponding to the arbitrary position in the field of view image displayed in the second region.

10. The system further includes a storage unit that stores position information indicating the position for each of the one or more player objects and field of view information indicating the direction of the view for each of the predetermined situations. The display control unit displays a series of coaching images for an arbitrary period of time based on the position information and field of view information stored in the storage unit. The apparatus according to claim 1.

11. The processing apparatus according to claim 10, wherein, by inputting the location information and field of view information of any of the one or more player objects to a trained prediction model obtained by machine learning the location information and field of view information for each predetermined situation stored in the memory unit, location prediction information and field of view prediction information that predicts the location information and field of view information in the future after the predetermined situation is obtained.

12. The processing apparatus according to claim 1, wherein the display control unit causes the first region to be displayed on a display provided in a first terminal device usable by the coach providing the coaching, and the second region to be displayed on a display provided in a second terminal device usable by the player receiving the coaching.

13. The processor causes one or more displays to show a coaching image for coaching in a sport played on a predetermined field, which includes a first region that displays an overhead image showing an overhead view of a virtual field corresponding to the predetermined field in a predetermined situation, and a second region that displays a field of view image showing the view from at least one player object among one or more player objects located in the virtual field. The processor receives an input for selecting a selected player object, which is one or more player objects located in the virtual field, and switches the view image of the second region to the view image from the selected player object and displays it. Processing method.

14. The processor is instructed to display at least the first region of a coaching image that includes a first region displaying an overhead image showing an overhead view of a virtual field corresponding to a predetermined field in a predetermined situation, and a second region displaying a field view image showing the view from at least one player object among one or more player objects located in the virtual field, for coaching in a sport performed in a predetermined field. The processor is instructed to accept an input for selecting a selected player object in order to switch the view image of the second region from the view image of a selected player object, which is one of the one or more player objects located in the virtual field. Processing program.

15. A processing system comprising a processing device used for coaching in a sport conducted on a predetermined field, a first terminal device usable by at least one coach providing the coaching, and a second terminal device usable by at least one player receiving the coaching, The processing device causes at least one of the displays of the first terminal device and the second terminal device to display a coaching image which includes a first region that displays an overhead image showing an overhead view of a virtual field corresponding to a predetermined field in a predetermined situation, and a second region that displays a view image showing the view from at least one player object among one or more player objects located in the virtual field. The first terminal device receives an operation input for selecting a selected player object, which is one or more player objects located in the virtual field. The processing device, based on the operation input, switches and displays the view image of the second region to the view image from the selected player object. Processing system.

Images