Hit position estimation system and hit position estimation method

Abstract

The hit position of a throwing tool can be estimated with higher precision. The hit position estimation system includes: a storage unit that stores a machine learning model in which machine learning has been completed, when image data generated on the basis of photographing a target when a throwing instrument is thrown in a throwing game and detection data generated on the basis of detecting contact between the thrown instrument and the target or other object are input, the target or other object is detected by the target or other object. Outputting an estimation result of a position hit by the throwing tool in the target; an imaging unit that generates image data on the basis of imaging the target when the throwing instrument is thrown in the throwing game; a detection unit that generates detection data on the basis of detecting contact between the thrown throwing tool and a target or another object; an acquisition unit that acquires an estimation result of a position hit by the throwing tool in the target, the estimation result being output by inputting the image data and the detection data into the machine learning model; and a display unit that displays the estimation result.

Description

Technical Field

[0001] The present invention relates to a technique for estimating the hitting position of a throwing implement in a throwing game. Background Art

[0002] Conventionally, a throwing game device has been known that is used to play a throwing game in which a throwing implement such as a dart is thrown at a target and the throwing skill is competed based on the score set according to the position where the throwing implement hits in the target. Regarding such a throwing game device, various techniques for determining the hitting position of the throwing implement have been proposed.

[0003] As a technique for estimating the hitting position of a throwing implement, a method using a machine learning model such as deep learning has also been considered. Regarding the technique for estimating the position of an object using a machine learning model, for example, in Patent Document 1, a position estimation device is described that estimates the position of an object based on an image captured by a monocular camera for bird's-eye view. The position estimation device uses a machine learning model that has been machine-learned using deep learning, performs image recognition on the acquired image, and detects the position of the detection target in the frame.

[0004] Prior Art Documents

[0005] Patent Documents

[0006] Patent Document 1: Japanese Unexamined Patent Application Publication No. 2022-125973 Summary of the Invention

[0007] The method described in the above Patent Document 1 only uses an image specifically obtained by photographing an object as the input to the machine learning model. In this regard, the inventor of the present invention has found that when a thrown throwing implement comes into contact with a target or other object, various sounds and impacts corresponding to the contact position are generated, and the detection results of these physical phenomena are also useful for estimating the hitting position of the throwing implement.

[0008] One aspect of the present invention has been completed in view of the above problems, and one of its purposes is to more accurately estimate the hitting position of a throwing implement.

[0009] A hit position estimation system according to one aspect of the present invention includes: a storage unit that stores a machine learning completed machine learning model, the machine learning completed machine learning model being configured to output an estimation result of the position where a throwing implement hits a target when image data generated by photographing a target when the throwing implement is thrown in a throwing game and detection data generated by detecting contact between the thrown throwing implement and the target or other object are input; a photographing unit that generates image data by photographing a target when the throwing implement is thrown in a throwing game; a detection unit that generates detection data by detecting contact between the thrown throwing implement and the target or other object; an acquisition unit that acquires an estimation result of the position where the throwing implement hits the target, which is output by inputting the image data and the detection data into the machine learning model; and a display unit that displays the estimation result.

[0010] According to the above structure, in the estimation of the hitting position of the throwing implement using the machine learning completed machine learning model, the input data includes not only image data but also detection data. Thus, the hitting position of the throwing implement can be estimated with higher accuracy.

[0011] Advantages of the Invention

[0012] According to one aspect of the present invention, the hitting position of the throwing implement can be estimated with higher accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a schematic front view showing an example of the appearance of the dart game device 30.

[0014] Figure 2 is a schematic diagram showing an example of the appearance of the dartboard 31.

[0015] Figure 3 is a block diagram showing an example of the structure of the dart game device 30.

[0016] Figure 4 is a functional block diagram showing an example of the functional units included in the control device 44.

[0017] Figure 5 is a diagram schematically showing an example of the input / output data of the machine learning model 121.

[0018] Figure 6 is a flowchart of an operation showing an example of a method for generating the machine learning model 121.

[0019] Figure 7 is a flowchart of an operation showing an example of a method for estimating the hitting position.

[0020] Figure 8 This is a diagram showing an example of the screen 200 displayed on the display device 34.

[0021] Figure 9 This is a diagram showing an example of the screen 300 displayed on the display device 34. Detailed implementation

[0022] (1) Structure (1-1) Overall structure of the dart game device 30

[0023] Refer to Figures 1 - 3 to describe the overall structure of the dart game device 30 of this embodiment. Figure 1 This is a schematic front view showing an example of the appearance of the dart game device 30. Figure 2 This is a schematic diagram showing an example of the appearance of the dartboard 31. Figure 3 This is a block diagram showing an example of the structure of the dart game device 30.

[0024] The dart game device 30 is a device for playing a dart game, which is an example of a throwing game, and is configured to have various structures inside and outside a substantially rectangular parallelepiped housing. The dart game device 30 includes, for example, a dartboard 31, a communication device 32, an operation device 33, a display device 34, a sound output device 35, a card slot 36, a card reader / writer 37, a coin insertion unit 38, a photographing device 39, a detection device 40, and a control device 44.

[0025] The dartboard 31 is an example of a target. As Figure 2 schematically shown, it is substantially circular and provided with a plurality of partitions defined according to the distance and direction from the center of the substantially circle. These partitions are defined, for example, by a frame erected on the dartboard 31. A plurality of receiving holes for receiving a dart, which is an example of a throwing implement, are provided in each partition provided on the dartboard 31. When the dart thrown by the player reaches a certain receiving hole, the dart can be held in a state where a part of it is inserted into the receiving hole. The partition including the receiving hole holding the dart becomes the partition where the player hits the dart in the throw, and a score corresponding to the partition is given to the player.

[0026] At the center of the dartboard 31, there is a substantially circular double bull partition DB and a single bull partition SB that concentrically surrounds it. Additionally, multiple substantially triangular inner single segments S1 are arranged around the single bull partition SB. Further, multiple substantially rectangular triple segments T are arranged around the inner single segments S1. Moreover, multiple substantially rectangular outer single segments S2 are arranged around the triple segments T. Additionally, multiple substantially rectangular double segments D are arranged around the outer single segments S2. Different numbers starting from the center of the dartboard 31 are attached to the inner single segments S1, triple segments T, outer single segments S2, and double segments D respectively, and the partition is determined based on the combination of this number and the type of the partition.

[0027] The communication device 32 is a communication interface for communicating with other information processing devices and the like. The communication device 32 follows the control of the control device 44 and communicates with other information processing devices and the like that are connected to be able to send and receive information with respect to the dart game device 30 via a communication line. The communication method based on the communication device 32 is not particularly limited and can be either wired communication or wireless communication. For example, it can be Bluetooth (registered trademark), wireless LAN (Local Area Network), wired LAN, WAN (Wide Area Network), public line network, mobile data communication network, or a combination thereof.

[0028] The operation device 33 is used for players and the like to perform various operations and is composed of, for example, operation buttons provided on the operation panel. In this embodiment, the operation device 33 is operated by a player, for example, when various dart games (such as Zero-One game, Cricket game, etc.) are selected.

[0029] The display device 34 is an example of an output device and has a function of displaying various screens related to the dart game based on the display data supplied from the control device 44. Specifically, the display device 34 can be, for example, a liquid crystal display or an organic EL display.

[0030] The sound output device 35 is another example of an output device and has a function of outputting (sound output) sounds related to the dart game based on the sound signal supplied from the control device 44. Specifically, the sound output device 35 is composed of, for example, a speaker that can amplify the sound signal through a digital amplifier and output it as sound.

[0031] The card slot 36 is an insertion port for inserting the game card held by the player. In the dart game device 30 in the present embodiment, four card slots 36 are provided so that up to four teams can play against each other (the maximum number of people is not limited). The game card can be configured to record, for example, the identification information (ID) of the player and the game history. The card reading and writing unit 37 is configured to be able to read and write such information recorded in the game card inserted into the card slot 36. In addition, the dart game device 30 is not limited to the method of recording the player's ID and game history in the game card. For example, a method of managing such information in a server or the like that can communicate with the dart game device 30 can also be adopted.

[0032] The coin insertion unit 38 has an insertion port for inserting coins and a sensor for detecting the coins inserted from the insertion port. In the present embodiment, the game can be played by the player inserting a specified number of coins from the coin insertion port. In addition, the dart game device 30 can also be configured to be able to play the game by using an electronic settlement payment method such as an electronic wallet that uses a prepaid payment means.

[0033] The photographing device 39 is provided at a specified position relative to the dartboard 31 and is configured as a camera having, for example, a lens and a photographing element. Figure 1 Among them, as the photographing device 39 of the dart game device 30, two photographing devices 39 provided at the upper left of the dartboard 31 and the photographing device 39 provided at the upper right of the dartboard 31 are shown. However, the number of the photographing devices 39 of the dart game device 30 is not limited to two, and can be one, or three or more.

[0034] The photographing device 39 and a later-described photographing control unit 111 included in the control device 44 together constitute a photographing unit. The photographing device 39 generates a photographing signal by photographing at least a part of the dartboard 31 based on a control signal (photographing control signal) supplied from the photographing control unit 111, and supplies the photographing signal to the photographing control unit 111. The photographing device 39 can, for example, continuously generate a photographing signal at each specified cycle during the photographing operation and supply the photographing signal to the photographing control unit 111. In addition, the photographing device 39 can also be configured to be able to perform pan (movement of the viewing angle) and zoom (enlargement and / or reduction).

[0035] The detection device 40 is a device for detecting the situation where a dart is thrown. The detection device 40 and a later-described detection control unit 112 included in the control device 44 together constitute a detection unit. The detection device 40 detects a change in a physical quantity generated by the contact between the thrown dart and the dartboard 31 or other objects based on a control signal (detection control signal) supplied from the detection control unit 112, and generates a detection signal, and supplies the detection signal to the detection control unit 112.

[0036] The detection device 40 can be configured as any sensor. In Figure 1 , as an example of the detection device 40, a microphone 41, an acceleration sensor 42, and a membrane switch 43 are shown. However, the dart game device 30 of the present embodiment only needs to include at least one of the microphone 41, the acceleration sensor 42, the membrane switch 43, and other detection devices as the detection device 40.

[0037] The microphone 41 is configured, for example, to be able to detect the sound generated by the contact of the thrown dart with the dartboard 31 or other objects and generate a sound waveform as a detection signal. In Figure 1 , as the microphone 41 included in the dart game device 30, the microphone 41a provided on the left side of the dartboard 31 and the microphone 41b provided on the right side of the dartboard 31 are shown. However, the number of microphones 41 included in the dart game device 30 is not limited to this, and can be 0, or one or more than three.

[0038] The microphone 41 can, for example, detect the sound generated by the contact of the dart with the dartboard 31. In this case, the microphone 41 can detect this sound regardless of whether the dart is held on the dartboard 31. As described above, each partition of the dartboard 31 is defined according to the distance and direction from the center of the dartboard 31. Therefore, the characteristic quantity inherent to each partition (for example, the characteristic quantity in the frequency spectrum of the sound waveform) can be included in the sound generated by the contact of the dart with each partition.

[0039] The microphone 41 can, for example, detect the sound generated by the contact of the dart with any structure other than the dartboard 31. The structure with which the dart comes into contact is not particularly limited. For example, in addition to the structure other than the dartboard 31 of the dart game device 30, a part of a building such as the floor, wall, and ceiling around the dart game device 30, it can also be any structure provided around the dart game device 30. Even for these arbitrary structures other than the dartboard 31, the characteristic quantity inherent to each structure (for example, the characteristic quantity in the frequency spectrum of the sound waveform) can be generated in the sound generated by the contact of the dart with these structures.

[0040] The acceleration sensor 42 is configured, for example, to be able to detect the impact (including vibration) generated by the contact of the thrown dart with the dartboard 31 or other objects and generate an impact waveform (vibration waveform) as a detection signal. In Figure 1 , as the acceleration sensor 42 included in the dart game device 30, the acceleration sensor 42a provided on the left side of the dartboard 31 and the acceleration sensor 42b provided on the right side of the dartboard 31 are shown. However, the number of acceleration sensors 42 included in the dart game device 30 is not limited to this, and can be 0, or one or more than three.

[0041] The acceleration sensor 42 can, for example, detect the impact generated due to the contact between the dart and the dartboard 31. The impact generated in the dartboard 31 is detected by the acceleration sensor 42, for example, after reaching the acceleration sensor 42 via various structures of the dart game device 30. In this case, the acceleration sensor 42 can detect the impact regardless of whether the dart is held on the dartboard 31. As described above, each partition provided on the dartboard 31 is defined according to the distance and direction from the center of the dartboard 31. Therefore, in the impact generated due to the contact between the dart and each partition, characteristic quantities inherent to each partition (for example, characteristic quantities in the spectrum of the impact waveform) can be generated.

[0042] The acceleration sensor 42 can, for example, detect the impact generated due to the contact between the dart and any structure other than the dartboard 31. The impact generated in any structure other than the dartboard 31 is detected by the acceleration sensor 42, for example, after reaching the acceleration sensor 42 via the dart game device 30 and other various structures. The structure contacted by the dart is not particularly limited. For example, in addition to the structure other than the dartboard 31 of the dart game device 30, a part of a building such as the floor, wall, ceiling, etc. around the dart game device 30, it can also be any structure provided around the dart game device 30. Even for these arbitrary structures other than the dartboard 31, characteristic quantities inherent to each structure (for example, characteristic quantities in the spectrum of the impact waveform) can be generated in the impact generated due to the contact between the dart and these structures.

[0043] The membrane switch 43 is configured to be able to detect the pressure on the dartboard 31 generated due to the contact between the thrown dart and the dartboard 31 (including the case where the dart is held), and generate an electrical signal as a detection signal. In addition, the method by which the membrane switch 43 detects the pressure can also include binary detection of whether the pressure is above a specified threshold value. The membrane switch 43 can be constituted, for example, by laminating a plurality of electrical contacts that are turned on by an externally applied pressure with a thin film sheet. For example, when the dart contacts the dartboard 31, the electrical contact is turned on, and an electrical signal as a detection signal is generated. The electrical contacts of the membrane switch 43 can, for example, also be able to generate an electrical signal inherent to the contact. In this case, based on the electrical signal, the throwing position of the dart can also be detected.

[0044] (1-2) Functional Structure of the Control Device 44

[0045] The control device 44 has a processor 45 and a memory 46, and comprehensively controls various structures of the dart game device 30. The processor 45 can be constituted by, for example, a CPU (Central Processing Unit), and can execute programs stored in the memory 46. The memory 46 can have, for example, a ROM (Read Only Memory) that stores various programs and the like, and a RAM (Random Access Memory) that functions as a storage area for data and a working area for the processor. In addition, the above programs can also be stored in non-transitory recording media such as a USB memory and a CD-ROM. Further, the above programs can also be supplied to the memory 46 via any transmission medium (such as a communication network or a broadcast wave) capable of transmitting the programs. In addition, one aspect of the present invention can also be implemented in the form of a data signal embedded in a carrier wave, which is embodied by electronic transmission of the above programs.

[0046] Figure 4 It is a functional block diagram showing an example of a functional part of the control device 44. The control device 44 has, for example, a control part 110 and a storage part 120 as functional parts implemented by the processor 45, the memory 46, and the like.

[0047] The storage part 120 is a storage device that stores data required for the processing of the dart game device 30. The storage part 120 stores various programs such as a program for playing a dart game and a program for the dart game device 30 to execute a method for estimating a hit position. In addition, the storage part 120 stores a machine learning model 121 and score information 122 showing the scores of each player in the throwing game.

[0048] The control part 110 comprehensively controls various structures of the dart game device 30 based on various programs stored in the storage part 120. The control part 110 has a shooting control part 111, a detection control part 112, a learning execution part 113, an estimation result acquisition part 114, and an output control part 115.

[0049] The shooting control part 111 controls the shooting by the shooting device 39 by supplying a control signal to the shooting device 39, and generates image data based on the shooting signal obtained from the shooting device 39. When the dart game device 30 has a plurality of shooting devices 39, the shooting control part 111 can generate image data for each shooting device 39, or can synthesize the respective shooting signals obtained from the plurality of shooting devices 39 to generate image data. The shooting control part 111 can also extract a part of the shooting signals obtained from the shooting device 39 to generate partial image data. The shooting control part 111 can also include time information indicating the shooting time in the image data.

[0050] The detection control unit 112 controls the detection performed by the detection device 40 by supplying a control signal to detection devices 40 such as the microphone 41, the acceleration sensor 42, and the membrane switch 43, and generates detection data based on the detection signal obtained from the detection device 40. For example, the detection control unit 112 can generate detection data of the sound waveform based on the sound waveform generated by the microphone 41. In addition, the detection control unit 112 can also generate detection data of the impact waveform based on the impact waveform generated by the acceleration sensor 42. In addition, the detection control unit 112 can also generate detection data of the electrical signal based on the electrical signal generated by the membrane switch 43. The detection control unit 112 can also include time information indicating the detection time in the detection data.

[0051] The learning execution unit 113 generates a machine learning model 121 by performing machine learning based on prescribed training data. Here, use Figure 5 to illustrate the outline of the machine learning model 121 of the present embodiment. Figure 5 is a diagram schematically showing an example of the input / output data of the machine learning model 121.

[0052] The machine learning model 121 can be configured, for example, as a multimodal model, that is, a model configured to include two types of data with different presentation forms (modalities) in the input data. The input data can include, for example, input data of the first presentation form (first input data) and input data of the second presentation form (second input data).

[0053] The presentation form (first presentation form) of the first input data can be an image. That is, the first input data can be, for example, image data generated by the shooting control unit 111, image data obtained by performing prescribed preprocessing on the image data. In Figure 5 the upper left part, as an example of the first input data, image data of a dartboard DB and a dart D1 being shot is shown.

[0054] The presentation form (second presentation form) of the second input data can be a presentation form other than an image. In particular, the second input data can be, for example, detection data (detection data of a sound waveform, detection data of an impact waveform, detection data of an electrical signal, etc.) generated by the detection control unit 112, detection data obtained by performing prescribed preprocessing on the detection data, and the like. In Figure 5 the lower left part, as an example of the second input data, sound waveform data when the dart D1 contacts (hits) the dartboard DB is shown.

[0055] The output data of the machine learning model 121 includes the estimated result of the hit position. Here, the estimated result of the hit position may include, for example, information indicating at least one estimated hit position. The information indicating the hit position may be, for example, information indicating the partition provided on the dartboard, or may be information indicating the coordinates of the hit position. The form of the coordinates is not particularly limited. For example, it may be two-dimensional coordinates on the image data as the input data, or three-dimensional coordinates in the space where the dartboard is provided. The estimated result of the hit position may also include the reliability of each hit position. The calculation method of the reliability is not particularly limited. For example, a method using an activation function such as the Sigmoid function or the Softmax function may be adopted. In other words, the reliability can also be said to be the generation probability of the hit position. On the Figure 5 right side, as an example of the output data (estimated result of the hit position), a display of the hit position "I19" of the inner single segment showing the number "19" hit by the dart D1 in the dartboard DB and the number "0.88" as the reliability of the estimated result are shown.

[0056] In this way, regarding the machine learning model 121, since data in a form other than an image is also included in the input data, the estimation accuracy of the hit position can be improved compared to the case where only data in an image form is included in the input data.

[0057] In addition, the machine learning model 121 may also be configured as a model that includes three or more types of data with different forms in the input data. In this machine learning model 121, for example, the first input data may be set as image data, the second input data may be set as a certain type of detection data (such as detection data of a sound waveform), and the input data of the third form (the third input data) may be set as another certain type of detection data (such as detection data of an impact waveform).

[0058] Returning to Figure 4 . The learning execution unit 113 generates the machine learning model 121 by performing machine learning based on the training data including the first input data and the second input data.

[0059] The first input data included in the training data may be the image data generated by the shooting control unit 111, or may be the image data acquired from other information processing devices or the like via the communication device 32. The image data acquired from other information processing devices or the like may not necessarily be the image data generated by shooting the dartboard 31, and may also be the image data generated by shooting the same or different types of other dartboards, or the image data generated by a prescribed computer graphics (CG) or the like. In the first input data such as the image data included in the training data, information indicating the hitting position of the dart on the dartboard may be included as a label. This information may be represented, for example, by the section on the dartboard where the dart hits, or may be represented by the coordinates of the hitting position of the dart.

[0060] The second input data included in the training data may be the detection data generated by the detection control unit 112, or may be the detection data acquired from other information processing devices or the like via the communication device 32. The detection data acquired from other information processing devices or the like may not necessarily be the detection data generated based on the detection signal detected when throwing a dart at the dartboard 31, and may also be the detection data generated using the same or different types of other dartboards or darts, or the detection data generated by a prescribed simulation. In the second input data such as the detection data included in the training data, information indicating the hitting position of the dart on the dartboard may be included as a label. This information may be represented, for example, by the section on the dartboard where the dart hits, or may be represented by the coordinates of the hitting position of the dart.

[0061] The machine learning model 121 can be arbitrarily configured using known techniques such as a method employing a deep Boltzmann machine or multiple classifiers. Specifically, the machine learning model 121 can be configured, for example, to have a first neural network corresponding to the first input data such as the image data, a second neural network corresponding to the second input data such as the detection data, and an integration layer that integrates the output of the first neural network and the output of the second neural network.

[0062] The estimation result acquisition unit 114 uses the machine learning model 121 to estimate the hitting position. Specifically, the estimation result acquisition unit 114 inputs prescribed input data into the machine learning model 121, for example, to obtain the estimation result of the hitting position as the output. The input data is configured corresponding to the structure of the machine learning model 121 described above. That is, the input data can be configured to include two types of data with different presentation forms (modalities). This input data may include, for example, the input data of the first presentation form (the first input data) and the input data of the second presentation form (the second input data).

[0063] The presentation form of the first input data (the first presentation form) can be, for example, an image. That is, the first input data can be, for example, image data generated by the shooting control unit 111, or image data obtained by performing a prescribed pre-processing on the image data. In addition, the presentation form of the second input data (the second presentation form) can be, for example, a presentation form other than an image. In particular, the second input data can be, for example, detection data (detection data of a sound waveform, detection data of an impact waveform, detection data of an electrical signal, etc.) generated by the detection control unit 112, or detection data obtained by performing a prescribed pre-processing on the detection data.

[0064] In addition, the input data can be configured to include three or more types of data with different presentation forms. For example, the first input data can be set as image data, the second input data can be set as a certain type of detection data (for example, detection data of a sound waveform), and the input data of the third presentation form (the third input data) can be set as another certain type of detection data (for example, detection data of an impact waveform).

[0065] The estimation result acquisition unit 114 can, for example, generate input data based on the image data generated by the shooting control unit 111 and the detection data generated by the detection control unit 112. The estimation result acquisition unit 114 can also compare the time information contained in the image data and the detection data respectively, and when the difference in the time information is less than a prescribed threshold value, establish a correspondence between the image data and the detection data with each other and set them as a set constituting the input data. The estimation result acquisition unit 114 can, for example, input the input data into the machine learning model 121, thereby obtaining the estimation result of the hit position output by the machine learning model 121.

[0066] The output control unit 115 controls the output based on various output devices. The output control unit 115 can, for example, have a function as a display control unit for controlling the display based on the display device 34. Specifically, the output control unit 115 generates display data of a prescribed screen to be displayed on the display device 34, and based on the display data, causes the display device 34 to display the prescribed screen. The screen displayed on the display device 34 can, for example, include the estimation result (numerical value, text, etc.) output by the machine learning model 121. In addition, the screen displayed on the display device 34 can include an image of the dartboard 31, or can be a state in which the estimation result of the hit position is overlapped on the image. Furthermore, the image of the dartboard 31 can be an image generated by shooting the dartboard 31 based on the image data input to the machine learning model 121, or can be a schematic image showing the dartboard 31 generated by CG or the like.

[0067] The output control unit 115 may have, for example, a function as a sound output control unit that controls the sound output of the sound output device 35. Specifically, the output control unit 115 generates prescribed sound data for causing the sound output device 35 to perform sound output, and causes the sound output device 35 to output prescribed sound based on the sound data. The sound output by the sound output device 35 may include, for example, a sound for reading out a prediction result (such as a numerical value or text) output by the machine learning model 121, a sound effect corresponding to the prediction result, and the like.

[0068] The output control unit 115 may also calculate, for example, a score obtained by the player by throwing based on the prediction result output by the machine learning model 121, and enter the score into the score information 122 stored in the storage unit 120.

[0069] (2) Actions (2-1) Machine learning model generation method

[0070] Figure 6 It is a flowchart of actions that illustrates an example of the generation method of the machine learning model 121. In addition, some steps may be executed in parallel or in a replaced order. Hereinafter, the case where the first input data is image data and the second input data is detection data will be described as an example.

[0071] (S101)

[0072] The learning execution unit 113 acquires image data and stores it in the storage unit 120. The image data may be the image data generated by the shooting control unit 111, and may include the time information obtained when the shooting control unit 111 generates the image data. Alternatively, the image data may be image data acquired via the communication device 32 from other information processing devices or the like. The image data acquired from other information processing devices or the like does not necessarily have to be image data generated by shooting the dartboard 31, and may be image data generated by shooting the same or different types of other dartboards, or image data generated by prescribed computer graphics (CG) or the like.

[0073] (S102)

[0074] The learning and execution unit 113 acquires the detection data and stores it in the storage unit 120. The detection data can be the detection data generated by the detection control unit 112, or can include the time information obtained when the detection control unit 112 generates the detection data. Alternatively, the detection data can also be the detection data acquired via the communication device 32 from other information processing devices or the like. The detection data acquired from other information processing devices or the like does not necessarily have to be the detection data generated based on the detection signal detected when a dart is thrown at the dartboard 31, and can also be the detection data generated using the same or different types of other dartboards or darts, or the detection data generated by a prescribed simulation.

[0075] (S103)

[0076] Based on the multiple image data and multiple detection data stored in the storage unit 120, the learning and execution unit 113 generates training data. For example, when the image data and the detection data are not associated, the learning and execution unit 113 compares the time information included in each of the image data and the detection data, and when the difference in the time information is less than a prescribed threshold value, associates the image data and the detection data with each other. In addition, after acquiring a label indicating the hit position (the hit section or coordinates), the learning and execution unit 113 assigns the label to the image data, the detection data, the training data, etc. according to the structure of the machine learning model 121. The label can also be information input by a player or an arbitrary administrator or the like using the operation device 33 of the dart game device 30 or the operation device of other information processing devices connected via the communication device 32.

[0077] (S104)

[0078] The learning and execution unit 113 generates a machine learning model 121 by performing machine learning based on the training data generated in step S103, and stores it in the storage unit 120.

[0079] (2-2) Method for estimating the hit position

[0080] Figure 7 is an operation flowchart for explaining an example of the method for estimating the hit position. In addition, a part of the steps can also be executed in parallel or in a replacement order. Hereinafter, the case where the first input data is image data and the second input data is detection data will be described as an example.

[0081] (S201)

[0082] The photographing control unit 111 generates image data based on the photographing signal obtained from the photographing device 39 by controlling the photographing device 39. The timing of generating the image data is not particularly limited. For example, it may be when a generation command for the image data is obtained via the communication device 32 or the like. The photographing control unit 111 includes time information indicating the photographing time in the image data.

[0083] (S202)

[0084] The detection control unit 112 generates detection data based on the detection signal obtained from the detection device 40 by controlling the detection device 40. The timing of generating the detection data is not particularly limited. For example, it may be when a generation command for the detection data is obtained via the communication device 32 or the like. The detection control unit 112 includes time information indicating the detection time in the detection data.

[0085] (S203)

[0086] The estimation result acquisition unit 114 generates input data based on the image data and the detection data. The estimation result acquisition unit 114 may compare the time information included in each of the image data and the detection data, and if the difference in the time information is less than a specified threshold, establish a correspondence between the image data and the detection data and set them as a set constituting the input data.

[0087] (S204)

[0088] The estimation result acquisition unit 114 may, for example, input the input data into the machine learning model 121, thereby obtaining the estimation result of the hit position output by the machine learning model 121. The estimation result of the hit position may include, for example, information indicating at least one estimated hit position. The information indicating the hit position may be, for example, information indicating the partition provided on the dartboard, or may be information indicating the coordinates of the hit position. The form of the coordinates is not particularly limited. In addition, the estimation result of the hit position may also include the probability (reliability) of each hit position.

[0089] (S205)

[0090] The output control unit 115 controls various output devices to output the estimation result. For example, the output control unit 115 generates display data for a specified screen to be displayed on the display device 34, and based on the display data, causes the display device 34 to display the specified screen.

[0091] Figure 8 is a diagram showing an example of the screen 200 displayed on the display device 34. The screen 200 includes a schematic image 201 of a dartboard and a display unit 203 for the estimation result of the hit position. The image 201 is generated by, for example, CG or the like. In the image 201, the hit position of the dart can be emphasized. In Figure 9In the example shown, a part of the outer single-fold partition of the number 10 is shaded. In the estimation result display unit 203, as the estimation result, it includes text showing the hit position "Hit position: outer single-fold (10)" and text showing the reliability "Reliability: 92%". In addition, the screen displayed on the display device 34 may also include an image generated by photographing the dartboard 31 based on the image data included in the input data for the machine learning model 121.

[0092] The output control unit 115 may also generate, for example, prescribed sound data, and based on this sound data, cause the sound output device 35 to output a sound reading out the estimation result (numerical value, text, etc.) output by the machine learning model 121, or an effect sound corresponding to the estimation result.

[0093] As described above, according to the present embodiment, in the estimation of the hit position of the dart using the machine learning model 121 after machine learning, not only image data is used as the input data, but also detection data is used as the input data. Thus, compared with the case where only images are used as the input data, the position can be estimated with higher accuracy.

[0094] (3) Supplementary Notes

[0095] 〔Supplementary Note 1〕

[0096] The input data of the machine learning model 121 may include at least one of detection data instead of image data.

[0097] For example, the input data of the machine learning model 121 may be sound waveform data. That is, the machine learning model 121 may be configured to output an estimation result of the position where the dart hits in the dartboard when sound generated by the thrown dart contacting the dartboard or other objects is input.

[0098] As described above, each partition provided on the dartboard is defined according to the distance and direction from the center of the dartboard. Therefore, the characteristic quantity inherent to each partition (for example, the characteristic quantity in the frequency spectrum of the sound waveform) can be included in the sound generated by the dart contacting each partition. In addition, even for any structure other than these dartboards, the characteristic quantity inherent to each structure (for example, the characteristic quantity in the frequency spectrum of the sound waveform) can be generated in the sound generated by the dart contacting these structures. Therefore, even if the machine learning model 121 includes sound waveform data instead of image data as the input data, it can output an estimation result of the hit position with sufficient accuracy.

[0099] 〔Supplementary Note 2〕

[0100] The dart game device 30 may also perform various processes corresponding to the judgment result after determining that the reliability included in the estimation result of the hit position is equal to or higher than a specified threshold value.

[0101] The output control unit 115 may also cause the output device to output specified information when it is determined that the reliability is less than the specified threshold value. For example, the output control unit 115 may cause the display device 34 to display Figure 9 the screen 300 shown. Here, in addition to Figure 8 the schematic image 201 of the dartboard included in the screen 200 shown and the display unit 203 of the estimation result of the hit position, the screen 300 also includes specified information 301 displayed when it is determined that the reliability is less than the specified threshold value. The information 301 includes the text "Please check the actual dart position and re-enter the hit position." This is an example of information urging the player to input the actual dart position by himself / herself. For example, after a player who sees this information confirms the actual hit position by observing the dartboard 31, the player can directly input the actual hit position or the score based on the actual hit position by operating the operation device 33 or the operation device of another information processing device connected to the dart game device 30 via the communication device 32. Regarding the information indicating the input actual hit position or score, for example, after the control unit 110 acquires it, the control unit 110 may update the score information 122 stored in the storage unit 120 based on this information.

[0102] In this way, the output control unit 115 may also cause the display device 34 to display information urging the input of the actual dart hit position when it is determined that the reliability is less than the specified threshold value. In addition, when it is determined that the reliability is less than the specified threshold value, the information displayed by the output control unit 115 on the display device 34 is not limited to the information urging the input of the hit position, and may also include information directly or indirectly indicating that the reliability of the estimation result of the hit position is low, or an image for specified display, etc.

[0103] In addition, the output control unit 115 may also cause the sound output device 35 to output a specified sound when it is determined that the reliability is less than the specified threshold value. The sound may be, for example, a sound reading out information urging the input of the actual dart hit position, or a sound reading out information directly or indirectly indicating that the reliability of the estimation result of the hit position is low, or a sound effect for specified display, etc.

[0104] 〔Supplementary Note 3〕

[0105] The dart game device 30 may also perform various processes corresponding to the judgment result after determining that the reliability included in the estimation result of the hit position is equal to or higher than a specified threshold value.

[0106] When it is determined that the reliability is less than a specified threshold value, the shooting control unit 111 may also generate image data (second image data) again. For example, the shooting area of the second image data may be the same as or different from the shooting area of the image data (first image data) included in the input data that is the basis for the determination. Specifically, it may be an area including the entire dartboard 31, or an area including at least a part of the dartboard 31.

[0107] The second image data may be generated by shooting. For example, the shooting control unit 111 may control the shooting device 39 to obtain a shooting signal with the same area as the first image data set as the shooting area, and generate the second image data based on the shooting signal. Additionally, for example, the shooting control unit 111 may also obtain a shooting signal after controlling the shooting device 39 to change the viewing angle or zoom ratio, and generate the second image data with a range different from the previous image data set as the shooting area based on the shooting signal. Specifically, the shooting control unit 111 may generate image data obtained by magnifying a partial range of the previous image data by increasing the magnification of the shooting device 39, or may generate the second image data with a wider angle than the previous image data by decreasing the magnification of the shooting device 39. In particular, when increasing the magnification of the shooting device 39, the shooting area of the shooting device 39 may be adjusted so that the hitting position estimated by the machine learning model 121 is further closer to the center of the image.

[0108] The second image data may also be generated based on the first image data. For example, the control unit 110 may generate the second image data by performing various processing operations such as image quality adjustment or enlargement / reduction on the first image data. In this case, the area of the second image data may be the same as or different from the area of the first image data. In particular, the area of the second image data may include at least a part of the dartboard 31.

[0109] When the second image data is generated, the learning execution unit 113 may update the image data included in the training data to the second image data and then perform machine learning again to update the machine learning model 121. And, the estimation result acquisition unit 114 may use the updated machine learning model 121 to obtain the estimation result again, and the output control unit 115 may cause the output device to output the estimation result obtained again.

[0110] 〔Supplementary Note 4〕

[0111] Part of the structure of the above-described dart game device 30 may also be provided in other information processing devices composed of a computer including a processor and a memory, such as a smartphone or a tablet terminal. In this case, the dart game device 30 can transmit and receive information to and from the other information processing device via a communication device 32 or the like. The other information processing device may also have at least a part of the structures such as a display device 34, a sound output device 35, a photographing device 39, and a detection device 40. In addition, the other information processing device may also have at least a part of the functional units such as a photographing control unit 111, a detection control unit 112, a learning execution unit 113, a estimation result acquisition unit 114, and an output control unit 115. In addition, the other information processing device may also store at least a part of a machine learning model 121 and score information 122 or the like.

[0112] The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope shown in the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention.

[0113] Description of Reference Numerals

[0114] 30... Dart game device, 31... Dartboard, 32... Communication device, 33... Operation device, 34... Display device, 35... Sound output device, 36... Card slot, 37... Card reader / writer, 38... Coin insertion unit, 39... Photographing device, 40... Detection device, 41, 41a, 41b... Microphone, 42, 42a, 42b... Acceleration sensor, 43... Membrane switch, 44... Control device, 45... Processor, 46... Memory, 110... Control unit, 111... Photographing control unit, 112... Detection control unit, 113... Learning execution unit, 114... Estimation result acquisition unit, 115... Output control unit, 120... Storage unit, 121... Machine learning model, 122... Score information.

Claims

1. A hit position estimation system, comprising: a storage unit that stores a machine learning model that has completed machine learning, the machine learning model that has completed machine learning being configured to output an estimation result of a position where a throwing implement hits a target in the target when image data generated by photographing the target when the throwing implement is thrown in a throwing game and detection data generated by detecting contact between the thrown throwing implement and the target or other object are input; a photographing unit that generates image data by photographing the target when the throwing tool is thrown in a throwing game; a detection unit that generates detection data based on detecting contact between the thrown throwing tool and the target or other object; an acquisition unit that acquires an estimated result of a position in the target hit by the throwing tool, which is output by inputting the image data and the detection data into the machine learning model; as well as A display unit displays the estimation result.

2. The hit position estimation system according to claim 1, wherein The detection unit detects a sound generated when the thrown throwing tool contacts the target or the other object.

3. The hit position estimation system according to claim 1, wherein The detection unit detects pressure on the target generated by the thrown throwing tool contacting the target.

4. The hit position estimation system according to claim 1, wherein The detection unit detects an impact caused by the thrown throwing tool contacting the target or the other object.

5. The hit position estimation system according to claim 1, wherein The machine learning model is configured to further output the reliability of the estimation result when the image data and the detection data are input. The acquisition unit further acquires the reliability output by inputting the image data and the detection data into the machine learning model, The display unit controls display content of the estimation result based on the reliability.

6. The hit position estimation system according to claim 5, wherein The display unit further displays the reliability.

7. The hit position estimation system according to claim 5, wherein When the reliability is less than a predetermined threshold value, the display unit displays information for urging the user to input a throwing result in the throwing game.

8. The hit position estimation system according to claim 5, wherein When the reliability is less than a predetermined threshold, the imaging unit generates second image data including at least a portion of the target when the throwing tool is thrown in the throwing game, The acquisition unit acquires a second estimation result of a position in the target hit by the throwing tool, which is output by inputting at least the second image data into the machine learning model, The display unit displays the second estimation result.

9. A hit position estimation method, in which an information processing device is caused to execute steps, wherein The information processing device is capable of accessing a storage unit storing a machine learning model that has been machine-learned, and the machine learning model that has been machine-learned is configured to output an estimated result of the position of the throwing tool in the target hit by the throwing tool when image data generated by photographing the target when the throwing tool is thrown in the throwing game and detection data generated by detecting the contact between the thrown throwing tool and the target or other objects are input, and the information processing device performs the following steps: A step of generating image data based on photographing the target when the throwing tool is thrown in a throwing game by controlling a photographing device; A step of generating detection data based on detecting contact between the thrown throwing tool and the target or other object by controlling a detection device; A step of obtaining an estimated result of a position in the target hit by the throwing tool, which is output by inputting the image data and the detection data into the machine learning model; and The step of causing the estimation result to be displayed on a display device.

10. A program for causing an information processing apparatus to function as a shooting control unit, a detection control unit, an acquisition unit, and a display control unit, wherein, The information processing device is capable of accessing a storage unit storing a machine learning model that has been machine learned, and the machine learning model that has been machine learned is configured to output an estimated result of the position where the throwing tool hits the target when image data generated by photographing the target when the throwing tool is thrown in the throwing game and detection data generated by detecting the contact between the thrown throwing tool and the target or other objects are input, The imaging control unit generates image data by controlling an imaging device to image the target when the throwing tool is thrown in a throwing game. The detection control unit generates detection data based on detecting the contact between the thrown throwing tool and the target or other object by controlling the detection device, The acquisition unit acquires an estimated result of a position in the target hit by the throwing tool, which is output by inputting the image data and the detection data into the machine learning model. The display control unit displays the estimation result on a display device.

11. A hit position estimation system, comprising: A storage unit that stores a machine learning-completed machine learning model, the machine learning-completed machine learning model being configured to output an estimation result of a position where a thrown implement hits in the target when a sound generated by contact between the thrown implement and the target or other object in a throwing game is input; a detection unit for detecting a sound generated by the thrown throwing tool contacting the target or the other object in the throwing game; an acquisition unit that acquires an estimated result of a position in the target hit by the throwing tool, which is output by inputting the sound into the machine learning model; and A display unit displays the estimation result.

12. The hit position estimation system according to claim 11, wherein, The detection unit is arranged at a predetermined position relative to the target, The target has a plurality of partitions defined according to distances and directions from a center of the target.

Images