Hit location estimation system and hit location estimation method

MYPI2025002904A0Undetermined Publication Date: 2024-05-14SEGA CORP +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
MYPI2025002904
Authority / Receiving Office
MY · MY
Patent Type
Applications
Current Assignee / Owner
Priority Date
2022-11-14
Filing Date
2023-11-07
Publication Date
2024-05-14

AI Technical Summary

Technical Problem

Existing methods for estimating the hit position of a throwing instrument in throwing games, such as darts, rely solely on image data, which limits accuracy, as they do not consider the physical phenomena like sounds and shocks generated upon contact with the target.

Method used

A hit position estimation system that incorporates both image data and detection data, including audio and impact waveforms, into a machine learning model to improve the accuracy of hit position estimation.

Benefits of technology

Enhances the accuracy of hit position estimation by utilizing multimodal input data, combining image recognition with physical detection data, leading to more precise results compared to using only image data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure PI2025002904_DRAWING_001
    Figure PI2025002904_DRAWING_001
  • Figure PI2025002904_DRAWING_002
    Figure PI2025002904_DRAWING_002
  • Figure PI2025002904_DRAWING_003
    Figure PI2025002904_DRAWING_003
Patent Text Reader

Abstract

A location hit by a throwing instrument is estimated with higher accuracy. A hit location estimation system (30) includes a storage unit (120) that stores a machine-trained machine learning model (121) configured to output, when image data generated on the basis of capturing of an image of a target (31) upon throwing of a throwing instrument in a throwing game and detection data generated on the basis of detection of contact of the thrown throwing instrument with the target (31) or another object is input to the machine learning model (121), a result of estimating a location on the target (31) hit by the throwing instrument, an image capturing unit (111) that generates the image data on the basis of the capturing of the image of the target (31) upon the throwing of the throwing instrument in the throwing game, a detection unit (112) that generates the detection data on the basis of the detection of the contact of the thrown throwing instrument with the target (31) or the other object, an acquisition unit (114) that acquires the result of estimating the location on the target (31) hit by the throwing instrument, the result being output by inputting the image data and the detection data to the machine learning model (121), and a display unit (115) that displays the estimation result.
Need to check novelty before this filing date? Find Prior Art

Description

Hit position estimation system and hit position estimation method

[0001] The present invention relates to a technique for estimating the target position of a throwing implement in a throwing game.

[0002] Conventionally, there have been known throwing game devices for playing throwing games in which players throw a throwing tool such as a dart at a target and compete in throwing skills with points set according to the position on the target where the throwing tool hits. With regard to such throwing game devices, various techniques have been proposed for determining the hit position of the throwing tool.

[0003] One possible technique for estimating the target position of a throwing implement is a method using a machine learning model such as deep learning. Regarding a technique for estimating the position of an object using a machine learning model, for example, Patent Document 1 describes a position estimation device that estimates the position of an object based on an image captured by a monocular bird's-eye view camera. The position estimation device uses a machine learning model trained by deep learning to perform image recognition on the acquired image and detect the position of the target in the frame.

[0004] Japanese Patent Application Laid-Open No. 2022-125973

[0005] The method described in Patent Document 1 above only uses images obtained by capturing images of objects as input to a machine learning model. In this regard, the inventors of the present invention have discovered that when a thrown throwing tool contacts a target or another object, various sounds, impacts, etc. are generated depending on the position of contact, and the detection results of these physical phenomena are also useful for estimating the hit position of the throwing tool.

[0006] One aspect of the present invention has been made in consideration of the above-mentioned problems, and one of its objects is to more accurately estimate the target position of a throwing tool.

[0007] A hit position estimation system according to one aspect of the present invention comprises: a memory unit that stores a machine-learned machine learning model configured to output an estimation result of the position at which the throwing instrument hits the target when image data generated based on imaging the target when thrown with a throwing instrument in a throwing game and detection data generated based on detecting that the thrown throwing instrument contacts the target or another object are input; an imaging unit that generates image data based on imaging the target when thrown with the throwing instrument in a throwing game; a detection unit that generates detection data based on detecting that the thrown throwing instrument contacts the target or another object; an acquisition unit that acquires the estimation result of the position at which the throwing instrument hits the target, which is output by inputting the image data and detection data into the machine learning model; and a display unit that displays the estimation result.

[0008] According to the above configuration, when estimating the target position of a throwing tool using a machine learning model, not only image data but also detection data is included in the input data, which allows the target position of a throwing tool to be estimated with higher accuracy.

[0009] According to one aspect of the present invention, the target position of a throwing implement can be estimated with greater accuracy.

[0010] 1 is a schematic front view showing an example of the appearance of the darts game device 30. FIG. 2 is a schematic view showing an example of the appearance of the darts board 31. FIG. 3 is a block diagram showing an example of the configuration of the darts game device 30. FIG. 4 is a functional block diagram showing an example of the functional units of the control device 44. FIG. 5 is a diagram schematically showing an example of input / output data of the machine learning model 121. FIG. 6 is an operation flow diagram for explaining an example of a method for generating the machine learning model 121. FIG. 7 is an operation flow diagram for explaining an example of a method for estimating a hit position. FIG. 8 is a diagram showing an example of a screen 200 displayed on the display device 34. FIG. 9 is a diagram showing an example of a screen 300 displayed on the display device 34.

[0011] (1) Configuration (1-1) Overall Configuration of the Dart Game Device 30 The overall configuration of the dart game device 30 according to this embodiment will be described with reference to Figures 1 to 3. Figure 1 is a schematic front view showing an example of the appearance of the dart game device 30. Figure 2 is a schematic view showing an example of the appearance of a dart board 31. Figure 3 is a block diagram showing an example of the configuration of the dart game device 30.

[0012] The darts game device 30 is a device for playing a darts game, which is an example of a throwing game, and is configured with various components inside and outside a substantially rectangular parallelepiped housing. The darts game device 30 includes, for example, a dart board 31, a communication device 32, an operation device 33, a display device 34, an audio output device 35, a card slot 36, a card RW unit 37, a coin insertion unit 38, an imaging device 39, a detection device 40, and a control device 44.

[0013] The dartboard 31 is an example of a target, and as shown schematically in Fig. 2, has a substantially circular shape with a plurality of segments defined by the distance and direction from the center of the substantially circle. Each of these segments is defined, for example, by a frame erected on the dartboard 31. Each segment of the dartboard 31 is provided with a plurality of receiving holes capable of receiving darts, which are an example of throwing tools. When a dart thrown by a player reaches one of the receiving holes, the dart can be held in a state in which a part of the dart is inserted into the receiving hole. The segment including the receiving hole in which the dart is held is the segment in which the player hit the target during the throw, and the player is awarded points according to the segment.

[0014] A substantially circular double bull segment DB is arranged at the center of the dartboard 31, and a single bull segment SB is arranged concentrically around it. A plurality of substantially triangular inner single segments S1 are arranged around the single bull segment SB. A plurality of substantially rectangular triple segments T are arranged around the inner single segment S1. A plurality of substantially rectangular outer single segments S2 are arranged around the triple segment T. A plurality of substantially rectangular double segments D are arranged around the outer single segment S2. The inner single segment S1, triple segment T, outer single segment S2, and double segment D are each assigned a different number depending on the direction from the center of the dartboard 31, and the segment is identified by the combination of the number and the type of segment.

[0015] The communication device 32 is a communication interface for communicating with other information processing devices, etc. The communication device 32 communicates with other information processing devices, etc. that are connected to the darts game device 30 so as to be able to send and receive information via a communication path under the control of the control device 44. The communication method used by the communication device 32 is not particularly limited, and may be either wired communication or wireless communication, for example, Bluetooth (registered trademark), a wireless LAN (Local Area Network), a wired LAN, a WAN (Wide Area Network), a public line network, a mobile data communication network, or a combination thereof.

[0016] The operation device 33 is used by a player or the like to perform various operations, and is configured, for example, with operation buttons provided on an operation panel. The operation device 33 in this embodiment is operated by a player, for example, when selecting various darts games (e.g., Zero One, Cricket, etc.).

[0017] The display device 34 is an example of an output device, and has a function of displaying various screens related to the darts game based on display data supplied from the control device 44. Specifically, the display device 34 may be, for example, a liquid crystal display or an organic EL display.

[0018] The audio output device 35 is another example of an output device, and has a function of outputting audio related to the darts game (audio output) based on an audio signal supplied from the control device 44. Specifically, the audio output device 35 is configured by, for example, a speaker or the like that can amplify an audio signal using a digital amplifier or the like and output the audible audio.

[0019] The card slot 36 is an insertion port into which a game card owned by a player is inserted. The darts game device 30 in this embodiment is provided with four card slots 36 so that a maximum of four teams can play against each other (the maximum number of players is not limited). The game card may be configured to record, for example, player identification information (ID), game history, etc. The card RW unit 37 is configured to be able to read and write such information recorded on a game card inserted into the card slot 36. Note that the darts game device 30 is not limited to a system in which the player ID, game history, etc. are recorded on the game card, and may employ a system in which such information is managed by a server or the like communicably connected to the darts game device 30.

[0020] The coin insertion unit 38 has a slot for inserting coins and a sensor for detecting coins inserted through the slot. In this embodiment, the game can be played when a player inserts a predetermined number of coins through the coin insertion slot. Note that the darts game device 30 may also be configured to allow game play by electronic payment using a prepaid payment method such as electronic money.

[0021] The imaging devices 39 are provided at predetermined positions relative to the dart board 31, and are configured as cameras having lenses, imaging elements, etc. Fig. 1 shows two imaging devices 39 provided in the dart game apparatus 30: one provided at the top left of the dart board 31, and the other provided at the top right of the dart board 31. However, the number of imaging devices 39 provided in the dart game apparatus 30 is not limited to two, and may be one, or three or more.

[0022] The imaging device 39 constitutes an imaging section together with an imaging control section 111 (described later) included in the control device 44. The imaging device 39 generates an imaging signal by capturing an image of at least a part of the dartboard 31 based on a control signal (imaging control signal) supplied from the imaging control section 111, and supplies the imaging signal to the imaging control section 111. For example, during imaging operation, the imaging device 39 may continuously generate imaging signals at predetermined intervals and supply these to the imaging control section 111. Note that the imaging device 39 may be configured to be capable of panning (moving the viewpoint) and zooming (enlarging and / or reducing).

[0023] The detection device 40 is a device for detecting that a dart has been thrown. The detection device 40 constitutes a detection section together with a detection control section 112 (described later) included in the control device 44. Based on a control signal (detection control signal) supplied from the detection control section 112, the detection device 40 detects a change in physical quantity caused by the thrown dart coming into contact with the dartboard 31 or another object, generates a detection signal, and supplies the detection signal to the detection control section 112.

[0024] The detection device 40 may be configured as any sensor. In Fig. 1, a microphone 41, an acceleration sensor 42, and a membrane switch 43 are shown as examples of the detection device 40. However, the darts game device 30 according to this embodiment may include at least one of the microphone 41, the acceleration sensor 42, the membrane switch 43, and other detection devices as the detection device 40.

[0025] The microphones 41 are configured to detect, for example, the sound generated when a thrown dart contacts the dartboard 31 or another object, and generate an audio waveform as a detection signal. In Fig. 1, the microphones 41 included in the dart game device 30 are shown as a microphone 41a provided on the left side of the dartboard 31 and a microphone 41b provided on the right side of the dartboard 31. However, the number of microphones 41 included in the dart game device 30 is not limited to this, and may be zero, one, or three or more.

[0026] The microphone 41 may detect, for example, a sound generated when a dart comes into contact with the dartboard 31. In this case, the microphone 41 may detect the sound regardless of whether or not a dart is held on the dartboard 31. As described above, each segment on the dartboard 31 is defined by its distance and direction from the center of the dartboard 31. Therefore, the sound generated when a dart comes into contact with each segment may include a feature unique to each segment (for example, a feature in the frequency spectrum of an audio waveform).

[0027] The microphone 41 may detect, for example, a sound generated when a dart comes into contact with any structure other than the dartboard 31. The structure with which the dart comes into contact is not particularly limited, and may be, for example, a structure of the dart game apparatus 30 other than the dartboard 31, a part of a building such as the floor, wall, or ceiling around the dart game apparatus 30, or any structure installed around the dart game apparatus 30. Even with these structures other than the dartboard 31, the sound generated when the dart comes into contact with these structures may have a feature specific to each structure (for example, a feature in the frequency spectrum of the audio waveform).

[0028] The acceleration sensor 42 is configured to detect, for example, an impact (including vibration) caused by a thrown dart coming into contact with the dartboard 31 or another object, and to generate an impact waveform (vibration waveform) as a detection signal. In Fig. 1, the acceleration sensors 42 included in the dart game device 30 are shown as an acceleration sensor 42a provided on the left side of the dartboard 31 and an acceleration sensor 42b provided on the right side of the dartboard 31. However, the number of acceleration sensors 42 included in the dart game device 30 is not limited to this, and may be zero, one, or three or more.

[0029] The acceleration sensor 42 may detect, for example, an impact caused by a dart contacting the dartboard 31. The impact generated on the dartboard 31 reaches the acceleration sensor 42 via, for example, various components of the dart game device 30, and is then detected by the acceleration sensor 42. In this case, the acceleration sensor 42 may detect the impact regardless of whether the dart is held on the dartboard 31. As described above, each segment provided on the dartboard 31 is defined by the distance and direction from the center of the dartboard 31. Therefore, the impact caused by the dart contacting each segment may produce a feature unique to each segment (for example, a feature in the frequency spectrum of the impact waveform).

[0030] The acceleration sensor 42 may detect, for example, an impact caused by a dart coming into contact with any structure other than the dartboard 31. The impact caused by the structure other than the dartboard 31 reaches the acceleration sensor 42 via, for example, the dart game device 30 or various other structures, and is then detected by the acceleration sensor 42. The structure with which the dart comes into contact is not particularly limited, and may be, for example, a structure of the dart game device 30 other than the dartboard 31, a part of a building such as the floor, wall, or ceiling around the dart game device 30, or any structure installed around the dart game device 30. Even with these structures other than the dartboard 31, the impact caused by the dart coming into contact with these structures may produce a characteristic quantity specific to each structure (for example, a characteristic quantity in the frequency spectrum of the impact waveform).

[0031] The membrane switch 43 is configured to detect pressure on the dartboard 31 caused by, for example, a thrown dart coming into contact with the dartboard 31 (including when the dart is held), and generate an electrical signal as a detection signal. The method by which the membrane switch 43 detects pressure may include binary detection of whether the pressure is equal to or greater than a predetermined threshold. The membrane switch 43 may be configured, for example, by stacking a membrane sheet and multiple electrical contacts that become conductive when pressure is applied from the outside. For example, when a dart comes into contact with the dartboard 31, the electrical contacts become conductive and generate an electrical signal as a detection signal. The electrical contacts of the membrane switch 43 may be capable of generating, for example, an electrical signal specific to each contact. In this case, the throwing position of the dart can also be detected based on the electrical signal.

[0032] (1-2) Functional Configuration of the Control Device 44 The control device 44 has a processor 45 and a memory 46 and comprehensively controls various components of the darts game device 30. The processor 45 may be configured, for example, by a CPU (Central Processing Unit) and may execute programs stored in the memory 46. The memory 46 may have, for example, a ROM (Read Only Memory) storing various programs and a RAM (Random Access Memory) functioning as a data storage area and a processor work area. The programs may be stored in non-transitory recording media such as a USB memory or a CD-ROM. The programs may also be supplied to the memory 46 via any transmission medium capable of transmitting the programs (such as a communication network or broadcast waves). One aspect of the present invention may be realized in the form of a data signal embedded in a carrier wave, embodied by electronic transmission of the programs.

[0033] 4 is a functional block diagram showing an example of functional units included in the control device 44. The control device 44 includes, for example, a control unit 110 and a storage unit 120 as functional units realized by a processor 45, a memory 46, etc.

[0034] The storage unit 120 is a storage device that stores data necessary for processing of the darts game device 30. The storage unit 120 stores various programs, such as a program for progressing the darts game and a program for executing a method for the darts game device 30 to estimate a hit position. The storage unit 120 also stores a machine learning model 121 and score information 122 that indicates the score of each player in the throwing game.

[0035] The control unit 110 comprehensively controls various components of the darts game device 30 based on various programs stored in the storage unit 120. The control unit 110 has an imaging control unit 111, a detection control unit 112, a learning execution unit 113, an estimation result acquisition unit 114, and an output control unit 115.

[0036] The imaging control unit 111 controls imaging by the imaging device 39 by supplying a control signal to the imaging device 39, and generates image data based on imaging signals acquired from the imaging device 39. When the darts game apparatus 30 has multiple imaging devices 39, the imaging control unit 111 may generate image data for each imaging device 39, or may generate image data by combining imaging signals acquired from the multiple imaging devices 39. The imaging control unit 111 may extract a portion of the imaging signals acquired from the imaging device 39 to generate partial image data. The imaging control unit 111 may include time information indicating the time of imaging in the image data.

[0037] The detection control unit 112 controls detection by the detection device 40 by supplying control signals to the detection device 40, such as the microphone 41, acceleration sensor 42, and membrane switch 43, and generates detection data based on detection signals acquired from the detection device 40. For example, the detection control unit 112 may generate detection data of an audio waveform based on an audio waveform generated by the microphone 41. The detection control unit 112 may also generate detection data of an impact waveform based on an impact waveform generated by the acceleration sensor 42. The detection control unit 112 may also generate detection data of an electrical signal based on an electrical signal generated by the membrane switch 43. The detection control unit 112 may include time information indicating the time of detection in the detection data.

[0038] The learning execution unit 113 executes machine learning using predetermined training data to generate a machine learning model 121. Here, an overview of the machine learning model 121 according to this embodiment will be described with reference to Fig. 5. Fig. 5 is a diagram schematically illustrating an example of input and output data of the machine learning model 121.

[0039] The machine learning model 121 may be configured as, for example, a multimodal model, i.e., a model including two pieces of data in different representation formats (modals) as input data. The input data may include, for example, input data in a first representation format (first input data) and input data in a second representation format (second input data).

[0040] The representation format of the first input data (first representation format) may be an image. That is, the first input data may be, for example, image data generated by the imaging control unit 111 or image data obtained by performing predetermined preprocessing on the image data. In the upper left of Fig. 5, image data of an image of a dartboard DB and a dart D1 is shown as an example of the first input data.

[0041] The representation format of the second input data (second representation format) may be a representation format other than an image. In particular, the second input data may be, for example, detection data generated by the detection control unit 112 (detection data of audio waveforms, detection data of impact waveforms, detection data of electrical signals, etc.), or detection data obtained by performing predetermined preprocessing on the detection data. The lower left part of Fig. 5 shows, as an example of the second input data, audio waveform data when the dart D1 contacts (hits) the dartboard DB.

[0042] The output data of the machine learning model 121 includes an estimation result of the hit position. Here, the estimation result of the hit position may include, for example, information indicating at least one estimated hit position. The information indicating the hit position may be, for example, information indicating a segment on the dartboard or information indicating the coordinates of the hit position. The format of the coordinates is not particularly limited, and may be, for example, two-dimensional coordinates on image data as input data or three-dimensional coordinates in the space in which the dartboard is installed. The estimation result of the hit position may include a reliability for each hit position. A method for calculating the reliability is not particularly limited, and may, for example, employ a method using an activation function such as a sigmoid function or a softmax function. In other words, the reliability can be considered as the probability of occurrence of the hit position. On the right side of Figure 5, as an example of output data (estimated hit position), there is shown the hit position "I19" indicating the inner single segment of the number "19" where dart D1 hit in the dartboard DB, and the number "0.88" which is the reliability of the estimation result.

[0043] In this way, since the machine learning model 121 includes data in expression formats other than images in the input data, the accuracy of estimating the hit position can be improved compared to when the input data includes only data in the image expression format.

[0044] The machine learning model 121 may be configured as a model whose input data includes three or more pieces of data with different expression formats. In the machine learning model 121, for example, the first input data may be image data, the second input data may be any of the detected data (e.g., detected data of a sound waveform), and the input data (third input data) in a third expression format may be any of the other detected data (e.g., detected data of a shock waveform).

[0045] Returning to Fig. 4, the learning execution unit 113 generates a machine learning model 121 by executing machine learning using training data including the first input data and the second input data.

[0046] The first input data included in the training data may be image data generated by the imaging control unit 111, or may be image data acquired from another information processing device or the like via the communication device 32. The image data acquired from another information processing device or the like does not necessarily have to be image data generated by imaging the dartboard 31, but may be image data generated by imaging another dartboard of the same or a different type, or image data generated by predetermined computer graphics (CG), or the like. The first input data such as image data included in the training data may include, as a label, information indicating the position of the dart on the dartboard where the dart hit. The information may be represented, for example, by the segment on the dartboard where the dart hit, or by the coordinates of the position of the dart hit.

[0047] The second input data included in the training data may be detection data generated by the detection control unit 112, or may be detection data acquired from another information processing device or the like via the communication device 32. The detection data acquired from another information processing device or the like does not necessarily have to be detection data generated based on a detection signal detected when a dart is thrown at the dartboard 31, but may be detection data generated using another dartboard or dart of the same or a different type, or detection data generated by a predetermined simulation. The second input data such as the detection data included in the training data may include, as a label, information indicating the hit position of the dart on the dartboard. The information may be represented, for example, by the segment on the dartboard where the dart hit, or by the coordinates of the hit position of the dart.

[0048] The machine learning model 121 can be configured arbitrarily using well-known techniques such as a deep Boltzmann machine or a method employing multiple classifiers. Specifically, the machine learning model 121 may be configured to include, for example, a first neural network corresponding to first input data such as image data, a second neural network corresponding to second input data such as detection data, and an integration layer that integrates the output of the first neural network and the output of the second neural network.

[0049] The estimation result acquisition unit 114 estimates the hit position using the machine learning model 121. Specifically, the estimation result acquisition unit 114, for example, inputs predetermined input data to the machine learning model 121 and acquires the estimated result of the hit position that is output thereby. The input data is configured according to the configuration of the machine learning model 121 described above. That is, the input data may be configured to include two pieces of data with different expression formats (modals). The input data may include, for example, input data in a first expression format (first input data) and input data in a second expression format (second input data).

[0050] The representation format of the first input data (first representation format) may be, for example, an image. That is, the first input data may be, for example, image data generated by the imaging control unit 111, or image data obtained by performing predetermined preprocessing on the image data. Furthermore, the representation format of the second input data (second representation format) may be, for example, a representation format other than an image. In particular, the second input data may be, for example, detection data generated by the detection control unit 112 (detection data of audio waveforms, detection data of shock waveforms, detection data of electrical signals, etc.), or detection data obtained by performing predetermined preprocessing on the detection data.

[0051] The input data may also be configured to include three or more pieces of data with different representation formats. For example, the first input data may be image data, the second input data may be any of the detected data (e.g., detected data of a sound waveform), and the input data in a third representation format (third input data) may be any of the other detected data (e.g., detected data of a shock waveform).

[0052] The estimation result acquisition unit 114 may generate input data based on, for example, image data generated by the imaging control unit 111 and detection data generated by the detection control unit 112. The estimation result acquisition unit 114 may compare time information included in the image data and the detection data, and if the difference in the time information is less than a predetermined threshold, may associate the image data and the detection data with each other to form a set that constitutes the input data. The estimation result acquisition unit 114 may, for example, input the input data to the machine learning model 121, and thereby acquire an estimation result of the hit position output by the machine learning model 121.

[0053] The output control unit 115 controls output by various output devices. The output control unit 115 may, for example, function as a display control unit that controls display by the display device 34. Specifically, the output control unit 115 generates display data for a predetermined screen to be displayed on the display device 34, and causes the display device 34 to display the predetermined screen based on the display data. The screen displayed by the display device 34 may include, for example, an estimation result (numerical values, text, etc.) output by the machine learning model 121. The screen displayed by the display device 34 may also include an image of the dartboard 31, and the estimation result of the hit position may be superimposed on the image. The image of the dartboard 31 may be an image generated by capturing an image of the dartboard 31 based on image data input to the machine learning model 121, or may be a schematic image of the dartboard 31 generated by CG or the like.

[0054] The output control unit 115 may have a function as, for example, an audio output control unit that controls audio output by the audio output device 35. Specifically, the output control unit 115 generates predetermined audio data to be output by the audio output device 35, and causes the audio output device 35 to output predetermined audio based on the audio data. The audio output by the audio output device 35 may include, for example, a voice reading out the estimation result (numerical values, text, etc.) output by the machine learning model 121, or sound effects corresponding to the estimation result.

[0055] The output control unit 115 may, for example, calculate the points earned by the player through the throw based on the estimation results output by the machine learning model 121, and register the points in the score information 122 stored in the memory unit 120.

[0056] (2) Operation (2-1) Machine Learning Model Generation Method Fig. 6 is an operational flow diagram for explaining an example of a method for generating the machine learning model 121. Note that some steps may be executed in parallel or in a different order. In the following, an example will be described in which the first input data is image data and the second input data is detection data.

[0057] (S101) The learning execution unit 113 acquires image data and stores it in the storage unit 120. The image data may be image data generated by the imaging control unit 111, and may include time information when the imaging control unit 111 generated the image data. Alternatively, the image data may be image data acquired from another information processing device or the like via the communication device 32. The image data acquired from another information processing device or the like does not necessarily have to be image data generated by imaging the dartboard 31, but may be image data generated by imaging another dartboard of the same or a different type, or image data generated by predetermined computer graphics (CG) or the like.

[0058] (S102) The learning execution unit 113 acquires the detection data and stores it in the storage unit 120. The detection data may be detection data generated by the detection control unit 112, and may include time information when the detection control unit 112 generated the detection data. Alternatively, the detection data may be detection data acquired from another information processing device or the like via the communication device 32. The detection data acquired from another information processing device or the like does not necessarily have to be detection data generated based on a detection signal detected when a dart is thrown at the dartboard 31, but may be detection data generated using another dartboard or dart of the same or a different type, or detection data generated by a predetermined simulation.

[0059] (S103) The learning execution unit 113 generates teacher data based on the multiple image data and multiple detection data stored in the storage unit 120. For example, if the image data and the detection data are not associated with each other, the learning execution unit 113 may compare time information included in the image data and the detection data, and associate the image data and the detection data with each other if the difference in the time information is less than a predetermined threshold. Furthermore, the learning execution unit 113 acquires labels indicating the hit positions (hit segments or coordinates) and assigns the labels to the image data, the detection data, the teacher data, etc. according to the configuration of the machine learning model 121. The labels may be information input by a player, an administrator, etc. using the operation device 33 of the darts game device 30 or an operation device of another information processing device connected via the communication device 32.

[0060] (S104) The learning execution unit 113 executes machine learning based on the training data generated in step S103 to generate a machine learning model 121, and stores the model in the storage unit 120.

[0061] (2-2) Hit Position Estimation Method Fig. 7 is an operational flow diagram for explaining an example of a hit position estimation method. Note that some steps may be executed in parallel or in a different order. In the following, an example will be described in which the first input data is image data and the second input data is detection data.

[0062] (S201) The imaging control unit 111 controls the imaging device 39 to generate image data based on an imaging signal acquired from the imaging device 39. The timing of generating the image data is not particularly limited, but may be, for example, when a command to generate image data is acquired via the communication device 32 or the like. The imaging control unit 111 includes time information indicating the time of imaging in the image data.

[0063] (S202) The detection control unit 112 controls the detection device 40 to generate detection data based on the detection signal acquired from the detection device 40. The timing for generating the detection data is not particularly limited, but may be, for example, when a command to generate the detection data is acquired via the communication device 32 or the like. The detection control unit 112 includes time information indicating the time of detection in the detection data.

[0064] (S203) The estimation result acquisition unit 114 generates input data based on the image data and the detection data. The estimation result acquisition unit 114 may compare time information included in the image data and the detection data, and if the difference between the time information is less than a predetermined threshold, may associate the image data and the detection data with each other to form a set that constitutes the input data.

[0065] (S204) The estimation result acquisition unit 114 may, for example, input input data to the machine learning model 121, thereby acquiring an estimation result of the hit position output by the machine learning model 121. The estimation result of the hit position may, for example, include information indicating at least one estimated hit position. The information indicating the hit position may, for example, be information indicating a segment provided on the dartboard, or information indicating the coordinates of the hit position. The format of the coordinates is not particularly limited. Furthermore, the estimation result of the hit position may include a probability (reliability) for each hit position.

[0066] (S205) The output control unit 115 controls various output devices to output the estimation results. For example, the output control unit 115 generates display data for a predetermined screen to be displayed on the display device 34, and causes the display device 34 to display the predetermined screen based on the display data.

[0067] FIG. 8 is a diagram showing an example of a screen 200 displayed on the display device 34. The screen 200 includes a schematic image 201 of a dartboard and a display section 203 showing the estimated hit position. The image 201 is generated, for example, by CG or the like. The hit position of the dart may be displayed in an emphasized manner in the image 201. In the example shown in FIG. 9, the outer single segment of the number 10 is shaded. The display section 203 of the estimated result includes, as the estimation result, text indicating the hit position, such as "Hit position: outer single (10)," and text indicating the reliability, such as "Confidence: 92%." The screen displayed on the display device 34 may also include an image generated by capturing an image of the dartboard 31 based on image data included in the input data for the machine learning model 121.

[0068] The output control unit 115 may, for example, generate predetermined audio data and, based on the audio data, cause the audio output device 35 to output audio such as a voice reading out the estimation results (numerical values, text, etc.) output by the machine learning model 121, or sound effects corresponding to the estimation results.

[0069] As described above, according to this embodiment, not only image data but also detection data is used as input data in estimating the dart hit position using the machine learning model 121. This allows for more accurate position estimation than when only images are used as input data.

[0070] (3) Additional Notes [Additional Note 1] The input data of the machine learning model 121 may not include image data, but may include at least one of the detection data.

[0071] For example, the input data of the machine learning model 121 may be audio waveform data. That is, the machine learning model 121 may be configured to output an estimation result of the position on the dartboard where the dart hits the target when the input data is a sound generated by a dart thrown in a dart game hitting a dartboard or another object.

[0072] As described above, each segment on the dartboard is defined by its distance and direction from the center of the dartboard. Therefore, the sound generated when a dart contacts each segment may contain features specific to that segment (e.g., features in the frequency spectrum of the audio waveform). Furthermore, even for any configuration other than the dartboard, the sound generated when a dart contacts the configuration may contain features specific to that configuration (e.g., features in the frequency spectrum of the audio waveform). Therefore, the machine learning model 121 can output a hit position estimation result with sufficient accuracy even when the input data does not include image data but includes audio waveform data.

[0073] [Additional Note 2] The darts game device 30 may determine whether or not the reliability included in the hit position estimation result is equal to or greater than a predetermined threshold, and then execute various processes according to the determination result.

[0074] The output control unit 115 may cause the output device to output predetermined information when the reliability is determined to be less than a predetermined threshold. For example, the output control unit 115 may cause the display device 34 to display a screen 300 shown in FIG. 9 . Here, the screen 300 includes the schematic image 201 of the dartboard and the display unit 203 showing the estimated hit position, as well as predetermined information 301 that is displayed when the reliability is determined to be less than a predetermined threshold, as shown in the screen 200 shown in FIG. 8 . The information 301 includes text such as "Please visually check the actual dart position and re-input the hit position." This is an example of information that prompts the player to input the actual dart position themselves. For example, after viewing this text, the player can visually check the actual hit position of the dartboard 31, and then directly input the actual hit position and the score based thereon by operating the operation device 33 or an operation device of another information processing device connected to the dart game device 30 via the communication device 32. The information indicating the actual hit position and score that was input may be acquired, for example, by the control unit 110, and the control unit 110 may then update the score information 122 stored in the memory unit 120 based on that information.

[0075] In this way, when the reliability is determined to be less than the predetermined threshold, the output control unit 115 may cause the display device 34 to display information prompting the user to input the actual hit position of the dart. Note that when the reliability is determined to be less than the predetermined threshold, the information displayed by the output control unit 115 on the display device 34 is not limited to information prompting the user to input the hit position, but may also include information directly or indirectly indicating that the reliability of the estimated hit position is low, images for predetermined effects, etc.

[0076] Furthermore, when it is determined that the reliability is less than a predetermined threshold, the output control unit 115 may cause the audio output device 35 to output a predetermined sound. The sound may be, for example, a sound that reads out information urging the player to input the actual hit position of the dart, a sound that reads out information that directly or indirectly indicates that the reliability of the estimated hit position is low, or a sound effect for a predetermined effect.

[0077] [Additional Note 3] The darts game device 30 may determine whether or not the reliability included in the estimation result of the hit position is equal to or greater than a predetermined threshold, and then execute various processes according to the determination result.

[0078] If the reliability is determined to be less than a predetermined threshold, the imaging control unit 111 may generate image data (second image data) again. The imaging area of ​​the second image data may be the same as or different from the imaging area of ​​the image data (first image data) included in the input data on which the determination was based, and specifically may be an area including the entire dartboard 31 or an area including at least a part of the dartboard 31.

[0079] The second image data may be generated by capturing an image. For example, the imaging control unit 111 may control the imaging device 39 to acquire an imaging signal having the same imaging area as the first image data, and generate the second image data based on the acquired imaging signal. Alternatively, for example, the imaging control unit 111 may control the imaging device 39 to change the viewpoint or zoom magnification, acquire an imaging signal, and generate the second image data having an imaging area different from the previous image data based on the acquired imaging signal. Specifically, the imaging control unit 111 may increase the magnification of the imaging device 39 to generate image data that enlarges a portion of the previous image data, or may decrease the magnification of the imaging device 39 to generate second image data with a wider angle than the previous image data. In particular, when increasing the magnification of the imaging device 39, the imaging area of ​​the imaging device 39 may be adjusted so that the hit position estimated by the machine learning model 121 is closer to the center of the image.

[0080] The second image data may be generated based on the first image data. For example, the control unit 110 may generate the second image data by performing various processing operations on the first image data, such as image quality adjustment and enlargement / reduction. In this case, the area of ​​the second image data may be the same as the area of ​​the first image data, or may be different from the area of ​​the first image data. In particular, the area of ​​the second image data may include at least a portion of the dartboard 31.

[0081] When the second image data is generated, the learning execution unit 113 may update the image data included in the teacher data to the second image data, and then execute machine learning again to update the machine learning model 121. Then, the estimation result acquisition unit 114 may use the updated machine learning model 121 to acquire the estimation result again, and the output control unit 115 may output the acquired estimation result to an output device.

[0082] [Additional Note 4] Some of the components of the darts game device 30 described above may be included in another information processing device configured by a computer including a processor and a memory, such as a smartphone or a tablet terminal. In this case, the darts game device 30 may transmit and receive information to the other information processing device via a communication device 32 or the like. The other information processing device may include at least some of the components of the display device 34, the audio output device 35, the imaging device 39, the detection device 40, etc. Furthermore, the other information processing device may include at least some of the functional units, such as the imaging control unit 111, the detection control unit 112, the learning execution unit 113, the estimation result acquisition unit 114, and the output control unit 115. Furthermore, the other information processing device may store at least some of the machine learning model 121, the score information 122, etc.

[0083] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention.

[0084] 30...dart game device, 31...dart board, 32...communication device, 33...operation device, 34...display device, 35...audio output device, 36...card slot, 37...card RW section, 38...coin insertion section, 39...imaging device, 40...detection device, 41, 41a, 41b...microphone, 42, 42a, 42b...acceleration sensor, 43...membrane switch, 44...control device, 45...processor, 46...memory, 110...control section, 111...imaging control section, 112...detection control section, 113...learning execution section, 114...estimation result acquisition section, 115...output control section, 120...storage section, 121...machine learning model, 122...score information

Claims

1. A hit position estimation system comprising: a memory unit that stores a machine-learned model configured to output an estimation result of the position at which the throwing instrument hits the target when image data generated based on imaging the target when thrown with a throwing instrument in a throwing game and detection data generated based on detecting the thrown throwing instrument contacting the target or another object are input; an imaging unit that generates image data based on imaging the target when thrown with the throwing instrument in a throwing game; a detection unit that generates detection data based on detecting the thrown throwing instrument contacting the target or another object; an acquisition unit that acquires the estimation result of the position at which the throwing instrument hits the target, which is output by inputting the image data and the detection data into the machine learning model; and a display unit that displays the estimation result.

2. A hit position estimation system as described in claim 1, wherein the detection unit detects the sound generated when the thrown throwing implement comes into contact with the target or the other object.

3. A hit position estimation system as described in claim 1, wherein the detection unit detects pressure on the target generated when the thrown throwing implement comes into contact with the target.

4. A hit position estimation system as described in claim 1, wherein the detection unit detects an impact caused by the thrown throwing implement coming into contact with the target or the other object.

5. The hit position estimation system of claim 1, wherein the machine learning model is configured to further output a reliability of the estimation result when the image data and the detection data are input, the acquisition unit further acquires the reliability output by inputting the image data and the detection data into the machine learning model, and the display unit controls the display content of the estimation result based on the reliability.

6. The hit position estimation system according to claim 5, wherein the display unit further displays the reliability.

7. A hit position estimation system as described in claim 5, wherein the display unit displays information to prompt the user to input the result of the throw in the throwing game if the reliability is less than a predetermined threshold.

8. The hit position estimation system described in claim 5, wherein the imaging unit generates second image data including at least a portion of the target when the throwing instrument is thrown in a throwing game if the reliability is less than a predetermined threshold, the acquisition unit acquires a second estimation result of the position at which the throwing instrument hits the target, which is output by inputting at least the second image data into the machine learning model, and the display unit displays the second estimation result.

9. A hit position estimation method in which an information processing device that can access a memory unit that stores a machine-learned machine learning model configured to output an estimation result of the position at which the throwing tool hits the target when image data generated based on capturing an image of the target when the throwing tool is thrown in a throwing game and detection data generated based on detecting that the thrown throwing tool contacts the target or another object are input, the information processing device performing the steps of: generating image data based on capturing an image of the target when the throwing tool is thrown in a throwing game, by controlling an imaging device; generating detection data based on detecting that the thrown throwing tool contacts the target or another object by controlling a detection device; obtaining an estimation result of the position at which the throwing tool hits the target, which is output by inputting the image data and the detection data into the machine learning model; and displaying the estimation result on a display device.

10. A program for causing an information processing device that can access a memory unit that stores a machine-learned model configured to output an estimation result of the position at which a throwing tool hits a target when image data generated based on capturing an image of the target when the throwing tool is thrown in a throwing game and detection data generated based on detecting that the thrown throwing tool contacts the target or another object, to function as: an imaging control unit that controls an imaging device to generate image data based on capturing an image of the target when the throwing tool is thrown in a throwing game; a detection control unit that controls a detection device to generate detection data based on detecting that the thrown throwing tool contacts the target or another object; an acquisition unit that acquires the estimation result of the position at which the throwing tool hits the target, which is output by inputting the image data and the detection data into the machine learning model; and a display control unit that displays the estimation result on a display device.

11. A hit position estimation system comprising: a memory unit that stores a machine-learned model configured to output an estimation result of the position at which the throwing instrument hits the target when a sound made by a throwing instrument thrown in a throwing game contacting a target or other object is input; a detection unit that detects the sound made by the throwing instrument thrown in a throwing game contacting the target or other object; an acquisition unit that acquires the estimation result of the position at which the throwing instrument hits the target, which is output by inputting the sound into the machine learning model; and a display unit that displays the estimation result.

12. The target position estimation system according to claim 11, wherein the detection unit is disposed at a predetermined position relative to the target, and the target has a plurality of segments defined by distances and directions from the center of the target.