Methods and musical instrument systems

Abstract

A method for detecting an object 114, such as a hand or fingers, in the vicinity of a musical instrument 102, which may be an electronic keyboard, the method comprising: receiving first data associate

Description

Technical Field The present application relates to methods and systems for detecting objects in the vicinity of a musical instrument using sensor data. Background Object detection models, particularly those leveraging neural networks, have become increasingly effective in identifying the presence or absence of objects in images or video streams. These models are commonly used for tasks such as detecting hands, faces, or other objects within a scene. Such models operate by generating probability scores or "confidence scores” for detected objects. When the confidence score surpasses a predefined confidence threshold, the model concludes that an object is present. Current object detection models face several challenges, for example how to select appropriate confidence thresholds. Setting the threshold too low can result in false positives, where objects are incorrectly identified as being present. Conversely, setting the threshold too high may lead to false negatives, where objects that are actually present are not detected. Selecting an appropriate confidence threshold remains an ongoing challenge. Summary The present disclosure relates to a method of detecting, via an object detection model (such as a model that utilises a neural network), whether an object, such as a user’s hand, is present in first sensor data obtained by a first sensor. For example, a camera or other imaging device may capture one or more images so that the presence of a hand can be detected in the image(s). Such a method may be useful in tracking the presence and / or position of a hand playing a musical instrument, such as various keyboard instruments, including an electronic keyboard, an acoustic piano, a digital piano and a synthesiser. The object detection model may use a confidence level requirement in the form of a threshold to determine whether the object is present. For example, an object in the first sensor data may be given a confidence score indicating the likelihood of the object being the object (i.e., a hand). If the confidence score is above the confidence threshold, the object may be determined to be present (for example, the object may be classified as the object). Methods described herein involve dynamically adjusting the confidence threshold using ground truth data, where the ground truth data is a secondary source of data indicating whether the object is present or not. For example, the ground truth data may be second sensor data obtained by a sensor on the musical instrument itself. For example, if a user plays a musical instrument (such as presses a key on a keyboard), this information can be used to adjust the confidence threshold to increase the likelihood of correctly identifying whether the object is present in future. As a first example, it may be determined, based on the first sensor data from the first sensor, that a user’s hand is not present, despite the second sensor data indicating that the musical instrument has been actuated / played. This is a false negative (a hand should have been detected in the first sensor data, but it was not) and could mean that the confidence threshold is currently set too high. To ensure that the hand is correctly detected in future sensor data from the first sensor, the confidence threshold may be lowered one or more times until the hand is correctly identified as being present. In a second example, it may be determined, based on the first sensor data from the first sensor, that a user’s hand is present, despite the second sensor data indicating that the musical instrument has not been actuated / played, or despite the second sensor data not been received or generated. This is a false positive (a false hand was detected in the first sensor data, but it should not have been), and could mean that the confidence threshold is currently set too low. To ensure that the false hand is not detected in future sensor data from the first sensor, the confidence threshold may be increased / raised one or more times until the hand is no longer identified as being present. Accordingly, in a first aspect of the present disclosure there is provided a method, comprising: (i) receiving first sensor data associated with a first sensor, the first sensor positioned to obtain sensor data from an area encompassing at least a portion of a musical instrument, (ii) determining, using an object detection model, a confidence score that the first sensor data comprises sensor data indicative of an object, (iii) determining, based on a comparison of the confidence score with a confidence threshold, whether the object is present, (iv) determining, based on second sensor data associated with a second sensor, whether the musical instrument has been played, and (v) adjusting the confidence threshold based on: (i) whether the object is determined to be present, and (ii) whether the musical instrument has been played. The method can therefore minimise false positives and / or false negatives. In addition, because the accuracy can be increased by adjusting the confidence threshold, the method does not require retraining of the object detection model. For example, if the object detection model involves use of a neural network, the neural network may not need retraining, such as via reinforcement learning. This has two distinct advantages, the first being that additional training time is not needed, reducing resource consumption. The second is a significantly reduced risk of incorrect inferences made by the neural network. Because the second sensor data is “ground truth” data, the result is a correction of incorrect assumptions made by the object detection model (such as a neural network), whereas with a model using reinforcement learning, if the additional input data provided is incorrect, or purely based on human preference, this can risk “reinforcing” incorrect output data. In examples, the object is a hand or one or more features associated with a hand, such as one or more fingers, or one or more stickers or rings specifically placed on the hand or fingers. In other examples, the object is a drum stick, plectrum, etc, or another part that is used to play an instrument. The object is therefore an object for playing / actuating the musical instrument, and may be a part of a human that is playing the musical instrument or a non-human part. In examples, the first sensor is part of the instrument. In other examples, the first sensor is a separate sensor, such as a sensor of a mobile device (e.g. smartphone). In examples, the first sensor is an imaging device configured to detect electromagnetic radiation of one or more particular wavelengths, such as visible light, infrared, ultraviolet, etc. The musical instrument may be a keyboard, a piano, a guitar or a drum, in examples. The “playing” of the musical instrument may include a button / key press, a button / key depress (i.e., letting go), a strumming, a hitting, a shaking, etc. In examples, “played” may mean that the instrument has been actuated. In examples, the musical instrument comprises the second sensor. For example, the second sensor may be a sensor associated with a key of a keyboard or piano, or it may be a sensor placed on the musical instrument, such as retrofitted to the instrument (for example, a sensor may be fitted underneath a key of a keyboard or piano). The confidence threshold may be a confidence threshold of the object detection model. The object detection model may be a machine learned object detection model and may have been trained using training data, such as a set of one or more training images. In examples, sensor data associated with a sensor may mean that the sensor data has been obtained by the sensor. The first sensor data may be raw sensor data obtained / captured by the first sensor (so the sensor data obtained by the first sensor may be the first sensor data) or the first sensor data may be processed sensor data (so the sensor data obtained by the first sensor may be processed to generate the first sensor data). As an example, one or more filters may be applied to the raw sensor data, such as a distortion correction filter to generate processed sensor data. In additional or alternative examples, one or more masks may be applied, to mask out one or more parts of an image obtained by the first sensor. Other examples are also possible. Similarly, the second sensor data may be raw sensor data obtained / captured by the second sensor or may be processed sensor data. In one example, raw sensor data may be second sensor data indicative of a pressure applied to the second sensor. Raw sensor data from the second sensor may be more useful than processed sensor data, such as a MIDI signal, because a MIDI signal may be generated at a certain point throughout a keystroke depending on the instrument preset selected, while with the raw sensor data, the sensor data could be received as soon as a user begins to press the key. The first sensor data may be received from the first sensor or it may be received from another entity. Similarly, the second sensor data may be received from the second sensor or it may be received from another entity. The first sensor data may be an image, in one example. The confidence score may be a value within a range of values, such as a value within a range of 0 to 100, 0 to 1, or hexadecimal values etc. The confidence score may be expressed as a percentage. The confidence threshold may be a particular value within the range of values, and may also be expressed as a percentage, in some examples. The second sensor data may be a signal. The second sensor data may be sensor data indicative of a musical note, such as a MIDI interface signal. The method may be a computer implemented method, and therefore be implemented by one or more processors, for example. The method steps may be performed by one or more local processors, and / or one or more remote processors. In one example, the method is performed by one or more processors of the musical instrument. In another example, the method is performed by one or more processors of a computing device communicatively coupled to the musical instrument. In one example, the computing device is a mobile device (e.g. smartphone). In a particular example, the mobile device comprises the first sensor. In examples, method may further comprise determining, based on the first sensor data, a location of the object and / or determining, based on the second sensor data, a location on the musical instrument that has been played. Once adjusted, the confidence threshold may be applied to future first sensor data to detect the presence of the object. Accordingly, in examples, the method further comprises receiving further first sensor data associated with the first sensor, determining, using the object detection model, a confidence score that the further first sensor data comprises sensor data indicative of the object, and determining, based on a comparison of the confidence score with the adjusted confidence threshold, whether the object is present. Adjusting the confidence threshold can improve the accuracy of correctly identifying the presence or absence of an object in sensor data received at a later time. The further first sensor data is sensor data associated with the first sensor that has been obtained at a time later than when the first sensor data was obtained. For example, it may be a later frame associated with video data obtained by the first sensor. In examples, the method may further comprise: (A) determining, based on further second sensor data associated with the second sensor, whether the musical instrument has been played and (B) further adjusting the confidence threshold based on: (i) whether the object is determined to be present, and (ii) whether the musical instrument has been played. Thus, the method may be repeated based on further sensor data. In examples, time delays after raising or lowering the confidence threshold could be implemented to prevent the confidence threshold from changing again too quickly, overshooting the required value, or affecting system performance. Accordingly, in examples, the method further comprises determining, based on further second sensor data associated with the second sensor, whether the musical instrument has been played, and after adjusting the confidence threshold, waiting a predetermined period of time before further adjusting the confidence threshold based on: (i) whether the object is determined to be present, and (ii) whether the musical instrument has been played. In examples where the confidence threshold is further adjusted, the further adjustment is based on the further first sensor data and the further second sensor data. In an example, the period of time may be 0.1 seconds, 0.5 seconds, 1 second, etc. As mentioned, in examples, adjusting the confidence threshold comprises increasing the confidence threshold in the event that: (i) the object is determined to be present, and (ii) the musical instrument is determined not to have been played. This results in the object being less likely to be detected in future, thereby avoiding false positives (i.e., the object is incorrectly identified in the first sensor data, when the second sensor data indicates that the musical instrument has not been played). In examples, the process of increasing the confidence threshold may be repeated until the object is no longer determined to be present. In some examples, an absence of the second sensor data may be indicative of the musical instrument having not been played. For example, the second sensor data may only be received and / or generated when the musical instrument is played. In other examples, the second sensor data may be received and / or generated regardless of whether the musical instrument is played, and so the musical instrument may be determined not to have been played based on an analysis of the second sensor data (for example, one or more characteristics or signatures may not be present in the second sensor data). Accordingly, in examples, the method further comprises determining that the second sensor data associated with the second sensor has not been received within a threshold period of time, and responsively determining that the musical instrument has not been played. As mentioned, the absence of the second sensor data being used to infer that the musical instrument has not been played can avoid having to analyse the sensor data, which may consume processing resources. In such cases, the second sensor data is therefore not indicative of the musical instrument having been played. In alternative examples, the method comprises receiving the second sensor data associated with the second sensor, and determining that the musical instrument has not been played based on an analysis of the second sensor data. Analysing the second sensor data may be a faster method of determining that the musical instrument has not been played (for example, it may avoid having to wait for the threshold period of time to pass). In such cases, the second sensor data is therefore not indicative of the musical instrument having been played. In examples, analysis of the second sensor data may comprise determining whether the second sensor data comprises data indicative of the musical instrument having been played (such as a MIDI interface signal) or may comprise determining whether the second sensor data comprises one or more characteristics or signatures indicative of the musical instrument having been played. Accordingly, determining that the musical instrument has not been played may comprise determining that the second sensor data does not comprise data indicative of the musical instrument having been played (such as a MIDI interface signal) or may comprise determining that the second sensor data does not comprise one or more characteristics or signatures indicative of the musical instrument having been played. In some examples, to make the object detection more robust, it may be possible to operate the system with separate “initialisation” and “kill” thresholds (referred to herein as “the second threshold” and “the threshold”, respectively). This would be the case where, for example, one neural network (e.g. object detection model) governs which objects are initially identified using an initialisation threshold, and then passes the first sensor data onto another network / model which decides when the objects should no longer be detected, using a kill threshold. It may therefore be possible to raise the kill threshold to remove the incorrect object (false positives), and lower the initialisation threshold to initialise the correct object (false negatives). This is instead of raising a single confidence threshold to remove the incorrect hand, and then afterward lowering it again, to try and capture the correct one. The second threshold may be used by one object detection model (or network) to decide or detect whether objects are present within the scene, and the (first) threshold can be used by another object detection model (or network), or in some cases the same model / network, to track objects and their movements across the scene once they have been detected. Here, an initialization / second threshold describes the confidence level above which an object is considered to have been detected, which is then passed onto a further detection model for tracking. A kill / first threshold describes the confidence level below which an object will no longer be considered to be as it was originally detected, and will no longer be tracked. Accordingly, in examples, the method may further comprise, prior to (i) determining, using the object detection model, the confidence score that the first sensor data comprises sensor data indicative of the object, and (ii) determining, based on the comparison of the confidence score with the confidence threshold, whether the object is present: (A) determining, using the object detection model or a second object detection model, a second confidence score that the first sensor data comprises sensor data indicative of the object, and determining, based on a comparison of the second confidence score with a second confidence threshold, that the object is present. In this example, the (first) object detection model is the model that uses the (first) threshold (also known as the kill threshold), and the second object detection model is the model that uses the second threshold (also known as the initialisation threshold). In examples, the steps of “(i) determining, using the object detection model, the confidence score that the first sensor data comprises sensor data indicative of the object, and (ii) determining, based on the comparison of the confidence score with the confidence threshold, whether the object is present” are performed responsive to determining, based on the comparison of the second confidence score with the second confidence threshold, that the object is present. In other words, such steps may only be performed when the object is determined to be present according to the second confidence score and second confidence threshold. In examples, the object detection model associated with the kill / first threshold may determine one or more components or features of the object, such as a finger, and / or may determine the location, velocity and / or orientation of the object. In examples, the method further comprises: raising the second confidence threshold based on a difference between the second confidence threshold and the second confidence score. Raising the second confidence threshold if the second confidence threshold is significantly below the second confidence score can prevent the second threshold from being too low (for example, it may have been lowered previously). In other examples, however, there may only be a single threshold and in some cases a single object detection model. As discussed above, in examples where the object is present and the musical instrument has not been played, the confidence threshold may be increased / raised. In other examples, however, the threshold may be lowered when the object is determined to not be present and when the instrument has been played. Accordingly, in examples, adjusting the confidence threshold comprises lowering the confidence threshold in the event that: (i) the object is determined not to be present, and (ii) the musical instrument has been played. This results in the object being more likely to be detected in future, thereby avoiding false negatives (i.e., the object is not identified in the first sensor data, but the second sensor data indicates that the musical instrument has been played, so the object should have been identified in the first sensor data). The process of lowering the confidence threshold may be repeated until the object is determined to be present, in some cases. In examples, the method comprises receiving the second sensor data associated with the second sensor, and determining that the musical instrument has been played by having received the second sensor data. The presence of the second sensor data being used to infer that the musical instrument has been played can avoid having to analyse the sensor data, which may consume processing resources. In such cases, the receipt of the second sensor data is therefore indicative of the musical instrument having been played. In alternative examples, the method comprises receiving the second sensor data associated with the second sensor and determining that the musical instrument has been played based on an analysis of the second sensor data. Analysing the second sensor data may allow further information about the signal to be determined, such as the location of the actuation and / or the musical note being played. In such cases, the second sensor data is therefore indicative of the musical instrument having been played. In examples, analysis of the second sensor data may comprise determining whether the second sensor data comprises data indicative of the musical instrument having been played (such as a MIDI interface signal) or may comprise determining whether the second sensor data comprises one or more characteristics or signatures indicative of the musical instrument having been played. Accordingly, determining that the musical instrument has been played may comprise determining that the second sensor data comprises data indicative of the musical instrument having been played (such as a MIDI interface signal) or may comprise determining that the second sensor data comprises one or more characteristics or signatures indicative of the musical instrument having been played. In some cases, an object may be detected but it is located in a position that is far away from where the instrument was played. The true object that is playing the instrument is therefore not being detected, as would be expected. To ensure the detection of the true object, the threshold may be increased. Accordingly, in examples, the method further comprises determining, based on the second sensor data, a location on the musical instrument that has been played, determining, based on the first sensor data, a location of the object, and determining, based on the location on the musical instrument that has been played and the location of the object, whether the location of the object is misaligned with the location on the musical instrument that has been played. Adjusting the confidence threshold therefore comprises increasing the confidence threshold in the event that: (i) the object is determined to be present, (ii) the musical instrument has been played, and (iii) the location of the object is misaligned with the location on the musical instrument. Increasing the confidence threshold therefore avoids erroneously detecting objects that are located relatively far from the location on the instrument that has been played. The object being detected is therefore not the correct one. In examples, determining whether the location of the object is misaligned with the location on the musical instrument may comprise comparing the location on the musical instrument that has been played to the location of the object. Comparing the locations may comprise determining a difference / distance between the locations. The locations may be defined in the same coordinate system or in different coordinate systems. The locations may be misaligned when a distance between the locations is greater than a distance threshold or when the locations do not coincide with each other. In examples, the location on the musical instrument may be associated with a musical note, such as a particular key on a keyboard. The location may be determined based on a MIDI signal, for example. The location of the object may be a location of the object relative to the musical instrument, in some examples. In examples, the method may first comprise: (i) receiving the second sensor data associated with the second sensor, and (ii) determining that the musical instrument has been played based on the second sensor data. In examples, after raising / increasing the confidence threshold, the true object may be detected in future first sensor data at the correct location (in other words, at a location that coincides with the location of the musical instrument that has been played). Accordingly, the method may further comprise: receiving further first sensor data associated with the first sensor, determining, using the object detection model, a second confidence score that the further first sensor data comprises sensor data indicative of the object, determining, based on a comparison of the second confidence score with the confidence threshold, that the object is present, determining, based on the further first sensor data, a location of the object, and determining, based on the location on the musical instrument and the location of the object, that the location of the object is aligned with the location on the musical instrument. An object may therefore be detected in the correct location within the further first sensor data, and so such a detection does not result in the confidence threshold being adjusted. In examples, the locations may be aligned when a distance between the locations is less than a distance threshold or when the locations coincide with each other. The further first sensor data is sensor data associated with the first sensor that has been obtained at a time later than when the first sensor data was obtained. For example, it may be a later frame associated with video data obtained by the first sensor. In one example, the location on the musical instrument may be determined based on further second sensor data associated with the second sensor (i.e., received at a later time than when the second sensor data was received / obtained). In some examples, a notification may be issued to a user to alert them that an erroneous object has been detected. For example, a mobile phone or other object may be located near to the instrument, which is being detected. The notification may alert the user to alter the environment, such as to make sure other objects (such as the mobile phone) are out of the field of view of the first sensor or to adjust the lighting in the room. Accordingly, in examples, the method may further comprise generating a user notification in the event that the location of the object is misaligned with the location on the musical instrument. The notification may be a text alert displayed on a display, or may be a sound or may be a haptic alert, or a combination, of any or all of these. In some cases, the adjustment of the threshold may be based on the location of the object. Accordingly, in examples, the method may further comprise, determining, based on the first sensor data, a location of the object and wherein adjusting the confidence threshold is further based on: (iii) the location of the object. Determining the location of the object may improve the robustness of the method. For example, if the object is determined to be located far from the musical instrument, then the confidence threshold may not be adjusted. The method may therefore only proceed if the object is determined to be close to the musical instrument (such as within a threshold distance). As another example, the amount the confidence threshold may be adjusted may be based on the location. For example, the change in the threshold may be smaller if the object is located further from the musical instrument, such as a reference point on the musical instrument. In some cases, this method is performed only if the object is determined to be present (to avoid performing this step in cases where the confidence score is below the confidence threshold). In examples, determining, based on the first sensor data, a location of the object comprises: determining one or more reference points on the musical instrument based on the first sensor data and determining the location of the object relative to the one or more reference points. The location relative to the musical instrument could be a lateral displacement from the musical instrument (so no, or minimal, occlusion of the instrument by the object from the first sensors perspective), and / or may be a vertical / perpendicular displacement from the musical instrument (for example, a hand may occlude the instrument (from the first sensors perspective, but be located high above the instrument), so no adjustment to the confidence threshold at that time, or a smaller adjustment). According to a second aspect of the present disclosure there is provided a method, comprising (i) receiving first sensor data associated with a first sensor, the first sensor positioned to obtain sensor data from an area encompassing at least a portion of a musical instrument, (ii) determining, using an object detection model, a confidence score that the first sensor data comprises sensor data indicative of an object, (iii) determining, based on a comparison of the confidence score with a confidence threshold, whether the object is present, (iv) determining, based on second sensor data associated with a second sensor, whether the musical instrument has been actuated, and (v) adjusting the confidence threshold based on: (i) whether the object is determined to be present, and (ii) whether the musical instrument has been actuated. The method may comprise any or all of the previously described method steps. According to a third aspect of the present disclosure there is provided a musical instrument system configured to perform any of the method steps previously described. According to a fourth aspect of the present disclosure there is provided a musical instrument system, comprising: a musical instrument, a first sensor positioned to obtain sensor data from an area encompassing at least a portion of the musical instrument, a second sensor, one or more processors, and a computer-readable medium having instructions stored thereon which, when executed by the one or more processors, cause the one or more processors to perform any of the method steps previously described. In examples, the computer-readable storage medium is non-transitory. The method step(s) defined throughout may be performed by the one or more processors or a device comprising the one or more processors (such as the musical instrument, or a local or remote computing device. For example, the method step(s) of receiving first sensor data and / or receiving second sensor data may be performed by the one or more processors or a device comprising the one or more processors (such as the musical instrument, or a local or remote computing device). Similarly, the method step(s) of determining a confidence score and / or determining whether the object is present and / or determining whether the musical instrument has been played / actuated and / or adjusting the confidence threshold is performed by the one or more processors or a device comprising the one or more processors (such as the musical instrument, or a local or remote computing device). The second sensor may be positioned to obtain sensor data associated with the musical instrument, such as sensor data associated with a portion of the musical instrument. In examples, the first sensor is an imaging device configured to detect electromagnetic radiation. The imaging device may also be configured to emit electromagnetic radiation, which is then reflected by one or more objects before being detected by the imaging device. The electromagnetic radiation detected by the imaging device may therefore be the electromagnetic radiation that the imaging device emitted after being reflected by the one or more objects. The imaging device may have a field of view, and may therefore obtain sensor data within the field of view. The field of view may encompass at least a portion of a musical instrument. In one example, the first sensor data is an image. In examples, the imaging device is an infrared imaging device. The electromagnetic radiation may therefore be infrared. In other examples, the imaging device may be a visible light imaging device. In examples, the musical instrument comprises the second sensor. The second sensor may detect or generate the second sensor data when an element of the musical instrument is played / actuated. In examples where the musical instrument is an electronic keyboard, the second sensor may be associated with a key of the electronic keyboard (for example, the second sensor may detect or generate the second sensor data when the key is pressed. In examples, the second sensor is a pressure sensor. A pressure sensor detects the application of a force. In a particular example, the second sensor data is indicative of the amount of force applied to the second sensor. In examples, the musical instrument comprises the first sensor. For example, the first sensor may be physically attached to, or integrally formed with, the musical instrument. In one example, the musical instrument is an electronic keyboard. In a particular example, the electronic keyboard comprises a plurality of keys that are playable by a user, and the first sensor is positioned to obtain sensor data from an area encompassing at least one or more keys of the plurality of keys. In examples, the first sensor is suspended or positioned above the plurality of keys. In examples, one of: (i) the musical instrument comprises the one or more processors and the computer-readable medium, (ii) a computing device comprises the one or more processors and the computer-readable medium, (iii) a computing device comprising the first sensor comprises the one or more processors and the computer-readable medium, and (iv) a remote computing device comprises the one or more processors and the computer-readable medium. The remote computing device may be alternatively referred to as a server. The computing device may be alternatively referred to as a local computing device. The computing device may be a mobile device, in some cases. In a particular example, the computing device comprising the first sensor is a mobile device. The mobile device may be a smartphone, a tablet, a laptop, etc. According to a fifth aspect of the present disclosure there is provided computer-readable storage medium having instructions stored thereon which, when executed by one or more processors, cause the one or more processors to perform any of the method steps previously described. Brief Description of the Figures Examples of the present disclosure will now be described with reference to the accompanying drawings: Figure 1 is a schematic diagram of an example musical instrument system; Figure 2 is a perspective view of a hand playing an electronic keyboard, the hand having not been identified by an object detection model; Figure 3 is a perspective view of a hand playing an electronic keyboard, the hand having been identified by an object detection model; Figure 4 is a perspective view of an electronic keyboard, where an erroneous hand has been identified by an object detection model; Figure 5 is a perspective view of a hand playing an electronic keyboard, the hand having not been identified by an object detection model; Figure 6 is a flow diagram of an example method; Figure 7 is a flow diagram of an example method; Figure 8 is a perspective view of an electronic keyboard, where an erroneous hand has been identified by an object detection model; Figure 9 is a flow diagram of another example method; Figure 10 is a flow diagram of showing an expanded version of the method of Figure 7; Figure 11 is a flow diagram of another example method utilising an initialisation threshold; Figure 12 is a flow diagram of another example method utilising a kill threshold; and Figure 13 is a flow diagram showing an expanded version of the method of Figure 7 or the method of Figure 10. Detailed Description Figure 1 is a schematic diagram showing an example of a musical instrument system 100. The musical instrument system 100 includes a musical instrument 102, a first sensor 104 positioned to obtain sensor data from an area encompassing at least a portion of the musical instrument 102, a second sensor 106, one or more processors 108, and a computer-readable medium 110 having instructions stored thereon which, when executed by the one or more processors 108, cause the one or more processors 108 to perform any of the method steps described herein. The first and second sensors 104, 106 are communicatively coupled to at least the one or more processors 108, either directly or indirectly. The musical instrument 102, in this example, is an electronic keyboard comprising a plurality of keys 112 that are playable by a user, in particular, by the hand 114 of the user. The first sensor 104 is positioned to obtain sensor data from an area encompassing at least one or more keys of the plurality of keys 112. For instance, the first sensor 104 may be suspended or otherwise positioned above the musical instrument 102. The first sensor has a field of view 116 that encompasses at least a portion of the musical instrument 102. In this example, the musical instrument 102 includes the second sensor 106. The second sensor 106 is associated with a key of the instrument 102. The first sensor 104 is responsible for collecting or generating first sensor data. This data, in a particular example, may be an image, such as a static image or one or more frames from a video captured by the first sensor 104. The first sensor 104 of this example is an imaging device configured to detect electromagnetic radiation, such as infrared or visible light. In some cases, the imaging device may also emit electromagnetic radiation, which then reflects off objects (e.g., the object 114) before being detected again by the device. The first sensor data may be received by the one or more processors 108. The first sensor data can be used by an object detection model to determine whether an object is present within the first sensor data. The second sensor 106 collects or generates second sensor data, which may be used by the one or more processors 108 to infer or determine whether the musical instrument 102 has been played. In a particular example, the second sensor 106 is a pressure sensor that detects the application of force, and the second sensor data may indicate the amount of force applied. A pressure sensor may be a Force Sensing Resistor (FSR), for example. The one or more processors 108, along with the computer-readable medium 110, form part of a computing device 118, which may be integrated into the musical instrument 102 or may be a separate entity, such as a mobile device, a local computing device, or a remote server. The processors 108 can perform various processes, including receiving sensor data, utilizing one or object detection models to analyse the first sensor data, and adjusting confidence thresholds based on the presence of objects and whether the musical instrument 102 has been played. The computer-readable medium 110 stores instructions that, when executed by the one or more processors 108, enable the performance of the methods described herein. As will be understood, an object detection model may be an algorithm designed to identify or locate specific objects within an image or video frame. The model may be trained on a large dataset of labelled images. Through this training process, the model leams to recognise patterns and features associated with various objects, such as hands or fingers. Once trained, the object detection model can analyse new, unseen images to predict the presence of these objects. In some cases, for each detected object, the model may output a bounding box that indicates the object's location within the image, along with a confidence score. This confidence score, typically expressed as a probability, represents the model's certainty that the detected object belongs to a specific class of objects. A confidence threshold can then be applied to filter out detections with low confidence scores, ensuring that only predictions above a certain level of certainty are considered valid. As discussed above, the present disclosure relates to detecting, via an object detection model, whether an object 114, such as a user’s hand, is present in first sensor data obtained by the first sensor 104. The object detection model may use a variable confidence threshold to determine whether the object 114 is present. For example, information derived from the first sensor data may be given a confidence score indicating the likelihood that the information corresponds to the presence of the object (i.e., a hand) in the first sensor data. If the confidence score is above the confidence threshold, the object 114 may be determined to be present. If the confidence score is below the threshold, the object may be determined not to be present. The present disclosure involves dynamically adjusting the confidence threshold using the second sensor data. The second sensor data may be referred to as “ground truth data”. Figure 2 illustrates an example where a hand 114 is playing the musical instrument 102. For example, a finger is pressing on a key of the musical instrument 102. Because the user is playing the instrument, second sensor data may be generated by the second sensor that is associated with the key and received by the one or more processors 108. In addition, the first sensor 104 is obtaining first sensor data, which is provided to an object detection model. The object detection model then determines a confidence score that the first sensor data comprises sensor data indicative of the object 114. In this particular example, the confidence score may be 90%, and the confidence threshold may be 95%. As such, the confidence score is below the current threshold, and the hand 114 is determined not to be present. The hand 114 is therefore determined not to be present, despite second sensor data indicating that the musical instrument is being played. To improve detection of the hand 114, the confidence threshold can be decreased so that the hand 114 is correctly detected. For example, the confidence threshold may be reduced / lowered to 85%. The confidence score of the object 114 may therefore now be above the adjusted confidence threshold, and therefore, based on the comparison of the confidence score with the (adjusted) confidence threshold, it may be determined that the object is present. Figure 3 illustrates an example where the object detection model now correctly identifies the hand 114 (illustrated by the bounding box drawn around the hand 114). Figures 2 and 3 therefore show how false negatives can be reduced. Figure 4 illustrates another example where the musical instrument 102 is not being played, specifically an example in which the hand 114 is not in the vicinity of the musical instrument 102. As for the Figures 2 and 3 examples, the first sensor 104 is obtaining first sensor data, which is provided to an object detection model. The object detection model then determines a confidence score that the first sensor data comprises sensor data indicative of any object. In this particular example, the confidence score may be 50%, and the confidence threshold may be 45%. As such, the confidence score is above the current threshold, and an object is determined to be present, as illustrated by the bounding box in Figure 4. An object is therefore determined to be present, despite the musical instrument 102 not being played. The one or more processors 108 can determine that there is no information in the second sensor data or the one or more processors 108 may not be receiving second sensor data, leading to a determination that the musical instrument 102 is not being played. A false positive is therefore being detected. To ensure that an object is not detected on the basis of the first sensor data, the confidence threshold can be increased. For example, the confidence threshold may be increased to 75%. The confidence score of an object may therefore now be below the adjusted confidence threshold, and therefore, based on the comparison of the confidence score with the (adjusted) confidence threshold, it may be determined that an object is not present, as would be expected. Figure 5 illustrates an example where the object detection model now correctly does not identify an object (illustrated by the absence of a bounding box). Figure 6 is a flow diagram of an example method which can be implemented by the musical instrument system 100 of Figure 1. In particular, in block 602, the method comprises, determining, based on first sensor data and a confidence threshold, whether an object is present. For example, block 602 may comprise: (i) receiving first sensor data associated with a first sensor, the first sensor positioned to obtain sensor data from an area encompassing at least a portion of a musical instrument, (ii) determining, using an object detection model, a confidence score that the first sensor data comprises sensor data indicative of an object and (iii) determining, based on a comparison of the confidence score with a confidence threshold, whether the object is present. As mentioned, the object may be determined to be present if the confidence score is above the threshold, and the object may be determined not to be present if the confidence score is below the threshold. The method proceeds to block 604 if the object is determined to be present. In block 604, the method comprises determining, based on second sensor data, whether the musical instrument has been played. For example, if no second sensor data is received (such as within a threshold period of time), it may be determined that the musical instrument has not been played. In another example, second sensor data may be received, and it may be determined that the musical instrument has not been played based on an analysis of the second sensor data. In either case, if it is determined that the musical instrument has not been played, the method proceeds to block 606. In block 606, the method comprises raising the threshold. As mentioned, raising the threshold can therefore eliminate false positives, given that an object has been detected, despite the instrument not being played. Conversely, if it is determined that the musical instrument has been played in block 604, the method returns to block 602. This means there is no adjustment to the confidence threshold. For example, the method may proceed again based on future / further first sensor data. The method proceeds to block 608 if the object is determined not to be present. In block 608, the method comprises determining, based on second sensor data, whether the musical instrument has been played. Block 608 may therefore be substantially the same as block 604. For example, if second sensor data is received (such as within a threshold period of time), it may be determined that the musical instrument has been played. In another example, second sensor data may be received, and it may be determined that the musical instrument has been played based on an analysis of the second sensor data. In either case, if it is determined that the musical instrument has been played, the method proceeds to block 610. In block 610, the method comprises lowering the threshold. As mentioned, lowering the threshold can therefore eliminate false negatives, given that no object has been detected, despite the instrument being played. Conversely, if it is determined that the musical instrument has not been played in block 608, the method returns to block 602. This means there is no adjustment to the confidence threshold. For example, the method may proceed again based on future / further first sensor data. Accordingly, in blocks 606 and 610, the confidence threshold is adjusted based on whether the object is determined to be present, and whether the musical instrument has been played. Figure 7 is a flow diagram of a more general method which can be implemented by the musical instrument system 100 of Figure 1. In particular, in block 702, the method comprises receiving first sensor data associated with the first sensor, the first sensor positioned to obtain sensor data from an area encompassing at least a portion of a musical instrument. In block 704, the method comprises, determining, using an object detection model, a confidence score that the first sensor data comprises sensor data indicative of an object. In block 706, the method comprises determining, based on a comparison of the confidence score with a confidence threshold, whether the object is present. Steps 702, 704 and 706 may be performed as part of block 602 in Figure 6, for example. In block 708, the method comprises determining, based on second sensor data associated with a second sensor, whether the musical instrument has been played. Block 708 may correspond to block 604 or 608 of Figure 6, for example. In block 710, the method comprises adjusting the confidence threshold based on: (i) whether the object is determined to be present, and (ii) whether the musical instrument has been played. Block 710 may correspond to block 606 or 610 of Figure 6, for example. As illustrated in Figure 7, the method may be repeated based on further first sensor data and / or further second sensor data. For example, a further image may be obtained by the first sensor 104. As such, in some examples, after adjusting the confidence threshold to obtain an adjusted confidence threshold, block 702 may comprise receiving further first sensor data associated with the first sensor. Block 704 may then comprise determining, using the object detection model, a confidence score (such as a different confidence score) that the further first sensor data comprises sensor data indicative of the object. Block 706 may then further comprise determining, based on a comparison of the confidence score with the adjusted confidence threshold, whether the object is present. In some cases, blocks 708 and 710 may also be repeated. For example, block 708 may comprise determining, based on further second sensor data associated with the second sensor, whether the musical instrument has been played. Block 710 may then comprise further adjusting the confidence threshold based on: (i) whether the object is determined to be present, and (ii) whether the musical instrument has been played. In a particular example, before repeating block 710 (after previously adjusting the confidence threshold), the method may involve waiting a predetermined period of time before further adjusting the confidence. As mentioned, this can prevent the confidence threshold from adjusting too quickly. As mentioned, in some examples, a location of the object 114 can be determined based on the first sensor data. For example, the location of the object 114 may be a location relative to one or more reference points, such as a reference point on the musical instrument, or in space. Figure 8 illustrates an example where an object 114 has been detected at a location 802 that is far away from a location 804 on the musical instrument 102 that has been played. In this case, a true hand 114a, is playing a particular key on the musical instrument 102, but the true hand 114a has not been determined to be present. It may therefore be useful to determine the location of each object that is detected and compare this to the location on the musical instrument that has been played. If there is misalignment between the two locations, as in this example, the confidence threshold may be increased. For example, this may mean that in future first sensor data, the erroneous object at location 802 is no longer detected. The conditions in the environment at that later time may mean that in the future first sensor data, the true hand 114a is detected. Figure 9 depicts the flow diagram of Figure 6 with additional steps in the form of blocks 902 and 904 that take account of the location of an object. Unlike in Figure 6, where the flow diagram returns to block 602 if it is determined that the musical instrument has been played, instead the method proceeds to block 902. In block 902, the method comprises determining, based on a location on the musical instrument that has been played and a location of the object, whether the location of the object is aligned (or misaligned) with the location on the musical instrument that has been played. In an example, block 902 comprises determining, based on the second sensor data, a location on the musical instrument that has been played and determining, based on the first sensor data, a location of the object. Determining whether the location of the object is aligned (or misaligned) with the location on the musical instrument that has been played, can therefore comprise comparing the two locations. For example, if the distance between the locations is above a threshold distance, it may be determined that the locations are misaligned, and if the distance is below the threshold distance, it may be determined that the locations are aligned. If it is determined that the location of the object is misaligned with the location on the musical instrument that has been played, then the process proceeds to block 904. In block 904 the method comprises raising the confidence threshold. As mentioned, this can result in the erroneous object at location 802 no longer being detected in future first sensor data and can result in the correct / true object 114a being correctly detected. As mentioned, in block 710 of Figure 7, the method comprises adjusting the confidence threshold. Block 710 may therefore correspond to block 904, in some examples. Conversely, if, in block 902, it is determined that the location of the object is aligned with the location on the musical instrument that has been played, then the process may return to block 602. This means there is no adjustment to the confidence threshold. For example, the method may proceed again based on future / further first sensor data. Figure 10 depicts an expanded version of the method of Figure 7. In particular, in block 1002, the method may involve performing blocks 702-708 of the method of Figure 7. In an example, the method progresses to block 1004 when the object is determined to be present (as determined in block 706) and it is determined that the musical instrument has been played (as determined in block 708). In block 1004, the method comprises determining, based on the second sensor data, a location on the musical instrument that has been played. Block 1004 may form part of block 902 of Figure 9. In block 1006, the method comprises determining, based on the first sensor data, a location of the object. Block 1006 may form part of block 902 of Figure 9. In block 1008, the method comprises, determining, based on the location on the musical instrument that has been played and the location of the object, whether the location of the object is misaligned with the location on the musical instrument that has been played. Block 1008 may form part of block 902 of Figure 9. In examples where the location of the object is misaligned with the location on the musical instrument (determined in block 1008), the method proceeds to block 1010, which involves increasing the confidence threshold. Block 1010 may correspond to block 710 in Figure 7, for example and block 904 of Figure 9. In some examples, block 1010 may further include generating a user notification in the event that the location of the object is misaligned with the location on the musical instrument. For example, the user may be asked to remove the object out of the field of view of the first sensor 104. For example, the object that is being determined to be present at location 802 may be a mobile phone, that is being interpreted as the user’s hand, for example. The notification may alternatively ask the user to change the lighting conditions, such as increase the brightness within the room. In examples, at least some of the blocks in Figure 10 can be repeated, such as based on further first sensor data received at a later time. For example, in an example, block 1002 comprises receiving further first sensor data associated with the first sensor, determining, using the object detection model, a second confidence score that the further first sensor data comprises sensor data indicative of the object and determining, based on a comparison of the second confidence score with the confidence threshold, that the object is present. In an example, the object that is determined to be present is the true object 114a of Figure 8. In block 1004, the method therefore comprises determining, based on the further first sensor data, a location of the object. In a particular example, in block 1006, the method comprises determining, based on the location on the musical instrument and the location of the object, that the location of the object is aligned with the location on the musical instrument. As such, no further adjustment to the confidence threshold may need to be made. In the previously described examples, there may be a single object detection model and a single confidence threshold that is adjusted. In other cases, however, there may be two object determine models, each utilising a separate confidence threshold. For example, one object detection model may utilise one confidence threshold, and another object detection model may utilise another confidence threshold. In a particular example, an object detection model may utilise an “initialization threshold”, and another object detection model may utilise a “kill threshold”. One model and threshold can be used to remove false positives, and the other model and threshold can be used to remove false negatives, for example. As an example, one object detection model may govern which objects are initially identified using the initialisation threshold, and then passes any object onto another object detection model which decides when they should no longer be detected, using a kill threshold. Adjustment of the initialisation and kill thresholds will now be described with reference to Figures 11 and 12. Figure 11 relates a method of adjusting the initialisation threshold, and Figure 12 relates to a method of adjusting the kill threshold. Referring to Figure 11, in block 602, the method comprises, determining, based on first sensor data and an initialisation threshold, whether an object is present. Block 602 of Figure 11 is substantially the same as previously described blocks 602, albeit now specifically relating to an initialisation threshold. For example, block 602 may comprise: (i) receiving first sensor data associated with a first sensor, the first sensor positioned to obtain sensor data from an area encompassing at least a portion of a musical instrument, (ii) determining, using an object detection model, an initialisation confidence score that the first sensor data comprises sensor data indicative of an object and (iii) determining, based on a comparison of the initialisation confidence score with the initialisation threshold, whether the object is present. As mentioned, the object may be determined to be present if the initialisation confidence score is above the initialisation threshold, and the object may be determined not to be present if the initialisation confidence score is below the initialisation threshold. Steps 702, 704 and 706 of Figure 7 may be performed as part of block 602 in Figure 11, for example. The method proceeds to block 608 if the object is determined not to be present. In block 608, the method comprises determining, based on second sensor data, whether the musical instrument has been played. Block 608 of Figure 11 is substantially the same as previously described blocks 608. If it is determined that the musical instrument has been played, the method proceeds to block 610. In block 610, the method comprises lowering the initialisation threshold. Block 610 of Figure 11 is substantially the same as previously described blocks 610, albeit now specifically relating to an initialisation threshold. As mentioned, lowering the initialisation threshold can therefore eliminate false negatives, given that no object has been detected, despite the instrument being played. Block 610 may correspond to block 710 of Figure 7. Conversely, if it is determined that the musical instrument has not been played in block 608, the method returns to block 602. This means there is no adjustment to the initialisation confidence threshold. For example, the method may proceed again based on future / further first sensor data. If, in block 602, it is determined that the obj ect is present, the process may proceed to the flow diagram of Figure 12, such as block 602 in Figure 12. In some examples, the method also proceeds to block 1102 if the object is determined to be present. In block 1102, the method comprises determining, whether the initialisation threshold is significantly below an initialisation confidence score. For example, if the difference between the current initialisation threshold and the initialisation confidence score is greater than a particular threshold, it may be determined that there is a high discrepancy between the initialisation threshold and the initialisation confidence score. If the initialisation threshold is significantly below an initialisation confidence score, then the method may progress to block 1104. In block 1104, the method comprises raising the initialisation threshold. This can avoid the initialisation threshold from becoming too low, and therefore act as a balance to increase the initialisation threshold, when it becomes too low. Figure 11 therefore relates to adjusting the initialisation threshold, and the method may be implemented using one object detection model / neural network. As mentioned, if an object is determined to be present in block 602 of Figure 11, the process may proceed to block 602 of Figure 12. Figure 12 is a flow diagram of another example method which can be implemented by the musical instrument system 100 of Figure 1. Block 602 of Figure 12 is substantially the same as previously described blocks 602, albeit now specifically relating to a kill threshold. In particular, in block 602, the method comprises, determining, based on first sensor data and a kill threshold, whether an object is present. For example, block 602 may comprise: (i) receiving the first sensor data associated with the first sensor, (ii) determining, using the object detection model (such as the same object detection model used in Figure 11, or a different object detection model), a confidence score that the first sensor data comprises sensor data indicative of an object and (iii) determining, based on a comparison of the confidence score with the kill threshold, whether the object is present. As mentioned, the object may be determined to be present if the confidence score is above the kill threshold, and the object may be determined not to be present if the confidence score is below the kill threshold. The step of determining whether the object is present based on a comparison of the confidence score with the kill threshold, may comprise determining that the object is the same object as previously detected. In this sense, a previously detected object would be one detected by the same object detection model within earlier first sensor data, such as an earlier video frame. Determining whether the objects are the same may comprise comparing one or more characteristics of the detected objects, such as location. For example, the object detection model may determine that the objects are the same by comparing the detected locations of the objects. In one example, the object detection model used in Figure 11 determines the presence of an object (in block 602 of Figure 11), whereas the object detection model used in Figure 12 may also track additional details of the object (in block 602 of Figure 12, for example), such as an orientation of the object, a velocity of the object and / or one or more particular features of the object, such as fingers. So, although both models may determine the presence of an object, each may be used for different purposes. The method proceeds to block 604 of Figure 12 if the object is determined to be present. In block 604, the method comprises determining, based on second sensor data, whether the musical instrument has been played. Block 604 of Figure 12 is substantially the same as previously described blocks 604. If it is determined that the musical instrument has not been played, the method proceeds to block 606. In block 606, the method comprises raising the kill threshold. Block 606 of Figure 12 is substantially the same as previously described blocks 606, albeit now specifically relating to a kill threshold. As mentioned, raising the kill threshold can therefore eliminate false positives, given that an object has been detected, despite the instrument not being played. If it is determined that the musical instrument has been played in block 604, the method proceeds to block 902. In block 902, the method comprises determining, based on a location on the musical instrument that has been played and a location of the object, whether the location of the object is aligned (or misaligned) with the location on the musical instrument that has been played. Block 902 of Figure 12 is substantially the same as previously described block 902. If it is determined that the location of the object is misaligned with the location on the musical instrument that has been played, then the process proceeds to block 904. In block 904 the method comprises raising the kill threshold. Block 904 of Figure 12 is substantially the same as previously described block 904, albeit now specifically relating to a kill threshold. As mentioned, this can result in an erroneous object being detected at location 802, for example, no longer being detected in future first sensor data and can result in a correct / true object 114a being correctly detected. Conversely, if, in block 902, it is determined that the location of the object is aligned with the location on the musical instrument that has been played, then the process may return to block 602 of Figure 12 or in some cases, block 602 of Figure 11. This means there is no adjustment to the kill threshold. For example, the method(s) may proceed again based on future / further first sensor data. If an object is not determined to be present in block 602, the method proceeds to block 1202, where the kill threshold is lowered. This can avoid the kill threshold from becoming too high, and therefore act as a balance to lower the kill threshold, when it becomes too high. Figure 13 depicts an expanded version of the method of Figure 7. In particular, the method steps of Figure 7, and in some cases Figure 10, may be performed using the kill threshold and the object detection model that is associated with the kill threshold. As such, prior to (i.e., before) performing the method steps of Figure 7 and Figure 10, blocks 1302 and 1304 may be implemented using the object detection model associated with the initialisation threshold (such as a second object detection model or the same object detection model). In particular, in block 1302, the method comprises determining, using the object detection model or a second object detection model, a second confidence score that the first sensor data comprises sensor data indicative of the object. In block 1304, the method further comprises determining, based on a comparison of the second confidence score with the initialisation threshold, that the object is present. Blocks 1302 and 1304 may therefore correspond to block 602 in Figure 11, for example. Because, in block 1304, an object was determined to be present based on the initialisation threshold, the process can proceed to block 1306. In block 1306, the method involves performing blocks 702-708 of the method of Figure 7 using the kill threshold or performing blocks 1002-1010 of the method of Figure 10 using the kill threshold. For example, blocks 702, 704 and 706 of Figure 7 may be performed as part of block 602 in Figure 12. Similarly, block 708 may correspond to block 604 of Figure 12. Block 710 may correspond to block 606 of Figure 12. Alternatively, block 1002 of Figure 10 may correspond to block 602 in Figure 12. Similarly, blocks 1004-1008 may correspond to block 902 in Figure 12, and block 1010 may correspond to block 904 in Figure 12. It will be appreciated that in any of the methods or flow diagrams described above, certain blocks may be performed in a different order to that which are described, or may be performed in parallel to other blocks. For example, block 708 may be performed before, during or after any of blocks 702-706. In some examples, certain blocks may be omitted, such as block 1102 and / or block 1202. It is to be understood that embodiments involve use of a confidence level requirement. The confidence level requirement may be a threshold, as is expressly disclosed in the foregoing examples. Alternatively, the confidence level may be a model selection, or a set of weights with each model or set of weights having a different confidence score output for the same input. More generally, it is to be understood that the confidence level requirement is indicative of a sensitivity that may be tuned to determine whether an object is present. The methods and processes described above may be implemented by one or more processors, operating in conjunction with software stored on a computer-readable medium. Such computer-readable media may include, but are not limited to, physical devices such as magnetic disks, optical disks, semiconductor memory devices, or other forms of non-volatile or volatile memory. The software stored on the medium comprises instructions that, when executed by the one or more processors, perform the steps of the methods described above. It is to be understood that any feature described in relation to any one embodiment may be used alone, or in combination with other features described, and may also be used in combination with one or more features of any other of the embodiments, or any combination of any other of the embodiments. Furthermore, equivalents and 5 modifications may also be employed without departing from the scope of the disclosure, which is defined in the accompanying claims.

Claims

1. A method, comprising:receiving first sensor data associated with a first sensor, the first sensor positioned to obtain sensor data from an area encompassing at least a portion of a musical instrument;determining, using an object detection model, a confidence score that the first sensor data comprises sensor data indicative of an object;determining, based on a comparison of the confidence score with a confidence threshold, whether the object is present;determining, based on second sensor data associated with a second sensor, whether the musical instrument has been played; andadjusting the confidence threshold based on: (i) whether the object is determined to be present, and (ii) whether the musical instrument has been played.

2. A method according to claim 1, further comprising:receiving further first sensor data associated with the first sensor;determining, using the object detection model, a confidence score that the further first sensor data comprises sensor data indicative of the object; anddetermining, based on a comparison of the confidence score with the adjusted confidence threshold, whether the object is present3. A method according to claim 2, further comprising:determining, based on further second sensor data associated with the second sensor, whether the musical instrument has been played; andafter adjusting the confidence threshold, waiting a predetermined period of time before further adjusting the confidence threshold based on: (i) whether the object is determined to be present, and (ii) whether the musical instrument has been played.

4. A method according to any preceding claim, wherein adjusting the confidence threshold comprises:increasing the confidence threshold in the event that: (i) the object is determined to be present, and (ii) the musical instrument is determined not to have been played.

5. A method according to claim 4, further comprising:determining that the second sensor data associated with the second sensor has not been received within a threshold period of time and responsively determining that the musical instrument has not been played.

6. A method according to claim 4, further comprising:receiving the second sensor data associated with the second sensor; and determining that the musical instrument has not been played based on an analysis of the second sensor data.

7. A method according to any of claims 4 to 6, further comprising:prior to (i) determining, using the object detection model, the confidence score that the first sensor data comprises sensor data indicative of the object, and (ii) determining, based on the comparison of the confidence score with the confidence threshold, whether the object is present:determining, using the object detection model or a second object detection model, a second confidence score that the first sensor data comprises sensor data indicative of the object; anddetermining, based on a comparison of the second confidence score with a second confidence threshold, that the object is present.

8. A method according to any of claims 1 to 3, wherein adjusting the confidence threshold comprises:lowering the confidence threshold in the event that: (i) the object is determined not to be present, and (ii) the musical instrument has been played.

9. A method according to claim 8, further comprising:receiving the second sensor data associated with the second sensor; anddetermining that the musical instrument has been played by having received the second sensor data.

10. A method according to claim 8, further comprising:receiving the second sensor data associated with the second sensor; anddetermining that the musical instrument has been played based on an analysis of the second sensor data.

11. A method according to any of claims 1 to 3, further comprising:determining, based on the second sensor data, a location on the musical instrument that has been played;determining, based on the first sensor data, a location of the object; anddetermining, based on the location on the musical instrument that has been played and the location of the object, whether the location of the object is misaligned with the location on the musical instrument that has been played;wherein adjusting the confidence threshold comprises:increasing the confidence threshold in the event that: (i) the object is determined to be present, (ii) the musical instrument has been played, and (iii) the location of the object is misaligned with the location on the musical instrument.

12. A method according to claim 11, further comprising:receiving further first sensor data associated with the first sensor;determining, using the object detection model, a second confidence score that the further first sensor data comprises sensor data indicative of the object;determining, based on a comparison of the second confidence score with the confidence threshold, that the object is present;determining, based on the further first sensor data, a location of the object; anddetermining, based on the location on the musical instrument and the location of the object, that the location of the object is aligned with the location on the musical instrument.

13. A method according to claims 11 or 12, further comprising generating a user notification in the event that the location of the object is misaligned with the location on the musical instrument.

14. A method according to any preceding claim, further comprising: determining, based on the first sensor data, a location of the object; wherein adjusting the confidence threshold is further based on: (iii) the location of the object.

15. A method, comprising:receiving first sensor data associated with a first sensor, the first sensor positioned to obtain sensor data from an area encompassing at least a portion of a musical instrument;determining, using an object detection model, a confidence score that the first sensor data comprises sensor data indicative of an object;determining, based on a comparison of the confidence score with a confidence threshold, whether the object is present;determining, based on second sensor data associated with a second sensor, whether the musical instrument has been actuated; andadjusting the confidence threshold based on: (i) whether the object is determined to be present, and (ii) whether the musical instrument has been actuated.

16. A musical instrument system configured to perform a method according to any preceding claim.

17. A musical instrument system, comprising:a musical instrument;a first sensor positioned to obtain sensor data from an area encompassing at least a portion of the musical instrument;a second sensor;one or more processors; anda computer-readable medium having instructions stored thereon which, when executed by the one or more processors, cause the one or more processors to perform a method according to any of claims 1 to 15.

18. A musical instrument system according to claim 17, wherein the first sensor is an imaging device configured to detect electromagnetic radiation.

19. A musical instrument system according to claim 18, wherein the imaging device is an infrared imaging device.

20. A musical instrument system according to any of claims 17 to 19, wherein the musical instrument comprises the second sensor.

21. A musical instrument system according to any of claims 17 to 20, wherein the second sensor is a pressure sensor.

22. A musical instrument system according to any of claims 17 to 21, wherein the musical instrument comprises the first sensor.

23. A musical instrument system according to any of claims 17 to 22, wherein the musical instrument is an electronic keyboard.

24. A musical instrument system according to claim 23, when appendant to claim 22, wherein the electronic keyboard comprises a plurality of keys that are playable by a user, wherein the first sensor is positioned to obtain sensor data from an area encompassing at least one or more keys of the plurality of keys.

25. A musical instrument system according to any of claims 17 to 24, wherein one ofthe musical instrument comprises the one or more processors and the computer-readable medium;a computing device comprises the one or more processors and the computer-readable medium;a computing device comprising the first sensor comprises the one or more processors and the computer-readable medium; and5 a remote computing device comprises the one or more processors and thecomputer-readable medium.

26. A musical instrument system according to claim 25, wherein the computing device comprising the first sensor is a mobile device.1027. A computer-readable storage medium having instructions stored thereon which, when executed by one or more processors, cause the one or more processors to perform a method according to any of claims 1 to 15.15s

Images