A guitar multifunction touch control method and device based on a composite gesture

Abstract

The application discloses a guitar multifunctional touch control method and device based on a composite gesture, and the method comprises the following steps: collecting gesture input data of a user on a touch sensing area on the surface of a guitar neck, wherein the touch sensing area is divided into a first function subarea, a second function subarea and a third function subarea; analyzing the relative position relationship characteristics between fingers in the gesture input data, matching the hand shape topological template in the prestorage to obtain a hand shape category recognition result, and then querying a control domain mapping table according to the hand shape category recognition result to obtain a target control domain; extracting contact position information from the gesture data to determine the function subarea to which the contact position information belongs, and obtaining a subarea positioning result; fusing and encoding the hand shape category recognition result, the target control domain and the subarea positioning result into a composite gesture feature vector, and then converting the composite gesture feature vector into a control instruction and outputting the control instruction. The application improves the intuitiveness and efficiency of guitar control, and realizes multifunctional control which is in harmony with the natural operation habits of a player.

Description

Technical Field

[0001] This invention relates to the field of electronic control technology for musical instruments, specifically to a multi-functional touch control method and device for guitars based on compound gestures. Background Technology

[0002] In guitar playing, real-time control of tone, volume, and effects parameters is crucial for enhancing musical expressiveness. Traditionally, players rely on pedals or separate control panels, which can disrupt the flow of the performance when complex fingerings are required. To address this, the industry has proposed integrating touch sensors into the guitar neck to trigger control commands by recognizing finger movements such as sliding and tapping. These solutions typically treat the touch area as a uniform interactive plane, and the definitions of gestures are relatively abstract, such as changing parameters by drawing circles or straight lines. However, in actual playing, a player's fingers naturally exhibit specific spatial positioning habits and morphological patterns on the guitar neck, such as pressing chords in different fret areas. The interaction logic of existing technologies, based on a uniform plane and abstract gestures, differs significantly from the player's inherent, highly structured operating habits. This not only requires players to memorize new, unnatural gestures, resulting in a high learning cost, but also, because it relies solely on the absolute coordinates of the fingers for recognition, players struggle to quickly locate the desired control area by touch alone, making them highly susceptible to interference from complex playing movements, thus affecting the intuitiveness and efficiency of operation. Summary of the Invention

[0003] In view of the above problems, the present invention provides a multi-functional touch control method and device for guitar based on compound gestures. By combining and encoding the topological shape of the fingers with the physical partitioning of the touch area on the guitar neck, multi-functional intuitive control that conforms to the player's natural operating habits is achieved.

[0004] To achieve the above objectives, in a first aspect, this application provides a multi-functional touch control method for a guitar based on composite gestures, comprising: The system acquires gesture input data performed by the user on the touch-sensitive area on the surface of the guitar neck. The touch-sensitive area is divided into a first functional area, a second functional area, and a third functional area. Finger topology analysis is performed on gesture input data to extract the relative positional relationship features between fingers. The relative positional relationship features are then matched with multiple pre-stored hand shape topology templates to generate hand shape category recognition results. Based on the hand shape category recognition result, the corresponding control domain is queried from the preset control domain mapping table and recorded as the target control domain. The types of control domains include volume control domain, effect parameter control domain and global parameter control domain. Extract contact location information from gesture input data, determine the corresponding functional zone based on the contact location information, and generate zone positioning results; The hand shape category recognition result, target control domain and partition localization result are fused and encoded to generate a composite gesture feature vector; Based on the preset instruction mapping relationship, the composite gesture feature vector is converted into the corresponding control instruction and output.

[0005] In some embodiments, finger topology analysis is performed on the gesture input data to extract the relative positional relationship features between fingers. These features are then matched with multiple pre-stored hand shape topology templates to generate hand shape category recognition results, including: Extract the coordinate data of each fingertip on the touch sensing area from the gesture input data, calculate the Euclidean distance between any two fingertips based on the coordinate data, and generate a fingertip distance matrix. Extract the orientation angle data of each fingertip from the gesture input data, calculate the angle between any two fingertips based on the orientation angle data, and generate a fingertip angle matrix; The fingertip distance matrix and the fingertip angle matrix are concatenated to generate a finger topological feature vector; Calculate the cosine similarity between the finger topological feature vector and each pre-stored hand topological template, which includes the index and middle fingers together template, the three fingers spread template, the fist template, and the thumb and index finger pinching template. The hand type corresponding to the hand type topology template whose cosine similarity value exceeds the preset matching threshold is determined as the hand type category recognition result.

[0006] In some embodiments, the fingertip distance matrix and the fingertip angle matrix are concatenated to generate a finger topological feature vector, including: Read the fingertip distance matrix. The fingertip distance matrix is ​​a symmetric matrix with each fingertip identifier as the row index and column index. Each element in the fingertip distance matrix is ​​the Euclidean distance value between the corresponding two fingertips. Read the fingertip angle matrix. The fingertip angle matrix is ​​a symmetric matrix with each fingertip identifier as the row index and column index. Each element in the fingertip angle matrix is ​​the angle value between the directions of the two corresponding fingertips. Extract the elements of the upper triangular region of the fingertip distance matrix in row-major order to generate a distance feature sequence; Extract the elements of the upper triangular region of the fingertip angle matrix in row-major order to generate an angle feature sequence; The distance feature sequence and the angle feature sequence are concatenated end to end according to a preset concatenation order to generate a one-dimensional vector form of finger topological feature vector.

[0007] In some embodiments, based on the hand shape category recognition result, the corresponding control domain is queried from a preset control domain mapping table, denoted as the target control domain, including: Read the hand shape category recognition results, which include the index and middle fingers together category, the three fingers spread apart category, the fist category, or the thumb and index finger pinched category. Load a preset control domain mapping table from the storage unit. The control domain mapping table records the correspondence between each hand shape category and each control domain. Among them, the index and middle fingers together category corresponds to the volume control domain, the three fingers spread category corresponds to the effect parameter control domain, the fist category corresponds to the global parameter control domain, and the thumb and index finger pinching category corresponds to the filter control domain. Using the hand shape category recognition result as the query key, a hash lookup operation is performed in the control domain mapping table to locate the mapping entry that matches the hand shape category recognition result; Extract the corresponding control domain identifier from the mapping entry, and determine the control domain indicated by the control domain identifier as the target control domain.

[0008] In some embodiments, using the hand shape category recognition result as the query key, a hash lookup operation is performed in the control domain mapping table to locate the mapping entry that matches the hand shape category recognition result, including: Read the hand shape category recognition result. The hand shape category recognition result includes a hand shape category name string. The hand shape category name string is one of the following: index finger and middle finger together, three fingers open, fist clenched, or thumb and index finger pinched together. Input the hand type category name string into the preset hash function. The hash function performs bytecode conversion operation on the hand type category name string to generate a hash key value corresponding to the hand type category name string. Using the hash key as an index, the access control domain mapping table is used to locate the target hash bucket corresponding to the hash key in the hash bucket array. Iterate through each mapping entry stored in the target hash bucket and compare the key field of each mapping entry with the hand type category name string byte by byte; Mapping entries that match the byte-by-byte comparison results are identified as mapping entries that match the hand type recognition results; Extract the corresponding control domain identifier from the mapping entry, and determine the control domain indicated by the control domain identifier as the target control domain, including: Read the key-value pair data structure in the mapping entry. The key-value pair data structure contains a key field and a value field. The key field stores the hand type recognition result, and the value field stores the control field identifier. Extract the control domain identifier from the value field. The control domain identifier is one of the following: volume control domain identifier, effect parameter control domain identifier, global parameter control domain identifier, or filter control domain identifier. The control domain indicated by the control domain identifier is identified as the target control domain, and the target control domain is associated with and stored with the hand shape category recognition result for subsequent fusion encoding steps.

[0009] In some embodiments, contact position information is extracted from gesture input data, the corresponding functional zone is determined based on the contact position information, and a zone positioning result is generated, including: Two-dimensional coordinate data of each contact point are extracted from the gesture input data. The two-dimensional coordinate data includes the longitudinal coordinate value along the length of the neck and the lateral coordinate value perpendicular to the length of the neck. Read the preset functional partition boundary parameters, which include the first boundary longitudinal coordinate value between the first functional partition and the second functional partition, and the second boundary longitudinal coordinate value between the second functional partition and the third functional partition. The longitudinal coordinate values ​​of each contact point are compared with the longitudinal coordinate values ​​of the first boundary and the second boundary, respectively. If the longitudinal coordinate value is less than the longitudinal coordinate value of the first boundary, then the corresponding contact point will be assigned to the first functional zone. If the longitudinal coordinate value is greater than or equal to the longitudinal coordinate value of the first boundary and less than the longitudinal coordinate value of the second boundary, then the corresponding contact point will be assigned to the second functional zone. If the longitudinal coordinate value is greater than or equal to the longitudinal coordinate value of the second boundary, the corresponding contact point will be assigned to the third functional zone. Summarize the functional zone affiliation information of each contact point and generate a zone location result that includes the correspondence between contact point identifiers and functional zone identifiers.

[0010] In some embodiments, the hand shape category recognition result, the target control domain, and the partition localization result are fused and encoded to generate a composite gesture feature vector, including: Read the hand shape category recognition result. The hand shape category recognition result includes a hand shape category identifier, which is one of the following: index finger and middle finger together, three fingers open, fist clenched, or thumb and index finger pinched together. Read the target control domain, which contains a control domain identifier. The control domain identifier is one of the following: volume control domain identifier, effect parameter control domain identifier, global parameter control domain identifier, or filter control domain identifier. Read the partition location result. The partition location result includes a function partition identifier, which is one of the first function partition identifier, the second function partition identifier, or the third function partition identifier. Map the hand type identifier to the first numerical code, the control domain identifier to the second numerical code, and the functional area identifier to the third numerical code; The first, second, and third numerical codes are concatenated according to a preset vector dimension order to generate a composite gesture feature vector in three-dimensional vector form.

[0011] In some embodiments, before converting the composite gesture feature vector into the corresponding control command, the method further includes: Collect string vibration data, which includes string amplitude data and string frequency data; Based on the string amplitude data and string frequency data, generate the string state modulation coefficient; The string state modulation coefficients are weighted and the composite gesture feature vector is weighted to generate the modulated composite gesture feature vector. The modulated composite gesture feature vector is used as the composite gesture feature vector; Based on a preset instruction mapping relationship, the composite gesture feature vector is converted into corresponding control instructions and output, including: Read the composite gesture feature vector, which is a three-dimensional vector containing the first dimension numerical code, the second dimension numerical code, and the third dimension numerical code; A preset instruction mapping table is loaded from the storage unit. The instruction mapping table records the correspondence between each composite gesture feature vector and each control instruction. The control instructions include MIDI control change instructions and digital signal control instructions. Using the composite gesture feature vector as the query index, a vector space nearest neighbor search is performed in the instruction mapping table to calculate the Euclidean distance between the composite gesture feature vector and the reference feature vector stored in each mapping entry of the instruction mapping table. Select the mapping entry with the smallest Euclidean distance and extract the corresponding control instruction identifier from the mapping entry; Based on the control instruction identifier, the corresponding control instruction data packet is retrieved from the control instruction library. The control instruction data packet contains an instruction type field, an instruction channel field, and an instruction parameter field. Control command data packets are sent to external effects devices or built-in digital signal processing modules via wireless Bluetooth MIDI protocol or wired USB MIDI protocol to achieve real-time adjustment of guitar tone, volume or effects parameters.

[0012] In some embodiments, acquiring gesture input data performed by a user on a touch-sensitive area of ​​the guitar neck surface includes: The capacitive touch sensor array in the touch sensing area is driven at a preset sampling frequency to collect data on the capacitance change of each sensing unit. Baseline calibration is performed on the capacitance change data by subtracting the background capacitance value under no-touch state from the capacitance change data to generate calibrated capacitance change data. Spatial filtering is performed on the calibrated capacitance change data to remove sensor unit data corresponding to isolated noise points and generate filtered capacitance change data. Identify continuous sensing unit regions where the capacitance change exceeds a preset contact threshold from the filtered capacitance change data, and mark the continuous sensing unit regions as valid contact regions. Extract the peak position of capacitance change of each sensing unit within the effective contact area, and convert the peak position of capacitance change into corresponding two-dimensional coordinate data; Touch duration data and touch motion trajectory data are extracted from the effective contact area. The touch duration data includes the contact start time and contact end time, and the touch motion trajectory data includes a two-dimensional coordinate sequence at each sampling time. Two-dimensional coordinate data, touch duration data, and touch motion trajectory data are integrated into a structured data package, which is then recorded as gesture input data.

[0013] In a second aspect, the present invention also provides a multi-functional touch control device for guitars based on composite gestures, applicable to the method described in the first aspect. The device includes a touch sensing module, a hand shape recognition module, a control domain query module, a partition positioning module, a fusion encoding module, and an instruction mapping module. The touch sensing module is disposed on the surface of the guitar neck and includes a capacitive touch sensor array. The area covered by the capacitive touch sensor array is divided into a first functional partition, a second functional partition, and a third functional partition. The hand shape recognition module is connected to the touch sensing module. The hand shape recognition module receives gesture input data output by the touch sensing module, performs finger topology analysis on the gesture input data, extracts the relative positional relationship features between fingers, and performs pattern matching between the relative positional relationship features and multiple pre-stored hand shape topology templates to generate a hand shape category recognition result. The control domain query module is connected to the hand shape recognition module. The control domain query module receives the hand shape category recognition result and, based on... Based on the hand shape category recognition result, the corresponding control domain is queried from the preset control domain mapping table and denoted as the target control domain. The control domain types include volume control domain, effect parameter control domain, and global parameter control domain. The partition positioning module is connected to the touch sensing module. The partition positioning module receives gesture input data, extracts contact position information from the gesture input data, determines the functional partition to which it belongs based on the contact position information, and generates the partition positioning result. The fusion encoding module is connected to the hand shape recognition module, the control domain query module, and the partition positioning module. The fusion encoding module receives the hand shape category recognition result, the target control domain, and the partition positioning result, and fuses and encodes the hand shape category recognition result, the target control domain, and the partition positioning result to generate a composite gesture feature vector. The command mapping module is connected to the fusion encoding module. The command mapping module receives the composite gesture feature vector, converts the composite gesture feature vector into the corresponding control command according to the preset command mapping relationship, and outputs it.

[0014] Unlike existing technologies, the above technical solution provides a method and device for multi-functional touch control of a guitar based on composite gestures. The method includes: collecting gesture input data performed by a user on a touch-sensitive area on the surface of the guitar neck, where the touch-sensitive area is divided into a first functional zone, a second functional zone, and a third functional zone; analyzing the relative positional relationship features between fingers in the gesture input data, matching them with a pre-stored hand shape topology template to obtain a hand shape category recognition result, and then querying a control domain mapping table based on the hand shape category recognition result to obtain a target control domain; extracting contact position information from the gesture data to determine the corresponding functional zone to obtain a zone positioning result; fusing and encoding the hand shape category recognition result, the target control domain, and the zone positioning result into a composite gesture feature vector, and then converting it into a control command output. This invention improves the intuitiveness and efficiency of guitar control, realizing multi-functional control that aligns with the player's natural operating habits. Attached Figure Description

[0015] Figure 1 This is a schematic diagram illustrating steps S101 to S106 of the method described in the specific implementation embodiment; Figure 2 This is a schematic diagram of the structure of the guitar multi-functional touch control device described in a specific embodiment.

[0016] The reference numerals used in the above figures are explained as follows: 1. Guitar multi-functional touch control device; 11. Touch sensing module; 12. Hand shape recognition module; 13. Control domain query module; 14. Partition positioning module; 15. Fusion encoding module; 16. Command mapping module. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] The processor described in the embodiments of this application can be implemented by hardware, firmware, software, or a combination thereof. It can be a circuit, one or more of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field-programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller, or a microprocessor. It also includes other physical, biological, or chemical structures that can implement the same or equivalent functions as the processors listed above, such as biological neurons, quantum computing units, DNA computing units, etc., so that the processor can execute some or all of the steps in the computer program or method involved in the various embodiments of this application, or any combination of the steps mentioned therein.

[0019] The computer program involved in the embodiments can be stored in a computer device readable storage medium, which includes, but is not limited to, disks, magnetic tapes, magnetic cards, floppy disks, flash memory, optical disks, optical cards, read-only memory (ROM), random access memory (RAM), erasable programmable ROM (EPROM), and electrically erasable programmable ROM (EEPROM), etc., and also includes other biological, physical, or chemical structures that can achieve the same or equivalent functions as the storage media listed above, such as DNA, RNA, proteins, and other units with information storage capabilities. In specific embodiments, the storage medium involved can be one of the above-mentioned media types, or a combination of the above-mentioned media types. In different embodiments, the computer program involved in the embodiments can be centrally stored in a single medium, or distributed and stored in multiple media. The memory containing the computer device readable storage medium can be non-volatile memory or random access memory. These computer device readable storage media can be built into the device, or can be connected to the device involved in the embodiments as an external device or part of an external device. In some embodiments, the memory having a computer device readable storage medium is deployed locally; in other embodiments, the memory may be deployed remotely from the processor, for example, as a network-attached memory accessed via RF circuitry or an external port and a communication network, wherein the communication network may be the Internet, one or more intranets, a local area network (LAN), a wide area network (WLAN), a storage area network (SAN), or a suitable combination thereof, as long as computer device access to the memory is enabled. Furthermore, the computer program involved in the embodiments may be stored in plaintext / ciphertext form, or it may be designed as training data, integrated and recombined through model training and implicitly stored in the parameter states of a deep neural network or other machine learning model.

[0020] Please see Figure 1 In a first aspect, this embodiment provides a multi-functional touch control method for a guitar based on compound gestures, including: S101. Obtain the gesture input data performed by the user on the touch sensing area on the surface of the guitar neck. The touch sensing area is divided into a first functional area, a second functional area and a third functional area. S102. Perform finger topology analysis on the gesture input data, extract the relative positional relationship features between fingers, perform pattern matching between the relative positional relationship features and multiple pre-stored hand shape topology templates, and generate hand shape category recognition results. S103. Based on the hand shape category recognition result, query the corresponding control domain from the preset control domain mapping table and record it as the target control domain. The types of control domains include volume control domain, effect parameter control domain and global parameter control domain. S104. Extract contact position information from gesture input data, determine the functional zone based on the contact position information, and generate zone positioning results; S105. The hand shape category recognition result, target control domain and partition positioning result are fused and encoded to generate a composite gesture feature vector; S106. Based on the preset instruction mapping relationship, convert the composite gesture feature vector into the corresponding control instruction and output it.

[0021] In step S101, the first, second, and third functional zones correspond to different physical sections of the guitar neck. This division is based on the player's functional expectations and blind operation positioning habits for different sections. For example, the first functional zone corresponds to the section near the nut on the guitar neck, the second functional zone corresponds to the middle fret section, and the third functional zone corresponds to the section near the body of the guitar. Gesture input data is collected by a touch sensing layer (such as a capacitive or resistive touch sensing layer) at a preset frequency, and converted into a digital signal containing information such as touch position, area, and duration. In some optional embodiments, the touch sensing layer can also integrate pressure sensing functionality to synchronously collect the pressure data during the user's touch operation.

[0022] In step S102, finger topology analysis constructs relative positional features describing the current hand shape by calculating the relative geometric relationships between multiple touch points, including spacing and angles. Multiple pre-stored hand topology templates are defined based on the geometric features of typical playing hand shapes (such as multiple fingers together, multiple fingers spread, single finger extended, etc.). By comparing the real-time extracted features with each template for similarity, the category corresponding to the template with the highest matching degree is determined as the user's current hand shape category recognition result.

[0023] In step S103, the control domain types include commonly used control domains such as volume control domain, effects parameter control domain, and global parameter control domain. The control domain mapping table is preset based on the performer's commonly used control logic, associating different hand shape categories with different functional areas such as volume control domain, effects parameter control domain, or global parameter control domain. For example, a hand shape with multiple fingers together is associated with volume control. The identified hand shape category is used as an index to search within the mapping table, thereby determining the currently active target control domain.

[0024] In step S104, the boundary coordinates of the functional zones are pre-calculated and stored in the system as position parameters based on the total length of the touch-sensing area and the preset proportion of each zone. By comparing the longitudinal coordinate value of the touch point with these boundary coordinate values, the specific zone to which it belongs is determined, thereby generating the zone positioning result.

[0025] In step S105, fusion encoding refers to combining the hand shape category recognition result, the target control domain, and the zonal positioning result—three identifiers representing different control dimensions—into a structured data object, namely a composite gesture feature vector, according to predetermined encoding rules. This vector integrates the discrete recognition results into a unified input, providing a complete decision-making basis for subsequent command mapping. It should be noted that if pressure data is collected, this pressure data can also be incorporated into the fusion encoding process as another dimension of information, thereby generating a composite gesture feature vector containing pressure information to further enrich the nuances of control expression.

[0026] In step S106, the instruction mapping relationship can be preset by the developer or user based on the target device's control protocol (such as MIDI specifications or custom digital protocols), establishing a one-to-one correspondence between composite gesture feature vectors and specific control instructions. Using the composite gesture feature vector as an index, a match is performed in this mapping relationship to find the corresponding control instruction, which is then sent to an external effects device or the built-in digital signal processing module for execution via wired or wireless means.

[0027] This embodiment introduces a hand shape recognition mechanism based on finger topology analysis and combines it with the physical functional zoning of the touch sensing area to construct a composite gesture recognition mechanism based on the combination of hand shape category and functional zoning. This method deeply integrates the user's natural operating habits with the system control logic. By fusing and encoding hand shape categories and zoning positioning results, it generates feature vectors that accurately describe complex control intentions, which are ultimately mapped to specific control commands. This significantly improves the intuitiveness, richness, and robustness of control, enabling performers to achieve real-time and efficient control of multiple guitar functions through natural and intuitive operations without interrupting their playing.

[0028] In some embodiments, finger topology analysis is performed on the gesture input data to extract the relative positional relationship features between fingers. These features are then matched with multiple pre-stored hand shape topology templates to generate hand shape category recognition results, including: Extract the coordinate data of each fingertip on the touch sensing area from the gesture input data, calculate the Euclidean distance between any two fingertips based on the coordinate data, and generate a fingertip distance matrix. Extract the orientation angle data of each fingertip from the gesture input data, calculate the angle between any two fingertips based on the orientation angle data, and generate a fingertip angle matrix; The fingertip distance matrix and the fingertip angle matrix are concatenated to generate a finger topological feature vector; Calculate the cosine similarity between the finger topological feature vector and each pre-stored hand topological template, which includes the index and middle fingers together template, the three fingers spread template, the fist template, and the thumb and index finger pinching template. The hand type corresponding to the hand type topology template whose cosine similarity value exceeds the preset matching threshold is determined as the hand type category recognition result.

[0029] In this embodiment, coordinate data is extracted from the position information of each touch point contained in the gesture input data. Euclidean distance is calculated based on the coordinate data of any two fingertips. The distance is obtained by taking the square root of the sum of the squares of the coordinate differences between the two points. The resulting fingertip distance matrix is ​​a symmetric matrix with the fingertip identifier as the row and column index, which is used to structurally store the straight-line distance relationship between all fingertip pairs.

[0030] Orientation angle data is also extracted from gesture input data. It can be deduced based on the shape of the fingertip touch area or the direction of the touch trajectory. The included angle is obtained by calculating the difference between the orientation angles of any two fingertips. The generated fingertip angle matrix and fingertip distance matrix have the same symmetrical structure and are used to store the orientation difference relationship between fingertip pairs.

[0031] The effective information in the fingertip distance matrix and fingertip angle matrix is ​​extracted and combined in sequence into a one-dimensional sequence, which is the finger topological feature vector. This vector integrates the geometric information of the two dimensions of distance and angle, and can uniquely represent the spatial shape of the current hand.

[0032] Cosine similarity assesses directional consistency by calculating the cosine of the angle between two vectors. Pre-stored hand shape topology templates correspond to feature vectors of standard hand shapes such as index and middle fingers together, three fingers spread, a fist, and thumb and index finger pinched together. Cosine similarity is calculated between each finger topology feature vector and each template vector to determine the degree of matching. A matching threshold is preset based on a balance between recognition accuracy and false recognition rate. Each cosine similarity value is compared to this threshold, and the hand shape category corresponding to the hand shape topology template with the highest similarity exceeding the preset matching threshold is selected as the hand shape category recognition result.

[0033] This embodiment constructs a multi-dimensional feature vector that integrates fingertip distance and angle, and uses cosine similarity for template matching, achieving accurate quantification and recognition of hand shape geometry. This method does not rely on the absolute coordinates of the touch point, effectively resisting interference from hand displacement and rotation during performance, and providing accurate and reliable input for subsequent control domain queries based on hand shape categories.

[0034] In some embodiments, the fingertip distance matrix and the fingertip angle matrix are concatenated to generate a finger topological feature vector, including: Read the fingertip distance matrix. The fingertip distance matrix is ​​a symmetric matrix with each fingertip identifier as the row index and column index. Each element in the fingertip distance matrix is ​​the Euclidean distance value between the corresponding two fingertips. Read the fingertip angle matrix. The fingertip angle matrix is ​​a symmetric matrix with each fingertip identifier as the row index and column index. Each element in the fingertip angle matrix is ​​the angle value between the directions of the two corresponding fingertips. Extract the elements of the upper triangular region of the fingertip distance matrix in row-major order to generate a distance feature sequence; Extract the elements of the upper triangular region of the fingertip angle matrix in row-major order to generate an angle feature sequence; The distance feature sequence and the angle feature sequence are concatenated end to end according to a preset concatenation order to generate a one-dimensional vector form of finger topological feature vector.

[0035] In this embodiment, the fingertip identifier in the fingertip distance matrix is ​​used to uniquely identify each detected fingertip. The matrix constructed using the fingertip identifier as the row and column index is a symmetric matrix, symmetric about the main diagonal with zero elements on the main diagonal. Each matrix element stores the Euclidean distance value between the corresponding two fingertips. The fingertip angle matrix is ​​constructed similarly, also as a symmetric matrix, with each matrix element storing the angle value between the directions of the corresponding two fingertips.

[0036] Since both the fingertip distance matrix and the fingertip angle matrix are symmetric matrices, their lower and upper triangular regions contain completely overlapping information. Extracting only the elements from the upper triangular region is sufficient to obtain non-redundant information for all fingertip pairs. Row-priority extraction involves starting from the first row of the matrix and sequentially extracting the elements located in the upper triangular portion of that row, then processing the next row, until all upper triangular elements have been extracted. The resulting distance feature sequence is a one-dimensional sequence containing the distance information between all fingertip pairs. The angle feature sequence is generated in the same way as the distance feature sequence, based on row-priority extraction of the upper triangular region of the fingertip angle matrix, and contains the angle information between all fingertip pairs.

[0037] The preset splicing order is a predefined sequence combination rule, such as placing the distance feature sequence first and the angle feature sequence last. By directly concatenating the two sequences end to end, a longer one-dimensional vector of finger topology feature vector that integrates distance and angle information is generated. This vector is the standardized input used for subsequent similarity matching with the hand topology template.

[0038] This embodiment eliminates data redundancy by extracting the upper triangular region of the symmetric matrix and generating a feature sequence in row-major order. It also integrates distance and angle information into a compact and structurally unified feature vector, providing a standardized data foundation for subsequent efficient and accurate template matching.

[0039] In some embodiments, based on the hand shape category recognition result, the corresponding control domain is queried from a preset control domain mapping table, denoted as the target control domain, including: Read the hand shape category recognition results, which include the index and middle fingers together category, the three fingers spread apart category, the fist category, or the thumb and index finger pinched category. Load a preset control domain mapping table from the storage unit. The control domain mapping table records the correspondence between each hand shape category and each control domain. Among them, the index and middle fingers together category corresponds to the volume control domain, the three fingers spread category corresponds to the effect parameter control domain, the fist category corresponds to the global parameter control domain, and the thumb and index finger pinching category corresponds to the filter control domain. Using the hand shape category recognition result as the query key, a hash lookup operation is performed in the control domain mapping table to locate the mapping entry that matches the hand shape category recognition result; Extract the corresponding control domain identifier from the mapping entry, and determine the control domain indicated by the control domain identifier as the target control domain.

[0040] In this embodiment, the hand shape category recognition result is determined by the aforementioned finger topology analysis process and is stored in the form of a string or identifier, containing four preset categories. The index and middle fingers together category refers to a hand shape formed by extending the index and middle fingers and bringing them close together, while the other fingers are naturally curved; the three fingers spread category refers to a hand shape formed by extending the index, middle, and ring fingers and spreading them apart; the fist category refers to a hand shape formed by bending all four fingers and clenching them, with only the index finger extended; and the thumb and index finger pinched category refers to a hand shape formed by pinching the tips of the thumb and index finger together in a circular shape. These four categories cover the most commonly used and easily distinguishable basic hand shapes when the performer performs functional control on the neck of the instrument, each corresponding to different finger topology features.

[0041] The control domain mapping table is pre-built and stored in non-volatile memory units based on the control logic commonly used by the performer. This mapping table records the correspondence between each hand shape category and each control domain. The volume control domain is used to adjust the output volume of the guitar; the effects parameter control domain is used to adjust effects parameters such as reverb depth, delay time, and distortion; the global parameter control domain is used to adjust overall parameters such as gain and wet / dry ratio; and the filter control domain is used to adjust filtering parameters such as cutoff frequency and formant. Among them, the index and middle fingers together category is associated with the volume control domain because of its stable shape and ease of maintenance; the three fingers spread category is associated with the effects parameter control domain because of its clear directionality; the fist clench category is associated with the global parameter control domain because of its decisive movement; and the thumb and index finger pinch category is associated with the filter control domain because of its precise movement.

[0042] Using the hand shape category recognition result as the query key, a hash algorithm is used to convert it into an index value. This index value is then used to directly locate the corresponding storage location in the control domain mapping table, quickly finding the mapping entry that matches the query key. The hash lookup operation utilizes the mapping relationship between key values ​​and storage locations, avoiding linear traversal of the entire mapping table, significantly improving query efficiency and meeting the low latency requirements of real-time performance scenarios. The control domain identifier is extracted from the value field of the located mapping entry. This identifier is a code or name that uniquely indicates a specific control domain. The control domain it indicates is determined as the target control domain and is associated with the hand shape category recognition result for storage, to be used in subsequent fusion encoding steps.

[0043] This embodiment establishes a predefined correspondence between hand shape category recognition results and control domains, and employs hash lookup technology to achieve rapid mapping. This logically associates hand shapes familiar to the performer with specific control functions, reducing the learning cost while ensuring efficient querying and meeting the low latency requirements of real-time performance scenarios.

[0044] In some embodiments, using the hand shape category recognition result as the query key, a hash lookup operation is performed in the control domain mapping table to locate the mapping entry that matches the hand shape category recognition result, including: Read the hand shape category recognition result. The hand shape category recognition result includes a hand shape category name string. The hand shape category name string is one of the following: index finger and middle finger together, three fingers open, fist clenched, or thumb and index finger pinched together. Input the hand type category name string into the preset hash function. The hash function performs bytecode conversion operation on the hand type category name string to generate a hash key value corresponding to the hand type category name string. Using the hash key as an index, the access control domain mapping table is used to locate the target hash bucket corresponding to the hash key in the hash bucket array. Iterate through each mapping entry stored in the target hash bucket and compare the key field of each mapping entry with the hand type category name string byte by byte; Mapping entries that match the byte-by-byte comparison results are identified as mapping entries that match the hand type recognition results; Extract the corresponding control domain identifier from the mapping entry, and determine the control domain indicated by the control domain identifier as the target control domain, including: Read the key-value pair data structure in the mapping entry. The key-value pair data structure contains a key field and a value field. The key field stores the hand type recognition result, and the value field stores the control field identifier. Extract the control domain identifier from the value field. The control domain identifier is one of the following: volume control domain identifier, effect parameter control domain identifier, global parameter control domain identifier, or filter control domain identifier. The control domain indicated by the control domain identifier is identified as the target control domain, and the target control domain is associated with and stored with the hand shape category recognition result for subsequent fusion encoding steps.

[0045] In this embodiment, the hand type name string is a readable text string, used as the input key for the hash lookup operation. A preset hash function can be pre-selected based on a common string hash algorithm, capable of mapping strings of arbitrary length to fixed-length numerical values. After the hand type name string is input into the preset hash function, the hash function performs bytecode conversion operations on each character in the hand type name string, converting it into a corresponding bytecode value. Then, through mixed calculations, these bytecode values ​​are integrated into a fixed-length value, which is the hash key corresponding to the hand type name string.

[0046] Using hash keys as indices, the bucket index within the predefined hash bucket array is calculated by performing a modulo operation on the length of the hash bucket array. Based on this bucket index, the hash bucket array is accessed to locate the target hash bucket corresponding to the hash key. The hash bucket array consists of multiple buckets, each storing mapping entries with the same or similar hash keys. This grouped storage method narrows the search range to a single bucket.

[0047] Iterate through each mapping entry stored in the target hash bucket, and compare the key field of each mapping entry with the hand type category name string byte by byte, starting from the first byte. When all bytes of the two strings are exactly the same, the comparison result is determined to be consistent, and the mapping entry is identified as the mapping entry that matches the hand type category recognition result.

[0048] This embodiment achieves rapid mapping from hand shape category recognition results to target control domains through hash lookup technology: the hand shape category name string is converted into a hash key value using a hash function, and the search range is narrowed to a single bucket by combining the group storage mechanism of the hash bucket array. Then, the mapping entries are accurately matched within the bucket by byte-by-byte comparison to resolve hash collisions. Finally, the control domain identifier is extracted from the value field of the key-value pair data structure to determine the target control domain. While ensuring search accuracy, the hash index avoids linear traversal of the entire mapping table, significantly improving query efficiency and meeting the low-latency requirements for control domain mapping in real-time performance scenarios.

[0049] In some embodiments, contact position information is extracted from gesture input data, the corresponding functional zone is determined based on the contact position information, and a zone positioning result is generated, including: Two-dimensional coordinate data of each contact point are extracted from the gesture input data. The two-dimensional coordinate data includes the longitudinal coordinate value along the length of the neck and the lateral coordinate value perpendicular to the length of the neck. Read the preset functional partition boundary parameters, which include the first boundary longitudinal coordinate value between the first functional partition and the second functional partition, and the second boundary longitudinal coordinate value between the second functional partition and the third functional partition. The longitudinal coordinate values ​​of each contact point are compared with the longitudinal coordinate values ​​of the first boundary and the second boundary, respectively. If the longitudinal coordinate value is less than the longitudinal coordinate value of the first boundary, then the corresponding contact point will be assigned to the first functional zone. If the longitudinal coordinate value is greater than or equal to the longitudinal coordinate value of the first boundary and less than the longitudinal coordinate value of the second boundary, then the corresponding contact point will be assigned to the second functional zone. If the longitudinal coordinate value is greater than or equal to the longitudinal coordinate value of the second boundary, the corresponding contact point will be assigned to the third functional zone. Summarize the functional zone affiliation information of each contact point and generate a zone location result that includes the correspondence between contact point identifiers and functional zone identifiers.

[0050] In this embodiment, the two-dimensional coordinate data is extracted from the position information of each contact point contained in the gesture input data. The vertical coordinate value extends along the length of the neck and is used to distinguish different functional zones. The horizontal coordinate value is perpendicular to the length of the neck and is used to distinguish different positions within the same zone.

[0051] The preset functional zone boundary parameters are predetermined based on the physical dimensions and zone design of the guitar neck. The setting of the first and second boundary longitudinal coordinate values ​​is based on the following: the first functional zone corresponds to the section of the neck near the nut, which is relatively narrow; the second functional zone corresponds to the middle fret section, which is the widest; and the third functional zone corresponds to the section near the body of the guitar. These boundary coordinate values ​​are calculated by measuring the ratio of the actual physical length of each zone on the neck to the total length of the touch-sensitive area and are stored in non-volatile memory.

[0052] Numerical comparisons are executed through conditional statements, comparing the longitudinal coordinates of each contact point with the longitudinal coordinates of the first and second boundaries to determine the positional range of the contact point relative to each boundary. Based on the comparison results, the contact points are assigned to the corresponding functional zones: those before the first boundary are assigned to the first functional zone, those between the first and second boundaries are assigned to the second functional zone, and those after the second boundary are assigned to the third functional zone.

[0053] Each detected contact point is assigned a unique number as a contact point identifier. The functional zone to which each contact point belongs is recorded in the form of a code as a functional zone identifier. The affiliation information of all contact points is summarized to generate a zone positioning result containing the correspondence between contact point identifiers and functional zone identifiers.

[0054] This embodiment achieves rapid determination of the functional zone to which the touch position belongs by comparing the longitudinal coordinate value of the contact point with the boundary parameters preset based on the physical structure of the guitar neck. This method utilizes the physical zone characteristics of the guitar neck, enabling the player to locate the desired control area blindly through tactile feedback. Simultaneously, it achieves efficient position determination through simple numerical comparison, providing reliable input for subsequent fusion coding based on zone information.

[0055] In some embodiments, the hand shape category recognition result, the target control domain, and the partition localization result are fused and encoded to generate a composite gesture feature vector, including: Read the hand shape category recognition result. The hand shape category recognition result includes a hand shape category identifier, which is one of the following: index finger and middle finger together, three fingers open, fist clenched, or thumb and index finger pinched together. Read the target control domain, which contains a control domain identifier. The control domain identifier is one of the following: volume control domain identifier, effect parameter control domain identifier, global parameter control domain identifier, or filter control domain identifier. Read the partition location result. The partition location result includes a function partition identifier, which is one of the first function partition identifier, the second function partition identifier, or the third function partition identifier. Map the hand type identifier to the first numerical code, the control domain identifier to the second numerical code, and the functional area identifier to the third numerical code; The first, second, and third numerical codes are concatenated according to a preset vector dimension order to generate a composite gesture feature vector in three-dimensional vector form.

[0056] In this embodiment, the hand type category identifier is a predefined abbreviation, which differs from the hand type category name string used for hash lookup. The hand type category name string is in readable text form and is used as input to the hash function; while the hand type category identifier is a simplified code form, dedicated to the subsequent numerical mapping process, determined by the aforementioned hand type recognition process and stored in the hand type recognition result. The control field identifier is also a predefined abbreviation, used as input to the numerical mapping, and originates from the same source as the control field identifier extracted from the hash mapping entry value field, but in this embodiment, it is used for encoding the mapping rather than looking up the output. The functional partition identifier is a predefined abbreviation, determined by the aforementioned partition positioning process and stored in the partition positioning result.

[0057] The hand type identifier, control domain identifier, and functional partition identifier are used as the query criteria to find the corresponding values ​​in the preset encoding mapping table. The encoding mapping table is pre-established with ordered numbering based on the category attributes of each identifier, thereby converting the discrete category identifiers into continuous numerical representations, resulting in the first numerical code, the second numerical code, and the third numerical code.

[0058] The preset vector dimension order is the predefined arrangement order of the numerical codes in the vector. The three numerical codes are arranged sequentially according to this order to generate an ordered array containing three numerical components, which is the composite gesture feature vector in three-dimensional vector form. This vector fuses the discrete recognition results of the three dimensions into a unified quantitative expression. Each unique three-dimensional combination corresponds to a specific composite gesture, providing a standardized input basis for subsequent command mapping.

[0059] This embodiment achieves the quantitative expression of complex gestures by mapping hand shape category identifiers, control domain identifiers, and functional partition identifiers to numerical codes and concatenating them into a three-dimensional vector in a preset order. This method compresses discrete information from three dimensions into a unified vector space, ensuring that each complex gesture has a unique structured description, providing a clear and efficient decision-making basis for subsequent mapping from feature vectors to control commands.

[0060] In some embodiments, before converting the composite gesture feature vector into the corresponding control command, the method further includes: Collect string vibration data, which includes string amplitude data and string frequency data; Based on the string amplitude data and string frequency data, generate the string state modulation coefficient; The string state modulation coefficients are weighted and the composite gesture feature vector is weighted to generate the modulated composite gesture feature vector. The modulated composite gesture feature vector is used as the composite gesture feature vector; Based on a preset instruction mapping relationship, the composite gesture feature vector is converted into corresponding control instructions and output, including: Read the composite gesture feature vector, which is a three-dimensional vector containing the first dimension numerical code, the second dimension numerical code, and the third dimension numerical code; A preset instruction mapping table is loaded from the storage unit. The instruction mapping table records the correspondence between each composite gesture feature vector and each control instruction. The control instructions include MIDI control change instructions and digital signal control instructions. Using the composite gesture feature vector as the query index, a vector space nearest neighbor search is performed in the instruction mapping table to calculate the Euclidean distance between the composite gesture feature vector and the reference feature vector stored in each mapping entry of the instruction mapping table. Select the mapping entry with the smallest Euclidean distance and extract the corresponding control instruction identifier from the mapping entry; Based on the control instruction identifier, the corresponding control instruction data packet is retrieved from the control instruction library. The control instruction data packet contains an instruction type field, an instruction channel field, and an instruction parameter field. Control command data packets are sent to external effects devices or built-in digital signal processing modules via wireless Bluetooth MIDI protocol or wired USB MIDI protocol to enable real-time adjustment of guitar tone, volume, or effects parameters.

[0061] In this embodiment, string vibration state data is acquired through a pickup or piezoelectric sensor located at the guitar bridge. The generation of the string state modulation coefficient is not a simple numerical combination, but rather based on the synergistic relationship between amplitude and frequency: for example, when the amplitude is large and the frequency is high, the modulation coefficient increases accordingly, making the control effect more significant; when the amplitude is small or the frequency is low, the modulation coefficient decreases accordingly, making the control effect softer. This design allows the modulation coefficient to reflect the player's current playing force and register position, providing a physical basis for subsequent weighted calculations. Optionally, the string state modulation coefficient is calculated by weighted summation of normalized amplitude and frequency values; the weighting coefficient can be preset according to different playing styles.

[0062] Furthermore, by combining the string state modulation coefficient with the values ​​of each dimension of the composite gesture feature vector through weighted calculations, the final control command dynamically changes with the string vibration state. For example, when the player plucks the string forcefully, the modulation coefficient increases, and the values ​​of each dimension in the composite gesture feature vector are correspondingly amplified, thus making the control effect produced by the same hand shape and zone combination more intense; conversely, the control effect is more delicate when the string is plucked gently. This modulation mechanism makes gesture control no longer a fixed mapping, but rather a real-time linkage with the performance state, enhancing musical expressiveness.

[0063] Preferably, the correspondence between each composite gesture feature vector and each control command in the command mapping table can be automatically generated through preset mapping rules, or it can be configured by developers or users according to actual needs (e.g., manually edited through accompanying software) to adapt to different playing styles and device configurations. Among the control commands, MIDI control change commands are used to be compatible with standard MIDI devices, and digital signal control commands are used to drive the built-in digital signal processing module. The two command formats can be selected according to the target device type.

[0064] During the nearest neighbor search in the vector space, Euclidean distance is used to measure the similarity between the composite gesture feature vector and each reference feature vector. Since the instruction mapping table may contain a large number of mapping entries, in real-time performance scenarios, the search process can be accelerated by constructing a spatial index structure for the reference feature vectors. For example, a tree structure can be used to partition the vector space, narrowing the search scope from global traversal to a local region, thus meeting low latency requirements. After selecting the mapping entry with the smallest Euclidean distance, the control instruction identifier is extracted from it, and then the corresponding control instruction data packet is called from the control instruction library.

[0065] The structure of the control command data packet meets the protocol requirements of the target device. The command type field identifies the function category of the command, the command channel field specifies the target device channel, and the command parameter field carries the specific control values. The control command data packet is sent to external effects devices or the built-in digital signal processing module via either the wireless Bluetooth MIDI protocol or the wired USB MIDI protocol. The two protocols are suitable for wireless and wired scenarios respectively, and can be flexibly selected according to the actual usage environment.

[0066] This embodiment modulates compound gestures by incorporating string vibration states, allowing the control effect to dynamically change with playing intensity and register, thus enhancing musical expressiveness. The configurable command mapping table adapts to the needs of different users and devices, while vector space nearest neighbor search combined with a spatial index structure ensures efficient real-time mapping. Support for multiple protocols ensures compatibility with external effects devices and the built-in digital signal processing module. This embodiment achieves precise control while providing performers with a richer and more natural interactive experience.

[0067] In some embodiments, acquiring gesture input data performed by a user on a touch-sensitive area of ​​the guitar neck surface includes: The capacitive touch sensor array in the touch sensing area is driven at a preset sampling frequency to collect data on the capacitance change of each sensing unit. Baseline calibration is performed on the capacitance change data by subtracting the background capacitance value under no-touch state from the capacitance change data to generate calibrated capacitance change data. Spatial filtering is performed on the calibrated capacitance change data to remove sensor unit data corresponding to isolated noise points and generate filtered capacitance change data. Identify continuous sensing unit regions where the capacitance change exceeds a preset contact threshold from the filtered capacitance change data, and mark the continuous sensing unit regions as valid contact regions. Extract the peak position of capacitance change of each sensing unit within the effective contact area, and convert the peak position of capacitance change into corresponding two-dimensional coordinate data; Touch duration data and touch motion trajectory data are extracted from the effective contact area. The touch duration data includes the contact start time and contact end time, and the touch motion trajectory data includes a two-dimensional coordinate sequence at each sampling time. Two-dimensional coordinate data, touch duration data, and touch motion trajectory data are integrated into a structured data package, which is then recorded as gesture input data.

[0068] In this embodiment, the sampling frequency is determined based on the typical operation speed and power consumption requirements of human-computer interaction. The capacitive touch sensor array covering the surface of the guitar neck is driven by this frequency. The capacitance value of each sensor unit at the current moment is read one by one and compared with the reference capacitance value in the non-touch state to obtain capacitance change data reflecting the capacitance change of each sensor unit.

[0069] Baseline calibration can eliminate capacitance drift caused by factors such as ambient temperature and humidity. By subtracting the background capacitance values ​​of each sensing unit that were pre-collected and stored in the non-touch state from the current capacitance change data, the calibrated capacitance change data is obtained, so that the data only reflects the real touch information.

[0070] Spatial filtering is used to eliminate spurious signals caused by electromagnetic interference or sensor noise. By comparing the value of each sensing unit in the calibrated capacitance change data with the values ​​of its neighboring units, the sensing unit data corresponding to isolated noise points that deviate significantly from the neighborhood average are removed, resulting in filtered capacitance change data, which improves the spatial continuity of the data.

[0071] The preset contact threshold is determined based on empirical values ​​or calibration processes and is used to distinguish between intentional touch and proximity sensing. Sensing units whose capacitance changes exceed the preset contact threshold are identified from the filtered capacitance change data. Adjacent sensing units of this type are merged into a continuous sensing unit region through connected component analysis, and this region is marked as an effective contact area.

[0072] Within the effective contact area, the location of the sensing unit with the largest capacitance change is searched and identified as the peak location of the capacitance change. Using a preset mapping relationship between physical location and coordinates, this peak location is converted into two-dimensional coordinate data on the touch sensing area. By tracking the positional changes of the effective contact area in consecutive sampling frames, the number of sampling cycles from the start to the end of the contact is recorded to obtain the touch duration data. Simultaneously, the two-dimensional coordinates at each sampling moment are recorded, forming a two-dimensional coordinate sequence, which is the touch motion trajectory data.

[0073] Two-dimensional coordinate data, touch duration data, and touch motion trajectory data are encapsulated according to a preset field format to generate a structured data packet containing touch point identifiers, spatial location, time information, and motion path. This structured data packet is recorded as gesture input data and can be called by subsequent finger topology analysis and partition positioning steps.

[0074] This embodiment eliminates environmental drift through baseline calibration, removes isolated noise through spatial filtering, extracts effective regions through threshold comparison and connected component analysis, determines precise coordinates through peak positioning, and acquires dynamic information through inter-frame tracking. Ultimately, it transforms the original capacitance signal into structured gesture input data, providing a reliable data foundation for subsequent high-level recognition and mapping.

[0075] Please see Figure 2In a second aspect, this embodiment also provides a multi-functional touch control device 1 for guitars based on composite gestures, applicable to the method described in the first aspect. The device includes a touch sensing module 11, a hand shape recognition module 12, a control domain query module 13, a partition positioning module 14, a fusion encoding module 15, and an instruction mapping module 16. The touch sensing module 11 is disposed on the surface of the guitar neck and includes a capacitive touch sensor array. The area covered by the capacitive touch sensor array is divided into a first functional partition, a second functional partition, and a third functional partition. The hand shape recognition module 12 is connected to the touch sensing module 11. The hand shape recognition module 12 receives gesture input data output by the touch sensing module 11, performs finger topology analysis on the gesture input data, extracts the relative positional relationship features between fingers, and performs pattern matching between the relative positional relationship features and multiple pre-stored hand shape topology templates to generate a hand shape category recognition result. The control domain query module 13 is connected to the hand shape recognition module 12 and receives hand shape categories. The recognition result is used to query the corresponding control domain from the preset control domain mapping table based on the hand shape category recognition result. This is denoted as the target control domain. The types of control domains include volume control domain, effect parameter control domain, and global parameter control domain. The partition positioning module 14 is connected to the touch sensing module 11. The partition positioning module 14 receives gesture input data, extracts contact position information from the gesture input data, determines the functional partition to which it belongs based on the contact position information, and generates the partition positioning result. The fusion encoding module 15 is connected to the hand shape recognition module 12, the control domain query module 13, and the partition positioning module 14. The fusion encoding module 15 receives the hand shape category recognition result, the target control domain, and the partition positioning result, and fuses and encodes the hand shape category recognition result, the target control domain, and the partition positioning result to generate a composite gesture feature vector. The command mapping module 16 is connected to the fusion encoding module 15. The command mapping module 16 receives the composite gesture feature vector, converts the composite gesture feature vector into the corresponding control command according to the preset command mapping relationship, and outputs it.

[0076] In this embodiment, the capacitive touch sensor array is integrated via a flexible circuit board. The area covered by this array is divided into a first functional zone, a second functional zone, and a third functional zone through electrode wiring layout or software configuration. This is used to convert the user's touch operation into gesture input data containing information such as touch position, area, and duration. This embodiment constructs a processing link from touch input to control command output by sequentially connecting the touch sensing module 11, hand shape recognition module 12, control domain query module 13, zone positioning module 14, fusion encoding module 15, and command mapping module 16. This enables real-time and intuitive control of guitar tone, volume, or effects parameters, allowing the performer to complete multi-functional operations without interrupting their performance.

[0077] By adopting the above technical solution, this invention differs from existing technologies and has the following beneficial effects: By analyzing the finger topology of gesture input data, extracting the relative positional relationship features between fingers, and performing pattern matching with multiple pre-stored hand shape topology templates, a hand shape category recognition result is generated. Simultaneously, contact position information is extracted from the gesture input data to determine the corresponding functional area, generating a partition positioning result. Then, the hand shape category recognition result, the target control domain obtained from the control domain mapping table based on this result, and the partition positioning result are fused and encoded to generate a composite gesture feature vector. Finally, according to a preset instruction mapping relationship, this composite gesture feature vector is converted into a corresponding control instruction and output. The above technical solution constructs a composite gesture recognition mechanism based on the combination of hand shape category and functional area, deeply integrating the performer's natural operating habits with the system control logic. Through fusion encoding, multi-dimensional recognition results are integrated into a unified feature vector, significantly improving the intuitiveness and richness of control. This allows performers to achieve real-time and efficient control of guitar tone, volume, or effects parameters through natural and intuitive operations without interrupting performance.

[0078] Finally, it should be noted that although the above embodiments have been described in the text and drawings of this application, this should not limit the scope of patent protection of this application. Any technical solutions that are based on the essential concept of this application and utilize the content described in the text and drawings of this application, resulting in equivalent structural or procedural substitutions or modifications, as well as the direct or indirect application of the technical solutions of the above embodiments to other related technical fields, are all included within the scope of patent protection of this application.

Claims

1. A multi-functional touch control method for guitar based on compound gestures, characterized in that, include: The system acquires gesture input data performed by the user on the touch-sensitive area on the surface of the guitar neck, wherein the touch-sensitive area is divided into a first functional area, a second functional area, and a third functional area. The gesture input data is analyzed for finger topology, and the relative positional relationship features between fingers are extracted. The relative positional relationship features are then matched with multiple pre-stored hand shape topology templates to generate hand shape category recognition results. Based on the hand shape category recognition result, the corresponding control domain is queried from the preset control domain mapping table and recorded as the target control domain. The types of the control domain include volume control domain, effect parameter control domain and global parameter control domain. The touch position information is extracted from the gesture input data, and the corresponding functional zone is determined based on the touch position information to generate a zone positioning result. The hand shape category recognition result, the target control domain, and the partition positioning result are fused and encoded to generate a composite gesture feature vector; According to the preset instruction mapping relationship, the composite gesture feature vector is converted into the corresponding control instruction and output.

2. The multi-functional touch control method for guitar based on compound gestures according to claim 1, characterized in that, The gesture input data is subjected to finger topology analysis to extract the relative positional relationship features between fingers. These features are then matched with multiple pre-stored hand shape topology templates to generate hand shape category recognition results, including: Extract the coordinate data of each fingertip on the touch sensing area from the gesture input data, calculate the Euclidean distance between any two fingertips based on the coordinate data, and generate a fingertip distance matrix. Extract the orientation angle data of each fingertip from the gesture input data, calculate the angle between any two fingertips based on the orientation angle data, and generate a fingertip angle matrix; The fingertip distance matrix and the fingertip angle matrix are concatenated to generate a finger topological feature vector; Calculate the cosine similarity between the finger topological feature vector and each pre-stored hand topological template, wherein the hand topological template includes a template for index and middle fingers together, a template for three fingers spread apart, a template for making a fist, and a template for thumb and index finger pinching together; The hand type corresponding to the hand type topology template whose cosine similarity value exceeds the preset matching threshold is determined as the hand type recognition result.

3. The multi-functional touch control method for guitar based on composite gestures according to claim 2, characterized in that, The fingertip distance matrix and the fingertip angle matrix are concatenated to generate a finger topological feature vector, including: Read the fingertip distance matrix, which is a symmetric matrix with each fingertip identifier as the row and column index, and each matrix element in the fingertip distance matrix is ​​the Euclidean distance value between two corresponding fingertips; Read the fingertip angle matrix, which is a symmetric matrix with each fingertip identifier as the row and column index, and each matrix element in the fingertip angle matrix is ​​the angle value between the directions of the corresponding two fingertips; Extract the elements of the upper triangular region of the fingertip distance matrix in row-major order to generate a distance feature sequence; Extract the elements of the upper triangular region of the fingertip angle matrix in row-major order to generate an angle feature sequence; The distance feature sequence and the angle feature sequence are concatenated end-to-end according to a preset concatenation order to generate a one-dimensional vector form of finger topological feature vector.

4. The multi-functional touch control method for guitar based on compound gestures according to claim 1, characterized in that, Based on the hand shape category recognition result, the corresponding control domain is queried from the preset control domain mapping table, denoted as the target control domain, including: Read the hand shape category recognition result, which includes the index and middle fingers together category, three fingers spread category, fist category, or thumb and index finger pinched category; A preset control domain mapping table is loaded from the storage unit. The control domain mapping table records the correspondence between each hand shape category and each control domain. The category of index and middle fingers together corresponds to the volume control domain, the category of three fingers spread corresponds to the effect parameter control domain, the category of fist clenching corresponds to the global parameter control domain, and the category of thumb and index finger pinching corresponds to the filter control domain. Using the hand shape category recognition result as the query key, a hash lookup operation is performed in the control domain mapping table to locate the mapping entry that matches the hand shape category recognition result; Extract the corresponding control domain identifier from the mapping entry, and determine the control domain indicated by the control domain identifier as the target control domain.

5. The multi-functional touch control method for guitar based on compound gestures according to claim 4, characterized in that, Using the hand shape category recognition result as the query key, a hash lookup operation is performed in the control domain mapping table to locate the mapping entry that matches the hand shape category recognition result, including: Read the hand shape category recognition result, which includes a hand shape category name string. The hand shape category name string is one of the following: index finger and middle finger together, three fingers spread, fist clenched, or thumb and index finger pinched together. The hand type category name string is input into a preset hash function, and the hash function performs bytecode conversion operation on the hand type category name string to generate a hash key value corresponding to the hand type category name string; Using the hash key value as an index, access the preset hash bucket array in the control domain mapping table, and locate the target hash bucket in the hash bucket array that corresponds to the hash key value; Traverse each mapping entry stored in the target hash bucket and compare the key field of each mapping entry with the hand type category name string byte by byte; The mapping entries that match the byte-by-byte comparison results are determined as the mapping entries that match the hand type recognition results; Extracting the corresponding control domain identifier from the mapping entry, and determining the control domain indicated by the control domain identifier as the target control domain, includes: Read the key-value pair data structure in the mapping entry. The key-value pair data structure includes a key field and a value field. The key field stores the hand type recognition result, and the value field stores the control domain identifier. Extract the control domain identifier from the value field. The control domain identifier is one of the following: volume control domain identifier, effect parameter control domain identifier, global parameter control domain identifier, or filter control domain identifier. The control domain indicated by the control domain identifier is identified as the target control domain, and the target control domain is associated with and stored with the hand shape category recognition result for subsequent fusion encoding steps.

6. The multi-functional touch control method for guitar based on compound gestures according to claim 1, characterized in that, Contact position information is extracted from the gesture input data, and the corresponding functional zone is determined based on the contact position information to generate a zone positioning result, including: Two-dimensional coordinate data of each contact point are extracted from the gesture input data. The two-dimensional coordinate data includes longitudinal coordinate values ​​along the length of the neck and transverse coordinate values ​​perpendicular to the length of the neck. Read the preset functional partition boundary parameters, which include the first boundary longitudinal coordinate value between the first functional partition and the second functional partition, and the second boundary longitudinal coordinate value between the second functional partition and the third functional partition; The longitudinal coordinate values ​​of each contact point are compared with the longitudinal coordinate values ​​of the first boundary and the second boundary, respectively. If the longitudinal coordinate value is less than the longitudinal coordinate value of the first boundary, then the corresponding contact point is assigned to the first functional partition. If the longitudinal coordinate value is greater than or equal to the first boundary longitudinal coordinate value and less than the second boundary longitudinal coordinate value, then the corresponding contact point is assigned to the second functional partition. If the longitudinal coordinate value is greater than or equal to the longitudinal coordinate value of the second boundary, then the corresponding contact point is assigned to the third functional partition. Summarize the functional zone affiliation information of each contact point to generate the zone positioning result containing the correspondence between contact point identifiers and functional zone identifiers.

7. The multi-functional touch control method for guitar based on compound gestures according to claim 1, characterized in that, The hand shape category recognition result, the target control domain, and the partition localization result are fused and encoded to generate a composite gesture feature vector, including: Read the hand shape category recognition result, which includes a hand shape category identifier. The hand shape category identifier is one of the following: index finger and middle finger together, three fingers open, fist clenched, or thumb and index finger pinched together. Read the target control domain, which contains a control domain identifier, which is one of the following: volume control domain identifier, effect parameter control domain identifier, global parameter control domain identifier, or filter control domain identifier; Read the partition location result, which includes a functional partition identifier, and the functional partition identifier is one of a first functional partition identifier, a second functional partition identifier, or a third functional partition identifier; The hand type identifier is mapped to a first numerical code, the control domain identifier is mapped to a second numerical code, and the functional partition identifier is mapped to a third numerical code. The first numerical code, the second numerical code, and the third numerical code are concatenated according to a preset vector dimension order to generate the composite gesture feature vector in three-dimensional vector form.

8. The multi-functional touch control method for guitar based on compound gestures according to claim 1, characterized in that, Before converting the composite gesture feature vector into the corresponding control command, the method further includes: Collect string vibration state data, which includes string amplitude data and string frequency data; Based on the string amplitude data and the string frequency data, a string state modulation coefficient is generated; The string state modulation coefficient is weighted and the composite gesture feature vector is weighted to generate the modulated composite gesture feature vector. The modulated composite gesture feature vector is used as the composite gesture feature vector; According to a preset instruction mapping relationship, the composite gesture feature vector is converted into a corresponding control instruction and output, including: Read the composite gesture feature vector, which is a three-dimensional vector containing a first-dimensional numerical code, a second-dimensional numerical code, and a third-dimensional numerical code; A preset instruction mapping table is loaded from the storage unit. The instruction mapping table records the correspondence between each composite gesture feature vector and each control instruction. The control instructions include MIDI control change instructions and digital signal control instructions. Using the composite gesture feature vector as a query index, a vector space nearest neighbor search is performed in the instruction mapping table to calculate the Euclidean distance between the composite gesture feature vector and the reference feature vectors stored in each mapping entry of the instruction mapping table. Select the mapping entry with the smallest Euclidean distance, and extract the corresponding control instruction identifier from the mapping entry; According to the control instruction identifier, the corresponding control instruction data packet is retrieved from the control instruction library. The control instruction data packet includes an instruction type field, an instruction channel field, and an instruction parameter field. The control command data packet is sent to an external effects device or a built-in digital signal processing module via wireless Bluetooth MIDI protocol or wired USB MIDI protocol to complete the real-time adjustment of guitar tone, volume or effects parameters.

9. The multi-functional touch control method for guitar based on composite gestures according to claim 1, characterized in that, Acquire gesture input data performed by the user on the touch-sensitive area of ​​the guitar neck surface, including: The capacitive touch sensor array in the touch sensing area is driven at a preset sampling frequency to collect capacitance change data on each sensing unit. The capacitance change data is subjected to baseline calibration processing. The background capacitance value under no-touch state is subtracted from the capacitance change data to generate calibrated capacitance change data. Spatial filtering is performed on the calibrated capacitance change data to remove sensor unit data corresponding to isolated noise points, and filtered capacitance change data is generated. From the filtered capacitance change data, identify continuous sensing unit regions where the capacitance change exceeds a preset contact threshold, and mark the continuous sensing unit regions as valid contact regions. Extract the peak position of capacitance change of each sensing unit within the effective contact area, and convert the peak position of capacitance change into corresponding two-dimensional coordinate data; Touch duration data and touch motion trajectory data are extracted from the effective contact area. The touch duration data includes the contact start time and the contact end time, and the touch motion trajectory data includes a two-dimensional coordinate sequence at each sampling time. The two-dimensional coordinate data, the touch duration data, and the touch motion trajectory data are integrated into a structured data packet, which is then denoted as the gesture input data.

10. A multi-functional touch control device for guitars based on compound gestures, characterized in that, The apparatus applicable to the method of any one of claims 1 to 9 comprises: A touch sensing module is disposed on the surface of the guitar neck. The touch sensing module includes a capacitive touch sensor array, and the area covered by the capacitive touch sensor array is divided into a first functional zone, a second functional zone, and a third functional zone. A hand shape recognition module is connected to the touch sensing module. The hand shape recognition module receives gesture input data output by the touch sensing module, performs finger topology analysis on the gesture input data, extracts the relative positional relationship features between fingers, performs pattern matching between the relative positional relationship features and multiple pre-stored hand shape topology templates, and generates hand shape category recognition results. The control domain query module is connected to the hand shape recognition module. The control domain query module receives the hand shape category recognition result and queries the corresponding control domain from the preset control domain mapping table according to the hand shape category recognition result. This is denoted as the target control domain. The types of the control domain include volume control domain, effect parameter control domain, and global parameter control domain. A partition positioning module is connected to the touch sensing module. The partition positioning module receives the gesture input data, extracts contact position information from the gesture input data, determines the functional partition to which the partition belongs based on the contact position information, and generates a partition positioning result. The fusion encoding module is connected to the hand shape recognition module, the control domain query module, and the partition positioning module. The fusion encoding module receives the hand shape category recognition result, the target control domain, and the partition positioning result, and performs fusion encoding on the hand shape category recognition result, the target control domain, and the partition positioning result to generate a composite gesture feature vector. The instruction mapping module is connected to the fusion encoding module. The instruction mapping module receives the composite gesture feature vector, converts the composite gesture feature vector into corresponding control instructions according to a preset instruction mapping relationship, and outputs them.

Images