Mobile terminal-based music production somatosensory control method, system, device and medium

Abstract

The present application relates to a kind of mobile terminal-based music production somatosensory control method, system, equipment and medium, it relates to digital music production control technical field, its method includes: opening synchronous thread, synchronous thread includes: first thread, for obtaining the device posture information sent by mobile operating system; Based on device posture information and preset geometric solution algorithm, determine the continuous control angle without deadlock;Based on the continuous control angle without deadlock, determine the first MIDI control parameter value;Second thread, for obtaining the touch event of user, determine the second MIDI control parameter value based on touch event and user configuration information;Based on the first MIDI control parameter value and the second MIDI control parameter value, determine target MIDI control parameter set, and send target MIDI control parameter set to all connected MIDI output port.The present application provides professional level mobile control experience with music production intuition, ready-to-use, powerful expression.

Description

Technical Field

[0001] This invention relates to the field of digital music production control technology, and in particular to a music production motion control method, system, device and medium based on a mobile terminal. Background Technology

[0002] In the field of professional music production, the development of digital technology has greatly promoted the progress of music composition. The emergence of MIDI technology has enabled music creators to more precisely control various parameters of music, such as dynamics, expression, and panning, bringing more possibilities and creativity to music production.

[0003] To control MIDI continuous controller signals, existing control solutions mainly fall into two categories. One category is controllers based on physical hardware, such as MIDI keyboards, consoles, and fader panels. These physical hardware devices are designed with precise control and tactile feedback in mind, allowing users to experience the adjustment of musical parameters through actual operation. The other category is controllers based on general-purpose software. These solutions are controlled through a graphical interface on a computer or mobile device, such as virtual faders and sliders. Users can adjust musical parameters on the software interface through clicks and swipes, offering advantages in convenience and low cost.

[0004] However, physical hardware-based controllers are expensive, inconvenient to carry, and require additional connections and configurations, making them difficult to adapt to the mobile and lightweight creative trends. General-purpose software-based controllers are limited by the precision and real-time performance of touchscreens and the lack of physical feedback, making it difficult to meet the needs of professional music production for delicate and continuous parameter adjustments. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a method, system, device and medium for music production motion control based on a mobile terminal, in order to solve at least one of the above-mentioned technical problems.

[0006] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: Firstly, this application provides a motion-sensing control method for music production based on a mobile terminal, employing the following technical solution: A motion-sensing control method for music production based on a mobile terminal includes: Start a synchronization thread, which includes a first thread and a second thread; The first thread is used to acquire device posture information sent by the mobile operating system in real time, the device posture information including the device's gravity vector components; determine a deadlock-free continuous control angle based on the device posture information and a preset geometric calculation algorithm; and determine the first MIDI control parameter value based on the deadlock-free continuous control angle, a preset angle range mapping method, a preset smoothing algorithm, and user configuration information. The second thread is used to acquire user touch events in real time, the touch events including touch position and touch state; and to determine the value of the second MIDI control parameter based on the touch events and the user configuration information. Based on the first MIDI control parameter value of the first thread and the second MIDI control parameter value of the second thread, a target MIDI control parameter set is determined, and the target MIDI control parameter set is sent to all connected MIDI output ports.

[0007] The beneficial effects of this invention are as follows: Enabling a synchronous thread and setting the first and second threads to process in parallel ensures that motion data acquisition, calculation, and touch event response do not interfere with each other, achieving efficient coordination between motion control and touch control in terms of time and logic, and improving control real-time performance; The first thread acquires the device posture information processed by the system and performs geometric calculation and angle mapping, avoiding numerical jumps or deadlocks, and ensuring the continuity and stability of the control signal; The second thread acquires user touch events and generates control parameter values, enriching the control methods; The parameter values ​​generated by the two types of threads are merged and sent to all connected MIDI output ports, realizing a plug-and-play connection experience and reliable signal transmission, providing a professional-grade mobile control experience that is intuitive for music production, ready to use, and highly expressive.

[0008] Based on the above technical solution, the present invention can be further improved as follows.

[0009] Furthermore, determining the deadlock-free continuous control angle based on the device attitude information and a preset geometric calculation algorithm includes: Based on the gravity vector components of the device, determine the Y-axis and Z-axis components of the device in the preset device coordinate system; Based on the Y-axis component, the Z-axis component, and the two-parameter arctangent function, the deadlock-free continuous control angle value is determined.

[0010] The beneficial effects of adopting the above-mentioned further solution are as follows: by first determining the Y-axis and Z-axis components of the device in the preset device coordinate system, and then combining the two-parameter arctangent function to determine the deadlock-free continuous control angle value, the problem of numerical jumps or deadlocks in the traditional angle calculation around ±90° is avoided, ensuring the continuity and stability of the control angle, thus providing a basis for the subsequent accurate determination of the first MIDI control parameter value, enabling the mobile terminal to provide more accurate motion control input for the music production process.

[0011] Furthermore, the determination of the first MIDI control parameter value based on the deadlock-free continuous control angle, the preset angle range mapping method, the preset smoothing algorithm, and user configuration information includes: Based on a first-order low-pass filter algorithm or a Kalman filter algorithm, the continuous control angle without deadlock is smoothed to obtain the smoothed continuous control angle. Based on the angle mapping range defined in the user configuration information, the same smoothed continuous control angle is mapped to the first MIDI control parameter value of multiple different controllers; The user configuration information is stored locally on the mobile terminal or on a cloud server that is associated with and synchronized through the user account.

[0012] The advantages of adopting the above-mentioned further solution are as follows: using a first-order low-pass filtering algorithm to smooth the continuous control angle without deadlock can suppress hand-held micro-shakes and ensure the stability and continuity of the control signal; mapping the smoothed continuous control angle to the first MIDI control parameter values ​​of multiple different controllers according to the user configuration information can meet diverse music production control needs; the user configuration is stored locally on the mobile terminal or synchronized with the user account, which can realize seamless switching of control configuration between different DAWs or different projects, making it convenient for users.

[0013] Furthermore, it also includes: Based on user preferences in the user configuration information, the filtering strength coefficients of the first-order low-pass filtering algorithm or the Kalman filtering algorithm are dynamically adjusted.

[0014] The beneficial effects of adopting the above-mentioned further scheme are: by dynamically adjusting the filtering strength coefficient of the first-order low-pass filtering algorithm according to the user's preference for filtering strength, the personalized needs of different users for signal smoothness can be met, effectively suppressing handheld micro-shakes and improving the stability and accuracy of control signals.

[0015] Furthermore, determining the second MIDI control parameter value based on the touch event and the user configuration information includes: The touch events are subject to accidental touch arbitration detection; When the touch event is detected by arbitration, the touch position and touch state are mapped to the corresponding second MIDI control parameter values ​​based on the user configuration information; The step of performing accidental touch arbitration detection on the touch event includes: Detects whether the touch pressure exceeds a preset pressure threshold; And / or, detect whether the touch duration has reached the valid duration; And / or, detect whether the device motion acceleration exceeds a preset motion intensity threshold when a touch event occurs.

[0016] The beneficial effects of adopting the above-mentioned further solutions are as follows: Anti-accidental touch arbitration detection of touch events can avoid erroneous MIDI control parameter values ​​due to accidental touches, ensuring the accuracy and stability of control; when a touch event passes arbitration detection, the touch position is mapped to the corresponding second MIDI control parameter value, enabling accurate control of music production according to the user's intent; arbitration detection by detecting touch pressure, touch duration, and device motion acceleration can accurately determine whether a touch event is intentional from multiple perspectives, further improving the effectiveness of anti-accidental touch measures.

[0017] Furthermore, determining the target MIDI control parameter set based on the first MIDI control parameter value of the first thread and the second MIDI control parameter value of the second thread includes: Timestamp the first MIDI control parameter value and the second MIDI control parameter value respectively; Based on timestamps, the first MIDI control parameter value and the second MIDI control parameter value are merged into a unified message queue in chronological order of occurrence to obtain a unified MIDI message queue. Based on the mutex lock mechanism and the unified MIDI message queue, the target MIDI control parameter set is determined.

[0018] The beneficial effects of adopting the above-mentioned further scheme are: timestamp the first MIDI control parameter value and the second MIDI control parameter value and merge them into a unified message queue in chronological order, which can ensure the precise coordination of continuous motion control and discrete touch events on the timeline and avoid signal corruption; determining the target MIDI control parameter set based on the mutex lock mechanism and the unified MIDI message queue can guarantee thread safety and the accuracy of parameter determination.

[0019] Furthermore, sending the target MIDI control parameter set to all connected MIDI output ports includes: Based on the native MIDI service of the mobile operating system, a list of available MIDI output ports is discovered and maintained. The list includes all MIDI output ports and the port types of the MIDI output ports, including USB-MIDI interfaces, network MIDI session members, and virtual MIDI ports. The target MIDI control parameter set is sequentially sent to each MIDI output port by traversing the port list.

[0020] The advantages of adopting the above-mentioned further solution are: it can automatically discover and dynamically maintain a full list of MIDI output ports, including USB-MIDI, network MIDI, and virtual ports, fundamentally solving the pain points of traditional solutions that require manual configuration and have poor compatibility. By employing a mechanism of traversing the port list and broadcasting data, it ensures that the same control signal can be sent synchronously and indiscriminately to all active output channels. It ensures that control signals can be stably received in various complex music production environments, thus providing users with a seamless, stable, and efficient professional-grade mobile control experience.

[0021] Secondly, this application provides a music production motion-sensing control system based on a mobile terminal, which adopts the following technical solution: A motion-sensing control system for music production based on a mobile terminal, comprising: An initialization module is used to start a synchronization thread, which includes a first thread and a second thread. The first thread includes: An attitude sampling module is used to acquire device attitude information sent by the mobile operating system in real time, the device attitude information including the device's gravity vector components; The continuous angle calculation module is used to determine the deadlock-free continuous control angle based on the device attitude information and the preset geometric calculation algorithm. The smooth mapping and configuration module is used to determine the first MIDI control parameter value based on the deadlock-free continuous control angle, the preset angle value range mapping method, the preset smoothing algorithm and user configuration information; The second thread includes: The acquisition module is used to acquire user touch events in real time, the touch events including touch position and touch state; A configuration module is used to determine the value of the second MIDI control parameter based on the touch event and the user configuration information; The broadcast sending module is used to determine a target MIDI control parameter set based on the first MIDI control parameter value of the first thread and the second MIDI control parameter value of the second thread, and send the target MIDI control parameter set to all connected MIDI output ports.

[0022] Thirdly, this application provides an electronic device that adopts the following technical solution: An electronic device includes a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executed as described in any of the first aspects of a mobile terminal-based music production motion control method.

[0023] Fourthly, this application provides a computer-readable storage medium, which adopts the following technical solution: A computer-readable storage medium storing a computer program capable of being loaded by a processor and executing the method for music production motion control based on a mobile terminal as described in any of the first aspects.

[0024] Additional aspects and advantages of this application will be set forth in part in the description which follows, and will become apparent from the description or may be learned by practice of this application. Attached Figure Description

[0025] Figure 1 A flowchart illustrating a motion-sensing control method for music production based on a mobile terminal, provided as an embodiment of the present invention; Figure 2 A schematic diagram of a mobile terminal-based music production motion control system is provided in one embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an electronic device provided in one embodiment of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article, unless otherwise specified, generally indicates that the preceding and following related objects have an "or" relationship.

[0028] This application provides a motion-sensing control method for music production based on a mobile terminal. This method can be executed by an electronic device, which can be a controller or a mobile terminal device, such as a laptop, mobile phone, or tablet computer, but is not limited thereto.

[0029] This invention integrates the CoreMotion framework: used to acquire system-fused device motion data at the optimal frequency supported by the device (typically 60Hz or higher), particularly the gravity vector attribute in the CMDeviceMotion object. This vector is stable and has eliminated linear acceleration interference, providing an ideal basis for tilt sensing.

[0030] Integrated CoreMIDI framework: used to create virtual MIDI ports, discover and manage network MIDI sessions (MIDINetworkSession), and communicate with connected USB-MIDI interfaces to create and send MIDI messages.

[0031] like Figure 1 As shown, a motion-sensing control method for music production based on a mobile terminal mainly includes: S1, Start a synchronization thread, which includes a first thread and a second thread; The first thread is used to acquire device posture information sent by the mobile operating system in real time, the device posture information including the device's gravity vector components; determine a deadlock-free continuous control angle based on the device posture information and a preset geometric calculation algorithm; and determine the first MIDI control parameter value based on the deadlock-free continuous control angle, a preset angle range mapping method, a preset smoothing algorithm, and user configuration information. The second thread is used to acquire user touch events in real time, the touch events including touch position and touch state; and to determine the value of the second MIDI control parameter based on the touch events and the user configuration information. S2, based on the first MIDI control parameter value of the first thread and the second MIDI control parameter value of the second thread, determine the target MIDI control parameter set and send the target MIDI control parameter set to all connected MIDI output ports.

[0032] In this embodiment, the first thread acquires device attitude information sent by the mobile operating system in real time. This attitude information includes the device's gravity vector components. In mobile terminals, this information is typically acquired through built-in sensors, such as accelerometers and gyroscopes. Accelerometers measure the device's acceleration in various directions, thereby calculating the gravity vector components; gyroscopes measure the device's rotational angular velocity. These sensors act like the terminal's "eyes," constantly sensing changes in the device's attitude. Besides common accelerometers and gyroscopes, auxiliary devices such as geomagnetic sensors can also be used to acquire attitude information more accurately.

[0033] Furthermore, the first thread, based on the device attitude information and a preset geometric calculation algorithm, determines the deadlock-free continuous control angle, including: Based on the gravity vector components of the device, determine the Y-axis and Z-axis components of the device in the preset device coordinate system; Based on the Y-axis component, the Z-axis component, and the two-parameter arctangent function, the deadlock-free continuous control angle value is determined.

[0034] In this embodiment, the Y-axis and Z-axis components of the device in a preset device coordinate system are first determined based on the device's gravity vector components. To map the device's tilt around its horizontal axis (left-right axis) into a continuous control quantity, instead of directly using attitude Euler angles (pitch / roll) output, the components of the gravity vector in the YZ plane are geometrically calculated. For example, the pitch angle can be calculated as θ = atan2(-gy,-gz), and the radians can be converted into angle values. The atan2 function output range is (-π,π], which can maintain continuous angle output even when the device approaches ±90° attitude, thereby avoiding numerical jumps and deadlock. In embodiments requiring only unidirectional control, the portion θ < 0 can be clipped to make θ ≥ 0.

[0035] This is analogous to defining the position of an object on specific coordinate axes in a three-dimensional space. Then, based on these components and the two-parameter arctangent function, a deadlock-free, continuous control angle value is determined. Using the two-parameter arctangent function avoids numerical jumps or deadlocks around ±90°, ensuring the continuity of angle values. For example, in traditional angle calculation methods, when the device approaches a ±90° angle, the angle value may suddenly jump or lock, leading to discontinuous control. This method, using the two-parameter arctangent function, effectively solves this problem, making control smoother. Alternatively, other methods such as curve fitting can also be used to achieve continuous angle calculation.

[0036] Optionally, the smooth mapping and configuration step determines the first MIDI control parameter value based on the deadlock-free continuous control angle, the preset angle value range mapping method, the preset smoothing algorithm, and user configuration information.

[0037] The specific operation is as follows: based on a first-order low-pass filter algorithm or a Kalman filter algorithm, the deadlock-free continuous control angle is smoothed to obtain the smoothed continuous control angle. The function of the first-order low-pass filter algorithm is to remove high-frequency noise from the signal, making the control angle more stable. For example, when holding a mobile terminal, slight hand tremors can cause fluctuations in the control angle; the first-order low-pass filter algorithm can effectively suppress these fluctuations. The filter strength coefficient can be dynamically adjusted according to user preferences. If the user wants more sensitive control, the filter strength coefficient can be decreased; if the user wants more stable control and reduces the impact of jitter, the filter strength coefficient can be increased.

[0038] Subsequently, based on the angle mapping range defined in the user configuration information, the same smoothed continuous control angle is mapped to the first MIDI control parameter values ​​of multiple different controllers. For example, the controller value range [0,127] or the 14-bit control value range, and the corresponding first MIDI control parameter values ​​are generated according to the user configuration.

[0039] User configuration information is stored locally on the mobile device or synchronized with the user account, allowing users to use the same configuration across different devices or scenarios. This ensures that the control configuration is bound to the user or device, completely independent of any external DAW project files, achieving "set once, take effect everywhere".

[0040] In this embodiment, a structured configuration model is defined within the electronic device to store user-personalized control schemes. This model includes at least: Target MIDI channel number (1-16); Target controller number (CC Number, e.g., 1 for modulation, 11 for expression); The status of this control channel's activation; The range of angle-to-MIDI value mappings (minimum angle, maximum angle); Optional curve types (linear, logarithmic, etc.).

[0041] The second thread also contains multiple processing parts.

[0042] Optionally, a second thread determines a second MIDI control parameter value based on the touch event and the user configuration information, including: The touch events are subject to accidental touch arbitration detection; When the touch event is detected by arbitration, the touch position and touch state are mapped to the corresponding second MIDI control parameter values ​​based on the user configuration information; The step of performing accidental touch arbitration detection on the touch event includes: Detects whether the touch pressure exceeds a preset pressure threshold; And / or, detect whether the touch duration has reached the valid duration; And / or, detect whether the device motion acceleration exceeds a preset motion intensity threshold when a touch event occurs.

[0043] In this application, a capacitive touchscreen is generally used to detect touch events on the screen of a mobile terminal. When a user touches the screen with their finger, the touchscreen detects a change in capacitance, thereby determining the location and state of the touch. Other types of screens, such as resistive touchscreens, can also be used to detect touch events.

[0044] The configuration section determines the second MIDI control parameter values ​​based on touch events and user configuration information. This includes performing anti-accidental touch arbitration detection on touch events: checking if the touch pressure exceeds a preset pressure threshold (too little pressure indicates an accidental touch and won't trigger the corresponding control operation); checking if the touch duration is sufficient (too short a duration indicates an accidental touch); and checking if the device's motion acceleration exceeds a preset motion intensity threshold (to prevent accidental touches if the device is moving rapidly). When a touch event passes the arbitration detection, the touch position and touch state are mapped to the corresponding second MIDI control parameter values ​​based on the user configuration information.

[0045] Finally, based on the first MIDI control parameter value of the first thread and the second MIDI control parameter value of the second thread, the target MIDI control parameter set is determined. Specifically, the first and second MIDI control parameter values ​​are timestamped, and then, based on the timestamps, they are merged into a unified message queue in chronological order, resulting in a unified MIDI message queue. To ensure data consistency and security, a mutex mechanism and the unified MIDI message queue are used to determine the target MIDI control parameter set. The mutex mechanism acts like a "gatekeeper," ensuring that only one thread can access and modify the unified MIDI message queue at any given time.

[0046] Furthermore, the target MIDI control parameter set is sent to all connected MIDI output ports, including: Based on the native MIDI service of the mobile operating system, a list of available MIDI output ports is discovered and maintained. The list includes all MIDI output ports and the port types of the MIDI output ports, including USB-MIDI interfaces, network MIDI session members, and virtual MIDI ports. The target MIDI control parameter set is sequentially sent to each MIDI output port by traversing the port list.

[0047] The above implementation, based on the mobile operating system's native MIDI service, discovers and maintains a list of available MIDI output ports. This list includes all MIDI output ports and their port types, such as USB-MIDI interfaces, network MIDI session members, and virtual MIDI ports. The port list is then iterated through, and the target MIDI control parameter set is sequentially sent to each MIDI output port. This ensures that control signals are received regardless of the port through which the music production equipment is connected.

[0048] The CoreMIDI functions, such as MIDIGetNumberOfDestinations() and MIDIGetDestination(), enumerate all available MIDI output endpoints in the current system.

[0049] Add the identified valid endpoints (such as USB-MIDI interfaces, virtual input ports for other applications, and network MIDI session members) to an active endpoint list.

[0050] When a target MIDI control parameter set needs to be sent, this list of active endpoints is iterated through, and the same MIDI data packet is sent sequentially to each endpoint in the list using the MIDISend() function. This "one-to-many" broadcast mode ensures that control signals from the mobile terminal can be reliably received regardless of which port the music production host (such as Logic Pro on a Mac) listens through (direct USB connection or network MIDI under the same Wi-Fi).

[0051] As an optional implementation method in this application, it further includes: Based on the tactile reference provided by the device's physical bezel, guide the user's finger to touch the initial position on the screen; Real-time acquisition of touch events and device motion status data; identification and arbitration of touch intent; differentiation between intentional operation and unintentional touch. For touch events that are arbitrated, the corresponding music control function is triggered according to the mapping configuration, and tactile and / or auditory feedback is provided to confirm the success of the operation; Monitor user operating habits and device usage scenarios to dynamically adjust the layout of functional controls, haptic feedback strategies, and accidental touch arbitration thresholds on the touchscreen.

[0052] The layout methods for functional controls on a touchscreen include: The high-frequency core function controls are fixedly placed in the four corners of the screen, using the physical border of the device as an absolute tactile reference point. Differentiated tactile identifiers are set for different functional areas or key controls. The tactile identifiers include simulated texture on the surface of the control and specific patterns of navigational vibrations triggered when a finger slides across the boundaries of different functional areas, resulting in an interactive interface that supports users to perform spatial positioning and navigation through touch.

[0053] In the above embodiments, the global gesture layer of the interactive interface is typically a transparent view covering the entire screen, used to capture specific global gestures (such as a three-finger double tap or an edge swipe in a specific area). These gestures are used to trigger high-priority, low-frequency operations, such as globally enabling / disabling motion control or switching control configuration sets. This layer view itself does not contain fine-grained controls.

[0054] Furthermore, the layout of functional controls, haptic feedback strategies, and accidental touch arbitration thresholds in the touchscreen are dynamically adjusted, including: Retrieve the current user's historical operation records and the currently active music production scene mode; Based on the usage frequency of each functional control, determine the typical touch pressure and movement speed of the user; Based on the connected DAW software, time or location information, determine whether the current mode is live performance music production mode, studio arrangement music production mode or practice music production mode. The most frequently used function controls are automatically adjusted to the most easily accessible screen areas, and the intensity of haptic feedback is fine-tuned based on the touch pressure according to user habits. For live performance music production mode, the anti-mistouch arbitration threshold is automatically increased to enhance anti-interference capabilities, resulting in updated function control layout configuration, haptic feedback parameters, and anti-mistouch arbitration threshold.

[0055] By acquiring user operating habits and current music production scenario modes (such as live performance, recording and arrangement), and adjusting the three core parameters accordingly, this multi-parameter linkage adaptive strategy fundamentally solves the industry problem that static interfaces of mobile controllers cannot balance flexibility and stability in changing usage environments.

[0056] The following example, using a live electronic music performance, illustrates how this invention can be deeply integrated into the professional music production process: The performer placed the iPhone on a handheld stand and connected it to a MacBook running Ableton Live via a USB-C to Lightning cable.

[0057] In the iPhone app, the performer has pre-configured a control scheme: tilting the wrist forward and backward controls CC11 (expression) of channel 1, tilting the wrist left and right controls CC1 (modulation) of channel 1, and the four large buttons on the screen are mapped to triggers for different timbres.

[0058] Once the performance begins, the performer's focus is primarily on the Live Session View and the audience. When a soaring melody needs to be introduced, he doesn't need to look at the iPhone screen; he simply locates and presses the button in the upper left corner of the screen (to trigger the corresponding timbre) using his fingertips, while simultaneously smoothly adjusting the expression parameters by naturally tilting his wrist forward, causing the timbre to gradually intensify.

[0059] During performance, the performer can adjust the modulation depth in real time through tilting movements to enhance the vibrato effect, based on the musical mood. In some embodiments, the application can simplify touch interaction or provide a locked entry point in performance mode to reduce interference from accidental touches.

[0060] After the performance, the control configuration remains saved on the iPhone. The next time the performer uses Logic Pro in the studio for arrangement, they simply open the same iPhone app, select the same configuration, and connect via Wi-Fi MIDI to continue using this motion control solution without needing to remap any controllers in the Logic Pro project. This example demonstrates how the present invention solves the problem of configuration reuse across projects and DAWs, and how simplified interaction in performance mode meets the high concentration requirements of live performances.

[0061] This invention, by enabling a synchronous thread and setting the first and second threads to process in parallel, ensures that motion data acquisition, processing, and touch event response do not interfere with each other, achieving efficient coordination between motion control and touch control in terms of time and logic, and improving control real-time performance. The first thread acquires the device posture information processed by the system and performs geometric calculation and angle mapping, avoiding numerical jumps or deadlocks, and ensuring the continuity and stability of the control signal. The second thread acquires user touch events and generates control parameter values, enriching the control methods. The parameter values ​​generated by the two types of threads are merged and sent to all connected MIDI output ports, realizing a plug-and-play connection experience and reliable signal transmission, providing a professional-grade mobile control experience that is intuitive for music production, ready to use, and highly expressive.

[0062] Figure 2 A schematic diagram of a mobile terminal-based music production motion control system 200 is shown.

[0063] like Figure 2 As shown, a mobile terminal-based music production motion control system 200 mainly includes: Initialization module 201 is used to start a synchronization thread, which includes a first thread and a second thread; The first thread includes: The attitude sampling module 2011 is used to acquire device attitude information sent by the mobile operating system in real time, the device attitude information including the device's gravity vector components; The continuous angle calculation module 2012 is used to determine the deadlock-free continuous control angle based on the device attitude information and the preset geometric calculation algorithm. The smooth mapping and configuration module 2013 is used to determine the first MIDI control parameter value based on the deadlock-free continuous control angle, the preset angle value range mapping method, the preset smoothing algorithm and user configuration information; The second thread includes: The acquisition module 2014 is used to acquire user touch events in real time, the touch events including touch position and touch state; Configuration module 2015 is used to determine the value of the second MIDI control parameter based on the touch event and the user configuration information; The broadcast sending module 202 is used to determine a target MIDI control parameter set based on the first MIDI control parameter value of the first thread and the second MIDI control parameter value of the second thread, and send the target MIDI control parameter set to all connected MIDI output ports.

[0064] In one example, the module in any of the above devices may be one or more integrated circuits configured to implement the above methods, such as one or more application-specific integrated circuits (ASICs), or one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs), or a combination of at least two of these integrated circuit forms.

[0065] For example, when modules in a device can be implemented via a processing element scheduler, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor capable of calling programs. Alternatively, these modules can be integrated together as a system-on-a-chip (SOC).

[0066] In this application, various objects such as messages / information / devices / network elements / systems / apparatus / actions / operations / processes / concepts may be named. It is understood that these specific names do not constitute a limitation on the relevant objects. The names may be changed depending on the scenario, context, or usage habits. The understanding of the technical meaning of the technical terms in this application should be mainly determined from their functions and technical effects embodied / performed in the technical solution.

[0067] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0068] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0069] Figure 3 This is a structural block diagram of an electronic device 300 according to an embodiment of this application.

[0070] like Figure 3 As shown, the electronic device 300 includes a processor 301 and a memory 302, and may further include one or more of an information input / output (I / O) interface 303, a communication component 304, and a communication bus 305.

[0071] The processor 301 controls the overall operation of the electronic device 300 to complete all or part of the steps in the aforementioned mobile terminal-based music production motion control method. The memory 302 stores various types of data to support the operation of the electronic device 300. This data may include, for example, instructions for any application or method operating on the electronic device 300, as well as application-related data. The memory 302 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as one or more of Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0072] I / O interface 303 provides an interface between processor 301 and other interface modules, such as keyboards, mice, and buttons. These buttons can be virtual or physical. Communication component 304 is used to test wired or wireless communication between electronic device 300 and other devices. Wireless communication includes Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, or 4G, or a combination thereof. Therefore, the corresponding communication component 304 may include a Wi-Fi component, a Bluetooth component, and an NFC component.

[0073] The communication bus 305 may include a path for transmitting information between the aforementioned components. The communication bus 305 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The communication bus 305 may be divided into an address bus, a data bus, a control bus, etc.

[0074] The electronic device 300 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to execute the mobile terminal-based music production motion control method given in the above embodiments.

[0075] The following describes the computer-readable storage medium provided in the embodiments of this application. The computer-readable storage medium described below can be referred to in correspondence with the music production motion control method based on mobile terminals described above.

[0076] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described mobile terminal-based music production motion control method.

[0077] The computer-readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0078] The terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0079] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the foregoing application concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions claimed in this application.

Claims

1. A method for motion-sensing control of music production based on a mobile terminal, characterized in that, include: Start a synchronization thread, which includes a first thread and a second thread; The first thread is used to acquire device posture information sent by the mobile operating system in real time, the device posture information including the device's gravity vector components; determine a deadlock-free continuous control angle based on the device posture information and a preset geometric calculation algorithm; and determine the first MIDI control parameter value based on the deadlock-free continuous control angle, a preset angle range mapping method, a preset smoothing algorithm, and user configuration information. The second thread is used to acquire the user's touch events in real time, the touch events including touch location and touch state; Based on the touch event and the user configuration information, the value of the second MIDI control parameter is determined; Based on the first MIDI control parameter value of the first thread and the second MIDI control parameter value of the second thread, a target MIDI control parameter set is determined, and the target MIDI control parameter set is sent to all connected MIDI output ports.

2. The method for motion-sensing control of music production based on a mobile terminal according to claim 1, characterized in that, The process of determining deadlock-free continuous control angles based on the device attitude information and a preset geometric calculation algorithm includes: Based on the gravity vector components of the device, determine the Y-axis and Z-axis components of the device in the preset device coordinate system; Based on the Y-axis component, the Z-axis component, and the two-parameter arctangent function, the deadlock-free continuous control angle value is determined.

3. The method for motion-sensing control of music production based on a mobile terminal according to claim 1, characterized in that, The method for determining the first MIDI control parameter value based on the deadlock-free continuous control angle, a preset angle range mapping method, a preset smoothing algorithm, and user configuration information includes: Based on a first-order low-pass filter algorithm or a Kalman filter algorithm, the continuous control angle without deadlock is smoothed to obtain the smoothed continuous control angle. Based on the angle mapping range defined in the user configuration information, the same smoothed continuous control angle is mapped to the first MIDI control parameter value of multiple different controllers; The user configuration information is stored locally on the mobile terminal or on a cloud server that is associated with and synchronized through the user account.

4. The method for music production motion control based on a mobile terminal according to claim 3, characterized in that, Also includes: Based on user preferences in the user configuration information, the filtering strength coefficients of the first-order low-pass filtering algorithm or the Kalman filtering algorithm are dynamically adjusted.

5. The method for motion-sensing control of music production based on a mobile terminal according to claim 3, characterized in that, The step of determining the second MIDI control parameter value based on the touch event and the user configuration information includes: The touch events are subject to accidental touch arbitration detection; When the touch event is detected by arbitration, the touch position and touch state are mapped to the corresponding second MIDI control parameter values ​​based on the user configuration information; The step of performing accidental touch arbitration detection on the touch event includes: Detects whether the touch pressure exceeds a preset pressure threshold; And / or, detect whether the touch duration has reached the valid duration; And / or, detect whether the device motion acceleration exceeds a preset motion intensity threshold when a touch event occurs.

6. The method for motion-sensing control of music production based on a mobile terminal according to claim 1, characterized in that, The determination of the target MIDI control parameter set based on the first MIDI control parameter value of the first thread and the second MIDI control parameter value of the second thread includes: Timestamp the first MIDI control parameter value and the second MIDI control parameter value respectively; Based on timestamps, the first MIDI control parameter value and the second MIDI control parameter value are merged into a unified message queue in chronological order of occurrence to obtain a unified MIDI message queue. Based on the mutex lock mechanism and the unified MIDI message queue, the target MIDI control parameter set is determined.

7. The method for motion-sensing control of music production based on a mobile terminal according to claim 1, characterized in that, Sending the target MIDI control parameter set to all connected MIDI output ports includes: Based on the native MIDI service of the mobile operating system, a list of available MIDI output ports is discovered and maintained. The list includes all MIDI output ports and the port types of the MIDI output ports, including USB-MIDI interfaces, network MIDI session members, and virtual MIDI ports. The target MIDI control parameter set is sequentially sent to each MIDI output port by traversing the port list.

8. A motion-sensing control system for music production based on a mobile terminal, characterized in that, include: An initialization module is used to start a synchronization thread, which includes a first thread and a second thread. The first thread includes: An attitude sampling module is used to acquire device attitude information sent by the mobile operating system in real time, the device attitude information including the device's gravity vector components; The continuous angle calculation module is used to determine the deadlock-free continuous control angle based on the device attitude information and the preset geometric calculation algorithm. The smooth mapping and configuration module is used to determine the first MIDI control parameter value based on the deadlock-free continuous control angle, the preset angle value range mapping method, the preset smoothing algorithm and user configuration information; The second thread includes: The acquisition module is used to acquire user touch events in real time, the touch events including touch position and touch state; A configuration module is used to determine the value of the second MIDI control parameter based on the touch event and the user configuration information; The broadcast sending module is used to determine a target MIDI control parameter set based on the first MIDI control parameter value of the first thread and the second MIDI control parameter value of the second thread, and send the target MIDI control parameter set to all connected MIDI output ports.

9. An electronic device, characterized in that, Includes a processor, which is coupled to a memory; The processor is configured to execute a computer program stored in the memory to cause the electronic device to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Includes a computer program or instructions that, when run on a computer, cause the computer to perform the method as described in any one of claims 1-7.

Images