Graphical programming multi-mode interactive input controller and multi-mode interactive control method and system

Abstract

The invention relates to the field of programmable technology, and discloses a graphical programming multi-mode interactive input controller and a multi-mode interactive control method and system.The multi-mode interactive input controller is a thin-sheet panel-shaped control handle with touch input contact electrodes directly exposed outwards; comprising a PCB (printed circuit board) and a touch input contact electrode which is arranged on the PCB and is directly exposed outwards, and a metal induction working surface of the contact electrode is directly triggered when being contacted by a human body of a user or is triggered when being contacted by the human body of the user through an extension line and a prop which are connected with the exposed touch input contact electrode. And multi-mode interactive input or control of graphical programming of the indirect contact control handle is carried out. Through multi-aspect improvement design, the multi-mode interaction of a user in a large space range, a large degree of freedom, multiple application scenes and participation of multiple props can be supported, and the multi-mode interaction platform can be widely applied to manufacturing of various programmable electronic products.

Description

Technical Field

[0001] The present invention relates to the field of programmable interactive control technology, and in particular to a graphically programmed multi-mode interactive input controller, a multi-mode interactive control method and a system. Background Art

[0002] With the development of mobile devices and Internet technology, graphical programming has begun to expand to mobile terminals and online platforms. Currently, various mainstream online graphical programming platforms, such as Python, Mind+, Scratch, etc., are becoming increasingly popular, making graphical programming more popular. Current graphical programming tools are more diverse and professional, not only used in the field of education, but also beginning to be applied to industrial automation, robot programming and other fields. The transition of graphical programming from professional fields to popularization by the general public has lowered the threshold for programming learning, allowing non-professionals to participate in programming and innovation.

[0003] In the prior art, the Chinese invention patent application document with the publication number CN109634592A discloses a graphical Python programming interactive method, system, and electronic device, which provides a building block editing module that can write Python code by dragging, splicing, and combining pre-stored building blocks in a building block programming mode; its text code editing module displays and writes Python code in the form of text in a text code programming mode; its conversion module converts encapsulated building blocks and Python code to each other; its running module executes the written Python code; and its output module is used to output the running results of the Python code. This embodiment of the invention assists Python programming by comparing the graphical building blocks one by one with the Python code, reduces the difficulty of Python programming, facilitates beginners such as teenagers to learn Python programming, and realizes visual and graphical programming.

[0004] Graphical programmable electronic products and systems in the prior art are usually presented and used in a software + hardware mode. The software part mainly includes programming platform tools, built-in code input, compilation and execution programs; the hardware part mainly includes input control terminals and execution output terminals. For example, the invention application with announcement number CN106178505A discloses a graphical children's programming toy and method thereof, the toy includes a graphical programming system and a controlled robot; the graphical programming system includes a graphical programming module, a data processing module and a data sending module; the graphical programming module includes a graphical programming block selection unit, a graphical programming block combination unit, a programming block editing unit, and a programming execution unit; the data processing module is used to process the programming block combination result for data processing, which is convenient for later transmission; the data sending module is used to send the data processed to the controlled robot; the controlled robot includes a data receiving module, a processor, a driving device and an execution device. The present invention combines graphical programming with physical robot control, makes it easier for children to understand the programming language through a graphical programming interface, and gives children a realistic programming experience through a physical robot.

[0005] In the learning, programming input and program control of conventional graphical programming, input is generally performed through the screen, keyboard or buttons of the programming intelligent terminal, which is not easy to operate, flexible, interactive and interesting. In order to overcome this shortcoming, some technologies that use game controllers for auxiliary input have appeared on the market. For example, Chinese patent document CN210573679U discloses a programming handle, which includes a shell, direction control keys, an external analog input terminal, a USB connection, a switch button, a sound sensor, a light sensor, a vibration sensor and a handle. The direction control key is arranged at the upper left corner of the upper surface of the shell, the switch button is arranged at the upper right corner of the upper surface of the shell, a sound sensor, a light sensor and a vibration sensor are arranged inside the shell, the USB connection is arranged at the top of the shell, and the external analog input terminal is arranged at the top of the shell and is next to the USB connection. The programming handle is connected via USB. Although it solves the technical problem of charging the programming handle, it needs to be connected to programming devices such as tablet computers and smart phones via USB. They all adopt a shell design, which is large in size and lacks fashion sense and transparency in product design. Input can only be made by directly operating the buttons on the handle. At the same time, the mechanical pressing buttons such as the direction control keys are prone to poor contact or damage after long-term use. The length of the USB connection is limited, and the connection and use are inconvenient. The operating space and operation method are subject to many restrictions.

[0006] In summary, the input and control of the graphical programming in the prior art have the following problems: first, the operation mode of programming input or interactive control is single, and the user can only directly operate the programming terminal or operate the key input of the programming handle (that is, only direct input mode can be used); most of the existing programming input control handles are from existing wired game handles, which are often thick and heavy, inconvenient to carry, and the traditional press buttons have a limited lifespan, and are prone to the problem of extremely poor compatibility of key commands during the graphical programming process. The structure of the buttons or keys is fixed and fully enclosed; some programming input control handles use traditional Bluetooth handles, which often have high energy consumption and save power by turning off the switch; if the switch is forgotten to be turned off, students will be unable to use the handle due to lack of power during graphical programming. The situation can easily affect the learning of programming; due to the limitation of the connection method between the programming terminal and the programming handle, the user can only operate the main body of the programming terminal and the programming handle, and cannot operate the non-main body (such as props and other items) without leaving the main body of the programming terminal and the programming handle; the programming terminal and the programming handle are large in size, single in shape, closed in structure, lack of visualization and fun, and there are many limitations on the operating space, degree of freedom, application scenarios and fun for the graphical programming process and the control during the execution of the program after programming. It is impossible to realize multi-mode interactive programming input and program operation control of graphical programming with a large spatial range, large degree of freedom, multiple programming and multiple execution modes, and it cannot meet the various needs of different users, different programming contents, program control and program control effects in different scenarios. Summary of the invention

[0007] The purpose of the present invention is to address the above-mentioned deficiencies and provide a new open-design graphical programming multi-mode interactive input controller, a multi-mode interactive control method and a system, which support multiple input modes, including multiple indirect input modes of non-direct operation of the handle body, through the simultaneous improvement of hardware and operation modes, greatly increasing the usage space and freedom of users and products, so that the fun and application scenarios of graphical programming, program operation control technology and their products can be significantly improved, and users can realize graphical programming in a large space range, large degree of freedom, multiple application scenarios and multiple props, and support multi-programming input, multi-execution control of multi-mode interactive control, so as to solve the various problems of the above-mentioned prior art and meet the various needs of users for graphical programming technology and products.

[0008] The present invention provides the following technical solutions: A graphically programmed multi-mode interactive input controller, characterized in that it is a thin panel-shaped control handle with a touch input contact electrode directly exposed outwardly, comprising a PCB circuit board, and at least one touch input contact electrode directly exposed outwardly arranged on the PCB circuit board, the metal sensing working surface of the exposed touch input contact electrode can be directly triggered by the user's human body contact, or can be triggered by the user's human body contact time through an extension line or prop connected to the exposed touch input contact electrode, thereby performing graphically programmed multi-mode interactive input or control of the control handle.

[0009] The graphically programmed multi-mode interactive input controller is an interactive input controller triggered by a capacitive signal, which uses the metal contact sheet of each exposed touch input contact electrode as the sensing working surface and triggers the input signal based on the current induction when the human body touches it.

[0010] The graphically programmed multi-mode interactive input controller is an interactive input controller triggered by a resistance signal, and the input signal is triggered by the on-off state between the positive working surface and the negative working surface of each exposed touch input contact electrode; each exposed touch input contact electrode includes a positive metal contact piece and a negative metal contact piece that are arranged on a PCB circuit board, adjacent to each other but separated, as a group of sensing working surfaces, corresponding to the positive working surface and the negative working surface respectively. When the human body contacts the positive working surface and the negative working surface at the same time, the on-off state of the current between the positive and negative electrodes changes, triggering the input signal.

[0011] The graphically programmed multi-mode interactive input controller has a metal positive working surface and a negative working surface located on the same plane of a PCB circuit board, arranged to surround each other, and having a narrow annular insulating groove between them; a small area of ​​skin at the front end of a human finger can simultaneously contact the positive working surface and the negative working surface in the same group of sensing working surfaces, triggering an input signal.

[0012] The extension wires are conductive wires, one end of each of which is connected to the sensing working surface of an exposed touch input contact electrode on the PCB circuit board, and the other end is connected to the conductive key area of ​​the prop; when the human body touches the conductive key area of ​​the prop, the trigger signal is transmitted to the exposed touch input contact electrode through the extension wire to perform multi-mode interactive input or control of indirect contact graphical programming; the props are physical objects that are compatible with the graphical programming content or the scene of program execution control.

[0013] A graphical programming multi-mode interactive control system, characterized by the graphical programming multi-mode interactive input controller, the graphical programming intelligent terminal, and the electronic product for executing the program; In the graphical programming intelligent terminal, graphical programming software is provided for receiving and processing programming signals or control signals sent by the multi-mode interactive input controller, or forwarding the programming signals or control signals to the electronic product executing the program; The electronic product for executing the program is used to execute the programming signal or control signal sent by the multi-mode interactive input controller or the graphical programming intelligent terminal.

[0014] The graphical programming intelligent terminal is a handheld intelligent terminal device or a computer; the graphical programming intelligent terminal is used to provide a graphical programming environment, programming software, or directly execute a compiled graphical program; The electronic product for executing the program includes a graphical programming intelligent terminal or a programmable electronic building block; A multi-mode interactive input controller includes a PCB circuit board provided with touch input contact electrodes directly exposed outward, a metal sensing working surface for connecting any exposed touch input contact electrode on the PCB circuit board, an extension line that can be triggered by the user's human body contact time, and a prop.

[0015] A multi-mode interactive control method for graphical programming, comprising the following steps: S1. Set up the multi-mode interactive control hardware environment A multi-mode interactive input controller for graphical programming, a graphical programming intelligent terminal, and an electronic product for executing the program are respectively provided; S2. Set up the multi-mode interactive control software environment In the graphic programming intelligent terminal, set up the graphic programming software; Connect the multi-mode interactive input controller, the graphical programming intelligent terminal, and the electronic products that execute the program through wireless signals; S3, Multi-mode interactive control operation for graphical programming The user can directly trigger the multi-mode interactive input controller by touching any metal sensing working surface of any touch input contact electrode on the multi-mode interactive input controller through any conductive part of the body, or indirectly trigger the multi-mode interactive input controller through an extension line or a metal sensing working surface on a prop, and then select one of the multiple interactive input mode combinations formed by the interactive input controller triggered by a resistance signal or the interactive input controller triggered by a capacitance signal, and send an input or control signal to the graphical programming intelligent terminal or the electronic product that executes the program to complete the graphical programming; S4. Presentation of multi-mode interactive control results of graphical programming After receiving and processing, the graphic programming intelligent terminal directly executes the compiled graphic program, or sends it to the electronic product that executes the program for execution; After receiving the signal, the electronic product executing the program executes the control command input by the user and presents the interactive control result of the graphical programming. It realizes the multi-mode interactive control of graphical programming with multiple programming and multiple execution modes in a large space range, large degree of freedom, multiple application scenarios, and multiple props.

[0016] The beneficial effects of the graphical programming multi-mode interactive input controller, multi-mode interactive control method and system provided by the present invention include at least the following points: 1. The multi-mode interactive input controller, multi-mode interactive control method and system for graphical programming provided by the present invention focus on the various deficiencies caused by the closed design of the prior art, and design a thin panel-shaped control handle with touch input contact electrodes directly exposed to the outside to cooperate with the extension line and props, so as to improve the hardware and operation mode simultaneously, and realize the scenario-based and high-freedom coordination of software and hardware and input and control operations; the system provided by the present invention supports multiple input modes and can be applied to larger spaces and more scenarios, including multiple indirect input modes of multiple non-direct operation handle bodies, which greatly increases the use space and freedom of users and products, and significantly improves the fun and application scenarios of graphical programming, program operation control technology and its products. Users can realize graphical programming with a large space range, large degree of freedom, multiple application scenarios and multiple props, and support multi-programming input, multi-execution control multi-mode interactive control, which comprehensively solves various problems of the prior art and can simultaneously meet the various needs of users for graphical programming technology and products.

[0017] 2. The present invention not only supports multiple input modes, including multiple indirect input modes of the non-direct operation handle itself, but more importantly, it further integrates the capacitive signal triggering and the resistive signal triggering mode to further support the single-point (single-hand) operation mode. For each touch input contact electrode, no matter whether it adopts the capacitive signal triggering or the resistive signal triggering mode, the user can input or control it by single-handed touch. The user only needs to touch any contact electrode with one finger in one hand to trigger the signal input, and the signal input or control operation can be completed with one hand. The use space and freedom of users and products are greatly increased, so that the interest and application scenarios of programmable technology and its products are significantly improved, and the various needs of graphical programming technology and products can be met.

[0018] 3. The present invention completely breaks through the limitations of the prior art on the user's usage space, scenarios and freedom by introducing extension cords and props, cooperating with an open-design PCB circuit board, and directly exposing the touch input contact electrodes. The multi-mode interactive input controller is embedded in various props of scenario-based design, or connected and combined with props in other forms, so that the multi-mode interactive input controller can be fully integrated into the specific program running scenario, which greatly improves the fun and freedom of programmable technology and its products, enriches the application scenarios, and improves the user experience, so it can meet the various scenario-based needs of graphical programming technology and products.

[0019] 4. The multi-mode interactive input controller and each exposed touch input contact electrode provided by the present invention simplify the overall structural design through various structural improvements, miniaturize the whole machine, and reduce power consumption. The design of the switch key and the shell is cancelled, and each key is improved from a traditional press button to a resistive PAD (capacitive signal sensor) directly arranged on the PCB board, so that it has a compact structure, small size and light weight. No wiring is required during the programming process, and a paired Bluetooth connection is used to reduce communication failures during use. The control distance is long and the use is flexible, thereby improving the overall service life, work efficiency and stability of the touch-type wireless graphical programming touch display panel.

[0020] 5. The multi-mode interactive input controller provided by the present invention has improved structural designs in many aspects and can be powered for a long time by button batteries. It can be used without a casing, and the thickness of the handle is greatly reduced, thereby reducing the overall volume and weight and facilitating storage.

[0021] 6. The multi-mode interactive input controller provided by the present invention has a set of directional PADs and a set of control PADs, both of which are dot matrix touch sensors and have no physical buttons. Each PAD can correspond to different input signals (control instructions), has a compact structure, is easy to operate, and is sensitive to response.

[0022] 7. The multi-mode interactive input controller of the present invention is also provided with a set of single-color or multi-color indicator lights, which can indicate different programming inputs and program control states and functions.

[0023] 8. The multi-mode interactive input controller of the present invention improves each key from a traditional push button to a resistive PAD, thereby extending the lifespan; adopts a low-power Bluetooth 5.4 module in conjunction with a button battery to achieve thousands of hours of standby; cancels the setting of the power key and simplifies the structure; adopts a secure connection through Bluetooth pairing, which greatly reduces signal interference and can ensure smooth input control of graphical programming software. It can be widely used in signal input and control of various programmable electronic products such as electronic building blocks and robots.

[0024] 9. The overall technical solution provided by the present invention combines programming input with program operation control, combines capacitive signal triggering with resistive signal triggering, combines wired and wireless technologies, and combines direct operation of the controller body with operation of various props. Users can implement various programming and program execution control operations in a large spatial range, with a large degree of freedom, in multiple application scenarios, and with the participation of multiple props. In addition, there are various modes of programming input and interactive control, which are highly integrated with specific program operation scenarios, have strong interactivity, and are easy to operate, which helps to quickly improve the user's scientific literacy and innovation ability in graphical programming, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a schematic diagram of the composition and connection structure of a multi-mode interactive control system with graphical programming according to an embodiment of the present invention; Figure 2 It is a schematic diagram of the overall three-dimensional appearance structure of a multi-mode interactive input controller triggered by a capacitive signal according to an embodiment of the present invention; Figure 3 2 is a schematic diagram of the main structure of a multi-mode interactive input controller triggered by a capacitive signal according to an embodiment of the present invention; Figure 4 is a rear structural diagram of a capacitive signal-triggered multi-mode interactive input controller according to an embodiment of the present invention; Figure 5 is a bottom-up structural diagram of a capacitive signal-triggered multi-mode interactive input controller according to an embodiment of the present invention; Figure 6 It is a schematic diagram of the components and electrical connection structure of the PCB circuit of the multi-mode interactive input controller triggered by a capacitive signal according to an embodiment of the present invention; Figure 7 is a schematic diagram of a page for inputting codes in a graphical programming process according to an embodiment of the present invention; Figure 8 It is a schematic diagram of a game interface after executing a program after graphical programming according to an embodiment of the present invention; Fig. 9 is a schematic diagram of a page for inputting codes in a graphical programming process according to an embodiment of the present invention; Fig.10 is a schematic diagram of a game interface when the graphical programming of an embodiment of the present invention is executed; Fig.11 It is a schematic diagram of the overall three-dimensional appearance structure of the multi-mode interactive input controller triggered by a resistance signal according to an embodiment of the present invention; Fig.12 2 is a schematic diagram of the main structure of a multi-mode interactive input controller triggered by a resistance signal according to an embodiment of the present invention; Fig.131 is a schematic diagram of the rear view structure of the multi-mode interactive input controller triggered by a resistance signal according to an embodiment of the present invention; Fig.14 1 is a bottom-up structural diagram of a multi-mode interactive input controller triggered by a resistance signal according to an embodiment of the present invention; Fig.15 It is a schematic diagram of the touch input contact electrode structure and signal triggering principle of the multi-mode interactive input controller triggered by a resistance signal according to an embodiment of the present invention; Fig.16 It is a schematic diagram of a three-dimensional structure of a multi-mode interactive input controller triggered by a resistance signal according to an embodiment of the present invention, which is embedded in a prop for use together; Fig.17 is a schematic diagram of a page for inputting codes in a graphical programming process according to an embodiment of the present invention; Fig.18 It is a schematic diagram of a game interface after executing a program after graphical programming according to an embodiment of the present invention; Fig.19 It is a schematic diagram of the overall three-dimensional appearance structure of the multi-mode interactive input controller triggered by a resistance signal according to an embodiment of the present invention; Fig. 20 It is a schematic diagram of the assembly structure of a multi-mode interactive input controller triggered by a resistance signal according to an embodiment of the present invention; Fig.21 It is a structural schematic diagram of a multi-mode interactive input controller triggered by a resistance signal according to an embodiment of the present invention, which is connected with an extension line, a prop, a human body and a graphical programming intelligent terminal for programming input or control; Fig. 22 is a schematic diagram of a page for inputting codes in a graphical programming process according to an embodiment of the present invention; Fig.23 is a schematic diagram of a game interface after executing a program after graphical programming according to an embodiment of the present invention (the electronic product executing the program may also be an electronic piano built with electronic building block modules); Fig.24 It is a schematic diagram of the three-dimensional appearance structure of a multi-mode interactive input controller triggered by a resistance signal according to an embodiment of the present invention, which is embedded in a prop for use.

[0026] In the figure: Ct, multi-mode interactive input controller triggered by capacitive signal 1. PCB circuit board; 2. Touch input contact electrode; 21. Metal sensing working surface; 22. Electrical connection hole; 4. Touch dot matrix direction PAD; 5. Touch dot matrix control PAD; 6. Lug grip; 11. Extension cord; 12. Props; Rt, resistance signal triggered multi-mode interactive input controller 1. PCB circuit board; 2. Touch input contact electrode; 22. Electrical connection hole; 23. Positive metal sensing working surface of touch input contact electrode; 24. Negative metal sensing working surface of touch input contact electrode; 25. Annular insulating groove; 4. Touch dot matrix direction PAD; 5. Touch dot matrix control PAD; 7. LED indicator light; 8. Control box; 9. MCU; 10. External interface; 11. Extension cord; 12. Props. DETAILED DESCRIPTION

[0027] The present invention will be described in detail below through multiple embodiments in conjunction with the accompanying drawings.

[0028] Basic Example

[0029] See attached Figure 1 The graphically programmed multi-mode interactive input controller provided in this embodiment is a thin panel-shaped control handle with a touch input contact electrode directly exposed outwardly, comprising a PCB circuit board, and at least one touch input contact electrode directly exposed outwardly arranged on the PCB circuit board, the metal sensing working surface of the exposed touch input contact electrode can be directly triggered by the user's human body contact, or can be triggered by the user's human body contact time through an extension line or prop connected to the exposed touch input contact electrode, to perform direct contact graphically programmed multi-mode interactive input or control.

[0030] The graphically programmed multi-mode interactive input controller is an interactive input controller triggered by a capacitive signal or an interactive input controller triggered by a resistive signal; The extension wire is a conductive wire, and the interface at one end of each wire is connected to the sensing working surface of an exposed touch input contact electrode on the PCB circuit board, and the other end is connected to the conductive key area of ​​the prop, or a metal sensing working surface is directly set; when the human body touches the conductive key area of ​​the prop (or the specially set metal sensing working surface), the trigger signal is transmitted to the exposed touch input contact electrode through the extension wire, and the multi-mode interactive input or control of indirect contact graphical programming is performed; The props described are physical items that are compatible with the graphical programming content or the scene of program execution control, which include a conductive button area, which is a separately set conductive button, or a conductive prop itself, such as conductive fruits such as apples; for props that are not conductive themselves, a metal sensing working surface is set at the end of the extension line, covering the prop body, and is operated and triggered by the user's body (such as fingers, feet, etc.).

[0031] A graphical programming multi-mode interactive control system, comprising the aforementioned graphical programming multi-mode interactive input controller, a graphical programming intelligent terminal, and an electronic product for executing the program; In the graphical programming intelligent terminal, graphical programming software is provided for receiving and processing programming signals or control signals sent by the multi-mode interactive input controller, or forwarding the programming signals or control signals to the electronic product executing the program; The electronic product for executing the program is used to execute the programming signal or control signal sent by the multi-mode interactive input controller or the graphical programming intelligent terminal.

[0032] The graphical programming intelligent terminal is a handheld intelligent terminal device or a computer; the graphical programming intelligent terminal is used to provide a graphical programming environment, programming software, or directly execute a compiled graphical program; The electronic product for executing the program includes a graphical programming intelligent terminal or a programmable electronic building block; A multi-mode interactive input controller includes a PCB circuit board provided with touch input contact electrodes directly exposed outward, a metal sensing working surface for connecting any exposed touch input contact electrode on the PCB circuit board, an extension line that can be triggered by the user's human body contact time, and a prop.

[0033] A multi-mode interactive control method for graphical programming, comprising the following steps: S1. Set up the multi-mode interactive control hardware environment A multi-mode interactive input controller for graphical programming, a graphical programming intelligent terminal, and an electronic product for executing the program are respectively provided; S2. Set up the multi-mode interactive control software environment In the graphic programming intelligent terminal, set up the graphic programming software; Connect the multi-mode interactive input controller, the graphical programming intelligent terminal, and the electronic products that execute the program through wireless signals; S3, Multi-mode interactive control operation for graphical programming The user can directly trigger the multi-mode interactive input controller by touching any metal sensing working surface of any touch input contact electrode on the multi-mode interactive input controller through any conductive part of the body, or indirectly trigger the multi-mode interactive input controller through an extension line or a metal sensing working surface on a prop, and then select one of the multiple interactive input mode combinations formed by the interactive input controller triggered by a resistance signal or the interactive input controller triggered by a capacitance signal, and send an input or control signal to the graphical programming intelligent terminal or the electronic product that executes the program to complete the graphical programming; S4. Presentation of multi-mode interactive control results of graphical programming After receiving and processing, the graphic programming intelligent terminal directly executes the compiled graphic program, or sends it to the electronic product that executes the program for execution; After receiving the signal, the electronic product executing the program executes the control command input by the user and presents the interactive control result of the graphical programming. It realizes the multi-mode interactive control of graphical programming with multiple programming and multiple execution modes in a large space range, large degree of freedom, multiple application scenarios, and multiple props.

[0034] The multi-mode interactive control includes the following multiple programming and control modes implemented by the aforementioned multi-mode interactive input controller: A. The user directly touches the metal sensing working surface of the touch input contact electrode on the multi-mode interactive input controller triggered by the capacitive signal, sends a signal, and the graphical programming intelligent terminal receives and processes it and directly executes it; B. The user directly touches the metal sensing working surface of the touch input contact electrode of the multi-mode interactive input controller triggered by the capacitive signal, sends a signal, and the graphical programming intelligent terminal receives and processes it, and then forwards it to the electronic product that executes the program for execution; C. The user directly touches the metal sensing working surface on the prop connected by the extension line, sends a signal, and transmits it to the metal sensing working surface of the corresponding contact electrode on the multi-mode interactive input controller. The graphical programming intelligent terminal receives and processes it and executes it directly; D. The user directly touches the metal sensing working surface on the prop connected by the extension cord, sends a signal, and transmits it to the metal sensing working surface of the corresponding contact electrode on the multi-mode interactive input controller triggered by the capacitive signal. After the graphical programming intelligent terminal receives and processes it, it is forwarded to the electronic product that executes the program for execution; E. The user directly touches the metal sensing working surface of the touch input contact electrode on the multi-mode interactive input controller triggered by the resistance signal, sends a signal, and the graphical programming intelligent terminal receives and processes it and directly executes it; F. The user directly touches the metal sensing working surface of the input contact electrode of the multi-mode interactive input controller triggered by the resistance signal, and sends a signal, which is received and processed by the graphical programming intelligent terminal and then forwarded to the electronic product that executes the program for execution; G. The user directly touches the metal sensing working surface of the prop connected by the extension line, sends a signal, and transmits it to the metal sensing working surface of the corresponding contact electrode on the multi-mode interactive input controller triggered by the resistance signal. The graphical programming intelligent terminal receives and processes it and executes it directly; H. The user directly touches the metal sensing work surface on the prop connected by the extension cord, sends out a signal, which is transmitted to the metal sensing work surface of the corresponding contact electrode on the multi-mode interactive input controller triggered by the resistance signal. After the graphical programming intelligent terminal receives and processes it, it is forwarded to the electronic product that executes the program for execution.

[0035] The present invention breaks through many limitations of the prior art and can support users' graphical programming and multi-mode interactive control in a large spatial range, large degree of freedom, multiple application scenarios, and with the participation of multiple props, with specific multiple programming and multiple execution modes.

[0036] The present invention will be described in detail below with reference to a number of specific embodiments in conjunction with the accompanying drawings.

[0037] Example 1 See attached Figures 2 to 8 The graphically programmed multi-mode interactive input controller, multi-mode interactive control method and system provided by the present invention are based on the basic embodiments, and use a multi-mode interactive input controller triggered by a capacitive signal, and A and B modes in the multi-mode interactive control to perform graphical programming and program operation control of a specific "Astronaut Returns to the Capsule" game program.

[0038] The graphical programming multi-mode interactive control system provided in this embodiment includes the aforementioned graphical programming multi-mode interactive input controller and a graphical programming intelligent terminal (intelligent PAD), which is also an electronic product for executing programs.

[0039] In the intelligent PAD of the graphical programming intelligent terminal, there is a graphical programming software (the present embodiment adopts the tdprogram programming software developed based on the prior art, https: / / tdprogram.td-robot.com / v2 / index.html, after entering the main page of the tdprogram programming software, find the controller handle adding page of the graphical programming software, click to enter the Bluetooth search process of the handle; if the search is successful, click to complete the pairing; after the pairing is successful, the multi-mode interactive input controller can be used to input programming information, move direction, select, cancel, confirm and other related operations.

[0040] The graphically programmed multi-mode interactive input controller Ct provided in this embodiment is a thin panel-shaped control handle provided with a plurality of directly exposed touch input contact electrodes, comprising a PCB circuit board 1 having an overall ingot-shaped outline, and eight directly exposed touch input contact electrodes 2 arranged on the PCB circuit board 1; each touch input contact electrode 2 is provided with a circular metal sensing working surface 21 for sensing human body signals; each metal sensing working surface 21 is provided with an electrical connection hole 22 (through hole) at the center position of the circle for electrically connecting external extension lines and props; each exposed metal sensing working surface 21 of the touch input contact electrode 2 (specifically, the exposed copper sheet of the PCB circuit board) can be directly triggered when touched by the user's body to perform graphically programmed multi-mode interactive input or program control of the direct contact control handle.

[0041] In order to better realize the interactive input of graphical programming or program control, the multi-mode interactive input controller Ct of this embodiment is provided with two touch dot matrix PADs (capacitive signal sensor planar arrays) each consisting of four exposed touch input contact electrodes, and each part is electrically connected through a PCB circuit board; among them, the four contact electrodes located on the left side of the PCB circuit board 1 constitute the touch dot matrix direction PAD 3, and the four contact electrodes located on the right side constitute the touch dot matrix control PAD4; the function and shape of each touch dot matrix PAD are the same as those of the game controller in the prior art to adapt to user habits.

[0042] In order to enhance the performance and function of the multi-mode interactive input controller Ct, the PCB circuit board 1 is also provided with components such as MCU, operation circuit, power circuit, button battery, LED indicator, DC-DC converter, crystal oscillator, other sensors, Bluetooth communication module, Bluetooth antenna, etc. (not shown in the figure); In order to facilitate the user to hold it directly with hands, the PCB circuit board 1 is designed as a whole to be ingot-shaped, and lug handles 6 protruding outwards are symmetrically provided on the left and right sides of the upper edge.

[0043] See also Figure 7 According to the page prompt for entering the code during the graphical programming process displayed in the graphical programming intelligent terminal, graphical programming is performed for the two sets of touch dot matrix PADs on the multi-mode interactive input controller Ct.

[0044] See also Figure 8 After programming is completed, the compiled program is executed in the graphical programming intelligent terminal, and interactive control is performed through the two sets of touch dot matrix PADs of the multi-mode interactive input controller Ct; the user touches any exposed touch input contact electrode set on the multi-mode interactive input controller Ct with a finger, which triggers the corresponding control signal, which is received and executed by the graphical programming intelligent terminal, and then the execution result is presented on its screen.

[0045] In this embodiment, two sets of touch dot matrix PADs, MCU, battery, DC-DC converter, crystal oscillator, Bluetooth antenna, LED indicator, sensor, etc., which are arranged on the PCB circuit board 1, are electrically connected to each other through the PCB circuit board 1; The MCU includes a low-power Bluetooth module, a Bluetooth antenna and a PCB circuit board 1 are integrally printed, and the Bluetooth module is electrically connected to the Bluetooth antenna; the battery is a button battery, and the button battery is electrically connected to the DC-DC converter; the button battery model is one of CR2016, CR2025, and CR2032.

[0046] The DC-DC converter includes: a power IC; one end of the power IC is electrically connected to the battery, and the other end is electrically connected to the MCU; the power IC model is: one of BD70522GUL, SCT2325, and TPS62745DSSR; The Bluetooth module is a low-power Bluetooth 5.4 module, and the model is one of STM32WB55, STM32WB50, nRF54H20, and nRF54L15. Compared with previous Bluetooth versions, the Bluetooth 5.4 module has significantly improved transmission speed, lower power consumption, and stronger signal stability; The touch dot matrix direction PAD 3 includes four symmetrically arranged contact electrodes, which are arranged in a cross shape, corresponding to the top, bottom, left and right respectively; each contact electrode is connected to the MCU through a PCB circuit board, and is based on the current induction trigger input signal when the human body touches it, that is, it is a capacitive signal sensor.

[0047] The touch matrix control PAD 4 also includes four contact electrodes arranged in a centrally symmetrical manner, which correspond to the ABCD keys of a conventional game controller respectively; each contact electrode is connected to the MCU through a PCB circuit board, and is based on current induction triggering input signals when touched by the human body, that is, they are all capacitive signal sensors; in other embodiments, the number of contact electrodes of the touch matrix direction PAD 3 or the touch matrix control PAD 4 can be increased according to the needs of graphical programming input and control.

[0048] The touch input contact electrode 2 used in this embodiment is a capacitive inductive touch switch, which has the characteristics of low energy consumption and good waterproof performance.

[0049] In other embodiments, an LED indicator light 4 is further provided on the PCB circuit board for indicating various status information of the multi-mode interactive input controller Ct.

[0050] The multi-mode interactive input controller Ct provided by the present invention has undergone structural simplification and targeted improvements in many aspects. The buttons are improved from traditional push buttons to capacitive PADs, and are controlled by capacitive sensing signals, thereby avoiding the life of mechanical buttons and the problem of poor contact during use; a low-power Bluetooth module is used in conjunction with a button battery to support long-term standby; it can be used without a shell, and the product is open, transparent, and novel, and can be flexibly connected to an extension cord to connect to various scene-based external props; metal contact electrodes are used to eliminate the need for a mechanical switch key, greatly simplifying the overall structure; users can perform various settings and program input and control operations according to graphical programming software; the interactive input controller and the graphical programming intelligent terminal use Bluetooth pairing to greatly reduce external signal interference and improve the working stability during the input or control process.

[0051] In other embodiments, when the graphical programming intelligent terminal is different from the electronic product that executes the program, such as the electronic product that executes the program is an electronic building block, an intelligent robot, etc., a multi-mode interactive control method of mode B can be adopted, that is: the user directly contacts the metal sensing working surface of the input contact electrode of the multi-mode interactive input controller triggered by the capacitive signal to send a signal, which is received and processed by the graphical programming intelligent terminal and then forwarded to the electronic product that executes the program for execution.

[0052] Example 2 See also Figures 9 and 10 This embodiment is based on the above embodiment 1, and further uses extended wires and props, and adopts modes C and D to perform graphical programming input and control of the dance machine, indirectly contact the control handle, and compile input and execute the control of the "Dodge Master" game program.

[0053] The graphical programming input content of the dance machine of this embodiment is shown in Fig. 9 ; This embodiment uses the same multi-mode interactive input controller Ct and graphical programming intelligent terminal as in Embodiment 1, but introduces metal extension wires and props (dance mats). This embodiment changes the program operation control from the controller Ct to the dance mat (dance machine) control mode, greatly broadening the interactive operation mode, space and degree of freedom with the user.

[0054] First, the four physical buttons (up, down, left, and right) on the prop dance mat are electrically connected to the upper, lower, left, and right contact electrodes corresponding to the touch dot matrix direction PAD 3 through a 2-5 meter long metal extension line. Specifically, the physical buttons are electrically connected to the metal sensing working surfaces of each contact electrode on the touch dot matrix direction PAD 3 through an interface (such as an electric clip or a conductive adhesive head); When using mode C, the user directly touches the contact point or key area on the prop connected by the extension line, or the specially set metal sensing work surface, and transmits the signal to the corresponding contact electrode on the multi-mode interactive input controller Ct, and the multi-mode interactive input controller forwards it to the graphical programming intelligent terminal for receiving, and directly executes it after processing; in fact, there is no need to directly operate the contact electrode on the multi-mode interactive input controller Ct, which is an indirect operation; because the extension line and the prop can be adjusted according to the actual application scenario, it completely breaks through the limitations of the traditional method on the user's use space and operation freedom; Users can Fig. 9 According to the content of the prompt, the physical buttons on the prop dance mat are operated, and the signals are transferred through the multi-mode interactive input controller Ct, and the programming information is input into the graphical programming intelligent terminal until the programming is completed; During the program running process, the user also operates the physical buttons on the prop dance mat to send signals, which are transferred through the multi-mode interactive input controller Ct to input control information into the graphical programming intelligent terminal to control the interactive operation of the program.

[0055] In this embodiment, when the graphic programming intelligent terminal is different from the electronic product of the program, such as the compiled program is performed by a dance machine (including a host and a large screen), the D mode is used for graphic programming and program control. Specifically, the user directly touches the metal sensing working surface on the prop connected by the extension line, sends a signal, and transmits it to the metal sensing working surface of the corresponding contact electrode on the multi-mode interactive input controller. After the graphic programming intelligent terminal receives and processes it, it is forwarded to the electronic product that executes the program (such as the dance machine that executes the program, the large screen, etc.) for execution, thereby completely integrating the process and execution results of graphic programming with the actual application scenario (dance machine), greatly breaking through the limitations of traditional technology on the space and degree of freedom of user programming operations, and greatly improving the practicality and fun of programming.

[0056] In this embodiment, during the execution of the program, the user can control the game program by walking quickly on the dance mat according to the pre-compiled program settings, which greatly improves the fun of graphical programming. For example, the user only needs to step on the up button of the dance mat to jump and step on the down button of the dance mat to squat, which increases the fun of the game; it can also carry out sports in a larger space to play a role in physical exercise; this embodiment can also set the character to avoid obstacles and slide down in the program to enhance the interactivity and challenge of the game; by adding a gold coin generation mechanism and a scoring program, the player's desire for exploration and sense of accomplishment are stimulated; if the obstacle set in the program is encountered, the game end function is triggered.

[0057] Example 3 See also Figures 11 to 18 This embodiment is based on the above embodiments, and further uses extended wires and props, a capacitive signal-triggered multi-mode interactive input controller Rt, and modes E and F to perform graphical programming input and control of the dance machine, directly or indirectly contact the control handle, and compile input and execute the control of the "virtual reality table tennis" game program.

[0058] See also Figure 11-Figure 14The graphically programmed multi-mode interactive input controller provided in this embodiment is an interactive input controller Rt triggered by a resistance signal, and the input signal is triggered by the on-off between the positive working surface 23 and the negative (grounding) working surface 24 of each exposed touch input contact electrode 2; each of the exposed touch input contact electrodes 2 includes a positive metal contact sheet and a negative metal contact sheet that are arranged on the PCB circuit board 1, adjacent to each other but separated, as a group of touch-triggered sensing working surfaces, corresponding to the positive working surface 23 and the negative working surface 24 respectively; as shown in FIG. Fig.15 As shown, when the human body contacts the positive working surface 23 and the negative working surface 24 at the same time, the current on-off state between the positive and negative electrodes changes (conduction), triggering an input or control signal.

[0059] The metal positive working surface 23 and the negative working surface 24 are located on the same plane of the PCB circuit board 1, and are embedded on the inner side of the upper surface of the PCB circuit board, and are arranged around each other, and a narrow annular insulating groove 25 (narrow width, usually 1-2mm; height is the thickness of the metal contact sheet of the metal positive working surface 23 and the negative working surface 24) is provided between each other; a small area of ​​skin at the front end of the human finger can simultaneously contact the positive working surface 23 and the negative working surface 24 in the same group of sensing working surfaces to trigger input or control signals. In order to facilitate the prompting of programming information or display the status of the multi-mode interactive input controller, an LED indicator 7 is also provided on the PCB circuit board 1, or an LED array composed of multiple LED indicator lights 7 is provided, which is electrically connected to the MCU 9 provided on the back of the PCB circuit board 1. 4. Touch dot matrix direction PAD; 5. Touch dot matrix control PAD.

[0060] See also Fig.16 In order to enhance the scene effect and sense of integration during the programming input and operation control process of this embodiment, and to make the multi-mode interactive input controller easier to use and operate, the multi-mode interactive input controller Rt triggered by the resistance signal can also be embedded in the groove of the prop 12 made of cardboard or plastic for use; the positive and negative electrodes of the touch input contact electrodes 2 that need to be transferred to the prop 12 for operation can be connected to the corresponding contact electrode trigger sensing area set on the prop 12 in advance using an extension line; when in use, the contact electrodes on the multi-mode interactive input controller Rt can be directly touched, or the contact electrodes on the prop can be touched, and programming input and program control can be performed; the interactive input controller Rt and the smart PAD terminal that executes the program adopt Bluetooth wireless connection; the user directly operates the prop (the prop can also be prepared in the shape of a table tennis racket) to control the program and flexibly control the operation of the virtual reality table tennis game.

[0061] In this embodiment, the control handle can be directly operated to play the game, or the control handle of this embodiment can be embedded in the handle of a table tennis racket-shaped prop. By using the acceleration sensor, gyroscope, geomagnetic sensor, etc. set in the control handle or the prop, the position of the table tennis racket can be identified, providing a highly simulated virtual reality scene of an interactive table tennis game.

[0062] This embodiment simplifies the overall structural design of the controller through various structural improvements, miniaturizes the entire machine and reduces power consumption, eliminates the design of the switch key and the shell, and improves each key from a traditional push button to a resistive PAD directly set on the PCB board, making it compact in structure, small in size and light in weight; no wiring is required during the programming process, and a paired Bluetooth connection is used to reduce communication failures during use, and the control distance is long and the use is flexible, thereby improving the overall service life, work efficiency and stability of the resistive touch wireless graphical programming input control handle; at the same time, it can also be used with various props to match different programs and application scenarios, and interaction methods (including modes E and F) to improve user experience.

[0063] Example 4 See also Figures 19 to 23 This embodiment is based on the above-mentioned embodiment 3, and further designs a control box of a multi-mode interactive input controller, and arranges more functional components and interfaces in the control box to realize more functions and applications, and is easier to cooperate with extension lines and props, and adopts a multi-mode interactive input controller Rt triggered by a capacitive signal, and modes G and H to perform graphical programming input and control of the dance machine, directly or indirectly contact the control handle, and compile input and execute the control of the "virtual reality electronic piano performance" game program.

[0064] See also Figures 19 to 20 The graphically programmed multi-mode interactive input controller Rt provided in this embodiment, on the basis of the above-mentioned embodiment 3, further includes a control box 8, and the PCB circuit board 1 is embedded on the upper surface of the control box 8, and the two are detachably connected. The PCB circuit board 1 is also a thin circuit panel with touch input contact electrodes directly exposed to the outside; the PCB circuit board 1 is provided with a plurality of LED indicator lights 7, two touch dot matrix PADs, specifically a touch dot matrix direction PAD 4 and a touch dot matrix control PAD 5, and its structure and shape are the same as those in embodiment 3.

[0065] The control box 8 is in the shape of a rectangular box, and a groove matching the PCB circuit board 1 is provided on the top surface thereof, and the PCB circuit board 1 is detachably buckled in the groove; The control box 2 is provided with an electrically connected external battery, an acceleration sensor, a gyroscope, a geomagnetic sensor, a distance sensor and a plurality of external interfaces 10 (such as USB) and the like.

[0066] See also Fig.21 In this embodiment, a plurality of props 12 (elastic rubber blocks or plasticine blocks) are used to simulate the keys of different scales of an electronic keyboard, and are respectively connected to corresponding contact electrodes on an interactive input controller Rt through extension lines 11. The human body is directly connected to the negative electrode through the extension lines 11, and can indirectly operate the multi-mode interactive input controller by touching the sensing area on the props; the props are arranged in sequence to form the keys of an electronic keyboard, which can better simulate the performance of an electronic keyboard and greatly break through the limitations of the prior art on the operating space and freedom of the user.

[0067] See attached Fig.24 In other embodiments, referring to the above embodiments 1-4, other shapes of props can be freely replaced to select programs. Application scenarios and interactive modes, by connecting (or not connecting) the electrode touch sensing area on the props through an extension line, various vivid application scenarios can be created. Fig.23 It simulates a driving steering wheel or a rudder for fishing), and can simulate various driving and fishing program input and operation control.

[0068] In summary, the overall technical solution provided by the above-mentioned embodiments of the present invention simplifies the overall structural design of the controller through improvements in many aspects such as the controller and the interactive input mode, introduces extension cords and various props, combines programming input with program operation control, combines capacitive signal triggering with resistive signal triggering, combines wired and wireless technologies, and combines direct operation of the controller body with operation of various props. Users can implement various programming and program execution control operations in a large spatial range, large degree of freedom, multiple application scenarios, and with the participation of multiple props. In addition, the programming input and interactive control modes are diverse, with a high degree of integration with specific program operation scenarios, strong interactivity, and easy operation, which helps to quickly improve the user's scientific literacy and innovation ability in graphical programming, etc.

[0069] The above-mentioned embodiments simplify the overall structure by eliminating the controller shell and mechanical switch, etc.; the secure connection of Bluetooth pairing is adopted, which greatly reduces signal interference, expands the control distance, and can ensure the smooth input control of graphical programming software, and can be widely used in signal input and control of various programmable electronic products such as electronic building blocks and robots; the various parts, software and hardware are combined with each other to support users (including teachers and students) to perform various graphical programming in a large space range, large degree of freedom, multiple application scenarios, and with the participation of multiple props, as well as multi-mode interactive control during program execution.

[0070] The applicant has conducted multiple graphical programming teaching application tests on various implementation schemes of the controller, props, extension cords, programs, etc. of the present invention under confidentiality conditions. Actual tests have shown that the resistive touch wireless graphical programming input control handle has an average standby time of no less than 600 hours, and an average teaching use time of no less than 300 hours. The multi-mode interactive input controller provided by the present invention has sensitive response, fast response speed, moderate control distance, reliable and stable connection, stable and reliable operation, ensuring the accuracy of signal input and control results, and can significantly improve the interest, freedom and efficiency of programming teaching.

[0071] It should be noted that, in the description of the present invention, it should be understood that the terms "upper", "lower", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, or are the orientations or positional relationships in which the products of the present invention are usually placed when in use, or are the orientations or positional relationships commonly understood by those skilled in the art. These are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0072] The above are only some specific embodiments of the present invention, and the protection scope of the present invention is not limited thereto; any replacement and improvement made without violating the concept of the present invention shall fall within the protection scope of the present invention.

Claims

1. A graphically programmed multi-mode interactive input controller, characterized in that: It is a thin panel-shaped control handle with a touch input contact electrode directly exposed outward, which includes a PCB circuit board and at least one touch input contact electrode directly exposed outward arranged on the PCB circuit board. The metal sensing working surface of the exposed touch input contact electrode can be directly triggered by the user's human body contact, or can be triggered by the user's human body contact through an extension line or prop connected to the exposed touch input contact electrode, thereby performing multi-mode interactive input or control of the graphically programmed indirect contact control handle.

2. The multi-mode interactive input controller for graphical programming according to claim 1, characterized in that: It is an interactive input controller triggered by a capacitive signal, using the metal contact sheet of each exposed touch input contact electrode as the sensing working surface, and triggering the input signal based on the current induction when the human body touches it.

3. The multi-mode interactive input controller for graphical programming according to claim 1, characterized in that: It is an interactive input controller triggered by a resistance signal, and the input signal is triggered by the on-off state between the positive working surface and the negative working surface of each exposed touch input contact electrode; each exposed touch input contact electrode includes a positive metal contact piece and a negative metal contact piece that are arranged on a PCB circuit board, adjacent to each other but separated, as a group of sensing working surfaces, corresponding to the positive working surface and the negative working surface respectively. When the human body touches the positive working surface and the negative working surface at the same time, the on-off state of the current between the positive and the negative electrodes changes, triggering the input signal.

4. The multi-mode interactive input controller for graphical programming according to claim 3, characterized in that: The metal positive working surface and the negative working surface are located on the same plane of the PCB circuit board, are arranged to surround each other, and have a narrow annular insulating groove between them; a small area of ​​skin at the front end of a human finger can simultaneously contact the positive working surface and the negative working surface in the same group of sensing working surfaces to trigger an input signal.

5. The multi-mode interactive input controller for graphical programming according to claim 1, characterized in that: The extension wires are conductive wires, one end of each of which is connected to the sensing working surface of an exposed touch input contact electrode on the PCB circuit board, and the other end is connected to the conductive key area of ​​the prop; when the human body touches the conductive key area of ​​the prop, the trigger signal is transmitted to the exposed touch input contact electrode through the extension wire to perform multi-mode interactive input or control of indirect contact graphical programming; the props are physical objects that are compatible with the graphical programming content or the scene of program execution control.

6. The multi-mode interactive input controller for graphical programming according to any one of claims 1 to 5, characterized in that: The PCB circuit board is also provided with an MCU and a touch dot matrix PAD composed of a plurality of exposed touch input contact electrodes, and each part is electrically connected through the PCB circuit board.

7. A graphically programmed multi-mode interactive control system, characterized in that: It comprises a multi-mode interactive input controller for graphical programming as claimed in any one of claims 1 to 6, as well as a graphical programming intelligent terminal and an electronic product for executing the program; In the graphical programming intelligent terminal, graphical programming software is provided for receiving and processing programming signals or control signals sent by the multi-mode interactive input controller, or forwarding the programming signals or control signals to the electronic product executing the program; The electronic product for executing the program is used to execute the programming signal or control signal sent by the multi-mode interactive input controller or the graphical programming intelligent terminal.

8. The graphically programmed multi-mode interactive control system according to claim 7, characterized in that: The graphical programming intelligent terminal is a handheld intelligent terminal device or a computer; the graphical programming intelligent terminal is used to provide a graphical programming environment, programming software, or directly execute a compiled graphical program; The electronic product for executing the program includes a graphical programming intelligent terminal or a programmable electronic building block; A multi-mode interactive input controller includes a PCB circuit board provided with touch input contact electrodes directly exposed outward, a metal sensing working surface for connecting any exposed touch input contact electrode on the PCB circuit board, an extension line that can be triggered by the user's human body contact time, and a prop.

9. A multi-mode interactive control method for graphical programming, characterized in that: It includes the following steps: S1. Set up the multi-mode interactive control hardware environment A multi-mode interactive input controller for graphical programming, a graphical programming intelligent terminal, and an electronic product for executing the program are respectively provided; S2. Set up the multi-mode interactive control software environment In the graphic programming intelligent terminal, set up the graphic programming software; Connect the multi-mode interactive input controller, the graphical programming intelligent terminal, and the electronic products that execute the program through wireless signals; S3, Multi-mode interactive control operation for graphical programming The user can directly trigger the multi-mode interactive input controller by touching any metal sensing working surface of any touch input contact electrode on the multi-mode interactive input controller through any conductive part of the body, or indirectly trigger the multi-mode interactive input controller through an extension line or a metal sensing working surface on a prop, and then select one of the multiple interactive input mode combinations formed by the interactive input controller triggered by a resistance signal or the interactive input controller triggered by a capacitance signal, and send an input or control signal to the graphical programming intelligent terminal or the electronic product that executes the program to complete the graphical programming; S4. Presentation of multi-mode interactive control results of graphical programming After receiving and processing, the graphic programming intelligent terminal directly executes the compiled graphic program, or sends it to the electronic product that executes the program for execution; After receiving the signal, the electronic product executing the program executes the control command input by the user and presents the interactive control result of the graphical programming.

10. The multi-mode interactive control method of graphical programming according to claim 9, characterized in that: The multi-mode interactive control includes at least the following programming and control modes: A. The user directly touches the metal sensing working surface of the touch input contact electrode on the multi-mode interactive input controller triggered by the capacitive signal, sends a signal, and the graphical programming intelligent terminal receives and processes it and directly executes it; B. The user directly touches the metal sensing working surface of the touch input contact electrode of the multi-mode interactive input controller triggered by the capacitive signal, sends a signal, and the graphical programming intelligent terminal receives and processes it, and then forwards it to the electronic product that executes the program for execution; C. The user directly touches the metal sensing working surface on the prop connected by the extension line, sends a signal, and transmits it to the metal sensing working surface of the corresponding contact electrode on the multi-mode interactive input controller. The graphical programming intelligent terminal receives and processes it and executes it directly; D. The user directly touches the metal sensing working surface on the prop connected by the extension cord, sends a signal, and transmits it to the metal sensing working surface of the corresponding contact electrode on the multi-mode interactive input controller triggered by the capacitive signal. After the graphical programming intelligent terminal receives and processes it, it is forwarded to the electronic product that executes the program for execution; E. The user directly touches the metal sensing working surface of the touch input contact electrode on the multi-mode interactive input controller triggered by the resistance signal, sends a signal, and the graphical programming intelligent terminal receives and processes it and directly executes it; F. The user directly touches the metal sensing working surface of the input contact electrode of the multi-mode interactive input controller triggered by the resistance signal, and sends a signal, which is received and processed by the graphical programming intelligent terminal and then forwarded to the electronic product that executes the program for execution; G. The user directly touches the metal sensing working surface of the prop connected by the extension line, sends a signal, and transmits it to the metal sensing working surface of the corresponding contact electrode on the multi-mode interactive input controller triggered by the resistance signal. The graphical programming intelligent terminal receives and processes it and executes it directly; H. The user directly touches the metal sensing work surface on the prop connected by the extension cord, sends out a signal, which is transmitted to the metal sensing work surface of the corresponding contact electrode on the multi-mode interactive input controller triggered by the resistance signal. After the graphical programming intelligent terminal receives and processes it, it is forwarded to the electronic product that executes the program for execution.

Images