Virtual keyboard display method and device for graphical programming development environment of mobile terminal

Abstract

The invention discloses a virtual keyboard display method and a virtual keyboard display device for a graphical programming development environment of a mobile terminal, and the virtual keyboard display method for the graphical programming development environment of the mobile terminal comprises the following steps: in the process that the mobile terminal receives graphical programming operation of a user, displaying the graphical programming operation of the user; detecting whether the user programming codes comprise event type building blocks and / or detection type building blocks related to the keys or not; if the event type building blocks and / or the detection type building blocks related to the keys are contained, a virtual keyboard is displayed when the user programming code program runs; and determining a display mode of the virtual keyboard according to the key type selected in the event type building block and / or the detection type building block. Through the technical path of intelligent detection, dynamic adaptation and differential interaction, the technical bottleneck of key interaction experience in graphical programming of the mobile terminal is effectively overcome, and unification of functional integrity, space economy and interaction friendliness is achieved.

Description

Technical Field

[0001] This invention relates to the field of computer education and human-computer interaction technology, specifically providing a method and apparatus for displaying a virtual keyboard in a mobile graphical programming development environment. Background Technology

[0002] With the popularization of children's programming education, visual programming, with its low barrier to entry and high interactivity, has become a core medium for children and programming beginners to learn programming. Among them, Scratch IDE is one of the most widely used visual programming tools. When using Scratch IDE, users do not need to write traditional text code. They can simply drag and drop "blocks" (graphical modules that encapsulate specific program logic) from the interface to the script canvas and combine them to create and run programs. This block-based programming mode effectively lowers the technical threshold for learning programming, allowing children to focus more on the design and creative implementation of program logic.

[0003] With the widespread adoption of mobile devices, Scratch IDE is also widely used on tablets, smartphones, and other mobile terminals to meet users' needs for programming anytime, anywhere. In Scratch programming, the "When ( ) key is pressed" event block and the "When ( ) key is pressed?" detection block are important functional modules for implementing user interaction. Users can use these blocks to set the program's response to specific keyboard keys. On a personal computer (PC), due to the physical keyboard, users can directly press the corresponding keys to trigger events and experience the complete project effect while the program is running. However, on mobile devices, due to the lack of a physical keyboard, a virtual keyboard is necessary to experience programming projects that rely on key interactions.

[0004] Currently, the mobile ScratchIDE has many shortcomings and defects in implementing virtual keyboard functionality, mainly reflected in: The virtual keyboard's incomplete content leads to a lack of user experience: In the current implementation, the virtual keyboard does not fully cover all selectable key parameters in the blocks (such as space, arrow keys, number keys, letter keys, etc.). This means that a user may select a key parameter while programming, but after the program runs, there is no corresponding key on the virtual keyboard to operate on it. This prevents the project's effect from being fully experienced on mobile devices, seriously affecting the fun and educational purpose of programming.

[0005] The lack of intelligent keyboard display distracts the user: Existing virtual keyboard solutions are typically static and fixed, unable to dynamically adjust based on the actual key types used in the user's program. Regardless of which key(s) are used in the user's code, the virtual keyboard tends to display a complete interface including all possible keys. This "one-size-fits-all" approach displays a large number of irrelevant key controls within the limited screen space of mobile devices, not only taking up valuable project preview space but also causing unnecessary visual interference and operational confusion for users, thus degrading the user experience.

[0006] Poor human-computer interaction experience: Fixed-size virtual keyboards may appear too small on some devices, making operation difficult, or they may obscure key elements on other screen ratios. Their display and collapse mechanisms are also often not smooth or natural.

[0007] Therefore, existing virtual keyboard solutions for mobile ScratchIDEs cannot intelligently and accurately adapt to the user's programming content, resulting in technical shortcomings such as incomplete functionality, unfriendly interaction, and poor user experience. A new technical solution is urgently needed that can dynamically generate an adapted virtual keyboard based on the user's programming intent to solve these problems.

[0008] In view of this, this invention patent is hereby proposed. Summary of the Invention

[0009] To address the aforementioned technical problems, this invention proposes a virtual keyboard display method and apparatus for mobile graphical programming development environments, specifically employing the following technical solution: In a first aspect, the present invention provides a method for displaying a virtual keyboard for a mobile graphical programming development environment, comprising: During the process of receiving the user's graphical programming operation on the mobile device, detect whether the user's programming code contains event class blocks and / or detection class blocks related to key presses; If it contains event-type blocks and / or detection-type blocks related to key presses, then a virtual keyboard will be displayed when the user's programming code runs. The display mode of the virtual keyboard is determined based on the key type selected in the event-type block and / or detection-type block.

[0010] As an optional implementation of this embodiment, in the virtual keyboard display method for a mobile graphical programming development environment, the step of detecting whether the user's programming code contains event-type blocks and / or detection-type blocks related to key presses is as follows: the content of the event-type blocks includes "when the ( ) key is pressed", and the content of the detection-type blocks includes "when the ( ) key is pressed?". The "( ) key" in the content of the blocks is selected and determined by a drop-down list.

[0011] As an optional implementation of this embodiment, in the virtual keyboard display method for a mobile graphical programming development environment, the step of determining the display mode of the virtual keyboard based on the key type selected in the event-type blocks and / or detection-type blocks includes: The virtual keyboard includes an arrow key display area, a number / space key display area, and an alphabet key display area; Based on the selected directional keys, and / or number / space keys, and / or letter keys in the event-type blocks and / or detection-type blocks, the corresponding keyboard display area of ​​the virtual keyboard is determined and displayed.

[0012] As an optional implementation of this embodiment, the virtual keyboard display method for a mobile graphical programming development environment includes: determining the shape and position of the display area of ​​the keyboard display area based on the key type selected in the event-type blocks and / or detection-type blocks.

[0013] As an optional implementation of this embodiment, the virtual keyboard display method for a mobile graphical programming development environment includes: If the selected event block and / or detection block contains at least one of the following keys: directional keys, number / space keys, and letter keys, then a full keyboard mode is displayed, which includes an directional keyboard display area, a number / space keyboard display area, and a letter keyboard display area. If the selected event block and / or detection block includes directional keys and number / space keys, then the combination mode of directional keys and number / space keys is displayed. The combination mode of directional keys and number / space keys includes a directional keyboard display area and a number / space keyboard display area. If the selected event block and / or detection block includes letter keys and number / space keys, then the letter key and number / space key combination mode is displayed, which includes a letter keyboard display area and a number / space keyboard display area. If the selected event-type block and / or detection-type block includes both directional and alphabetic keys, then the directional key and alphabetic key combination mode is displayed, which includes a directional keyboard display area and an alphabetic keyboard display area.

[0014] As an optional implementation of this embodiment, in the virtual keyboard display method for a mobile graphical programming development environment, the overall display area of ​​the virtual keyboard is fixed to the bottom area of ​​the mobile screen and the keyboard display area is displayed in a horizontal arrangement. The directional keyboard display area displays the directional keys in a cross-shaped layout, the numeric / space keyboard display area displays the numeric keys and space key in a rectangular grid layout, and the letter keyboard display area displays the letter keys in a rectangular grid layout. In full keyboard mode, the arrow key display area, the number / space key display area, and the letter key display area are arranged horizontally in a preset order. In combination mode, the included keyboard display areas are arranged horizontally or stacked vertically according to a preset layout. The overall display area size of the virtual keyboard is dynamically adjusted according to the number of keyboard display areas currently displayed; the more display areas there are, the larger the overall display area becomes.

[0015] As an optional implementation of this embodiment, in the virtual keyboard display method for a mobile graphical programming development environment, the step of detecting whether the user's programming code contains event-type blocks and / or detection-type blocks related to key presses during the process of receiving the user's graphical programming operation on the mobile terminal includes: Before the program runs, static analysis is performed on the graphical code blocks assembled by the user to identify whether they contain specific types of event-type building blocks and / or detection-type building blocks. The detection of whether the user-programmed code contains event-type blocks and / or detection-type blocks related to key presses includes: During program execution, the code execution flow is monitored in real time. When a specific type of event block and / or detection block is detected, the corresponding processing logic is triggered. The detection process includes the following steps: Parse the syntax tree structure of the user's programming code; Traverse the syntax tree to identify whether it contains event class blocks and / or detection class blocks of predefined types; When the building block is detected, the button parameter information set therein is extracted. The button parameter value includes at least one of the arrow keys, number keys, space key, or letter keys.

[0016] As an optional implementation of this embodiment, the virtual keyboard display method for a mobile graphical programming development environment includes: In the virtual keyboard display mode, the selected button in the drop-down menu of the blocks is highlighted as operable, while the unselected button is grayed out as inoperable.

[0017] In a second aspect, the present invention provides a virtual keyboard display device for a mobile graphical programming development environment, comprising: The key event detection module detects whether the user's programming code contains key-related event blocks and / or detection blocks during the process of receiving the user's graphical programming operation on the mobile terminal. The virtual keyboard display module, if it includes event-type blocks and / or detection-type blocks related to key presses, displays the virtual keyboard when the user's programming code program is running, and determines the display mode of the virtual keyboard according to the key type selected in the event-type blocks and / or detection-type blocks.

[0018] In a third aspect, the present invention provides a computer-readable recording medium storing a computer-executable program, which, when executed, implements the virtual keyboard display method for a mobile graphical programming development environment.

[0019] The present invention provides a virtual keyboard display method for mobile graphical programming development environments, which can produce the following beneficial technical effects: This system achieves intelligent, on-demand triggering and precise content adaptation of the virtual keyboard: by automatically detecting whether the user's code contains blocks for key-related events, the virtual keyboard is only triggered when necessary, avoiding unnecessary screen obstruction and resource consumption. Furthermore, by dynamically determining the keyboard display mode by identifying the specific key type selected in the blocks, it ensures that the key content displayed on the virtual keyboard perfectly matches the user's code logic, solving the problem of incomplete project experience caused by incomplete or overly generalized virtual keyboard content in existing technologies.

[0020] The system optimizes the utilization of limited screen space on mobile devices, enhancing the user experience. By dividing the virtual keyboard into different function display areas (such as arrow keys, number / space keys, and letter keys) and intelligently combining and displaying them based on the actual key usage (such as full keyboard mode, combination mode, or single mode), the default footprint of the virtual keyboard is significantly reduced. This dynamically adaptive layout only displays the keys necessary for the current program on the screen, minimizing the virtual keyboard's obstruction of the main visual area of ​​the project, and providing users with a clearer and more focused preview and interactive experience of programming effects.

[0021] It provides clear and intuitive human-computer interaction guidance: in the displayed virtual keyboard, the keys actually used in the code are highlighted and enabled, while unused keys in the same area are grayed out and disabled. This visual differentiation provides users with clear operation guidance. Users can immediately see which keys are currently active in the program, avoiding accidental operations and confusion, making the operation of key interaction items on mobile devices more intuitive and efficient.

[0022] This method enhances the compatibility and expressiveness of mobile programming projects: it ensures that Scratch projects written on mobile devices, including button interaction logic, achieve the same and complete performance as those on PCs. Users do not need to sacrifice project functionality due to platform switching, thus encouraging creative freedom, enhancing the functionality and practicality of mobile graphical programming tools, and playing a positive role in promoting programming education on mobile platforms.

[0023] In summary, the virtual keyboard display method for mobile graphical programming development environments of the present invention effectively overcomes the technical bottleneck of key interaction experience in mobile graphical programming through the technical path of "intelligent detection-dynamic adaptation-differentiated interaction", and achieves a unity of functional integrity, space economy and interactive friendliness. Attached Figure Description

[0024] Figure 1 A flowchart of a virtual keyboard display method for a mobile graphical programming development environment according to an embodiment of the present invention; Figure 2 Examples of dropdown lists for event-type building blocks and detection-type building blocks in embodiments of the present invention; Figure 3 Examples of event-type building blocks and detection-type building blocks in this invention; Figure 4 Example of full keyboard mode demonstration in this invention; Figure 5 Example 1 of the combined keyboard mode demonstration of this invention; Figure 6 Example 2 of the combined keyboard mode demonstration of this invention; Figure 7 Example 3 of the combined keyboard mode demonstration of this invention; Figure 8 A schematic diagram of the structure of the electronic device according to an embodiment of the present invention; Figure 9 A schematic diagram of a computer-readable recording medium according to an embodiment of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0026] Therefore, the following detailed description of embodiments of the present invention is not intended to limit the scope of the claimed invention, but merely illustrates some embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0027] It should be noted that, unless otherwise specified, the embodiments and features and technical solutions in the present invention can be combined with each other.

[0028] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0029] In the description of this invention, it should be noted that the terms "upper," "lower," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of this invention is in use, or the orientation or positional relationship commonly understood by those skilled in the art. These terms are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention. In addition, the terms "first," "second," etc., are only used to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0030] See Figure 1 As shown, this embodiment provides a method for displaying a virtual keyboard in a mobile graphical programming development environment, including: During the process of receiving the user's graphical programming operation on the mobile device, detect whether the user's programming code contains event class blocks and / or detection class blocks related to key presses; If it contains event-type blocks and / or detection-type blocks related to key presses, then a virtual keyboard will be displayed when the user's programming code runs. The display mode of the virtual keyboard is determined based on the key type selected in the event-type block and / or detection-type block.

[0031] This embodiment provides a virtual keyboard display method for mobile graphical programming development environments, which can produce the following beneficial technical effects: This system achieves intelligent, on-demand triggering and precise content adaptation of the virtual keyboard: by automatically detecting whether the user's code contains blocks for key-related events, the virtual keyboard is only triggered when necessary, avoiding unnecessary screen obstruction and resource consumption. Furthermore, by dynamically determining the keyboard display mode by identifying the specific key type selected in the blocks, it ensures that the key content displayed on the virtual keyboard perfectly matches the user's code logic, solving the problem of incomplete project experience caused by incomplete or overly generalized virtual keyboard content in existing technologies.

[0032] This method enhances the compatibility and expressiveness of mobile programming projects: it ensures that Scratch projects written on mobile devices, including button interaction logic, achieve the same and complete performance as those on PCs. Users do not need to sacrifice project functionality due to platform switching, thus encouraging creative freedom, enhancing the functionality and practicality of mobile graphical programming tools, and playing a positive role in promoting programming education on mobile platforms.

[0033] In summary, the virtual keyboard display method for mobile graphical programming development environments provided in this embodiment effectively overcomes the technical bottleneck of key interaction experience in mobile graphical programming through the technical path of "intelligent detection-dynamic adaptation-differentiated interaction", and achieves a unity of functional integrity, space economy and interactive friendliness.

[0034] Furthermore, in the virtual keyboard display method for a mobile graphical programming development environment in this embodiment, the step of detecting whether the user's programming code contains event-type blocks and / or detection-type blocks related to key presses is as follows: the content of the event-type blocks includes "when the ( ) key is pressed", and the content of the detection-type blocks includes "when the ( ) key is pressed?". The "( ) key" in the content of the blocks is selected and determined by a drop-down list.

[0035] like Figure 2 As shown, the drop-down lists of event blocks and detection blocks in this embodiment are displayed in the following order: space, ↑, ↓, ←, →, 1, 2, 3, 4, w, a, s, d, with the space bar selected by default.

[0036] The virtual keyboard in this embodiment is triggered when the start button is clicked and the code detects a key-related event block [When the () key is pressed] and a detection block [When the () key is pressed?], such as... Figure 3 As shown, a virtual keyboard is displayed; the keyboard disappears after the program ends.

[0037] like Figures 4 to 7As shown in the figure, this embodiment describes a virtual keyboard display method for a mobile graphical programming development environment. The step of determining the virtual keyboard display mode based on the key types selected in the event-type blocks and / or detection-type blocks includes: The virtual keyboard includes an arrow key display area, a number / space key display area, and an alphabet key display area; Based on the selected directional keys, and / or number / space keys, and / or letter keys in the event-type blocks and / or detection-type blocks, the corresponding keyboard display area of ​​the virtual keyboard is determined and displayed.

[0038] Furthermore, the virtual keyboard display method for a mobile graphical programming development environment in this embodiment includes: determining the shape and position of the display area of ​​the keyboard display area based on the key type selected in the event-type blocks and / or detection-type blocks.

[0039] The technical solution of this embodiment dynamically determines the shape and position of the keyboard display area according to the type of keys selected by the user, which can produce the following significant technical effects: It greatly optimizes screen space utilization and visual harmony: By adaptively determining the shape of the display area based on the inherent logic and operating habits of key types (such as arrow keys, number keys, and letter keys) (e.g., arrow keys use a cross-shaped layout, and number keys use a grid layout), the layout of each keyboard area best matches its functional characteristics, making operation intuitive. At the same time, it dynamically adjusts the relative positions of each area (e.g., side-by-side, stacked, centered), ensuring that the virtual keyboard as a whole is highly adapted to the mobile screen size and current orientation, eliminating the visual abruptness and operational inconvenience caused by rigid layouts, disproportionate proportions, or improper positions, achieving a unity of functionality and aesthetics.

[0040] This effectively avoids obstructing the project's running effects and enhances the user experience: the intelligent positioning mechanism places the virtual keyboard in the most reasonable position on the screen (such as at the bottom) and can be fine-tuned according to important elements of the program's running screen. This minimizes obstruction of the main visual area of ​​the running project, allowing users to observe the program's dynamic response effects without hindrance while operating the keys, thus achieving an immersive programming and experience environment similar to operating a physical keyboard on a PC.

[0041] It provides an ergonomic and cognitively intuitive interface: different button types correspond to different optimal operating areas and gestures (e.g., directional keys are suitable for thumb sliding at the bottom of the screen, while number keys are suitable for clicking). By setting the display shape and position according to button type, this solution places the most frequently used operating areas in the most easily accessible and operable screen positions (e.g., the thumb's hot zone), reducing operator fatigue. At the same time, the layout, which conforms to the user's mental model (e.g., the D-pad for directional control), significantly reduces learning costs and user confusion, making the interaction more natural, efficient, and accurate.

[0042] This achieves deep integration of the virtual keyboard with mobile devices and application scenarios: the adaptive mechanism transforms the virtual keyboard from an isolated control floating above the application into an organic component that intelligently integrates into the current operating environment (such as screen resolution, orientation, and project interface layout). This deep integration ensures a consistent and high-quality user experience across various complex mobile devices and usage scenarios, enhancing the professionalism and robustness of the entire development environment.

[0043] In summary, the technical solution of this embodiment not only solves the basic input function problem by intelligently determining the shape and position of the keyboard display area, but also brings multi-dimensional improvements in space utilization, visual interaction, human-computer interaction and system integration, ultimately significantly enhancing the overall experience of graphical programming on mobile devices.

[0044] This embodiment provides a virtual keyboard display method for mobile graphical programming development environments, including: If the selected event block and / or detection block contains at least one of the following keys: directional keys, number / space keys, and letter keys, then a full keyboard mode is displayed, which includes an directional keyboard display area, a number / space keyboard display area, and a letter keyboard display area. If the selected event block and / or detection block includes directional keys and number / space keys, then the combination mode of directional keys and number / space keys is displayed. The combination mode of directional keys and number / space keys includes a directional keyboard display area and a number / space keyboard display area. If the selected event block and / or detection block includes letter keys and number / space keys, then the letter key and number / space key combination mode is displayed, which includes a letter keyboard display area and a number / space keyboard display area. If the selected event-type block and / or detection-type block includes both directional and alphabetic keys, then the directional key and alphabetic key combination mode is displayed, which includes a directional keyboard display area and an alphabetic keyboard display area.

[0045] Specifically, the virtual keyboard display in this embodiment is mainly divided into three groups: [arrow keys], [1, 2, 3, 4 + space], and [WASD]. In each group, only the keys involved in the programming code block are highlighted and clickable, while the keys not involved are grayed out. The keyboard display mode is as follows: (1) such as Figure 4 As shown, the trigger condition for full keyboard mode is: ① When all three sets of key contents are involved in the code block (i.e., they exist in the [number / space key], [arrow keys], and [letter keys]); (2) such as Figure 5 As shown, the [arrow keys] + [1, 2, 3, 4 + space] combination mode is triggered under the following conditions: ① When the code block contains only the arrow keys; ② When the code block contains only numbers or spaces; ③ When the code block contains any one arrow key and any one numeric space key, and does not contain any of the letters w, a, s, or d.

[0046] (3) such as Figure 6 As shown, the trigger condition for the combination pattern of

wasd

[0047] (4) such as Figure 7 As shown, the [arrow keys] + [WASD] combination mode is triggered under the following conditions: ① When the code block contains any one arrow key and any one letter key, but no number or space key.

[0048] like Figures 4 to 7 As shown in the figure, this embodiment describes a virtual keyboard display method for a mobile graphical programming development environment, characterized in that... The virtual keyboard's overall display area is fixed at the bottom of the mobile screen and is displayed in a horizontal arrangement. The directional keyboard display area displays the directional keys in a cross-shaped layout, the numeric / space keyboard display area displays the numeric keys and space key in a rectangular grid layout, and the letter keyboard display area displays the letter keys in a rectangular grid layout. In full keyboard mode, the arrow key display area, the number / space key display area, and the letter key display area are arranged horizontally in a preset order. In combination mode, the included keyboard display areas are arranged horizontally or stacked vertically according to a preset layout. The overall display area size of the virtual keyboard is dynamically adjusted according to the number of keyboard display areas currently displayed; the more display areas there are, the larger the overall display area becomes.

[0049] Furthermore, the virtual keyboard display method for a mobile graphical programming development environment in this embodiment includes: In the virtual keyboard display mode, the selected button in the drop-down menu of the blocks is highlighted as operable, while the unselected button is grayed out as inoperable.

[0050] In this embodiment, the virtual keyboard display method for a mobile graphical programming development environment determines the specific implementation of displaying the corresponding keyboard display area based on the key type selected in the event-type blocks and / or detection-type blocks. This method has the following technical features: 1. Intelligent decision-making logic for display modes The virtual keyboard's display mode is not randomly determined, but dynamically generated based on logical judgments of the key combinations selected in the user's code. Its core decision-making logic includes: Full Keyboard Mode: This mode is triggered only when the user code contains arrow keys, number / space keys, and letter keys simultaneously, displaying all three keyboard areas (arrow area, number / space area, and letter area) to the user.

[0051] Combination Keyboard Mode: When user code contains only two types of keys, the system intelligently hides the unused third type of keyboard area, displaying only the relevant two areas. For example: If only arrow keys and number / space keys are available, the "arrow keys + number / space keys" combination mode will be displayed.

[0052] If only letter keys and number / space keys are present, the "letter key + number / space key" combination mode will be displayed.

[0053] If only arrow keys and letter keys are available, the "arrow keys + letter keys" combination mode will be displayed.

[0054] Single Keyboard Mode: When user code uses only a certain type of key (such as only using the arrow keys), the system will only display a single keyboard display area corresponding to that type of key, maximizing the saving of screen space.

[0055] 2. Layout and arrangement rules of the keyboard display area To adapt to diverse screen sizes on mobile devices and provide a consistent interactive experience, the layout of each keyboard display area follows preset rules: Horizontal priority arrangement: When screen space allows, multiple keyboard areas that need to be displayed at the same time (such as in combo mode) should be arranged horizontally side by side to conform to the user's visual habits of left and right operation.

[0056] Adaptive Adjustment: The system dynamically adjusts the size and spacing of each display area based on the available width of the screen. On narrow-screen devices (or in portrait mode), if there is insufficient horizontal space, it can automatically switch to a vertical stacked layout to ensure all buttons are clearly tactile.

[0057] Fixed core area: The numeric / space keyboard display area is usually regarded as the core input area because it contains the most frequently used "space" key. Its relative position (such as center) in various combination modes remains fixed to provide a stable operation anchor point.

[0058] 3. Fine-grained management of button status Within each displayed keyboard area, not all keys have equal access permissions. The system will perform a second, more refined assessment: Highlight available keys: Only the specific keys that are actually selected and used in the user code (e.g., the user only selected the "up arrow" and "space bar") are highlighted and clickable.

[0059] Gray off unused keys: Disable keys that are not used by the code in the same area (e.g., in the direction area, the "left, right, and down arrows" are not used) by making them gray and semi-transparent.

[0060] This design ensures a high degree of consistency between the virtual keyboard interface and the user's program logic, providing users with extremely clear and precise operation guidance.

[0061] Therefore, in this embodiment of the virtual keyboard display method for a mobile graphical programming development environment, determining the specific implementation method for displaying the corresponding keyboard display area based on the key type selected in the event-type building blocks and / or detection-type building blocks can produce the following significant technical effects: It maximizes screen space utilization efficiency: Through a three-level dynamic adjustment mechanism of "full mode - combined mode - single mode", the system can accurately allocate screen resources according to the actual needs of the user's code. The full keyboard is only displayed when absolutely necessary; in most cases, only 1-2 keyboard areas, or even a single key area, are displayed. This greatly reduces the obstruction of the virtual keyboard from the project's running screen, solving the core pain point of limited screen space on mobile devices and providing users with a wider field of view to observe the program's operation.

[0062] It offers a highly intelligent and adaptive interactive experience: This solution is no longer a simple "present" or "absent" virtual keyboard, but an intelligent system capable of understanding the user's programming intent. By analyzing the code content, it automatically infers the most suitable keyboard layout and key states, adapting to different screen sizes. This "what you think is what you see, what you use is what you get" interaction significantly reduces the user's cognitive load and operational costs, making the mobile programming experience smoother and more efficient than ever before.

[0063] Enhanced accuracy and guidance: By differentiating between "activated" and "disabled" states for buttons, the interface itself becomes a clear "user manual." Users can immediately identify valid operation targets without trial and error or guesswork, completely avoiding the confusion and frustration caused by accidentally pressing invalid buttons. This visual guidance greatly improves the accuracy and user-friendliness of human-computer interaction, especially suitable for Scratch's target users (teenagers and beginners).

[0064] Enhanced functional completeness and platform consistency: This solution ensures that any Scratch project using button interaction can be operated via corresponding virtual buttons on mobile devices, compensating for the functional deficiencies of mobile devices compared to PCs. This enables cross-platform project creation and experience, maintains the consistency and integrity of the Scratch programming environment, and strongly supports the promotion and development of mobile programming education.

[0065] As an optional implementation of this embodiment, the virtual keyboard display method for a mobile graphical programming development environment is characterized in that, during the process of receiving the user's graphical programming operation on the mobile terminal, detecting whether the user's programming code contains event-type blocks and / or detection-type blocks related to key presses includes: Before the program runs, static analysis is performed on the graphical code blocks assembled by the user to identify whether they contain specific types of event-type building blocks and / or detection-type building blocks. The detection of whether the user-programmed code contains event-type blocks and / or detection-type blocks related to key presses includes: During program execution, the code execution flow is monitored in real time. When a specific type of event block and / or detection block is detected, the corresponding processing logic is triggered. The detection process includes the following steps: Parse the syntax tree structure of the user's programming code; Traverse the syntax tree to identify whether it contains event class blocks and / or detection class blocks of predefined types; When the building block is detected, the button parameter information set therein is extracted. The button parameter value includes at least one of the arrow keys, number keys, space key, or letter keys.

[0066] This embodiment also provides a virtual keyboard display device for a mobile graphical programming development environment, including: The key event detection module detects whether the user's programming code contains key-related event blocks and / or detection blocks during the process of receiving the user's graphical programming operation on the mobile terminal. The virtual keyboard display module, if it includes event-type blocks and / or detection-type blocks related to key presses, displays the virtual keyboard when the user's programming code program is running, and determines the display mode of the virtual keyboard according to the key type selected in the event-type blocks and / or detection-type blocks.

[0067] Figure 8 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. The electronic device includes a processor and a memory. The memory is used to store a computer-executable program. When the computer program is executed by the processor, the processor executes the virtual keyboard display method for a mobile graphical programming development environment according to the embodiment.

[0068] like Figure 8 As shown, the electronic device is embodied in the form of a general-purpose computing device. There can be one or more processors working collaboratively. This invention also does not preclude distributed processing, meaning that processors can be distributed across different physical devices. The electronic device of this invention is not limited to a single entity, but can also be the sum of multiple physical devices.

[0069] The memory stores a computer-executable program, typically machine-readable code. The computer-readable program can be executed by the processor to enable the electronic device to perform the method of the present invention, or at least some steps of the method.

[0070] The memory includes volatile memory, such as random access memory (RAM) and / or cache memory, and may also be non-volatile memory, such as read-only memory (ROM).

[0071] Optionally, in this embodiment, the electronic device further includes an I / O interface for exchanging data with external devices. The I / O interface can represent one or more of several bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0072] It should be understood that Figure 8The electronic device shown is merely one example of the present invention, and the electronic device of the present invention may also include elements or components not shown in the above examples. For example, some electronic devices also include display units such as displays, and some electronic devices also include human-computer interaction elements such as buttons and keyboards. Any electronic device capable of executing a computer-readable program in memory to implement the method of the present invention or at least some steps of the method can be considered as an electronic device covered by the present invention.

[0073] Figure 9 This is a schematic diagram of a computer-readable recording medium according to an embodiment of the present invention. Figure 9 As shown, a computer-readable recording medium stores a computer-executable program. When the computer-executable program is executed, it implements the virtual keyboard display method for a mobile graphical programming development environment according to embodiments of the present invention. The computer-readable recording medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable recording medium may also be any readable medium other than a readable recording medium, which can send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the readable recording medium can be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0074] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0075] From the above description of the embodiments, those skilled in the art will readily understand that the present invention can be implemented by hardware capable of executing specific computer programs, such as the system of the present invention, and the electronic processing unit, server, client, mobile phone, control unit, processor, etc. included in the system. The present invention can also be implemented by computer software executing the methods of the present invention, for example, by control software executed by a microprocessor, electronic control unit, client, server, etc. However, it should be noted that the computer software executing the methods of the present invention is not limited to execution in one or a specific set of hardware entities; it can also be implemented in a distributed manner by unspecified hardware. For computer software, the software product can be stored on a computer-readable recording medium (such as a CD-ROM, USB flash drive, portable hard disk, etc.) or distributed across a network, as long as it enables electronic devices to execute the methods according to the present invention.

[0076] The above embodiments are only used to illustrate the present invention and are not intended to limit the technical solutions described herein. Although the present invention has been described in detail with reference to the above embodiments, the present invention is not limited to the specific embodiments described above. Therefore, any modifications or equivalent substitutions to the present invention, as well as all technical solutions and improvements that do not depart from the spirit and scope of the invention, are covered within the scope of the claims of the present invention.

Claims

1. A method for displaying a virtual keyboard in a mobile graphical programming development environment, characterized in that, include: During the process of receiving the user's graphical programming operation on the mobile device, detect whether the user's programming code contains event class blocks and / or detection class blocks related to key presses; If it contains event-type blocks and / or detection-type blocks related to key presses, then a virtual keyboard will be displayed when the user's programming code runs. The display mode of the virtual keyboard is determined based on the key type selected in the event-type block and / or detection-type block.

2. The virtual keyboard display method for a mobile graphical programming development environment according to claim 1, characterized in that, The detection method checks whether the user programming code contains event-type blocks and / or detection-type blocks related to key presses. The content of the event-type blocks includes "when the ( ) key is pressed", and the content of the detection-type blocks includes "when the ( ) key is pressed?". The "( ) key" in the content of the blocks is selected by a drop-down list.

3. The virtual keyboard display method for a mobile graphical programming development environment according to claim 2, characterized in that, The step of determining the virtual keyboard display mode based on the key type selected in the event-type block and / or detection-type block includes: The virtual keyboard includes an arrow key display area, a number / space key display area, and an alphabet key display area; Based on the selected directional keys, and / or number / space keys, and / or letter keys in the event-type blocks and / or detection-type blocks, the corresponding keyboard display area of ​​the virtual keyboard is determined and displayed.

4. The virtual keyboard display method for a mobile graphical programming development environment according to claim 3, characterized in that, include: The shape and position of the keyboard display area are determined based on the key type selected in the event-type blocks and / or detection-type blocks.

5. The virtual keyboard display method for a mobile graphical programming development environment according to claim 4, characterized in that, include: If the selected event block and / or detection block contains at least one of the following keys: directional keys, number / space keys, and letter keys, then a full keyboard mode is displayed, which includes an directional keyboard display area, a number / space keyboard display area, and a letter keyboard display area. If the selected event block and / or detection block includes directional keys and number / space keys, then the combination mode of directional keys and number / space keys is displayed. The combination mode of directional keys and number / space keys includes a directional keyboard display area and a number / space keyboard display area. If the selected event block and / or detection block includes letter keys and number / space keys, then the letter key and number / space key combination mode is displayed, which includes a letter keyboard display area and a number / space keyboard display area. If the selected event-type block and / or detection-type block includes both directional and alphabetic keys, then the directional key and alphabetic key combination mode is displayed, which includes a directional keyboard display area and an alphabetic keyboard display area.

6. The virtual keyboard display method for a mobile graphical programming development environment according to claim 5, characterized in that, The virtual keyboard's overall display area is fixed at the bottom of the mobile screen and is displayed in a horizontal arrangement. The directional keyboard display area displays the directional keys in a cross-shaped layout, the numeric / space keyboard display area displays the numeric keys and space key in a rectangular grid layout, and the letter keyboard display area displays the letter keys in a rectangular grid layout. In full keyboard mode, the arrow key display area, the number / space key display area, and the letter key display area are arranged horizontally in a preset order. In combination mode, the included keyboard display areas are arranged horizontally or stacked vertically according to a preset layout. The overall display area size of the virtual keyboard is dynamically adjusted according to the number of keyboard display areas currently displayed; the more display areas there are, the larger the overall display area becomes.

7. The virtual keyboard display method for a mobile graphical programming development environment according to claim 3, characterized in that, During the process of receiving the user's graphical programming operation on the mobile terminal, detecting whether the user's programming code contains event-type blocks and / or detection-type blocks related to key presses includes: Before the program runs, static analysis is performed on the graphical code blocks assembled by the user to identify whether they contain specific types of event-type building blocks and / or detection-type building blocks. The detection of whether the user-programmed code contains event-type blocks and / or detection-type blocks related to key presses includes: During program execution, the code execution flow is monitored in real time. When a specific type of event block and / or detection block is detected, the corresponding processing logic is triggered. The detection process includes the following steps: Parse the syntax tree structure of the user's programming code; Traverse the syntax tree to identify whether it contains event class blocks and / or detection class blocks of predefined types; When the building block is detected, the button parameter information set therein is extracted. The button parameter value includes at least one of the arrow keys, number keys, space key, or letter keys.

8. The virtual keyboard display method for a mobile graphical programming development environment according to claim 2, characterized in that, include: In the virtual keyboard's display mode, the selected button from the drop-down menu in the blocks is highlighted as operable, while the unselected button is grayed out as inoperable.

9. A virtual keyboard display device for a mobile graphical programming development environment, characterized in that, include: The key event detection module detects whether the user's programming code contains key-related event blocks and / or detection blocks during the process of receiving the user's graphical programming operation on the mobile terminal. The virtual keyboard display module, if it includes event-type blocks and / or detection-type blocks related to key presses, displays the virtual keyboard when the user's programming code program is running, and determines the display mode of the virtual keyboard according to the key type selected in the event-type blocks and / or detection-type blocks.

10. A computer-readable recording medium storing a computer-executable program, characterized in that, When the computer executable program is executed, it implements the virtual keyboard display method for a mobile graphical programming development environment as described in any one of claims 1-8.

Images