Graphical programming building block voice reading method and device

Abstract

The invention discloses a graphical programming building block voice reading method and device, and the method is characterized in that the method comprises the steps: receiving a building block voice reading mode starting instruction sent by a user; in response to the instruction, activating a building block voice reading function in a graphical programming interface; detecting a voice reading instruction triggered by a user for the target building block; and in response to the voice reading instruction, performing voice reading on the function or content represented by the target building block. According to the graphical programming building block voice reading method, by adding the voice reading function of the building blocks, the basic problem that young users read the building blocks is solved, the dimension of man-machine interaction is enriched through an intelligent and non-intrusive interactive design, the learning efficiency is improved, meanwhile, the experience of core operation is guaranteed, and the user experience is improved. The method has remarkable technical progress and positive popularization and application value.

Description

Technical Field

[0001] This invention relates to the field of computer graphical programming education and human-computer interaction technology, specifically providing a method and device for graphical programming block-based speech reading. Background Technology

[0002] With the popularization of children's programming education, graphical programming, with its low barrier to entry and high interactivity, has become a core medium for children and programming beginners to learn programming. Among them, Scratch IDE is one of the most widely used graphical programming tools. When using Scratch IDE, users do not need to write traditional text code; they simply drag and drop "blocks" (graphical modules encapsulating specific program logic) from the interface to the script canvas and combine them to create and run programs. This block-based programming mode effectively lowers the technical threshold for learning programming, allowing young users to focus more on the design and creative implementation of program logic.

[0003] However, existing Scratch programming environments primarily rely on visual interaction, and their functionality and content understanding heavily depend on the user's ability to recognize text and images. Currently, this technological field suffers from the following main defects and limitations: First, it is not user-friendly for young children and those with limited literacy skills. Scratch's core users include many preschool or lower elementary school children who have not yet fully mastered reading. Faced with text labels on the blocks, these users cannot understand their meaning independently and must rely on teachers or parents for assistance. This seriously hinders their independent learning and exploration process and increases the initial barrier to learning programming.

[0004] Secondly, existing programming environments lack effective multimodal interactive assistance features, especially voice-assisted support. Current mainstream Scratch IDEs (Integrated Development Environments) do not have the function of converting block content into speech and reading it aloud. User interaction with the program relies entirely on the visual channel, which not only limits the user base but also fails to fully utilize the auditory channel to enhance understanding and memory, thus contradicting the modern educational philosophy of multisensory learning.

[0005] Finally, there are potential conflicts in the interaction logic. Even if we envision adding voice functionality to the blocks, designing the trigger mechanism is a major challenge. If we simply define clicking or touching the blocks as the trigger for reading aloud, it will seriously conflict with the inherent and core "drag and drop" operation of the blocks, causing users to frequently trigger the reading aloud accidentally when trying to move the blocks, thereby disrupting the continuity of the programming flow and reducing the user experience.

[0006] Therefore, there is an urgent need in this field for an innovative solution that can intelligently provide voice reading assistance in a graphical programming environment, while effectively avoiding conflicts with existing interactive operations (such as drag and drop), thereby truly meeting the needs of young users and improving the accessibility and friendliness of programming learning.

[0007] In view of this, this invention patent is hereby proposed. Summary of the Invention

[0008] To address the aforementioned technical problems, this invention proposes a graphical programming-based block-based speech reading method and apparatus, specifically employing the following technical solution: In a first aspect, the present invention provides a graphical programming method for reading aloud using building blocks, comprising: Receives user commands to activate the block-based voice reading mode; In response to the instruction, the block-based voice reading function is activated in the graphical programming interface; Detect the user's voice command triggered by the target block; In response to the voice reading instruction, the function or content represented by the target block is read aloud.

[0009] As an optional embodiment of the present invention, in a graphical programming method for reading aloud using building blocks, activating the reading aloud function in the graphical programming interface includes: Traverse all current blocks in the graphical programming interface and identify the set of blocks that support voice reading; A visual reading prompt icon is displayed on each block in the set of blocks that support voice reading. The visual reading prompt icon switches between the display states of the block when it is not being read aloud, when it is being read aloud, and when it is being read aloud.

[0010] As an optional embodiment of the present invention, in a graphical programming block-based voice reading method of the present invention, the block set supporting voice reading has a visual reading prompt display area and a block drag operation area. The visual reading prompt icon is displayed in the visual reading prompt icon display area; When a user triggers a command to operate the visual reading prompt display area, the function or content represented by the target block is read aloud. When a user presses on the drag operation area of ​​the building block and drags it, the target building block moves in response to the user's drag command.

[0011] As an optional embodiment of the present invention, a graphical programming block-based speech reading method of the present invention includes: Receive operation instructions from the user on the target block; The input characteristic parameters of the operation command are collected and analyzed. These input characteristic parameters include at least one of the following: operation duration, operation pressure value, touch contact area, and the rate of change of contact area or pressure value during the operation: Based on the input feature parameters, a predefined classification algorithm is used to determine whether the operation instruction is a touch operation or a press operation. If the operation is determined to be a touch operation, the voice reading function for the target block is triggered; If the operation is determined to be a pressing operation, the drag-and-drop function of the target block is triggered. The judgment is made by a predefined classification algorithm, which adopts a machine learning classification model based on multiple feature parameters. The input feature parameters include operation duration, pressure value and contact area. The output of the model is the probability of touch operation or press operation.

[0012] As an optional embodiment of the present invention, in a graphical programming block-based speech reading method of the present invention, the function or content represented by the target block is read aloud, including: Extract the text content from the target block; The text content is semantically parsed and sentences are reorganized to generate a text that conforms to natural language habits; The speech synthesis engine is invoked to convert the read-out text into a speech signal and output it.

[0013] As an optional embodiment of the present invention, in a graphical programming block-based speech reading method of the present invention, the function or content represented by the target block is read aloud, including: When the target block contains a drop-down selection list, determine whether the target block has selected an option from the drop-down list; If an option in the drop-down list is selected, the content of the currently selected option will be read aloud. If no option in the drop-down list is selected, it will further determine whether the drop-down selection list is in the expanded state. If the drop-down selection list is expanded, the text of the currently focused option will be read aloud. If the drop-down selection list is not expanded, a prompt voice will be played to guide the user in making a selection.

[0014] As an optional embodiment of the present invention, in a graphical programming block-based speech reading method of the present invention, the function or content represented by the target block is read aloud, including: When the target building block contains symbolic or graphic identifiers, the speech reading process includes: Identify the type of symbolic or graphic identifier; Query the predefined semantic mapping table of symbol and graphic identifiers to obtain the corresponding natural language descriptions; The natural language description is incorporated into the reading content for output.

[0015] As an optional embodiment of the present invention, in a graphical programming block-based speech reading method of the present invention, the function or content represented by the target block is read aloud, including: When the target building block is a nested structure, the speech reading process traverses the building block structure from the outside to the inside and from the left to the right. Extract the text content of each level of building blocks; After combining all the text content to generate a complete semantic sentence, it is read aloud.

[0016] As an optional embodiment of the present invention, in a graphical programming method for reading aloud from a block, the present invention identifies block blocks in the current programming interface that do not support reading aloud. For building blocks that do not support voice reading, no voice reading prompt will be displayed, or an unavailable prompt will be displayed. The building blocks that do not support voice reading include at least one of the following: Blank placeholder blocks; Purely graphical, text-free building blocks; The content of system function blocks cannot be effectively described using natural language. User-defined blocks that have been set to disable the text-to-speech function.

[0017] In a second aspect, the present invention provides a graphically programmable block-based speech reading device, comprising: The voice reading command receiving module receives the user's command to start the block voice reading mode; The voice reading function activation module, in response to the instruction, activates the block voice reading function in the graphical programming interface; The voice reading function control module detects the voice reading command triggered by the user on the target block, and responds to the voice reading command by reading the function or content represented by the target block.

[0018] The present invention provides a graphical programming method for reading aloud using building blocks. By adding the reading aloud function to the building blocks, it achieves the following technical effects: This invention significantly lowers the learning threshold for graphical programming, enhancing the inclusivity and accessibility of education. By introducing a voice reading function, the text information of the building blocks is converted into auditory signals, enabling young children who have not yet mastered reading skills or users with reading disabilities to independently understand the meaning of programming instructions. This reduces reliance on teacher or parental assistance, supports users' independent learning and exploration, and effectively expands the scope of applicable users.

[0019] This invention provides a multimodal human-computer interaction method, optimizing the user experience. Traditional graphical programming environments rely solely on visual interaction, while this invention adds an auditory interaction channel, forming a bimodal information transmission of "visual + auditory". This design not only better aligns with the cognitive patterns of multisensory learning, helping to deepen users' understanding and memory of programming logic, but also makes the programming process more vivid and interesting, thereby enhancing users' learning interest and immersion.

[0020] This invention achieves accurate recognition of interactive intents, avoiding functional conflicts. Instead of simply utilizing the original drag-and-drop operation events, it activates voice reading through specific trigger commands (such as clicking a specific icon, distinguishing between touch and press). This design effectively isolates the trigger conditions for "reading aloud" and "drag and drop" operations logically or spatially, thus completely avoiding the operational conflict of accidentally triggering the reading function when the user intends to move the blocks, ensuring the smoothness and stability of the core programming flow.

[0021] The solution enhances the intelligence and practicality of the functionality. It incorporates targeted speech processing logic for special building blocks (such as those with dropdown lists, directional arrows, or nested structures), enabling the generation of complete sentences that conform to natural language habits for reading aloud. This demonstrates that the function is not simply text-to-speech, but rather integrates semantic understanding and content reorganization, resulting in more accurate and natural readings and significantly improving the function's practical value and intelligence.

[0022] In summary, this invention not only solves the basic problem of young users recognizing building blocks, but also enriches the dimensions of human-computer interaction through an intelligent and non-intrusive interactive design. It improves learning efficiency while ensuring the experience of core operations, demonstrating significant technological progress and positive value for promotion and application. Attached Figure Description

[0023] Figure 1 A flowchart of a graphical programming-based block-based speech reading method according to an embodiment of the present invention; Figure 2 A schematic diagram of the structure of the electronic device according to an embodiment of the present invention; Figure 3 A schematic diagram of a computer-readable recording medium according to an embodiment of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0025] Therefore, the following detailed description of embodiments of the present invention is not intended to limit the scope of the claimed invention, but merely illustrates some embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0026] It should be noted that, unless otherwise specified, the embodiments and features and technical solutions in the embodiments of the present invention can be combined with each other.

[0027] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0028] In the description of this invention, it should be noted that the terms "upper," "lower," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of this invention is in use, or the orientation or positional relationship commonly understood by those skilled in the art. These terms are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention. In addition, the terms "first," "second," etc., are only used to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0029] See Figure 1 As shown, this embodiment provides a graphical programming-based block-based speech reading method, including: Receives user commands to activate the block-based voice reading mode; In response to the instruction, the block-based voice reading function is activated in the graphical programming interface; Detect the user's voice command triggered by the target block; In response to the voice reading instruction, the function or content represented by the target block is read aloud.

[0030] This embodiment of a graphical programming block-based speech reading method, by adding speech reading functionality to the blocks, achieves the following technical effects: (1) It significantly lowers the learning threshold of graphical programming and enhances the inclusiveness and accessibility of education. By introducing a voice reading function, this invention converts the text information of the building blocks into auditory signals, enabling young children who have not yet mastered the ability to read or users with reading disabilities to independently understand the meaning of programming instructions. This reduces the reliance on teachers or parents for assistance, supports users' independent learning and exploration, and effectively expands the scope of applicable users.

[0031] (2) It provides a multimodal human-computer interaction method, which optimizes the user experience. Traditional graphical programming environments rely solely on visual interaction, while this invention adds an auditory interaction channel, forming a bimodal information transmission of "visual + auditory". This design not only conforms to the cognitive laws of multisensory learning, helping to deepen users' understanding and memory of programming logic, but also makes the programming process more vivid and interesting, thereby enhancing users' learning interest and immersion.

[0032] In summary, the graphical programming-based block-based voice reading method of this embodiment not only solves the basic problem of young users recognizing building blocks, but also enriches the dimensions of human-computer interaction through an intelligent and non-intrusive interactive design. It improves learning efficiency while ensuring the experience of core operations, and has significant technological progress and positive promotion and application value.

[0033] In this embodiment of the graphical programming block-based speech reading method, activating the block-based speech reading function in the graphical programming interface includes: Traverse all current blocks in the graphical programming interface and identify the set of blocks that support voice reading; A visual reading prompt icon is displayed on each block in the set of blocks that support voice reading. The visual reading prompt icon switches between the display states of the block when it is not being read aloud, when it is being read aloud, and when it is being read aloud.

[0034] This embodiment proactively reveals the boundaries and availability of the functionality to the user by traversing the interface and displaying prompts only on blocks that support reading aloud. Users can immediately see which blocks can be read aloud without trial-and-error clicking, thus avoiding confusion or frustration from clicking on blocks that do not support reading aloud, and greatly reducing cognitive load.

[0035] This embodiment provides users with rich and timely interactive status information by displaying visual reading prompts that switch between different states (not being read, reading in progress, and already read). For example, the "reading in progress" state (e.g., icon animation, highlighting) clearly informs the user that the system has correctly received the instruction and is processing it, effectively alleviating the uncertainty during the waiting process; while the "read already read" state serves as a visual memory aid, helping users, especially younger users, to track their learning path and avoid repetitive operations.

[0036] The "traversal recognition" step described in this embodiment is a crucial preprocessing step. The system completes the screening of all blocks at the very beginning of function activation, rather than making a judgment only when the user clicks. This "pre-calculation" strategy distributes the computational burden to the initial stage, avoiding the delays and stutters that may occur when performing instant recognition at the high-frequency clicks of user interaction. This ensures the immediacy and smoothness of subsequent reading trigger responses, improving the overall user experience.

[0037] This embodiment identifies the set of blocks that support voice reading in a programmatic manner, ensuring the accuracy of function triggering. It fundamentally prevents program errors or logical anomalies that may be caused by mistakenly assigning the reading function to blocks without text content or that cannot be read (such as blank placeholders or pure graphic blocks), thus ensuring the stability and robustness of the system.

[0038] In summary, the technical solution of this embodiment does not simply implement the function, but significantly improves the discoverability, usability, responsiveness and performance of the function through careful preprocessing and dynamic visual design. This allows the voice reading function to be organically integrated from an isolated feature into the entire graphical programming interactive environment, reflecting profound human-computer interaction design considerations.

[0039] As an optional implementation of this embodiment, in a graphical programming block-based voice reading method of this embodiment, the block set supporting voice reading has a visual reading prompt display area and a block drag operation area. The visual reading prompt icon is displayed in the visual reading prompt icon display area; When a user triggers a command to operate the visual reading prompt display area, the function or content represented by the target block is read aloud. When a user presses on the drag operation area of ​​the building block and drags it, the target building block moves in response to the user's drag command.

[0040] This embodiment clearly separates the trigger areas for the two high-frequency operations, "speech reading" and "drag and drop," in physical space (display area vs. operation area), ensuring that every touch or click by the user has a clear intent. The system no longer needs to rely on complex algorithms to guess whether the user wants to read aloud or move, thus completely avoiding the problem of accidental triggering and ensuring the efficiency and smoothness of the programming process.

[0041] This embodiment binds functions to fixed screen areas, adhering to the classic design principle of "function visibility." Users (especially children) can very quickly establish conditioned reflexes between "clicking an icon" and "hearing a sound," and between "pressing the block" and "moving the block." This design eliminates the need for textual explanations; teaching is accomplished through the interface itself, making the interaction natural and intuitive.

[0042] This embodiment provides a separate, well-designed indicator display area for the "read aloud" function, essentially creating a clear and easy-to-click "button" for this operation. This is more precise than requiring users to click any part of the block to trigger the reading, reduces the difficulty of operation, and effectively improves the success rate of interaction and user experience, especially on touchscreen devices.

[0043] This embodiment enhances the system's predictability and the user's sense of control. The scheme establishes a stable and clear interaction mapping rule: "Operation A always occurs in area A, and operation B always occurs in area B." This high degree of consistency gives users a strong ability to predict interface behavior, making them feel that the entire system is under their complete control, thereby building confidence and encouraging them to explore and interact more.

[0044] In summary, the technical solution described in this embodiment cleverly solves the core conflict problem in multi-functional integration through a simple and efficient spatial partitioning design. It is not only a technical implementation but also an excellent interactive design, with its effects directly reflected in a leap in operational accuracy, a reduction in learning costs, an optimization of user experience, and the establishment of user confidence, enabling the voice reading function to be seamlessly and harmoniously integrated into the original drag-and-drop programming paradigm.

[0045] As another optional implementation of this embodiment, a graphical programming block-based speech reading method of this embodiment includes: Receive operation instructions from the user on the target block; The input characteristic parameters of the operation command are collected and analyzed. These input characteristic parameters include at least one of the following: operation duration, operation pressure value, touch contact area, and the rate of change of contact area or pressure value during the operation: Based on the input feature parameters, a predefined classification algorithm is used to determine whether the operation instruction is a touch operation or a press operation. If the operation is determined to be a touch operation, the voice reading function for the target block is triggered; If the operation is determined to be a pressing operation, the drag-and-drop function of the target block is triggered. The judgment is made by a predefined classification algorithm, which adopts a machine learning classification model based on multiple feature parameters. The input feature parameters include operation duration, pressure value and contact area. The output of the model is the probability of touch operation or press operation.

[0046] Compared to solutions that differentiate intentions by dividing specific functional areas, the above-mentioned technical solution in this embodiment allows users to trigger different functions in the same area of ​​the building blocks through different operation methods (tap / press), without the need to add extra buttons or labels to the UI, maintaining the visual simplicity of the building blocks themselves and the cleanliness of the programming area, and reducing the cognitive burden on users.

[0047] The technical solution described in this embodiment employs a machine learning model based on multiple feature parameters (such as duration, pressure value, and contact area) for comprehensive judgment, rather than relying on a single, fixed threshold. This multi-dimensional, non-linear analysis method can more accurately capture the subtle differences in sensor data between touch and press operations, thereby significantly reducing the false judgment rate and avoiding recognition errors that may occur when faced with different user operating habits or different hardware devices due to a single threshold condition, resulting in extremely accurate function triggering.

[0048] Therefore, the graphical programming block-based speech reading method of this embodiment, by introducing machine learning algorithms, elevates the operation intention recognition from a simple logical judgment problem to an intelligent and adaptive pattern recognition problem, thereby achieving significant technical progress in recognition accuracy, user experience, interface design and system scalability, and providing core technical support for realizing natural, smooth and intelligent human-computer interaction.

[0049] In this embodiment of a graphical programming block-based speech reading method, the function or content represented by the target block is read aloud, including: Extract the text content from the target block; The text content is semantically parsed and sentences are reorganized to generate a text that conforms to natural language habits; The speech synthesis engine is invoked to convert the read-out text into a speech signal and output it.

[0050] In this embodiment, when performing block-based speech reading, the original text on the blocks is not mechanically and word-for-word read aloud. Instead, semantic parsing and sentence reconstruction are performed first. This process can identify and correct grammatical omissions, unclear pronoun references, and fragmented structures in the original text, generating complete sentences that conform to natural language habits. This makes the output speech no longer rigid code fragments, but fluent and meaningful instructions or descriptions, greatly reducing the user's auditory cognitive load, especially facilitating understanding and learning for younger users. Therefore, this embodiment significantly improves the naturalness and comprehensibility of speech reading when performing block-based speech reading, achieving a leap from "text reading" to "semantic reading."

[0051] As an optional implementation of this embodiment, a graphical programming block-based speech reading method of this embodiment reads aloud the function or content represented by the target block, including: When the target block contains a drop-down selection list, determine whether the target block has selected an option from the drop-down list; If an option in the drop-down list is selected, the content of the currently selected option will be read aloud. If no option in the drop-down list is selected, it will further determine whether the drop-down selection list is in the expanded state. If the drop-down selection list is expanded, the text of the currently focused option will be read aloud. If the drop-down selection list is not expanded, a prompt voice will be played to guide the user in making a selection.

[0052] This embodiment has the following technical features when performing voice reading for special target blocks with drop-down selection lists: This invention achieves precise voice feedback for dynamically interactive elements, greatly improving the usability and accessibility of complex blocks. Instead of treating dropdown lists as simple text labels, it deeply integrates their state machine (selection status, expansion status) and interaction context (current focus). By judging different states and triggering corresponding voice content (reading the selected item, reading the focused item, playing guiding prompts), the system can provide users with precise auditory feedback that is completely synchronized with changes in the visual interface. This allows visually impaired or illiterate users to clearly perceive changes in the dropdown list's state and available content, achieving truly accessible interaction.

[0053] It provides proactive interactive guidance, reducing the learning curve and operational confusion for users. When the drop-down list is not expanded and no selection is made, the system will proactively play a prompt voice (such as "Please select an item") to guide the user to the next step. This is an intelligent, context-aware help system. It effectively prevents users from getting stuck due to not understanding the drop-down list function or not knowing how to operate it. It is especially user-friendly for young beginners, guiding them to successfully complete programming tasks and improving their learning autonomy and fluency.

[0054] This solution optimizes the flow and efficiency of the interaction, creating a seamless multimodal user experience. It ensures consistent voice support throughout the entire process of manipulating the dropdown list—from gazing at unselected items to expanding and browsing options, and finally making a selection. Voice feedback is no longer an isolated, one-off event triggered only upon final selection; instead, it permeates the entire exploration and decision-making process. This tightly weaves visual actions with auditory feedback, reducing cognitive gaps for the user and making the interaction flow more natural and efficient.

[0055] This design embodies a highly intelligent and human-centered approach. The core of this technical solution lies in its decision-making logic. It simulates the thought process of an "intelligent assistant": first, it checks if there is a final choice (for maximum efficiency); if not, it checks if the user is currently browsing (providing immediate feedback); if still not, it infers that the user may be encountering difficulties and provides guidance. This hierarchical judgment logic demonstrates the system's deep understanding of user intent and proactive service capabilities, far exceeding simple, mechanical text reading, showcasing a high level of human-computer interaction intelligence.

[0056] In summary, this technical solution upgrades the voice reading function from static text playback to a dynamic interactive guide by refining the state recognition and response of the common UI component, dropdown lists. Its effectiveness lies not only in accurate reading but also in comprehension and guidance, thus achieving significant technological advancements in improving accessibility, lowering learning barriers, ensuring smooth operation, and demonstrating system intelligence. This greatly enhances the product's user experience and market competitiveness.

[0057] As another optional implementation of this embodiment, a graphical programming block-based speech reading method of this embodiment reads aloud the function or content represented by the target block, including: When the target building block contains symbolic or graphic identifiers, the speech reading process includes: Identify the type of symbolic or graphic identifier; Query the predefined semantic mapping table of symbol and graphic identifiers to obtain the corresponding natural language descriptions; The natural language description is incorporated into the reading content for output.

[0058] This embodiment has the following technical features when performing voice reading for special target building blocks containing symbolic or graphic identifiers: This invention achieves intelligent speech conversion of non-text programming elements, greatly expanding the coverage and practicality of speech reading functionality. By recognizing symbols or graphic identifiers and querying a predefined semantic mapping table, this invention successfully solves the problem that purely graphical building blocks cannot be effectively read aloud. This allows non-text information such as arrow directions, mathematical symbols, logic icons, and status indicators to be accurately converted into natural language descriptions (such as "move right," "loop infinitely," and "play sound"), thereby ensuring that the complete semantics of programming instructions can be transmitted through the auditory channel, significantly improving the practicality and universality of the function.

[0059] This approach improves the accuracy and naturalness of voice reading, ensuring consistency between auditory information and visual intent. Unlike mechanical recitations of simple symbol names (such as "right arrow") or filenames (such as "icon_sound.png"), this solution outputs functional descriptions (such as "turn right" or "play sound") through a semantic mapping table. This semantic-based conversion transforms the read-aloud content from obscure symbol names into actionable instructions that users can directly understand, greatly enhancing the accuracy and comprehensibility of voice feedback and ensuring a perfect match between what is heard and what is seen.

[0060] This solution enhances accessibility support for special user groups, improving the inclusivity and equity of programming education. For visually impaired or cognitively impaired users, graphic symbols are a significant barrier in visual interfaces. This solution reveals the functional meaning behind the symbols through voice, enabling them to understand and manipulate these building blocks. This breaks down the barriers of visual dependence, providing all users with equal access to programming tools. It is a crucial step towards inclusive design and has significant social value.

[0061] It offers high scalability and maintainability, reducing the cost of system upgrades and adaptations. Employing a core architecture of "semantic mapping tables," it manages the correspondence between symbols and descriptions in data form, rather than hard-coding it into the program logic. When new symbols need to be supported or the wording of existing descriptions needs to be changed, only the mapping table data needs to be updated, without modifying the core code. This design allows the system to easily adapt to the needs of different languages ​​(internationalization), different block library versions, or different educational scenarios, greatly reducing long-term maintenance costs and ensuring the technology's lifecycle.

[0062] This ensures consistency and standardization of interactive feedback, optimizing the multimodal learning experience. A centralized mapping table manages the reading descriptions of the same symbol uniformly, ensuring that the same symbol is read with the same description throughout the entire programming environment. This consistency avoids user confusion caused by hearing different descriptions, helps quickly establish solid cognitive associations, and thus optimizes the experience and effectiveness of visual-auditory co-learning.

[0063] In summary, this optional implementation successfully upgrades the voice reading function from "text broadcasting" to "semantic explanation" through intelligent semantic conversion of non-text information. The technical effects it brings in terms of functional coverage, expression accuracy, barrier-free access, system scalability, and user experience are significant and outstanding, providing an important and beneficial supplement to the core invention.

[0064] As another optional implementation of this embodiment, a graphical programming block-based speech reading method of this embodiment reads aloud the function or content represented by the target block, including: When the target building block is a nested structure, the speech reading process traverses the building block structure from the outside to the inside and from the left to the right. Extract the text content of each level of building blocks; After combining all the text content to generate a complete semantic sentence, it is read aloud.

[0065] This embodiment provides a graphical programming method for reading aloud the text of building blocks, which identifies building blocks in the current programming interface that do not support reading aloud. For building blocks that do not support voice reading, no voice reading prompt will be displayed, or an unavailable prompt will be displayed. The building blocks that do not support voice reading include at least one of the following: Blank placeholder blocks; Purely graphical, text-free building blocks; The content of system function blocks cannot be effectively described using natural language. User-defined blocks that have been set to disable the text-to-speech function.

[0066] The technical solution described in this embodiment significantly improves the clarity and user-friendliness of the user interface, effectively avoiding user confusion and misoperation. By intelligently identifying and distinguishing blocks that do not support voice reading and providing clear visual prompts (not displayed or displayed as unavailable), the system proactively and clearly conveys the boundaries of the function to the user. Users can intuitively understand which blocks have the reading function without making ineffective trial-and-error clicks, thereby avoiding the confusion and frustration caused by the lack of feedback, greatly reducing cognitive load and improving interaction efficiency.

[0067] The technical solution described in this embodiment achieves precise and efficient allocation of system resources, optimizes rendering performance, and reduces computational overhead. This solution avoids indiscriminately loading listener events and rendering prompts for all blocks. By pre-filtering out blocks that do not support text-to-speech and excluding them from the processing flow, the system saves unnecessary memory usage and CPU cycles, concentrating limited resources on blocks that support text-to-speech. This differentiated management ensures smooth interface rendering and interactive responses, effectively maintaining overall system performance, especially in complex projects with a large number of blocks.

[0068] The technical solution described in this embodiment enhances the rationality of handling special building blocks and the robustness of the system. This solution clearly defines several types of building blocks unsuitable for reading aloud (such as blank blocks, pure graphic blocks, and system function blocks), reflecting profound functional design considerations. For example, reading aloud a blank placeholder or a system icon that cannot be effectively described in words (such as the advanced settings gear) is neither practically meaningful nor likely to interfere with the user. Excluding these at the source prevents triggering meaningless reading requests and potential program errors, ensuring functional rigor and system robustness.

[0069] The technical solution described in this embodiment provides flexible user-defined control capabilities, respecting users' individual needs. By including "blocks whose reading function is disabled by the user" in the unsupported scope, users are given the authority to disable the reading function of specific blocks based on their own habits or specific scenarios (such as the need for a quiet environment). This design not only increases the product's flexibility but also embodies a user-centered design philosophy, ensuring that functions serve people rather than forcing users to adapt to functions, thereby improving user satisfaction and product usability.

[0070] The technical solution described in this embodiment enhances the rigor and accessibility of assistive function design. For visually impaired users who rely on voice reading, clear functional boundaries are crucial. Clearly indicating which modules cannot provide voice feedback is just as important as providing feedback itself. This helps users quickly build accurate mental models, understand the system's capabilities, and formulate effective operational strategies, avoiding wasting time and effort on elements where feedback is unavailable, reflecting a higher level of accessibility design considerations.

[0071] In summary, the technical solution described in this embodiment, by identifying and differentiating "blocks that do not support reading aloud," has produced significant beneficial effects in improving the clarity of interaction, optimizing system performance, ensuring the rationality of processing, empowering users, and enhancing accessibility. It does not simply implement "what can be done," but rather meticulously defines "what cannot be done" and "how to elegantly inform users of what cannot be done," reflecting a mature, complete, and user-experience-first design philosophy. This is a crucial foundation for ensuring the successful application of core functions.

[0072] This embodiment also provides a graphically programmable block-based speech reading device, including: The voice reading command receiving module receives the user's command to start the block voice reading mode; The voice reading function activation module, in response to the instruction, activates the block voice reading function in the graphical programming interface; The voice reading function control module detects the voice reading command triggered by the user on the target block, and responds to the voice reading command by reading the function or content represented by the target block.

[0073] Figure 2 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. The electronic device includes a processor and a memory. The memory is used to store a computer-executable program. When the computer program is executed by the processor, the processor executes a graphical programming block-based speech reading method according to an embodiment.

[0074] like Figure 2 As shown, the electronic device is embodied in the form of a general-purpose computing device. There can be one or more processors working collaboratively. This invention also does not preclude distributed processing, meaning that processors can be distributed across different physical devices. The electronic device of this invention is not limited to a single entity, but can also be the sum of multiple physical devices.

[0075] The memory stores a computer-executable program, typically machine-readable code. The computer-readable program can be executed by the processor to enable the electronic device to perform the method of the present invention, or at least some steps of the method.

[0076] The memory includes volatile memory, such as random access memory (RAM) and / or cache memory, and may also be non-volatile memory, such as read-only memory (ROM).

[0077] Optionally, in this embodiment, the electronic device further includes an I / O interface for exchanging data with external devices. The I / O interface can represent one or more of several bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0078] It should be understood that Figure 2 The electronic device shown is merely one example of the present invention, and the electronic device of the present invention may also include elements or components not shown in the above examples. For example, some electronic devices also include display units such as displays, and some electronic devices also include human-computer interaction elements such as buttons and keyboards. Any electronic device capable of executing a computer-readable program in memory to implement the method of the present invention or at least some steps of the method can be considered as an electronic device covered by the present invention.

[0079] Figure 3 This is a schematic diagram of a computer-readable recording medium according to an embodiment of the present invention. Figure 3 As shown, a computer-readable recording medium stores a computer-executable program. When the computer-executable program is executed, it implements a graphical programming block-based speech reading method according to an embodiment of the present invention. The computer-readable recording medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable recording medium may also be any readable medium other than a readable recording medium, which can send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the readable recording medium can be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0080] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0081] From the above description of the embodiments, those skilled in the art will readily understand that the present invention can be implemented by hardware capable of executing specific computer programs, such as the system of the present invention, and the electronic processing unit, server, client, mobile phone, control unit, processor, etc. included in the system. The present invention can also be implemented by computer software executing the methods of the present invention, for example, by control software executed by a microprocessor, electronic control unit, client, server, etc. However, it should be noted that the computer software executing the methods of the present invention is not limited to execution in one or a specific set of hardware entities; it can also be implemented in a distributed manner by unspecified hardware. For computer software, the software product can be stored on a computer-readable recording medium (such as a CD-ROM, USB flash drive, portable hard disk, etc.) or distributed across a network, as long as it enables electronic devices to execute the methods according to the present invention.

[0082] The above embodiments are only used to illustrate the present invention and are not intended to limit the technical solutions described herein. Although the present invention has been described in detail with reference to the above embodiments, the present invention is not limited to the specific embodiments described above. Therefore, any modifications or equivalent substitutions to the present invention, as well as all technical solutions and improvements that do not depart from the spirit and scope of the invention, are covered within the scope of the claims of the present invention.

Claims

1. A graphical programming method for block-based speech reading, characterized in that, include: Receives user commands to activate the block-based voice reading mode; In response to the instruction, the block-based voice reading function is activated in the graphical programming interface; Detect the user's voice command triggered by the target block; In response to the voice reading instruction, the function or content represented by the target block is read aloud.

2. The graphical programming block-based speech reading method according to claim 1, characterized in that, Activating the block-based speech reading function in the graphical programming interface includes: Traverse all current blocks in the graphical programming interface and identify the set of blocks that support voice reading; A visual reading prompt icon is displayed on each block in the set of blocks that support voice reading. The visual reading prompt icon switches between the display states of the block when it is not being read aloud, when it is being read aloud, and when it is being read aloud.

3. The graphical programming block-based speech reading method according to claim 2, characterized in that, The set of building blocks that support voice reading has a visual reading prompt display area and a building block drag operation area; The visual reading prompt icon is displayed in the visual reading prompt icon display area; When a user triggers a command to operate the visual reading prompt display area, the function or content represented by the target block is read aloud. When a user presses on the drag operation area of ​​the building block and drags it, the target building block moves in response to the user's drag command.

4. The graphical programming-based block-based speech reading method according to claim 2, characterized in that, include: Receive operation instructions from the user on the target block; The input characteristic parameters of the operation command are collected and analyzed. These input characteristic parameters include at least one of the following: operation duration, operation pressure value, touch contact area, and the rate of change of contact area or pressure value during the operation: Based on the input feature parameters, a predefined classification algorithm is used to determine whether the operation instruction is a touch operation or a press operation. If the operation is determined to be a touch operation, the voice reading function for the target block is triggered; If the operation is determined to be a pressing operation, the drag-and-drop function of the target block is triggered. The judgment is made by a predefined classification algorithm, which adopts a machine learning classification model based on multiple feature parameters. The input feature parameters include operation duration, pressure value and contact area. The output of the model is the probability of touch operation or press operation.

5. The graphical programming method for block-based speech reading according to claim 2, characterized in that, The function or content represented by the target building block is read aloud, including: Extract the text content from the target block; The text content is semantically parsed and sentences are reorganized to generate a text that conforms to natural language habits; The speech synthesis engine is invoked to convert the read-out text into a speech signal and output it.

6. The graphical programming-based block-based speech reading method according to claim 2, characterized in that, The function or content represented by the target building block is read aloud, including: When the target block contains a drop-down selection list, determine whether the target block has selected an option from the drop-down list; If an option in the drop-down list is selected, the content of the currently selected option will be read aloud. If no option in the drop-down list is selected, it will further determine whether the drop-down selection list is in the expanded state. If the drop-down selection list is expanded, the text of the currently focused option will be read aloud. If the drop-down selection list is not expanded, a prompt voice will be played to guide the user in making a selection.

7. The graphical programming method for block-based speech reading according to claim 2, characterized in that, The function or content represented by the target building block is read aloud, including: When the target building block contains symbolic or graphic identifiers, the speech reading process includes: Identify the type of symbolic or graphic identifier; Query the predefined semantic mapping table of symbol and graphic identifiers to obtain the corresponding natural language descriptions; The natural language description is incorporated into the reading content for output.

8. The graphical programming method for block-based speech reading according to claim 2, characterized in that, The function or content represented by the target building block is read aloud, including: When the target building block is a nested structure, the speech reading process traverses the building block structure from the outside to the inside and from the left to the right. Extract the text content of each level of building blocks; After combining all the text content to generate a complete semantic sentence, it is read aloud.

9. The graphical programming method for block-based speech reading according to claim 2, characterized in that, Identify blocks in the current programming interface that do not support voice reading; For building blocks that do not support voice reading, no voice reading prompt will be displayed, or an unavailable prompt will be displayed. The building blocks that do not support voice reading include at least one of the following: Blank placeholder blocks; Purely graphical, text-free building blocks; The content of system function blocks cannot be effectively described using natural language. User-defined blocks that have been set to disable the text-to-speech function.

10. A graphically programmable block-based speech reading device, characterized in that, include: The voice reading command receiving module receives the user's command to start the block voice reading mode; The voice reading function activation module, in response to the instruction, activates the block voice reading function in the graphical programming interface; The voice reading function control module detects the voice reading command triggered by the user on the target block, and responds to the voice reading command by reading the function or content represented by the target block.

Images