Braille reading system based on phased array ultrasonic tactile feedback

Abstract

The invention provides a braille reading system based on phased array ultrasonic tactile feedback, and the system comprises a visual processing unit which is configured to collect the graphic and text information of a reading material, and transmits the graphic and text information to an upper computer; the upper computer is configured to distinguish a text region and a graphic region in the graphic and text information through a target detection algorithm; processing and generating a static focus coordinate sequence corresponding to the braille dot matrix; discretization processing is carried out on the graph area, feature points are extracted, and a dynamic focus track point sequence corresponding to the graph contour is generated; the tactile control driving unit is used for receiving the static focus coordinate sequence or the dynamic focus track point sequence, operating a phase optimization algorithm to calculate phase delay and modulation information, and generating multiple paths of driving signals after power amplification; and the ultrasonic transducer array is configured to respond to the driving signal to emit ultrasonic waves, and the ultrasonic waves are focused in the space through the phased array to form a tactile focus, so that non-contact braille and graph tactile perception is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of human-computer interaction, computer vision and embedded technology, and particularly relates to a Braille reading system based on phased array ultrasonic tactile feedback. BACKGROUND

[0002] With the progress of science and technology and the improvement of social concern for the visually impaired, a series of devices to help the visually impaired live conveniently have been developed, among which the Braille reader is a hot research topic in recent years, and is committed to helping the visually impaired read more conveniently and efficiently. Traditional Braille readers are mostly based on mechanical structures or piezoelectric ceramic driving technology. For example, the mainstream dynamic Braille display forms a raised dot Braille through the physical lifting of an array of micro needles. Such devices, relying on mature mechanical control technology, have strong stability and accuracy in static text display, and can adapt to different environmental conditions such as temperature and humidity. For a long time, they have been an important tool for the visually impaired to obtain text information. However, the essential characteristics of mechanical or piezoelectric driving determine that the information update rate is limited by the physical inertia, and the single character refresh delay is usually hundreds of milliseconds, which is prone to lag during dynamic interaction, and it is difficult to meet the display needs of fast scrolling reading or dynamic content (such as real-time data charts).

[0003] Although with the development of artificial intelligence and micro-electro-mechanical systems, some intelligent Braille readers try to optimize the driving circuit or introduce adaptive control algorithms to improve the refresh speed, and can realize real-time generation of dynamic Braille, but such devices still cannot break through the expression limitation of "dot matrix characters". In today's information environment, graphical content has become an important carrier of information transmission. For example, geometric figures, map outlines in educational scenarios, interface icons, traffic signs in daily scenarios, and even data charts in technical literature, all rely on continuous lines, area filling or three-dimensional contours for information transmission. However, the existing Braille readers can only simulate the local features of simple graphics through discrete raised dots, and cannot present the overall structure and detail association of the graphics, which leads to significant obstacles for the visually impaired in understanding spatial relationships and morphological features, greatly limiting their ability to acquire multi-element information.

[0004] Furthermore, all existing Braille readers are contact-based devices, with their core interaction relying on direct contact between the hand and the raised surface. From a public health perspective, this contact design makes the device surface susceptible to the accumulation of bacteria, viruses, and other pathogens, significantly increasing the risk of cross-infection, especially in shared environments such as libraries and schools. This problem is even more pronounced during periods of high incidence of infectious diseases. From a maintenance perspective, frequent contact with mechanical pin arrays or piezoelectric components can lead to wear and jamming. This not only requires regular cleaning and lubrication to ensure reliability but may also exacerbate component wear due to individual differences in usage habits (such as the pressure applied), further increasing maintenance costs. Simultaneously, contact-based interaction places high demands on user posture, requiring the hand to maintain a stable pressing position. Prolonged use can easily cause hand fatigue, affecting the comfort and sustainability of the reading experience. In conclusion, the limitations of current Braille reading systems in terms of the graphical representation capabilities of the tactile interface and the contact-based interaction design make it difficult to meet the actual needs of visually impaired individuals for safe, efficient, and diverse information access. Summary of the Invention

[0005] To address the shortcomings and deficiencies of existing technologies, this invention provides a Braille reading system based on phased array ultrasonic tactile feedback, aiming to solve problems such as hygiene and safety hazards, insufficient graphical content expression capabilities, and limited dynamic interactive performance of traditional contact Braille readers. The system includes a vision processing unit, a host computer, a tactile control drive unit, and an ultrasonic transducer array. The vision processing unit acquires the text and graphic information of the reading material and transmits it to the host computer. The host computer uses a target detection algorithm to distinguish between text and graphic regions, converting the text region into a static focal coordinate sequence of the corresponding Braille dot matrix, and discretizing the graphic region and extracting feature points to generate a dynamic focal trajectory point sequence of the corresponding graphic contour. After receiving the focal coordinate sequence, the tactile control drive unit runs a greedy search algorithm based on a linear synthesis scheme (LSS-Greedy), which performs equalization on the continuous phase space... By using interval discretization sampling and selecting the maximum phase of the target focal sound pressure for each transducer, the phase delay and modulation information of each ultrasonic transducer unit are calculated, thereby generating a multi-channel power-amplified drive signal. The ultrasonic transducer array responds to the drive signal by emitting ultrasonic waves, which are focused in space by a phased array to form a tactile focal point. Differentiated ultrasonic energy modulation technology is used for Braille and graphics: amplitude modulation (AM) is used for Braille dots to achieve stable tactile perception, and temporal modulation technology with path planning (PRO-STM) is used for graphic contours to achieve continuous contour perception, ultimately realizing non-contact tactile feedback for Braille and graphics.

[0006] The system's visual processing unit includes a camera module and communication transmission circuitry, which, along with a book-positioning fixture, ensures the stability of text and image information acquisition. The host computer processes text using OCR recognition technology, converting it into standard Braille encoding. For graphics, it extracts contour features such as vertices and inflection points to generate continuous trajectories. The tactile control drive unit consists of a main control computing subunit based on an STM32F103ZET6 development board and a signal drive subunit centered on an FPGA, capable of generating 256 synchronous PWM drive signals. The ultrasonic transducer array comprises 256 piezoelectric ceramic transducers, each 10mm in diameter and 40kHz in frequency, forming a sound pressure focus in the user's palm area that meets the tactile perception threshold. This invention, through a dual-mode tactile rendering strategy of "static encoding + dynamic trajectory," combined with efficient phase optimization and differentiated energy modulation technology, overcomes the limitations of traditional devices in expressing graphical content. The non-contact design reduces the risk of cross-infection and maintenance costs in public settings, making it widely applicable in public accessibility facilities, special education, and other fields, providing visually impaired individuals with a safe and rich way to access information.

[0007] The specific technical solution adopted by this invention to solve its technical problem is as follows:

[0008] A Braille reading system based on phased array ultrasonic tactile feedback includes:

[0009] The visual processing unit is configured to collect the graphic and textual information of the reading material and transmit the graphic and textual information to the host computer.

[0010] The host computer is communicatively connected to the vision processing unit and configured as follows:

[0011] The object detection algorithm distinguishes between text regions and graphic regions in graphic information.

[0012] The text region is processed to generate a static focus coordinate sequence of the corresponding Braille dot matrix;

[0013] The graphic region is discretized and feature points are extracted to generate a dynamic focus trajectory point sequence corresponding to the graphic contour;

[0014] The tactile control drive unit is communicatively connected to the host computer and is configured to receive the static focus coordinate sequence or the dynamic focus trajectory point sequence, run a phase optimization algorithm to calculate the phase delay and modulation information, and generate a multi-channel power-amplified drive signal based on the phase delay and modulation information.

[0015] An ultrasonic transducer array, electrically connected to the tactile control drive unit, is configured to emit ultrasonic waves in response to the drive signal and focus them in space through a phased array to form a tactile focal point, thereby achieving non-contact Braille and graphic tactile perception.

[0016] Furthermore, the visual processing unit includes a camera module and a communication transmission circuit. The camera module is used to acquire image information of the reading material, and the communication transmission circuit is used to transmit the image and text information to the host computer in real time.

[0017] The system also includes a book placement and fixing component, which is used to fix the reading material so that the pages of the reading material remain flat and in a uniform position.

[0018] Furthermore, the processing of the text region by the host computer includes: converting the text in the text region into text data through OCR recognition technology, mapping the text data into standard Braille coding, and finally generating a static focus coordinate sequence based on the dot matrix distribution of the standard Braille coding;

[0019] The discretization processing of the graphic region by the host computer includes: extracting the contour feature points and key structural points of the graphic region. The feature points include the vertices, inflection points and contour sampling points of the graphic. The dynamic focus trajectory point sequence generates a continuous coordinate path according to the connection order of the feature points.

[0020] Furthermore, the tactile control driving unit includes a main control calculation subunit and a signal driving subunit. The main control calculation subunit is based on the STM32F103ZET6 development board and is used to run a greedy search algorithm based on the linear synthesis scheme (LSS-Greedy). By discretizing the continuous phase space [0,2π) at equal intervals and selecting the maximum phase of the target focal sound pressure for each transducer, it calculates and outputs the phase delay and modulation information of each ultrasonic transducer unit. The signal driving subunit is based on an FPGA and is used to calculate the initial address of the multi-way lookup table according to the phase delay and modulation information, generate 256 synchronous PWM drive signals, and amplify the power of the PWM drive signals through a power amplifier circuit.

[0021] Furthermore, the ultrasonic transducer array consists of 256 piezoelectric ceramic ultrasonic transducers arranged in an array configuration;

[0022] The ultrasonic transducer array forms a tactile focal point through phased array focusing. Its sound pressure intensity meets the tactile perception threshold of the human hand's mechanoreceptors, and the focusing area of ​​the tactile focal point is limited to the user's palm area to ensure that the user can clearly perceive the Braille dot matrix and graphic outline through the palm.

[0023] Furthermore, the dynamic focus trajectory point sequence generated by the host computer is suitable for haptic rendering of geometric shapes, UI elements, and logo graphics;

[0024] The geometric shapes include circles, triangles, and rectangles; the UI elements include sliders and buttons; the identifiers include arrows and five-pointed stars; and the dynamic focus trajectory point sequence adapts to the contour complexity of different shapes by adjusting the focus movement speed and path density.

[0025] Furthermore, the system also includes a Braille interaction area, which is a dedicated area for the ultrasonic transducer array to form a tactile focus. The Braille interaction area is provided with a position marking structure to guide the user to place their dominant hand horizontally within the interaction area with the palm facing upward to receive tactile feedback.

[0026] Furthermore, the ultrasonic transducer array achieves tactile rendering through different ultrasonic energy modulation techniques:

[0027] For the Braille dot matrix corresponding to the static focal coordinate sequence, amplitude modulation (AM) technology is used to adjust the sound pressure amplitude of the ultrasonic wave to form a stable Braille tactile perception in the user's palm area.

[0028] For the graphic contour corresponding to the dynamic focus trajectory point sequence, a temporal control technique with path planning (PRO-STM) is adopted. By performing weighted clustering, path optimization and gradient descent optimization on the feature points after the graphic is discretized, a smooth focus motion path is generated. Then, by temporally controlling the sound pressure of ultrasonic waves and the focus movement speed, continuous tactile perception of the graphic contour is realized.

[0029] Furthermore, the implementation of the time-based control technology with path planning specifically includes:

[0030] Determine the reference speed for focus movement, path length constraints, and the number of simultaneous focal points;

[0031] The graph is discretized into sound pressure sample points to construct the target sound pressure distribution map and model the quasi-static pressure field.

[0032] Representative stress point sets are extracted by weighted k-means clustering, and the Traveling Salesman Problem (TSP) is solved using the 2-opt algorithm to generate closed paths. If the path length exceeds the constraint, the point set is split and the solution is repeated, retaining the multiple paths with the highest stress integral.

[0033] To improve resolution, the retained path is sampled. A comprehensive objective function is constructed based on pressure intensity, path length, self-intersection, and curvature. The path is optimized using a gradient descent algorithm to ensure that the path meets the conditions of non-self-intersection, length constraint, and smoothness.

[0034] The optimized path is converted into a three-dimensional path format to generate a focal position sequence adapted to the update frequency of the ultrasonic transducer array.

[0035] Furthermore, a Braille reading method based on phased array ultrasonic tactile feedback includes the following steps:

[0036] S1: Collect text and graphic information of reading materials: Collect text and graphic information of reading materials through the vision processing unit, and transmit the text and graphic information to the host computer;

[0037] S2: Differentiate and process graphic information: The host computer uses a target detection algorithm to distinguish between text regions and graphic regions in the graphic information; processes the text regions to generate a static focus coordinate sequence of the corresponding Braille dot matrix; discretizes the graphic regions and extracts feature points to generate a dynamic focus trajectory point sequence of the corresponding graphic contour.

[0038] S3: Calculate phase parameters and generate drive signals: The tactile control drive unit receives the static focus coordinate sequence or dynamic focus trajectory point sequence, runs a phase optimization algorithm to calculate phase delay and modulation information, and generates multi-channel power amplified drive signals based on the phase delay and modulation information;

[0039] S4: Forming non-contact tactile feedback: The ultrasonic transducer array responds to the driving signal and emits ultrasonic waves, which are focused in space by a phased array to form a tactile focus, so as to realize non-contact Braille and graphic tactile perception.

[0040] And a computer device including a processor and a memory, the memory storing a computer program, wherein when the processor executes the computer program, it implements the method described above.

[0041] A non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described above.

[0042] Compared with existing technologies, the present invention and its preferred embodiment achieve non-contact Braille and graphic tactile perception by adopting phased array ultrasonic tactile feedback technology, effectively avoiding the hygiene and safety hazards caused by direct contact with traditional contact devices, reducing the risk of cross-infection in public use scenarios, and reducing the need for surface cleaning and maintenance of equipment, thus improving ease of use.

[0043] Through the collaborative work of the visual processing unit and the host computer, the system can automatically distinguish between text and graphic areas in reading materials. It adopts a dual-mode processing strategy of static focal coordinate sequence and dynamic trajectory point sequence, and further optimizes the tactile perception effect by combining differentiated ultrasonic energy modulation technology: Amplitude modulation (AM) technology is used for Braille dot matrix to ensure the stability and uniformity of static tactile feedback, allowing visually impaired users to accurately perceive Braille details; For graphic outlines, a temporal control technology with path planning (PRO-STM) is used. Through weighted clustering, path optimization and gradient descent adjustment, a smooth and continuous focal motion trajectory is generated. This breaks through the limitation of existing devices that can only display discrete dot matrix characters or simple graphics, enabling visually impaired users to clearly understand the outline structure, corner features and even the spatial relationships of complex UI elements (such as sliders and arrows), significantly enriching the dimensions and accuracy of information acquisition.

[0044] The haptic control drive unit, in conjunction with the ultrasonic transducer array, achieves low complexity and high real-time performance in phase delay calculation through a greedy search algorithm based on a linear synthesis scheme (LSS-Greedy) and precise control of multiple drive signals. This algorithm is adapted to the computing power characteristics of STM32 embedded hardware, avoids the combinatorial explosion problem of traditional global optimization algorithms, and can quickly output the optimal phase parameters of each transducer. Combined with 256 synchronous PWM drive signals generated by the FPGA, it ensures rapid response and stable focusing of the haptic focus. This not only solves the information update delay problem caused by physical drive in traditional devices, but also improves the smoothness of dynamic interaction, maintaining the continuity of haptic perception even in fast graphics rendering scenarios.

[0045] Furthermore, the system's integrated visual recognition and real-time processing capabilities support the dynamic capture and conversion of unknown external graphic content, enhancing the initiative of visually impaired individuals in acquiring information independently. The hardware compatibility of the phase optimization algorithm and the scalability of the energy modulation technology further improve the system's adaptability to different scenarios, providing a more flexible and efficient tactile interaction solution for public accessibility facilities, special education, and other fields. In particular, in visually impaired education, it allows students to more intuitively "touch" mathematical geometric figures, geographical map outlines, and other content that is difficult to present with traditional equipment, thus facilitating the implementation of tactile teaching. Attached Figure Description

[0046] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0047] Figure 1 This is a schematic diagram of the system structure according to an embodiment of the present invention;

[0048] Figure 2 This is a flowchart illustrating the system's working logic in an embodiment of the present invention.

[0049] Figure 3 This is a schematic diagram of the static display of Braille and the motion trajectory of the discrete graphics in an embodiment of the present invention. Detailed Implementation

[0050] To make the features and advantages of the present invention more apparent and understandable, specific embodiments are described below in detail:

[0051] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0052] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0053] To address the hygiene risks associated with contact-based interaction and the inability to display graphical content in existing Braille reading devices, this invention proposes a non-contact tactile feedback system based on an ultrasonic array, combined with image acquisition and recognition technology, to achieve a safer, more efficient, and feature-rich Braille reading experience. This invention utilizes a phased-array controlled ultrasonic transmitter array to precisely focus the sound waves emitted by each array element onto a target point in space. By modulating the sound energy intensity at the focal point, it stimulates the mechanoreceptors in the human hand to generate tactile perception, thus rendering Braille and graphical information without physical contact. This system effectively overcomes the limitations of traditional devices in terms of cross-infection risk, graphical display capabilities, and dynamic interactive performance, providing a Braille reading solution that supports complex interfaces and non-contact operation, significantly expanding the reading information available to visually impaired individuals.

[0054] Based on phased array ultrasonic haptic technology, the reader provided by this invention consists of a visual recognition module, a host computer, a main control and computing module, a signal generator and driving module, and an ultrasonic array. It acquires text and image information through a camera, distinguishes them using a target detection algorithm, and generates spatial focal coordinates using both static encoding and dynamic trajectory strategies. The main control and computing module, along with the signal generator and driving module, drives the ultrasonic array to form a focal point in space, achieving haptic rendering in the user's palm. This reader combines hygiene and safety with dynamic interaction and graphic display capabilities, and can be widely used in public accessibility facilities, special education, and human-computer interaction fields, providing visually impaired individuals with a safer and richer way to access information.

[0055] The technical solution for text content recognition and conversion in this invention is as follows: First, a book fixing module fixes the book in a designated position, ensuring the page is flat and uniformly positioned, providing good physical conditions for the subsequent visual recognition module to clearly and accurately identify and collect content. Then, the visual recognition module acquires the book page content through a camera module and preprocesses the images. The acquired image data is transmitted to a host computer, which uses a target detection algorithm to distinguish and discriminate the image content, accurately differentiating continuous text areas from graphic areas. Subsequently, different recognition methods are used to recognize and convert the text and graphic image content. For text, it is converted into standard Braille encoding and further mapped to a sequence of focal coordinates in three-dimensional space; for corresponding graphic images, the content is discretized, feature points are extracted, and a sequence of feature trajectory points for dynamic rendering is generated. This method of recognition separation and the use of different strategies enable the reading system to acquire information in diverse ways. Through the above recognition separation strategy and differentiated processing paths, the system achieves the recognition and conversion of different text content.

[0056] This invention provides a tactile feedback generation technology solution for non-contact reading. Specifically, based on ultrasonic phased array technology, it generates perceptible tactile stimulation in the user's palm area through sound field modulation, thereby achieving non-contact reading. The specific process is as follows: The host computer determines the text content to be recognized, selects the appropriate rendering strategy (static rendering for Braille dots, dynamic trajectory rendering for graphics), and sends the target focus coordinate information to the main control calculation module. After receiving the specified coordinate information, the main control calculation module runs a phase optimization algorithm to obtain the corresponding phase delay and modulation information of each transducer unit, and then packages the information and sends it to the signal generator and drive module. This module uses an FPGA as its core, calculates different initial lookup addresses based on the phase parameters of each channel, achieves precise phase delay, and generates multi-channel synchronous PWM drive signals. The drive circuit amplifies the power of this signal, thereby exciting the ultrasonic transducer array to emit ultrasonic waves. The ultrasonic transducer array sends ultrasonic waves according to instructions, generating a specified sound field, forming a tactile focus with sufficient sound pressure, thereby generating corresponding tactile stimulation, allowing the user to clearly perceive the rendered Braille or graphic content through their palm.

[0057] The key features of this invention include:

[0058] 1) The proposed Braille reading system achieves contactless reading through the construction of an ultrasonic tactile system, which is safe and hygienic, greatly reducing the risk of cross-infection of bacteria and viruses from contact devices, and has enormous public health value in the current era. At the same time, it eliminates the need for frequent disinfection of the equipment, reducing maintenance costs.

[0059] 2) The proposed Braille reading system uses a visual module to not only acquire text content but also analyze image information, enabling users to read beyond just text content. They can also perceive simple UI elements such as charts, geometric shapes, buttons, and sliders, greatly enriching the information dimension and providing visually impaired people with more reading experience and imagination.

[0060] 3) The proposed Braille reading system is simple in structure and comprehensive in function. Through ultrasonic tactile stimulation, it provides blind users with a unique reading experience. Its non-contact nature makes it particularly suitable for use in public service settings, such as libraries, museums, and government service halls. It helps visually impaired individuals independently read notices, business forms, museum exhibit descriptions, and more. It can also be used in special education schools and online education platforms. Through this new type of Braille reader, students can not only read Braille textbooks but also "touch" geometric figures in mathematics and map outlines in geography for the first time. It provides an unprecedented tactile teaching tool for the education of visually impaired individuals.

[0061] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0062] like Figure 1 As shown. This embodiment provides a Braille reading system based on phased array ultrasonic tactile feedback, including: a host computer (Raspberry Pi) 1, a visual recognition module 2, a main control computing module 3, a signal generator and drive module 4, an ultrasonic transducer array 5, a Braille interaction area 6, a support and bus 7, and a book placement and fixing point 8.

[0063] The host computer 1 is responsible for visual processing and information recognition, as well as data conversion and process management. It also communicates with the main control computer. Specifically, host computer 1 processes the information collected by the visual recognition module, separating text and graphic information and employing different conversion and control strategies. For text, it converts it into a standard Braille sequence and uses a static control mode to generate the coordinates of the corresponding Braille points. For graphics, it discretizes the collected graphics and uses a dynamic control mode to generate the focus motion trajectory that needs continuous rendering. Finally, it packages the above information and sends it to the main control calculation module 3.

[0064] The visual recognition module 2 includes a camera module and a communication transmission circuit, which transmits the scanned text image information to the host computer 1.

[0065] The main control computing module 3 primarily serves as a communication hub, mainly used to parse the instructions sent by the host computer. Based on the processed information sent by the host computer, it runs the corresponding phase control algorithm to calculate the corresponding time delay and modulation information, and then sends the calculated drive data to the signal generator and drive module 4. This main control computing module uses a development board based on STM32F103ZET6, equipped with an ARM Cortex-M3 core, with a maximum operating frequency of 72MHz, enabling it to efficiently handle complex tasks.

[0066] The signal generator and drive module 4 mainly realizes power amplification and drive, and receives phase delay and modulation information from the main control calculation module 3 to generate 256 high-voltage drive signals with high precision to excite the ultrasonic transducer array 5 to work.

[0067] The ultrasonic transducer array 5 consists of 256 piezoelectric ceramic ultrasonic transducers arranged at a frequency of 40 kHz and a diameter of 10 mm. The ultrasonic transducer array emits ultrasonic waves according to a corresponding delay control signal, which are focused at the desired control point to form tactile feedback at the corresponding location, thus realizing non-contact reading.

[0068] Braille interaction area 6 is the functional area where users receive tactile feedback. When working, users need to place their dominant hand in this designated position to ensure that they can receive tactile stimulation stably and accurately during use, resulting in a better reading experience.

[0069] The book placement fixing point 8 is a book fixing device. When the user selects a specific book, he / she can open it to the desired reading page and then use the fixing device to stabilize it, thereby ensuring that the page is flat and in a uniform position, which makes it easier for the visual recognition module to clearly and correctly identify and collect the content.

[0070] like Figure 2 The diagram shows the logical flow of this invention for enabling non-contact reading for the blind:

[0071] First, select the book you want to read and fix it on the reader device. Then, place your dominant hand horizontally on the tactile interaction area (palm facing up).

[0072] Then, the system power is turned on to complete the initialization of each module. This includes: opening the host computer communication port, camera driver, activating the main controller, and enabling the signal generator and ultrasonic transducer array.

[0073] The user can then open the book to the desired text. The visual recognition module captures an image of the text and transmits it to the host computer. The host computer uses an object detection algorithm to determine the text type and selects different strategy commands accordingly. For text paths, a static control strategy is used, converting text characters into corresponding Braille codes and generating corresponding static focus coordinates. For graphic paths, a dynamic trajectory strategy is used, discretizing the graphic and extracting feature information to generate a corresponding dynamic focus control trajectory. The parsed information and strategy are then sent to the main control computing module. The object detection algorithm is a conventional existing technology, and those skilled in the art can implement it using existing mature models such as YOLO / SSD.

[0074] The main control calculation module is responsible for receiving information sent by the host computer, namely the corresponding control points of the phased array ultrasonic phased array, running the phase optimization algorithm to calculate the phase delay and modulation information of the tactile points to be generated, and sending it to the signal generator and drive module in accordance with the corresponding control strategy.

[0075] This embodiment considers real-time phase calculation on embedded hardware platforms such as STM32, requiring an optimization algorithm with high computational efficiency and low resource requirements. Traditional global optimization methods have high computational complexity and stringent requirements on processor computing power and memory resources, making it difficult to meet the real-time requirements of embedded controllers like STM32. Therefore, this embodiment selects a highly efficient phase optimization algorithm that combines robustness, scalability, and high practicality: the Greedy Search Algorithm for Linear Synthesis Scheme (LSS-Greedy). Based on the traditional Linear Synthesis Scheme (LSS), this algorithm models the sound field reconstruction problem as a combinatorial optimization problem. By discretizing the original continuous phase space [0, 2π) at equal intervals and employing a greedy strategy to optimize the phase shift of each focal point one by one, high-quality sound field synthesis is achieved with limited computational resources. The specific implementation steps are as follows:

[0076] 1. Initialize the output amplitude of all transducers to 0;

[0077] 2. Starting with the first transducer, set its amplitude to 1, and select the phase that maximizes the sound pressure at the target focus from the 32 discretized phase values;

[0078] 3. Record the optimal phase and output state of the transducer;

[0079] 4. Repeat the above process for subsequent transducers in sequence until the phase optimization of all transducers is completed.

[0080] The pseudocode for the algorithm is as follows:

[0081] Algorithm LSS-Greedy

[0082] 1. Input:

[0083] 2. Output:

[0084] 3. Discretize

[0085] 4. For

[0086] 5. For

[0087] 6. Obtain

[0088] Current sound pressure.

[0089] 7. End for

[0090] 8.

[0091] 9. End for

[0092] This algorithm is particularly suitable for STM32 embedded environments and has the following outstanding advantages:

[0093] 1. Low computational complexity: The algorithm complexity is only linearly related to the number of focal points and the discretization accuracy, effectively avoiding the combinatorial explosion problem caused by the increase in the number of transducers in traditional methods;

[0094] 2. Strong real-time performance: The greedy search mechanism has a simple structure and makes rapid judgments, making it very suitable for execution in the STM32 Cortex-M series core at a fixed cycle, meeting the stringent requirements of haptic rendering for real-time phase updates;

[0095] 3. Good hardware compatibility: The algorithm logic is easy to implement in the embedded C language environment, making it easy to port and deploy in the entire STM32 series of chips.

[0096] The signal generator will continuously process the information and generate a drive signal for each ultrasonic transducer channel to continuously excite the ultrasonic transducer array. The ultrasonic transducers emit ultrasonic waves according to the corresponding instructions, forming a specific sound field in the tactile interaction area. That is, a corresponding static Braille array or dynamic image rendering is generated in the palm of the blind person's hand. By focusing the ultrasonic energy density, the blind person's palm can generate corresponding tactile stimulation, thereby achieving the effect of non-contact reading.

[0097] As a preferred embodiment, the ultrasonic tactile modulation technology adopts segmented selection. When displaying static Braille, amplitude modulation (AM) technology is used, and when rendering graphics, a time-limited modulation technology with path planning (PRO-STM) is used.

[0098] The specific technical implementation process is as follows:

[0099] 1. Preparation process: Based on the principles and characteristics of STM, determine the reference speed for focus movement, path length constraints, and the number of simultaneous focus points, etc.

[0100] 2. Obtain a two-dimensional image, discretize it to form a two-dimensional point set, and treat each pixel of the image as a sound pressure sample point to construct a target sound pressure distribution map. Based on this sound pressure distribution map, model the quasi-static pressure field.

[0101] 3. Weighted k-means clustering is used to extract a representative pressure point set D from the sound pressure sample points in the 2D graph. The pressure points in D are used as the points to be visited in the Traveling Salesman Problem (Tsp). The 2-opt algorithm is used to solve the problem and a closed path to visit all points is constructed. If the length of the closed path is greater than the path length constraint of the STM, the pressure point set is carefully segmented, the covariance matrix of the point set is calculated, the eigenvector corresponding to the largest eigenvalue is found as the expansion direction to divide the subset, and the closed path is solved for each subset. The above process is repeated until all paths meet the length constraint. Finally, the three paths with the highest pressure integral are retained for further optimization.

[0102] 4. Sample the three paths obtained above, increase the number of path points to improve resolution, and construct a comprehensive objective function with cost terms such as pressure intensity, path length, self-intersection, path distance, and curvature. Then, use the gradient descent algorithm for optimization and apply the line search technique to ensure that each iteration along the gradient direction can effectively reduce the objective function value, thereby gradually obtaining an optimized path that satisfies the conditions of non-self-intersection, length constraint, and smoothness.

[0103] 5. The optimized path is converted into a three-dimensional path format suitable for ultrasound equipment. Based on the equipment update frequency and requirements, the corresponding focus position sequence is generated and output to achieve the effect of graphic rendering.

[0104] PRO-STM optimizes the focal path to generate tactile stimulation over a larger area of ​​skin and allows for more precise control over the shape and detail of the stimulation. It can utilize the movement and variation of the focal path to create more complex and intricate patterns or shapes on the skin, enabling users to perceive richer tactile information.

[0105] In this embodiment, the reproduction of text content involves converting the corresponding text characters into the corresponding Braille dot matrix pattern (e.g., ...). Figure 3 (As shown). The bolded dots in the diagram are the control points that the phased array ultrasonic array needs to focus on. By precisely controlling these focal points, corresponding tactile Braille characters can be formed in the user's palm area.

[0106] For graphics, the system first discretizes the graphic outline (e.g., Figure 3 As shown in the diagram, the appropriate control strategy is then selected to render the text according to the predetermined motion trajectory, so that the text achieves an accurate graphic tactile effect in the user's hand. Although the diagram only shows the rendering strategies for basic geometric shapes such as circles, triangles, rectangles and straight lines, this Braille reader can also render various rich graphic styles such as arrows, sliders, and five-pointed stars. This function significantly expands the expressiveness of the content being read and greatly enriches the user's non-contact reading experience.

[0107] The differences and advantages of the above solutions in the embodiments of the present invention compared with the prior art include:

[0108] 1. Most existing Braille-assisted sensing systems rely on pre-stored Braille symbols or 3D graphic data on a PC. They can only respond to user interaction commands based on preset content and reproduce tactile sensations, unable to process unknown or continuous external graphic information in real time. Furthermore, visually impaired users often find it difficult to operate the PC independently to set content and issue interaction commands, limiting the system's practicality and autonomy. This invention introduces a visual recognition module and target detection algorithm to automatically distinguish and process text and graphic regions in books or images. It possesses the ability to dynamically capture, recognize, and convert unknown, continuously input graphic content into tactile information in real time, thus significantly expanding the system's applicable scenarios and the initiative in information acquisition.

[0109] 2. Existing systems mostly focus on generating static, tactile virtual Braille dots or 3D graphics in space. Essentially, they construct tactile forms by rapidly switching multiple static focal points. While this can achieve basic graphic representation, it lacks support for continuous trajectories, limiting the expression of complex graphics and dynamic interactions. This invention innovatively adopts a dual-mode rendering strategy of "static encoding + dynamic trajectory": text content is converted into Braille dots and rendered using static focal points; for graphic content, feature extraction and discretization are used to generate continuously moving focal trajectory. This dynamic trajectory rendering method can more realistically and naturally reproduce the outline and structure of graphics, effectively improving the continuity and recognition accuracy of tactile perception, and expanding the capabilities of non-contact tactile feedback in expressing graphical information.

[0110] Based on the same inventive concept, this invention also provides a computer device, comprising: one or more processors, and a memory for storing one or more computer programs; the programs include program instructions, and the processor executes the program instructions stored in the memory. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, used to implement one or more instructions, specifically for loading and executing one or more instructions stored in a computer storage medium to implement the above-described method.

[0111] It should be further explained that, based on the same inventive concept, the present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, performs the above-described method. This storage medium can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0112] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0113] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

[0114] This invention is not limited to the preferred embodiment described above. Anyone inspired by this invention can derive various other forms of Braille reading systems based on phased array ultrasonic tactile feedback. All equivalent variations and modifications made within the scope of the claims of this invention shall fall within the scope of this invention.

Claims

1. A Braille reading system based on phased array ultrasonic tactile feedback, characterized in that, include: The visual processing unit is configured to collect the graphic and textual information of the reading material and transmit the graphic and textual information to the host computer. The host computer is communicatively connected to the vision processing unit and configured as follows: The object detection algorithm distinguishes between text regions and graphic regions in graphic information. The text region is processed to generate a static focus coordinate sequence of the corresponding Braille dot matrix; The graphic region is discretized and feature points are extracted to generate a dynamic focus trajectory point sequence corresponding to the graphic contour. The tactile control drive unit is communicatively connected to the host computer and is configured to receive the static focus coordinate sequence or the dynamic focus trajectory point sequence, run a phase optimization algorithm to calculate the phase delay and modulation information, and generate a multi-channel power-amplified drive signal based on the phase delay and modulation information. An ultrasonic transducer array, electrically connected to the tactile control drive unit, is configured to emit ultrasonic waves in response to the drive signal and focus them in space through a phased array to form a tactile focal point, thereby achieving non-contact Braille and graphic tactile perception.

2. The Braille reading system based on phased array ultrasonic tactile feedback according to claim 1, characterized in that: The visual processing unit includes a camera module and a communication transmission circuit. The camera module is used to acquire image information of the reading material, and the communication transmission circuit is used to transmit the image and text information to the host computer in real time. The system also includes a book placement and fixing component, which is used to fix the reading material so that the pages of the reading material remain flat and in a uniform position.

3. The Braille reading system based on phased array ultrasonic tactile feedback according to claim 1, characterized in that: The processing of the text region by the host computer includes: converting the text in the text region into text data through OCR recognition technology, mapping the text data into standard Braille code, and finally generating a static focus coordinate sequence based on the dot matrix distribution of the standard Braille code; The discretization processing of the graphic region by the host computer includes: extracting the contour feature points and key structural points of the graphic region. The feature points include the vertices, inflection points and contour sampling points of the graphic. The dynamic focus trajectory point sequence generates a continuous coordinate path according to the connection order of the feature points.

4. The Braille reading system based on phased array ultrasonic tactile feedback according to claim 1, characterized in that: The tactile control drive unit includes a main control calculation subunit and a signal drive subunit. The main control calculation subunit is based on an STM32F103ZET6 development board and is used to run a greedy search algorithm based on a linear synthesis scheme. It calculates and outputs the phase delay and modulation information of each ultrasonic transducer unit by sampling the continuous phase space [0,2π) at equal intervals and selecting the maximum phase of the target focal sound pressure for each transducer. The signal drive subunit is based on an FPGA and is used to calculate the initial address of the multi-way lookup table according to the phase delay and modulation information, generate 256 synchronous PWM drive signals, and amplify the power of the PWM drive signals through a power amplifier circuit.

5. A Braille reading system based on phased array ultrasonic tactile feedback according to claim 1, characterized in that: The ultrasonic transducer array consists of 256 piezoelectric ceramic ultrasonic transducers arranged in an array. The ultrasonic transducer array forms a tactile focal point through phased array focusing. Its sound pressure intensity meets the tactile perception threshold of the human hand's mechanoreceptors, and the focusing area of ​​the tactile focal point is limited to the user's palm area to ensure that the user can clearly perceive the Braille dot matrix and graphic outline through the palm.

6. A Braille reading system based on phased array ultrasonic tactile feedback according to claim 1, characterized in that: The dynamic focus trajectory point sequence generated by the host computer is suitable for tactile rendering of geometric shapes, UI elements, and logo graphics; The geometric shapes include circles, triangles, and rectangles; the UI elements include sliders and buttons; the identifiers include arrows and five-pointed stars; and the dynamic focus trajectory point sequence adapts to the contour complexity of different shapes by adjusting the focus movement speed and path density.

7. A Braille reading system based on phased array ultrasonic tactile feedback according to claim 1, characterized in that: The system also includes a Braille interaction area, which is a dedicated area for the ultrasonic transducer array to form a tactile focus. The Braille interaction area is equipped with a position marking structure to guide the user to place their dominant hand horizontally within the interaction area with the palm facing upward to receive tactile feedback.

8. A Braille reading system based on phased array ultrasonic tactile feedback according to claim 1, characterized in that: The ultrasonic transducer array achieves tactile rendering through different ultrasonic energy modulation techniques: For the Braille dot matrix corresponding to the static focal coordinate sequence, amplitude modulation technology is used to adjust the sound pressure amplitude of the ultrasonic wave to form a stable Braille tactile perception in the user's palm area. For the graphic contour corresponding to the dynamic focus trajectory point sequence, a time-controlled technique with path planning is adopted. By performing weighted clustering, path optimization and gradient descent optimization on the feature points after the graphic is discretized, a smooth focus motion path is generated. Then, by time-controlled ultrasonic sound pressure and focus movement speed, continuous tactile perception of the graphic contour is realized.

9. A Braille reading system based on phased array ultrasonic tactile feedback according to claim 8, characterized in that: The implementation of the time-based control technology with path planning specifically includes: Determine the reference speed for focus movement, path length constraints, and the number of simultaneous focal points; The graph is discretized into sound pressure sample points to construct the target sound pressure distribution map and model the quasi-static pressure field. Representative stress point sets are extracted by weighted k-means clustering. The 2-opt algorithm is used to solve the traveling salesman problem to generate closed paths. If the path length exceeds the constraint, the point set is split and the solution is repeated. The paths with the highest stress integral are retained. To improve resolution, the retained path is sampled. A comprehensive objective function is constructed based on pressure intensity, path length, self-intersection, and curvature. The path is optimized using a gradient descent algorithm to ensure that the path meets the conditions of non-self-intersection, length constraint, and smoothness. The optimized path is converted into a three-dimensional path format to generate a focal position sequence adapted to the update frequency of the ultrasonic transducer array.

10. A Braille reading method based on phased array ultrasonic tactile feedback, characterized in that, Includes the following steps: S1: Collect text and graphic information of reading materials: Collect text and graphic information of reading materials through the vision processing unit, and transmit the text and graphic information to the host computer; S2: Differentiate and process text and image information: The host computer uses a target detection algorithm to distinguish between text areas and graphic areas in the text and image information; The text region is processed to generate a static focus coordinate sequence of the corresponding Braille dot matrix; the graphic region is discretized and feature points are extracted to generate a dynamic focus trajectory point sequence of the corresponding graphic contour. S3: Calculate phase parameters and generate drive signals: The tactile control drive unit receives the static focus coordinate sequence or dynamic focus trajectory point sequence, runs a phase optimization algorithm to calculate phase delay and modulation information, and generates multi-channel power amplified drive signals based on the phase delay and modulation information; S4: Forming non-contact tactile feedback: The ultrasonic transducer array responds to the driving signal and emits ultrasonic waves, which are focused in space by a phased array to form a tactile focus, so as to realize non-contact Braille and graphic tactile perception.

Images