A high-precision real-time blind-Chinese translation method and system

Abstract

A high-precision real-time Braille-Chinese translation method includes the following steps: A camera equipped with a specific lateral parallel light source is used to acquire Braille images, and the Braille images are preprocessed to obtain a first image; the first image is input into a model to obtain the bounding box information of all detected Braille bumps, and the Braille coordinates and dimensions are extracted; using the coordinates of all detected blind spots, least-squares linear regression is used to fit the overall trend of the Braille lines, the tilt angle is calculated, a rotation matrix is ​​generated using the tilt angle, and an affine transformation is performed on all blind spot coordinates to obtain a corrected second image; the second image is structurally segmented to obtain a set of Braille dots; based on the conversion rules between Braille and Chinese characters, the content of the Braille dot set is converted into Chinese text. This invention constructs a complete algorithm chain from raw blind spot detection to structured analysis and finally to Chinese text translation. It achieves high-precision and highly robust automatic Braille recognition and translation.

Description

Technical Field

[0001] This invention relates to the field of Braille translation technology, and in particular to a high-precision real-time Braille-Chinese translation method and system. Background Technology

[0002] We live in an era of information overload, where information has become an indispensable core element driving social development and promoting personal growth. From the convenience of daily life to education, employment, and cultural entertainment, the ability to acquire and utilize information profoundly impacts an individual's quality of life and social participation. For the blind and visually impaired, a large and unique segment of society, the lack or limitation of visual function creates obstacles to their equal and efficient access to and use of information. Traditional information presentation methods that primarily rely on visual means, such as printed books, graphical interfaces, and complex physical signage, often exclude them from mainstream information exchange channels, creating a significant "information gap."

[0003] Currently, my country's Braille book resources are relatively scarce, with outdated content and insufficient variety and quantity to meet the needs of the vast visually impaired population. Statistics show that on average, over 60,000 blind people own only one type of Braille book, and about 150 blind people own one Braille publication. Meanwhile, although some progress has been made in building accessible urban environments, problems such as blocked tactile paving and a lack of clear and effective Braille signage in public places and everyday goods remain widespread, causing significant inconvenience to independent travel, information access, and daily life for visually impaired individuals. Furthermore, some visually impaired individuals find the process of learning and using Braille too lengthy and cumbersome, leading more and more to turn to technical devices and audio recordings as alternatives, resulting in a decline in Braille literacy rates. However, assistive technologies and audio recordings are not suitable for all situations encountered by visually impaired individuals in daily life, and they may require assistance from others to decipher Braille. In addition, those who collaborate with blind individuals but are unfamiliar with Braille may also require two-way Braille interaction. Enabling visually impaired individuals to conveniently, independently, and efficiently read and understand physical Braille information (such as books, medicine instructions, and public signs) remains a key challenge. Traditional Braille reading relies on touch, which has limited efficiency. Most existing Braille digitization solutions depend on high-resolution, high-cost scanners. While these devices can provide clear and regular images, their high price, large size, and complex operation often require assistance, limiting the ability of visually impaired users to independently identify and read physical Braille information around them anytime, anywhere in their daily lives. Summary of the Invention

[0004] To address the aforementioned shortcomings, the present invention aims to propose a high-precision real-time Braille-Chinese translation method and system, thereby solving the problems of low accuracy and robustness in existing Braille translation methods.

[0005] To achieve this objective, the present invention adopts the following technical solution: A high-precision real-time blind-Chinese translation method includes the following steps: Step S1: Use a camera equipped with a horizontal lateral parallel light source to capture Braille images, and preprocess the Braille images to obtain the first image; Step S2: Input the first image into the pre-trained YOLOv8 model, obtain the bounding box information of all detected Braille bumps, and extract the Braille coordinates and dimensions; Step S3: Using all detected blind spot coordinates, the overall tilt trend of the Braille lines is fitted by least squares linear regression, the tilt angle is calculated, a rotation matrix is ​​generated through the tilt angle, and an affine transformation is performed on all blind spot coordinates to obtain the corrected second image. Step S4: Perform structural segmentation on the second image to obtain the set of blind symbol points; Step S5: Based on the conversion rules between Braille and Chinese characters, convert the content of the Braille dot set into Chinese content.

[0006] Preferably, the horizontal lateral parallel light in step S1 includes single-sided parallel light input and multiple double-sided lateral parallel light input; The conditions for single-sided parallel light input are: a light source is projected onto the Braille document from one side, and the angle requirements for the projection light source are: the light source is projected at an angle parallel to the reference line with the Braille document as the horizontal reference line, or the angle between the light source and the reference line is less than the angle threshold. The conditions for parallel light input from both sides are as follows: a light source is projected onto the Braille document from both sides, and the angle requirements for the projected light source are: with the Braille document as the horizontal baseline, the light source is projected at an angle parallel to the baseline, or the angle between the light source and the baseline is less than the angle threshold.

[0007] Preferably, the preprocessing in step S1 includes: The Braille image is processed into a computer-readable format using the standard image processing library OpenCV. This requires ensuring that the image data type and color channels conform to the processing requirements of the YOLOv8 model, converting it to grayscale, and resizing it while maintaining the BGR / RGB format. Cropping the background that is unrelated to the Braille document; First, apply Gaussian filtering and median filtering, then execute the nonlocal mean denoising algorithm.

[0008] Preferably, step S3 is as follows: Step S31: Treat all detected blind spot coordinates (x, y) as a point cloud dataset, perform PCA processing on the dataset, and obtain the maximum feature vector in the PCA processing; The angle between the largest eigenvalue vector and the horizontal axis of the coordinate system is used as the first angle. The image is rotated using the first angle to obtain a preliminary image. Step S32: Extract the Y coordinates of blind spots in the preliminary image, and use the "ward" method to construct a one-dimensional feature for clustering, thereby obtaining multiple clusters; Step S33: Perform linear regression on each cluster using the least squares method to fit an optimal straight line model and obtain the slope of the straight line model; Step S34: Obtain the average slope, construct a two-dimensional rotation matrix using the average slope, and perform an affine transformation on the initial image using the two-dimensional rotation matrix to obtain the corrected second image.

[0009] Preferably, the specific steps of step S4 are as follows: Step S41: Obtain the vertical distance between adjacent points or clusters of points on the Y coordinate in the second image as the first distance. Determine whether the first distance exceeds the adaptive line spacing parameter. If it does not exceed the parameter, determine that the point or cluster of points belongs to the same text line. Step S42: Based on the Y coordinate within the text line and the vertical point spacing parameter within the adaptive blind character, obtain the upper and lower baselines of the text line respectively; Step S43: For text lines within the baseline, obtain the distance between adjacent points in the X coordinate as the second distance, and determine whether the second distance exceeds the adaptive word spacing parameter or the derived word spacing threshold. If it does not exceed the parameter, then define adjacent points as the same word group. Step S44: For the phrase, obtain the distance between adjacent points in the X coordinate as the third distance, and determine whether the third distance exceeds the adaptive blind character spacing parameter or the blind character spacing threshold derived from this value. If it does not exceed the parameter, it is defined as a single blind character. Step S45: For a single blind symbol, divide the left column candidate points and the right column candidate points by using the adaptive horizontal spacing parameter within the blind symbol, and determine the position of the candidate points in the Y-axis direction within the baseline.

[0010] Preferably, the following steps need to be performed before performing step S4: Step A: Identify all detected blind spots in the second image, obtain the horizontal Euclidean distance between all adjacent blind spots, and construct a dataset of observed actual pixel distances; Step B: Construct an objective function to quantify the total deviation of all observed actual pixel distances from their nearest theoretical target pixel distance, as shown in the following expression: Where E(x) is the objective function value, N is the total number of observed distances, and M is the theoretical number of target distances, which is 8. The distance to the i-th observed pixel. The mapping value of the j-th theoretical target distance under the current geometric parameter x, where min is the value for each observed distance. Find the closest theoretical distance ; Step C: Using a univariate optimization algorithm, search within a preset range of reasonable x values ​​to find the x_optimal that minimizes the objective function; Step D: By calculating the relative proportions of x_optimal with the various spacing parameters of Braille in the national standard, the adaptive line spacing parameter, the adaptive vertical dot spacing parameter within the Braille character, the adaptive word spacing parameter, the word gap threshold derived from this value, the adaptive Braille character spacing parameter, the Braille character gap threshold derived from this value, and the adaptive horizontal spacing parameter within the Braille character are obtained.

[0011] Preferably, step S5 is as follows: Step S51: Convert the dot combination of each identified blind symbol into its corresponding basic linguistic or semiotic unit according to the conversion rules of Chinese characters, and use it as the basic unit; Step S52: According to the spelling rules of Chinese Braille, combine the initials, finals and tone marks of the basic units into complete Chinese Pinyin syllables with tones; Step S53: After removing the tone information from the Chinese Pinyin syllables, input them into the Pinyin to Chinese character engine to generate one or more possible candidate Chinese character sequences for the input Chinese Pinyin syllables; Step S54: Input the candidate Chinese character sequence into the Chinese character to Pinyin library to obtain its standardized Pinyin representation with numerical tones, as the candidate Chinese Pinyin sequence; Step S53: Compare the Chinese Pinyin syllables with the candidate Chinese Pinyin sequences, and obtain the Chinese characters whose candidate Chinese Pinyin sequences match the Chinese Pinyin syllables as the translation results.

[0012] A high-precision real-time blind-Chinese translation system, using the aforementioned high-precision real-time blind-Chinese translation method, includes an image acquisition module, an extraction module, a correction module, a segmentation module, and a conversion module; The image acquisition module is used to acquire Braille images using a camera equipped with a horizontal lateral parallel light source, and to preprocess the Braille images to obtain the first image; The extraction module is used to input the first image into the pre-trained YOLOv8 model, obtain the bounding box information of all detected Braille bumps, and extract the Braille coordinates and dimensions; The correction module is used to fit the overall tilt trend of the Braille line using least squares linear regression with all detected blind spot coordinates, calculate the tilt angle, generate a rotation matrix through the tilt angle, and perform affine transformation on all blind spot coordinates to obtain the corrected second image. The segmentation module is used to perform structural segmentation on the second image and obtain the set of blind symbol points; The conversion module is used to convert the content of the Braille dot set into Chinese content based on the conversion rules between Braille and Chinese characters.

[0013] One of the above technical solutions has the following advantages or beneficial effects: This invention constructs a complete algorithm chain from raw blind spot detection in an image to structured analysis and finally, Chinese text translation. By combining advanced deep learning object detection technology, innovative adaptive spacing parameter optimization and hierarchical analysis strategies, and a refined translation mechanism assisted by tone information, this invention lays a solid core algorithmic foundation for realizing a high-precision and highly robust automatic Braille recognition and translation system. Attached Figure Description

[0014] Figure 1 This is a flowchart of one embodiment of the method of the present invention.

[0015] Figure 2 This is a schematic diagram of the structure of an industrial camera and a side-lighting device according to an embodiment of the present invention.

[0016] Figure 3 This is a comparison diagram of the effects of having a side light source and not having a side light source in one embodiment of the present invention.

[0017] Figure 4 This is a schematic diagram of a bilateral side light source according to an embodiment of the present invention.

[0018] Figure 5 This is a schematic diagram illustrating the segmentation effect of one embodiment of the present invention.

[0019] Figure 6 This is a schematic diagram of Braille translation into pinyin according to an embodiment of the present invention.

[0020] Figure 7 This is an illustration of the effect of Braille translation into Chinese according to an embodiment of the present invention.

[0021] Figure 8 This is a schematic diagram of the structure of one embodiment of the system of the present invention. Detailed Implementation

[0022] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0023] In the description of embodiments of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of embodiments of the present invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0024] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0025] like Figures 1-7 As shown, a high-precision real-time blind-Chinese translation method includes the following steps: Step S1: Use a camera equipped with a horizontal lateral parallel light source to capture Braille images, and preprocess the Braille images to obtain the first image; Step S2: Input the first image into the pre-trained YOLOv8 model, obtain the bounding box information of all detected Braille bumps, and extract the Braille coordinates and dimensions; Step S3: Using all detected blind spot coordinates, the overall tilt trend of the Braille lines is fitted by least squares linear regression, the tilt angle is calculated, a rotation matrix is ​​generated through the tilt angle, and an affine transformation is performed on all blind spot coordinates to obtain the corrected second image. Step S4: Perform structural segmentation on the second image to obtain the set of blind symbol points; Step S5: Based on the conversion rules between Braille and Chinese characters, convert the content of the Braille dot set into Chinese content.

[0026] This invention first addresses the accurate detection of blind spots—the basic unit of Braille—by introducing an advanced YOLOv8 object detection model. By analyzing the challenges of low contrast, blurriness, uneven lighting, geometric deformation, and blind spot morphological differences in Braille images in natural scenes, the necessity of deep learning methods is demonstrated. The selection criteria for the YOLOv8 model and its detection principles (including the backbone network, neck network, detection head, and post-processing mechanisms) are detailed. Emphasis is placed on the construction of a self-made dataset for this task, the model training process (including loading pre-trained weights, parameter configuration, mosaic enhancement strategies, etc.), and the extraction of blind spot coordinates. Experimental results show that YOLOv8 can effectively and robustly detect blind spots from complex backgrounds.

[0027] Secondly, after obtaining the set of blind spot coordinates, this chapter proposes an adaptive multi-level structured analysis method to address the structural analysis challenge caused by image geometric distortion. The core of this method lies in first autonomously learning feature spacing parameters (such as the optimal scale factor corresponding to the relative proportions of dot spacing, character spacing, phrase spacing, and line spacing) from the detected point coordinates in the current image through an optimization model. Subsequently, using these adaptive spacing parameters, a hierarchical process (page preprocessing and rotation correction, line segmentation, inline baseline determination, phrase segmentation, blind character segmentation, and analysis of internal points within the blind character) precisely organizes the disordered point cloud into a logically meaningful sequence of Braille units. This method significantly improves the robustness and accuracy of analysis under non-ideal conditions.

[0028] Building upon structured Braille information, this chapter then details the complete Braille translation process. This process comprises three key steps: 1) The parsed blind symbol dot matrix is ​​accurately mapped to basic linguistic units such as pinyin initials, finals, tones, and punctuation marks by looking up a table; 2) Based on the spelling rules of Chinese Braille, these basic unit sequences are synthesized into a complete stream of Chinese Pinyin syllables with tones; 3) A mature Pinyin-to-Chinese character engine is used to generate candidate Chinese character sequences. These sequences are then rigorously matched and verified with the tone information preserved in the original Braille text. This effectively resolves ambiguity of homophones (words) and ultimately outputs accurate Chinese text, while also taking into account the handling of special symbols and formats.

[0029] In summary, this invention constructs a complete algorithm chain from raw blind spot detection in images to structured parsing and finally, Chinese text translation. By combining advanced deep learning object detection technology, innovative adaptive spacing parameter optimization and hierarchical parsing strategies, and a refined translation mechanism assisted by tone information, this invention lays a solid core algorithmic foundation for achieving a high-precision and highly robust automatic Braille recognition and translation system.

[0030] Preferably, the horizontal lateral parallel light in step S1 includes single-sided parallel light input and multiple double-sided lateral parallel light input; The conditions for single-sided parallel light input are: a light source is projected onto the Braille document from one side, and the angle requirements for the projection light source are: the light source is projected at an angle parallel to the reference line with the Braille document as the horizontal reference line, or the angle between the light source and the reference line is less than the angle threshold. The conditions for parallel light input from both sides are as follows: a light source is projected onto the Braille document from both sides, and the angle requirements for the projected light source are: with the Braille document as the horizontal baseline, the light source is projected at an angle parallel to the baseline, or the angle between the light source and the baseline is less than the angle threshold.

[0031] Braille text is visually represented by raised dots on a background, making it extremely difficult to recognize, especially double-sided Braille. The text printed on the back of the paper appears as recessed dots, which are almost invisible to the touch and do not interfere with the sensation of the raised dots on the front, but are visually difficult to distinguish from the front. The height difference of these dots is its core physical feature, but this height difference is often not significant in two-dimensional images, especially under diffused lighting conditions. The Braille dots and their background (usually homogeneous paper) are completely identical in material and color, lacking natural grayscale or color differences.

[0032] To address this, this invention introduces an active physical feature enhancement strategy at the image acquisition front end—an illumination method based on specific lateral light sources. This does not rely on uncontrollable ambient light, but rather optimizes the visual representation of blind spots from a physical perspective through a carefully designed, cost-effective lighting system, providing higher-quality input information for subsequent digital recognition algorithms.

[0033] The core of the unilateral parallel light strategy lies in illuminating the Braille surface with light at a low grazing angle. When light is incident at an angle close to parallel to the paper (angle threshold less than 15°), the tiny protrusions (Braille) act like hills in terrain, significantly blocking and reflecting the incident light. Shadow generation: The protruding portion of the Braille block sidelight, casting a clear, elongated shadow on the side facing away from the light source. This shadowed area receives far less light than the directly illuminated background, creating a low-grayscale (dark) area in the image. Highlight formation: The sloping portion of the Braille facing the light source may exhibit higher brightness than the surrounding flat background due to a near-specular reflection angle with the incident light or because its curved surface converges the light, creating a highlight area. This "shadow-highlight" effect generated by a single sidelight significantly alters the local grayscale distribution of the Braille and its surrounding area. What was originally a Braille with extremely low contrast now forms a clear grayscale gradient through the dark shadow on one side and the bright highlight on the other. Essentially, side lighting cleverly transforms the physical height difference (the protrusion of the blind spot) into a visually perceptible difference in grayscale or brightness. This enhancement is far superior to frontal lighting (which tends to provide uniform illumination, reducing detail) or completely diffused lighting (which lacks directionality and makes it difficult to create clear shadows), greatly improving the discernibility and detectability of blind spots. Figure 2 As shown.

[0034] While a single-sided light source can effectively enhance contrast, its effect is limited by the specific orientation and intensity of the light source, and may produce asymmetrical enhancement effects on different edges of blind spots. To obtain a more stable and robust feature enhancement effect, this invention can also use a bilateral side-lit parallel light strategy, such as... Figure 3 As shown, the system consists of two symmetrically or angled light sources, illuminating the Braille area from the left and right sides at lower angles, respectively. This design offers a deeper advantage: dual feature generation and increased redundancy. Each blind spot is now simultaneously illuminated from two directions. This means that a shadow (and highlight) is formed on the left side of the blind spot by the right light source, while a shadow (and highlight) is formed on the right side by the left light source. Thus, two sets of approximately symmetrical "shadow-highlight" features are formed around each blind spot. This feature redundancy significantly increases the probability of blind spots being detected. Even if the feature produced by one light source is not clear enough for some reason (such as localized paper reflectivity or minor obstructions), the feature produced by the other light source can still provide useful information.

[0035] Preferably, the preprocessing in step S1 includes: The Braille image is processed into a computer-readable format using the standard image processing library OpenCV. This requires ensuring that the image data type and color channels conform to the processing requirements of the YOLOv8 model, converting it to grayscale, and resizing it while maintaining the BGR / RGB format. It can crop backgrounds unrelated to Braille documents, requiring less computation and potentially avoiding interference from distant backgrounds in subsequent analysis.

[0036] Considering the noise present in the camera image (such as sensor noise and compression artifacts), Gaussian filtering and median filtering are applied first, and then a nonlocal mean denoising algorithm is executed.

[0037] Preferably, step S3 is as follows: Step S31: Treat all detected blind spot coordinates (x, y) as a point cloud dataset, perform PCA processing on the dataset, and obtain the maximum feature vector in the PCA processing; The angle between the largest eigenvalue vector and the horizontal axis of the coordinate system is used as the first angle. The image is rotated using the first angle to obtain a preliminary image. Step S32: Extract the Y coordinates of blind spots in the preliminary image, and use the "ward" method to construct a one-dimensional feature for clustering, thereby obtaining multiple clusters; Step S33: Perform linear regression on each cluster using the least squares method to fit an optimal straight line model and obtain the slope of the straight line model; Step S34: Obtain the average slope, construct a two-dimensional rotation matrix using the average slope, and perform an affine transformation on the initial image using the two-dimensional rotation matrix to obtain the corrected second image.

[0038] After obtaining the coordinates of all potential blind spots in the image using a preliminary target detection model (such as YOLO), a preliminary, coarse global tilt angle estimation can be performed. The basic assumption at this stage is that even with local deformations, the entire page or the main Braille block can be macroscopically approximated as a rectangular region with a specific orientation, and the detected blind spots are sampling points within this region arranged in a specific pattern (the row and column structure of the Braille). When performing the coarse tilt angle estimation, the coordinates of the detected blind spots are first processed using PCA: 1. Calculate the mean center of the blind spot coordinates.

[0039] 2. Calculate the covariance matrix of the blind spot coordinates. This matrix describes the correlation between the X and Y coordinates and their respective variances.

[0040] 3. Perform eigenvalue decomposition on the covariance matrix to obtain eigenvalues ​​and corresponding eigenvectors.

[0041] The eigenvector represents the main directional axis of the data point distribution, while the corresponding eigenvalues ​​represent the variance of the data in these directions. Therefore, the largest eigenvalue vector indicates the direction of maximum variance in the data point distribution. In the context of Braille dot matrix, since Braille lines are typically shorter than their height and the dots within each line are densely packed, the expected distribution range (variance) of data points along the direction perpendicular to the Braille lines is the largest. Therefore, it is reasonable to assume that the direction of the largest eigenvalue vector is approximately perpendicular to the overall direction of the Braille lines. Then, using the angle between the largest eigenvalue vector and the horizontal axis (X-axis) of the coordinate system as the first angle, image rotation is performed to obtain a preliminary image.

[0042] After obtaining the initial image, in order to obtain a more accurate correction angle, this invention adopts a refinement strategy based on line-by-line analysis and linear fitting: First, the Y-coordinate values ​​of all blind points in the dataset are extracted. These Y-coordinates constitute a one-dimensional feature for clustering. Second, an agglomerative hierarchical clustering algorithm is applied to the extracted Y-coordinate set. In this algorithm, we choose "ward" as the linking method. The "ward" method is mainly suitable for minimizing the increment of the sum of squares within a cluster when merging clusters, and it usually produces relatively compact clusters of similar size. The key parameter for clustering is distance_threshold, which is set to 250 in this implementation based on experimental testing. During the clustering process, clusters are only merged when the ward distance between two clusters is less than this threshold. When the distance between all potential merged cluster pairs exceeds 250, the clustering process terminates, and the algorithm automatically determines the final number of clusters based on this threshold. Blind points that are close in location along the Y-axis (i.e., points with similar Y-coordinate values) will be automatically assigned to the same cluster. Each resulting cluster represents a candidate Braille line segment, effectively organizing vertically dispersed point groups into a set of row-structured clusters. Each cluster represents a single Braille line segment. Then, a linear regression is performed on the point set of each line segment using the least squares method to fit an optimal linear model y = mx + c, where c is a constant. , Represents the x and y coordinates (pixel coordinates) of the i-th point. The average of the horizontal and vertical coordinates of all points in the current line of Braille is used. However, because points in a single line may have local deformations (such as slight curvature) or contain noise points, the fitted slope of a single line may not perfectly reflect the true tilt of the entire text. To obtain a more robust and accurate overall tilt angle, the average of the slopes fitted to all valid line segments is calculated. This averaging process effectively suppresses the influence of local noise and outliers, resulting in a fine estimate of the tilt angle in the dominant direction of the entire Braille area. This average slope is the final angle used for correction.

[0043] The construction of the two-dimensional rotation matrix and the PCA processing are existing technologies, and will not be explained in detail here.

[0044] Preferably, the specific steps of step S4 are as follows: Step S41: Obtain the vertical distance between adjacent points or clusters of points in the Y coordinate of the second image as the first distance. Determine whether the first distance exceeds the adaptive line spacing parameter. If it does not exceed the parameter, determine that the point or cluster of points is the same text line. When the vertical distance between adjacent (in the Y direction) points or clusters of points exceeds the threshold defined by the adaptive line spacing parameter (e.g., 10 × x_optimal or its derived threshold), it is considered that there is a line boundary.

[0045] Step S42: Based on the Y coordinate within the text line and the vertical point spacing parameter within the adaptive blind character, obtain the upper and lower baselines of the text line respectively; based on the vertical point spacing parameter within the adaptive blind character (its scale is usually equivalent to 2.5 × x_optimal or adjusted proportionally), further segment the points within the line in the Y direction to identify the logical horizontal baselines corresponding to the upper, middle, and lower rows of points of the blind character. Since the baseline of the middle layer intersects with the blind character, the dividing line cannot be distinguished, so only the upper and lower baselines are needed in the end.

[0046] Step S43: For text lines within the baseline, obtain the distance between adjacent points in the X coordinate as the second distance. Determine whether the second distance exceeds the adaptive phrase spacing parameter or the phrase gap threshold derived from this value. If it does not exceed, define adjacent points as the same phrase. Analyze the distance between adjacent points (or the centroid of potential blind symbol units) in the horizontal direction. When this distance exceeds the threshold defined by the adaptive phrase spacing parameter (e.g., 12 × x_optimal or the derived phrase gap threshold, such as (8.5 × x_optimal + 9.5 × x_optimal) / 2 = 9 × x_optimal), it is identified as a phrase boundary.

[0047] Step S44: For the phrase, obtain the distance between adjacent points in the X coordinate as the third distance. Determine whether the third distance exceeds the adaptive blind symbol spacing parameter or the blind symbol gap threshold derived from this value. If it does not exceed, it is defined as a single blind symbol. Analyze the spacing between adjacent points in the horizontal direction. When this distance exceeds the threshold defined by the adaptive blind symbol spacing parameter (e.g., 6 × x_optimal or the blind symbol gap threshold derived from this value, such as (3.5 × x_optimal + 6 × x_optimal) / 2 = 4.75 × x_optimal), it is identified as a blind symbol boundary.

[0048] Step S45: For a single Braille symbol, divide it into left and right column candidate points using the adaptive horizontal spacing parameter within the Braille symbol. The position of each candidate point is determined by its position along the Y-axis within the baseline. Based on the adaptive horizontal spacing parameter (2.5 × x_optimal), the point is divided into left and right column candidate points by comparing the position of the point relative to the horizontal center of the Braille symbol (approximately half of this parameter can be used as a discrimination threshold), or by finding an optimal column-to-column separation gap. Combining the left and right column division results with the previously determined upper and lower baselines of the row, the final determination is made of which of the standard six-dot Braille points (dots 1 to 6) each point belongs to, and the dot composition information of the Braille symbol is output, such as... Figure 4 As shown.

[0049] Preferably, the following steps need to be performed before performing step S4: Step A: Identify all detected blind spots in the second image, obtain the horizontal Euclidean distance between all adjacent blind spots, and construct a dataset of observed actual pixel distances; Step B: Construct an objective function to quantify the total deviation of all observed actual pixel distances from their nearest theoretical target pixel distance, as shown in the following expression: Where E(x) is the objective function value, N is the total number of observed distances, and M is the theoretical number of target distances, which is 8. The distance to the i-th observed pixel. The mapping value of the j-th theoretical target distance under the current geometric parameter x, where min is the value for each observed distance. Find the closest theoretical distance ; Step C: Using a univariate optimization algorithm (minimize_scalar from the SciPy library, employing a bounded optimization method), search within a preset range of reasonable x values ​​to find the x_optimal that minimizes the objective function; Step D: By calculating the relative proportions of x_optimal with the various spacing parameters of Braille in the national standard, the adaptive line spacing parameter, the adaptive vertical dot spacing parameter within the Braille character, the adaptive word spacing parameter, the word gap threshold derived from this value, the adaptive Braille character spacing parameter, the Braille character gap threshold derived from this value, and the adaptive horizontal spacing parameter within the Braille character are obtained.

[0050] The national standard Braille scheme provides clear physical size specifications for dot spacing, character spacing, phrase spacing, and line spacing. However, these ideal values, after being converted into a two-dimensional image, exhibit significant pixel-level differences due to factors such as shooting distance, angle, lens distortion, and paper deformation. Furthermore, these differences may vary across different regions of the same image. Traditional fixed threshold methods struggle to adapt to this variability, easily leading to segmentation errors. To overcome this limitation, this invention proposes an adaptive spacing parameter estimation method based on an optimization model and applies it to the subsequent hierarchical Braille structure parsing process. The core idea is that although absolute pixel spacing may change, the relative proportions between different types of spacing remain relatively stable within a local range. This method aims to learn an optimal "scale factor" x_optimal from the image data itself and, based on this, generate a set of adaptive spacing parameters that highly match the characteristics of the current image, thereby guiding subsequent accurate segmentation.

[0051] The structural segmentation of the second image in this invention has the following significant advantages compared to the traditional fixed threshold method: High adaptability and robustness: By dynamically learning the optimal scale factor from current image data and generating a complete set of adaptive spacing parameters, the system can effectively adapt to actual spacing changes caused by factors such as shooting conditions and paper deformation. This method directly utilizes the inherent structural proportions of the national standard Braille and intelligently maps them to the pixel space of the current image.

[0052] Improved segmentation accuracy: The accurately estimated adaptive spacing parameters provide a more reliable basis for subsequent segmentation tasks that is more in line with the actual situation of the current image, thereby significantly improving the accuracy of segmentation at each level of line, word, and blind symbol, and laying a solid foundation for the correct identification of points within the final blind symbol, avoiding the "one-size-fits-all" problem that may be caused by fixed thresholds.

[0053] Reduce manual intervention and improve automation: The core scaling factor is automatically determined through a data-driven optimization process, which significantly reduces the reliance on a large number of preset fixed thresholds and the need for complex manual adjustments, thereby enhancing the system's automation level and its generalization ability under different Braille scanning images or shooting conditions.

[0054] Preferably, step S5 is as follows: Step S51: Convert the dot combination of each identified blind symbol into its corresponding basic linguistic or semiotic unit according to the conversion rules of Chinese characters, and use it as the basic unit; Step S52: According to the spelling rules of Chinese Braille, combine the adjacent initials, finals, and tone marks of the basic unit into complete, toned Chinese Pinyin syllables; for example, ['pin1', 'yin1', 'shi4', 'yi1', 'ge4', 'li4', 'zi5', '。']. This sequence accurately reflects the phonetic information encoded by Braille; such as... Figure 5 As shown.

[0055] Step S53: After removing the tone information from the Chinese Pinyin syllables, input them into a Pinyin-to-Hanzi engine (such as the Pinyin2Hanzi.dag library based on a language model or dictionary). This engine will generate one or more possible candidate Chinese character sequences for the input Pinyin syllables based on its built-in language model (considering word collocation probabilities); for example, for ['pin', 'yin'], it may generate candidates such as ['pinyin'], ['pinyin'], etc. Typically, it will obtain several of the top-ranked candidates. Step S54: Input the candidate Chinese character sequence into the Chinese character to Pinyin library (such as pypinyin) to obtain its standardized Pinyin representation with numerical tones, as the candidate Chinese Pinyin sequence; (for example, pinyin('pinyin', style=Style.TONE3) yields ['pin1', 'yin1']); Step S53: Compare the Chinese Pinyin syllables with the candidate Chinese Pinyin sequences, and obtain the Chinese characters whose candidate Chinese Pinyin sequences match the Chinese Pinyin syllables as the translation results.

[0056] Converting the pinyin sequence into the final Chinese character text is the last step in the translation process. The main obstacle lies in the large number of homophones (words) in Chinese. To facilitate writing, Chinese pinyin omits consonants, vowels, or tones, especially tones, according to rules and personal preferences. According to linguistic research based on the largest corpus of Chinese pinyin, pinyin authors omit an average of up to 90% of the tones. While simplification helps streamline the writing process, the existence of homophones and a large number of homonymous characters hinders accurate comprehension. Even with tones, pinyin alone is often insufficient to uniquely identify the corresponding Chinese character.

[0057] Therefore, this invention uses a structured Braille dot matrix as a base for mapping phonetic components and symbols through a lookup table; then, based on the spelling rules of Chinese Braille, a standard Pinyin syllable sequence with precise tone information is synthesized; next, a Pinyin-to-Chinese character engine is used to generate candidate Chinese characters, and tone information derived from Braille is used for matching and verification to accurately resolve homophonic ambiguity; finally, punctuation marks are integrated and special formats are processed to generate the final fluent and accurate Chinese translation text. This process fully utilizes the encoded information of Braille itself, improving the accuracy and reliability of automatic translation in complex contexts, such as... Figure 6 As shown.

[0058] It is worth mentioning that when there are multiple Chinese characters whose pinyin sequence matches the pinyin syllables, the character with the highest ranking is selected for translation.

[0059] A high-precision real-time blind-Chinese translation system, using the aforementioned high-precision real-time blind-Chinese translation method, includes an image acquisition module, an extraction module, a correction module, a segmentation module, and a conversion module; The image acquisition module is used to acquire Braille images using a camera equipped with a horizontal lateral parallel light source, and to preprocess the Braille images to obtain the first image; The extraction module is used to input the first image into the pre-trained YOLOv8 model, obtain the bounding box information of all detected Braille bumps, and extract the Braille coordinates and dimensions; The correction module is used to fit the overall tilt trend of the Braille line using least squares linear regression with all detected blind spot coordinates, calculate the tilt angle, generate a rotation matrix through the tilt angle, and perform affine transformation on all blind spot coordinates to obtain the corrected second image. The segmentation module is used to perform structural segmentation on the second image and obtain the set of blind symbol points; The conversion module is used to convert the content of the Braille dot set into Chinese content based on the conversion rules between Braille and Chinese characters.

[0060] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0061] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A high-precision real-time blind-Chinese translation method, characterized in that, Includes the following steps: Step S1: Use a camera equipped with a horizontal lateral parallel light source to capture Braille images, and preprocess the Braille images to obtain the first image; Step S2: Input the first image into the pre-trained YOLOv8 model, obtain the bounding box information of all detected Braille bumps, and extract the Braille coordinates and dimensions; Step S3: Using all detected blind spot coordinates, the overall tilt trend of the Braille lines is fitted by least squares linear regression, the tilt angle is calculated, a rotation matrix is ​​generated through the tilt angle, and an affine transformation is performed on all blind spot coordinates to obtain the corrected second image. Step S4: Perform structural segmentation on the second image to obtain the set of blind symbol points; Step S5: Based on the conversion rules between Braille and Chinese characters, convert the content of the Braille dot set into Chinese text. In step S41: Obtain the vertical distance between adjacent points or clusters of points on the Y coordinate in the second image as the first distance, and determine whether the first distance exceeds the adaptive line spacing parameter. If it does not exceed the parameter, determine that the point or cluster of points is the same text line. Step S42: Based on the Y coordinate within the text line and the vertical point spacing parameter within the adaptive blind character, obtain the upper and lower baselines of the text line respectively; Step S43: For text lines within the baseline, obtain the distance between adjacent points in the X coordinate as the second distance, and determine whether the second distance exceeds the adaptive word spacing parameter or the derived word spacing threshold. If it does not exceed the parameter, then define adjacent points as the same word group. Step S44: For the phrase, obtain the distance between adjacent points in the X coordinate as the third distance, and determine whether the third distance exceeds the adaptive blind character spacing parameter or the blind character spacing threshold derived from this value. If it does not exceed the parameter, it is defined as a single blind character. Step S45: For a single blind symbol, divide the left column candidate points and the right column candidate points by using the adaptive horizontal spacing parameter within the blind symbol, and determine the position of the candidate points in the Y-axis direction within the baseline.

2. The high-precision real-time blind-Chinese translation method according to claim 1, characterized in that, The horizontal lateral parallel light in step S1 includes single-sided parallel light input and multiple double-sided lateral parallel light input; The conditions for single-sided parallel light input are: a light source is projected onto the Braille document from one side, and the angle requirements for the projection light source are: the light source is projected at an angle parallel to the reference line with the Braille document as the horizontal reference line, or the angle between the light source and the reference line is less than the angle threshold. The conditions for parallel light input from both sides are as follows: a light source is projected onto the Braille document from both sides, and the angle requirements for the projected light source are: with the Braille document as the horizontal baseline, the light source is projected at an angle parallel to the baseline, or the angle between the light source and the baseline is less than the angle threshold.

3. The high-precision real-time blind-Chinese translation method according to claim 1, characterized in that, The preprocessing in step S1 includes: The Braille image is processed into a computer-readable format using the standard image processing library OpenCV. This requires ensuring that the image data type and color channels conform to the processing requirements of the YOLOv8 model, converting it to grayscale, and resizing it while maintaining the BGR / RGB format. Cropping the background that is unrelated to the Braille document; First, apply Gaussian filtering and median filtering, then execute the nonlocal mean denoising algorithm.

4. The high-precision real-time blind-Chinese translation method according to claim 1, characterized in that, The steps of step S3 are as follows: Step S31: Treat all detected blind spot coordinates (x, y) as a point cloud dataset, perform PCA processing on the dataset, and obtain the maximum feature vector in the PCA processing; The angle between the largest eigenvalue vector and the horizontal axis of the coordinate system is used as the first angle. The image is rotated using the first angle to obtain a preliminary image. Step S32: Extract the Y coordinates of blind spots in the preliminary image, and use the "ward" method to construct a one-dimensional feature for clustering, thereby obtaining multiple clusters; Step S33: Perform linear regression on each cluster using the least squares method to fit an optimal straight line model and obtain the slope of the straight line model; Step S34: Obtain the average slope, construct a two-dimensional rotation matrix using the average slope, and perform an affine transformation on the preliminary image using the two-dimensional rotation matrix to obtain the corrected second image.

5. The high-precision real-time blind-Chinese translation method according to claim 1, characterized in that, Before performing step S4, the following steps also need to be performed: Step A: Identify all detected blind spots in the second image, obtain the horizontal Euclidean distance between all adjacent blind spots, and construct a dataset of observed actual pixel distances; Step B: Construct an objective function to quantify the total deviation of all observed actual pixel distances from their nearest theoretical target pixel distance, as shown in the following expression: Where E(x) is the objective function value, N is the total number of observed distances, and M is the theoretical number of target distances, which is 8. The distance to the i-th observed pixel. The mapping value of the j-th theoretical target distance under the current geometric parameter x, where min is the value for each observed distance. Find the closest theoretical distance ; Step C: Using a univariate optimization algorithm, search within a preset range of reasonable x values ​​to find the x_optimal that minimizes the objective function; Step D: By calculating the relative proportions of x_optimal with the various spacing parameters of Braille in the national standard, the adaptive line spacing parameter, the adaptive vertical dot spacing parameter within the Braille character, the adaptive word spacing parameter, the word gap threshold derived from this value, the adaptive Braille character spacing parameter, the Braille character gap threshold derived from this value, and the adaptive horizontal spacing parameter within the Braille character are obtained.

6. The high-precision real-time blind-Chinese translation method according to claim 5, characterized in that, Step S5 is as follows: Step S51: Convert the dot combination of each identified blind symbol into its corresponding basic linguistic or semiotic unit according to the conversion rules of Chinese characters, and use it as the basic unit; Step S52: According to the spelling rules of Chinese Braille, combine the initials, finals and tone marks of the basic units into complete Chinese Pinyin syllables with tones; Step S53: After removing the tone information from the Chinese Pinyin syllables, input them into the Pinyin to Chinese character engine to generate one or more possible candidate Chinese character sequences for the input Chinese Pinyin syllables; Step S54: Input the candidate Chinese character sequence into the Chinese character to Pinyin library to obtain its standardized Pinyin representation with numerical tones, as the candidate Chinese Pinyin sequence; Step S53: Compare the Chinese Pinyin syllables with the candidate Chinese Pinyin sequences, and obtain the Chinese characters whose candidate Chinese Pinyin sequences match the Chinese Pinyin syllables as the translation results.

7. A high-precision real-time blind-Chinese translation system, characterized in that, The high-precision real-time blind-Chinese translation method according to any one of claims 1 to 6 includes an image acquisition module, an extraction module, a correction module, a segmentation module, and a conversion module; The image acquisition module is used to acquire Braille images using a camera equipped with a horizontal lateral parallel light source, and to preprocess the Braille images to obtain the first image; The extraction module is used to input the first image into the pre-trained YOLOv8 model, obtain the bounding box information of all detected Braille bumps, and extract the Braille coordinates and dimensions; The correction module is used to fit the overall tilt trend of the Braille line using least squares linear regression with all detected blind spot coordinates, calculate the tilt angle, generate a rotation matrix through the tilt angle, and perform affine transformation on all blind spot coordinates to obtain the corrected second image. The segmentation module is used to perform structural segmentation on the second image and obtain the set of blind symbol points; The conversion module is used to convert the content of the Braille dot set into Chinese content based on the conversion rules between Braille and Chinese characters.

Images