A method and device for detecting smeared Braille squares based on block detection preprocessing

Through the block detection preprocessing method, using the Rotated-RetinaNet network and image preprocessing technology, the problems of low detection efficiency and imbalance in traditional Braille detection are solved, and efficient detection of Braille smearing is achieved.

CN118429979BActive Publication Date: 2024-11-15HANGZHOU HUIGUANG TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410529926.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-04-29
Publication Date
2024-11-15
Estimated Expiration
2044-04-29

AI Technical Summary

Technical Problem

Traditional Braille Testing Methods cannot effectively detect Braille Smear, resulting in inefficiency and category imbalance.

Method used

Using a block detection preprocessing method, the M1, M2 and M3 networks trained by Rotated-RetinaNet network obtain Braille block information, detect Braille square information and smearing situation respectively, and image preprocessing is performed by combining cv2.getRotationMatrix2D and cv2.warpAffine functions. and rotate to realize detection of the Braille smear.

Benefits of technology

It improves the detection effect of the Braille smear, avoids the problem of category imbalance, and significantly improves the detection performance in handwritten Braille scenes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118429979B_ABST
    Figure CN118429979B_ABST
Patent Text Reader

Abstract

The present invention provides a method and device for detecting smeared Braille squares based on block detection preprocessing, the method comprising: obtaining all Braille block information in a Braille picture to be detected based on an M1 network obtained through Rotated-RetinaNet network training; cropping all Braille blocks from the Braille picture to be detected based on the Braille block information to obtain a Braille block picture corresponding thereto; rotating the Braille block picture to a horizontal direction; detecting Braille square information existing in all Braille block pictures based on an M2 network obtained through Rotated-RetinaNet network training; sequentially detecting the smearing of Braille squares in each Braille block image based on an M3 network obtained through Rotated-RetinaNet network training; and obtaining a detection result of the smeared Braille square based on the Braille square information and the smearing of the Braille square. The method solves the problem that the detection effect of the traditional Braille square detection method on the smeared Braille square is limited, and avoids the problem of imbalanced detection categories of the smeared Braille square, greatly improving the detection effect of the smeared Braille square in the handwritten Braille scene.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of Braille square detection technology, and specifically to a method and apparatus for detecting smeared Braille squares based on block detection preprocessing. Background Technology

[0002] Sighted people, without specialized training, have low efficiency in reading Braille. Handwritten Braille inevitably contains errors, further impacting their reading efficiency. To convert Braille into easily readable Chinese documents or process it into speech, object detection technology is typically used to detect Braille squares. Traditional techniques directly apply object detection to Braille square detection. However, handwritten Braille using Braille writing tools often results in smudged Braille squares, which object detection technology cannot effectively detect. Furthermore, due to the complex features of smudged Braille squares, the small sample size of smudged Braille squares, and the large variety of smudged Braille squares leading to significant class imbalance, traditional Braille square detection methods have limited effectiveness.

[0003] The above-mentioned problems urgently need to be solved. Summary of the Invention

[0004] This invention aims to overcome at least one of the aforementioned drawbacks of the prior art. On one hand, it provides a method for detecting smeared Braille squares based on block detection preprocessing. The method includes: acquiring a Braille image to be detected; acquiring all Braille block information in the Braille image based on an M1 network trained using a Rotated-RetinaNet network; cropping all Braille blocks from the Braille image based on the Braille block information to obtain corresponding Braille block images; rotating the Braille block images to a horizontal orientation; detecting Braille square information present in all Braille block images based on an M2 network trained using a Rotated-RetinaNet network; sequentially detecting the smearing status of Braille squares in each Braille block image based on an M3 network trained using a Rotated-RetinaNet network; and obtaining a detection result for smeared Braille squares based on the Braille square information and the smearing status of the Braille squares.

[0005] Furthermore, the M1 network is trained by combining a Rotated-RetinaNet network with Braille images and corresponding Braille block annotation information.

[0006] Furthermore, the step of obtaining all Braille block information in the Braille image to be detected based on the M1 network trained by the Rotated-RetinaNet network includes: preprocessing the Braille image to be detected to adjust its format and size; inputting the preprocessed Braille image to be detected into the M1 network to obtain the Braille block information in the Braille image to be detected; the Braille block information includes the center coordinates of the Braille block, the width of the Braille block, the height of the Braille block, and the rotation angle of the Braille block.

[0007] Furthermore, the step of cropping all Braille blocks from the Braille image to be detected based on the Braille block information to obtain the corresponding Braille block image includes: cropping the Braille blocks from the Braille image to be detected based on the cv2.getRotationMatrix2D function, the cv2.warpAffine function, and the cv2.bitwise_and function.

[0008] Furthermore, rotating the Braille block image to a horizontal position includes: sequentially rotating the cropped Braille block image to a horizontal position using the center coordinates of the Braille block as the rotation center, based on the cv2.getRotationMatrix2D function and the cv2.warpAffine function.

[0009] Furthermore, the M2 network is trained by combining a Rotated-RetinaNet network with Braille images and corresponding Braille annotation information.

[0010] Furthermore, the step of detecting Braille square information in all Braille blocks based on the M2 network trained by the Rotated-RetinaNet network includes: preprocessing the Braille block image; inputting the preprocessed Braille block image into the M2 network to obtain Braille square information in the Braille block; the Braille square information includes the center coordinates of the Braille square, the width of the Braille square, the height of the Braille square, the rotation angle of the Braille square, and the Braille square category.

[0011] Furthermore, the step of sequentially detecting the smearing status of Braille squares in each Braille block based on the M3 network trained by the Rotated-RetinaNet network includes: the M3 network is trained by combining the Rotated-RetinaNet network with the Braille image and the corresponding Braille square smearing status annotation information; the Braille block image is preprocessed; and the preprocessed Braille block image is input into the M3 network to obtain the smearing status of all Braille squares contained in the Braille block image.

[0012] Furthermore, the step of obtaining the detection result of smeared Braille based on Braille square information and the smearing status of Braille squares includes: merging the Braille square information and the smearing status of Braille squares; mapping the merged result back to the Braille image to be detected; and integrating all Braille squares to obtain the detection result of smeared Braille.

[0013] Secondly, the present invention provides a device for detecting smudged Braille squares based on block detection preprocessing. The device includes: a unit for acquiring a Braille image to be detected, adapted to acquire a Braille image to be detected; a unit for acquiring Braille block information, adapted to acquire all Braille block information in the Braille image to be detected based on an M1 network trained by a Rotated-RetinaNet network; a unit for acquiring Braille block images, adapted to crop all Braille blocks from the Braille image to be detected based on the Braille block information to obtain corresponding Braille block images; a rotation unit, adapted to rotate the Braille block images to a horizontal direction; a unit for detecting Braille square information, adapted to detect Braille square information present in all Braille block images based on an M2 network trained by a Rotated-RetinaNet network; a unit for detecting smudged Braille squares, adapted to sequentially detect the smudged Braille squares in each Braille block image based on an M3 network trained by a Rotated-RetinaNet network; and a detection result generation unit, adapted to obtain a detection result of smudged Braille squares based on the Braille square information and the smudged Braille squares.

[0014] Thirdly, the present invention also provides a computer-readable storage medium storing one or more instructions, the computer instructions being used to cause the computer to perform the above-described method for detecting smeared Braille squares based on block detection preprocessing.

[0015] In another aspect, the present invention provides an electronic device, comprising: a memory and a processor; the memory storing at least one program instruction; the processor loading and executing the at least one program instruction to implement the above-described method for detecting smeared Braille squares based on block detection preprocessing.

[0016] The beneficial effects of this invention are as follows: This invention provides a method for detecting smeared Braille squares based on block detection preprocessing. The method includes: acquiring a Braille image to be detected; acquiring all Braille block information in the Braille image to be detected based on an M1 network trained using a Rotated-RetinaNet network; cropping all Braille blocks from the Braille image to be detected based on the Braille block information to obtain corresponding Braille block images; rotating the Braille block images to a horizontal orientation; detecting Braille square information present in all Braille block images based on an M2 network trained using a Rotated-RetinaNet network; sequentially detecting the smearing status of Braille squares in each Braille block image based on an M3 network trained using a Rotated-RetinaNet network; and obtaining a detection result of smeared Braille squares based on the Braille square information and the smearing status of the Braille squares. This paper improves existing Braille square detection methods by introducing a block detection preprocessing method to detect smudged Braille squares in Braille images and adding a network for detecting smudged Braille squares. By treating whether a Braille square is smudged as a separate detection module task, the paper solves the problem of limited detection effect of traditional Braille square detection methods on smudged Braille squares and avoids the problem of class imbalance in smudged Braille square detection, which greatly improves the detection effect of smudged Braille squares in handwritten Braille scenarios. Attached Figure Description

[0017] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0018] Figure 1 This is a flowchart of a method for detecting smeared Braille squares based on block detection preprocessing provided in an embodiment of the present invention.

[0019] Figure 2 This is a schematic diagram of Braille squares and Braille blocks contained in a Braille image to be detected, provided in an embodiment of the present invention.

[0020] Figure 3 This is a schematic diagram of a smeared Braille square detection device based on block detection preprocessing provided in an embodiment of the present invention.

[0021] Figure 4 This is a partial block diagram of the electronic device provided in the embodiments of the present invention. Detailed Implementation

[0022] Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but may also have additional steps not included in the figures. The process can correspond to a method, function, procedure, subroutine, subroutine, etc.

[0023] It should be understood that although the terms "first," "second," etc., may be used herein to describe various units, these units should not be limited by these terms. These terms are used merely to distinguish one unit from another. For example, without departing from the scope of the exemplary embodiments, a first unit may be referred to as a second unit, and similarly, a second unit may be referred to as a first unit. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0024] The present invention will now be described in detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0025] Example 1

[0026] like Figure 1 The diagram shown is a flowchart of a smeared Braille square detection method based on block detection preprocessing provided by the present invention.

[0027] As an example, combined Figure 2 The method includes:

[0028] S110: Obtain the Braille image to be tested.

[0029] S120: Obtain information on all Braille blocks in the Braille image to be detected based on the M1 network trained by the Rotated-RetinaNet network.

[0030] Preferably, the M1 network is trained using a Rotated-RetinaNet network combined with Braille images and corresponding Braille block annotation information. Specifically, the M1 network is trained using a Rotated-RetinaNet network (a rotated object detection version of RetinaNet that adds the angle of the predicted bounding box to the horizontal detection) combined with Braille images and corresponding Braille block annotation information (including the position of the Braille blocks), and is used to detect Braille blocks present in the image.

[0031] Preferably, the step of obtaining all Braille block information in the Braille image to be detected based on the M1 network trained by the Rotated-RetinaNet network includes: preprocessing the Braille image to be detected to adjust its format and size; inputting the preprocessed Braille image to be detected into the M1 network to obtain the Braille block information in the Braille image to be detected; the Braille block information includes the center coordinates of the Braille block, the width of the Braille block, the height of the Braille block, and the rotation angle of the Braille block. Specifically, the input Braille image to be detected is adjusted to the same input size and format, and then the M1 network is used to obtain all Braille block information {b1,b2,…,bk} contained in the image, where each Braille block bi is composed of {(xi,yi),wi,hi,anglei}, where (xi,yi) represents the center coordinates of the Braille block, wi represents the width of the Braille block, hi represents the height of the Braille block, and anglei represents the rotation angle of the Braille block.

[0032] S130: Based on the Braille block information, crop all Braille blocks from the Braille image to be detected to obtain the corresponding Braille block image.

[0033] Preferably, the step of cropping all Braille blocks from the Braille image to be detected based on the Braille block information to obtain the corresponding Braille block image includes: cropping the Braille blocks from the Braille image to be detected based on the cv2.getRotationMatrix2D function, the cv2.warpAffine function, and the cv2.bitwise_and function.

[0034] S140: Rotate the Braille block image to a horizontal orientation.

[0035] Preferably, rotating the Braille block image to a horizontal position includes: sequentially rotating the cropped Braille block image to a horizontal position using the center coordinates of the Braille blocks as the rotation center, based on the cv2.getRotationMatrix2D and cv2.warpAffine functions. Specifically, based on all detected Braille blocks {b1,b2,…,bk}, all Braille blocks are cropped from the original Braille image to be detected, and then these cropped Braille blocks are rotated to a horizontal position using their own rotation angle anglei and the center coordinates (xi, yi) as the rotation center.

[0036] S150: Detect Braille square information in all Braille block images based on the M2 network trained by Rotated-RetinaNet.

[0037] Preferably, the M2 network is trained using a Rotated-RetinaNet network combined with Braille images and corresponding Braille square annotation information. Specifically, the M2 network is trained using a Rotated-RetinaNet network combined with Braille images and corresponding Braille square annotation information (including the location and category of the Braille squares), and is used to detect Braille squares present in the images.

[0038] Preferably, the step of detecting Braille square information in all Braille blocks based on the M2 network trained by the Rotated-RetinaNet network includes: preprocessing the Braille block image; inputting the preprocessed Braille block image into the M2 network to obtain Braille square information in the Braille blocks; the Braille square information includes the center coordinates of the Braille square, the width of the Braille square, the height of the Braille square, the rotation angle of the Braille square, and the Braille square category. Specifically, the Braille block images are preprocessed to standardize the input size and format. Then, the images are input into the Rotated-RetinaNet detection network M2 to obtain all Braille squares {br1, br2, …, brm} contained in the Braille block images. Each Braille square brj consists of {(xj, yj), wj, hj, anglej, clsi}, where (xj, yj) represents the center coordinates of the Braille square, wj represents the width of the Braille square, hj represents the height of the Braille square, anglej represents the rotation angle of the Braille square, and clsi represents the category of the Braille square. There are a total of 63 categories of Braille squares, excluding blank Braille squares.

[0039] S160: Based on the M3 network trained by the Rotated-RetinaNet network, the smearing of Braille squares in each Braille block image is detected sequentially.

[0040] Preferably, the M3 network is trained using a Rotated-RetinaNet network combined with Braille images and corresponding Braille square smearing annotations. Specifically, the M3 network is trained using a Rotated-RetinaNet network combined with Braille images and corresponding Braille square smearing annotations (including the position and smearing status of the Braille squares), and is used to detect the smearing status of Braille squares in the image.

[0041] Preferably, the step of sequentially detecting the smearing status of Braille squares in each Braille block based on the M3 network trained by the Rotated-RetinaNet network includes: preprocessing the Braille block image; and inputting the preprocessed Braille block image into the M3 network to obtain the smearing status of all Braille squares contained in the Braille block image. Specifically, the Braille block image is preprocessed to unify the input size and format, and then the image is input into the Rotated-RetinaNet detection network M3 to obtain the smearing status {f1, f2, …, fn} of all Braille squares contained in the Braille block image. Each Braille square fj1 is composed of {(xj1, yj1), wj1, hj1, anglej1, modifiedj1}, where (xj1, yj1) represents the center coordinates of the Braille square, wj1 represents the width of the Braille square, hj1 represents the height of the Braille square, anglej1 represents the rotation angle of the Braille square, and modifiedj1 indicates whether the Braille square is smeared.

[0042] S170: The detection results of the smeared Braille square are obtained based on the Braille square information and the smearing status of the Braille square.

[0043] Preferably, the step of obtaining the detection result of smeared Braille squares based on Braille square information and smearing status of Braille squares includes: merging the Braille square information and smearing status of Braille squares; mapping the merged result back to the Braille image to be detected; and integrating all Braille squares to obtain the detection result of smeared Braille squares. Specifically, the Braille squares {br1, br2, …, brm} and the smearing status of Braille squares {f1, f2, …, fn} are merged based on the IoU threshold to obtain the integrated Braille square overall data {brs1, brs2, …, brsm}. Each Braille square brsj consists of {(xj, yj), wj, hj, anglej, clsj, modifiedj}, representing the center coordinates, width, height, rotation angle, category, and whether it is smeared. Based on the Braille block bi to which each Braille square brsj belongs, the Braille square is rotated with the center coordinates of the Braille block as the rotation point, and its position on the original image is mapped according to anglei. Then, the Braille squares in all Braille blocks are merged to obtain the final Braille square detection result (including the center coordinates, width and height, category, and whether it is smeared for each Braille square).

[0044] The above embodiments provide a method for detecting smudged Braille squares based on block detection preprocessing. The method adds block detection preprocessing and a smudged Braille square detection network to the traditional Braille square detection method to detect Braille squares in Braille images. The detection task of smudged Braille squares is divided into two sub-tasks: Braille square category detection and Braille square smudge detection. This effectively avoids the class imbalance problem caused by too many smudged Braille square categories and too few smudged samples, and significantly improves the detection effect of smudged Braille squares in handwritten Braille scenarios.

[0045] Example 2

[0046] Please see Figure 3 This embodiment provides a schematic diagram of a smeared Braille square detection device based on block detection preprocessing.

[0047] As an example, the device includes:

[0048] Unit 310 for acquiring Braille images to be detected is applicable to acquiring Braille images to be detected.

[0049] The Braille block information acquisition unit 320 is suitable for acquiring all Braille block information in a Braille image to be detected based on an M1 network trained by a Rotated-RetinaNet network.

[0050] The Braille block image acquisition unit 330 is adapted to crop all Braille blocks from the Braille image to be detected based on the Braille block information to obtain the corresponding Braille block image.

[0051] The rotation unit 340 is adapted to rotate the Braille block image to a horizontal position.

[0052] The Braille square information detection unit 350 is suitable for detecting Braille square information in all Braille block images based on the M2 network trained by the Rotated-RetinaNet network.

[0053] The Braille square smear detection unit 360 is suitable for sequentially detecting the smearing of Braille squares in each Braille block image based on the M3 network trained by the Rotated-RetinaNet network.

[0054] The detection result generation unit 370 is suitable for obtaining detection results of smeared Braille squares based on Braille square information and the smearing status of Braille squares.

[0055] It is not difficult to see that this embodiment is a system implementation corresponding to the first embodiment, and this embodiment can be implemented in conjunction with the first embodiment. The relevant technical details mentioned in the first embodiment are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the first embodiment.

[0056] It is worth mentioning that all units involved in this embodiment are logical units. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this invention, this embodiment does not introduce units that are not closely related to solving the technical problem proposed by this invention; however, this does not mean that other units are absent from this embodiment.

[0057] Example 3

[0058] This invention also proposes a storage medium storing a smeared Braille square detection method based on block detection preprocessing. When the smeared Braille square detection program based on block detection preprocessing is executed by a processor, it implements the steps of the smeared Braille square detection method based on block detection preprocessing as described above. Since this storage medium adopts all the technical solutions of all the above embodiments, it has at least all the beneficial effects brought about by the technical solutions of the above embodiments, which will not be repeated here.

[0059] Example 4

[0060] Please see Figure 4 The present invention also provides an electronic device, including: a memory and a processor; the memory stores at least one program instruction; the processor loads and executes the at least one program instruction to implement the smeared Braille square detection method based on block detection preprocessing provided in Embodiment 1.

[0061] The memory 602 and processor 601 are connected via a bus, which may include any number of interconnecting buses and bridges, connecting various circuits of one or more processors 601 and memory 602 together. The bus may also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. A bus interface provides an interface between the bus and the transceiver. The transceiver may be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by processor 601 is transmitted over a wireless medium via an antenna, which further receives data and transmits it to processor 601.

[0062] Processor 601 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory 602 can be used to store data used by processor 601 during operation.

[0063] The above descriptions are merely embodiments of the present invention. Commonly known structures and characteristics are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are aware of all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, based on the guidance provided in this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

Claims

1. A method for detecting smeared Braille squares based on block detection preprocessing, It is characterized in that The method comprises: Get the Braille image to be detected; The method comprises: preprocessing the braille image to be detected to adjust the format and size of the braille image to be detected; inputting the preprocessed braille image to be detected into the M1 network to obtain the braille block information in the braille image to be detected; the braille block information includes the center coordinates of the braille block, the width of the braille block, the height of the braille block and the rotation angle of the braille block; Based on the braille block information, cropping all braille blocks from the braille image to be detected obtains a braille block image corresponding thereto; Rotating the braille block image to a horizontal direction; Based on the M2 network obtained by training the Rotated-RetinaNet network, the Braille square information existing in all the Braille block images is detected, including: preprocessing the Braille block images; inputting the preprocessed Braille block images into the M2 network to obtain the Braille square information existing in the Braille blocks; the Braille square information includes the center coordinates of the Braille square, the width of the Braille square, the height of the Braille square, the rotation angle of the Braille square and the Braille square category, wherein the Braille square categories excluding the blank Braille squares have a total of 63 categories; Based on the M3 network trained by the Rotated-RetinaNet network, the smearing status of the Braille squares in each Braille block image is detected in turn. The detection results generated by the M3 network include: the center coordinates of the Braille square, the width of the Braille square, the height of the Braille square, the rotation angle of the Braille square, and whether the Braille square is smeared; A detection result of a smeared Braille square is obtained based on the Braille square information and the smearing condition of the Braille square, including: merging the Braille square information and the smearing condition of the Braille square; mapping the merged result back to the Braille image to be detected; integrating all Braille squares to obtain a detection result of the smeared Braille square, wherein the detection result includes the center point coordinates, width and height, category and whether smeared of each Braille square.

2. The method for detecting smeared Braille squares based on block detection preprocessing according to claim 1, It is characterized in that The M1 network is trained by combining the Rotated-RetinaNet network with the Braille image and the corresponding Braille block annotation information.

3. The method for detecting smeared Braille squares based on block detection preprocessing according to claim 1, It is characterized in that The step of cropping all the braille blocks from the to-be-detected braille picture based on the braille block information to obtain a braille block picture corresponding thereto comprises: The braille block is cropped from the braille image to be detected based on the cv2.getRotationMatrix2D function, the cv2.warpAffine function and the cv2.bitwise_and function.

4. The method for detecting smeared Braille squares based on block detection preprocessing according to claim 1, It is characterized in that The step of rotating the braille block image to a horizontal direction comprises: The braille block image is rotated to the horizontal direction with the central coordinates of the braille block as the rotation center based on the cv2.getRotationMatrix2D function and the cv2.warpAffine function.

5. The method for detecting smeared Braille squares based on block detection preprocessing according to claim 1, It is characterized in that The M2 network is trained by combining the Rotated-RetinaNet network with the Braille image and the corresponding Braille annotation information.

6. The method for detecting smeared Braille squares based on block detection preprocessing according to claim 1, It is characterized in that The M3 network obtained by training the Rotated-RetinaNet network sequentially detects the smearing of the braille squares in each braille block, including: The M3 network is trained by combining the Rotated-RetinaNet network with the Braille image and the corresponding Braille square smearing situation annotation information; Preprocess the Braille block image; The preprocessed Braille block image is input into the M3 network to obtain the smearing conditions of all Braille squares contained in the Braille block image.

7. A smeared Braille square detection device based on block detection preprocessing, It is characterized in that The device comprises: A unit for obtaining a Braille image to be detected is used for obtaining a Braille image to be detected; The unit for obtaining braille block information is adapted to obtain all braille block information in the braille picture to be detected based on the M1 network obtained by training the Rotated-RetinaNet network, including: preprocessing the braille picture to be detected to adjust the format and size of the braille picture to be detected; inputting the preprocessed braille picture to be detected into the M1 network to obtain the braille block information in the braille picture to be detected; the braille block information includes the center coordinates of the braille block, the width of the braille block, the height of the braille block and the rotation angle of the braille block; A braille block image acquisition unit is used to crop all braille blocks from the braille image to be detected based on the braille block information to obtain a braille block image corresponding thereto; A rotation unit, adapted to rotate the braille block image to a horizontal direction; A unit for detecting braille square information is used to detect the braille square information existing in all braille block images based on the M2 network obtained by training the Rotated-RetinaNet network, including: preprocessing the braille block image; inputting the preprocessed braille block image into the M2 network to obtain the braille square information existing in the braille block; the braille square information includes the center coordinates of the braille square, the width of the braille square, the height of the braille square, the rotation angle of the braille square and the braille square category, wherein the braille square categories excluding the blank braille squares have a total of 63 categories; The unit for detecting the smearing of the Braille square is adapted to detect the smearing of the Braille square in each Braille block image in turn based on the M3 network trained by the Rotated-RetinaNet network. The detection results generated by the M3 network include: the center coordinates of the Braille square, the width of the Braille square, the height of the Braille square, the rotation angle of the Braille square, and whether the Braille square is smeared; A detection result generating unit is adapted to obtain a detection result of a smeared Braille square based on Braille square information and the smearing condition of the Braille square, including: merging the Braille square information and the smearing condition of the Braille square; mapping the merged result back to the Braille image to be detected; integrating all Braille squares to obtain a detection result of the smeared Braille square, wherein the detection result includes the center point coordinates, width and height, category and whether smeared of each Braille square.