Method for generating and displaying a tactile image

Abstract

The present invention discloses a method for generating and displaying a tactile image. Step 1: Extract a target image from the acquired original visual two-dimensional digital image. Step 2: Perform edge detection on the target image to obtain a contour binary image of the target image. Step 3: Correct the contour binary image based on a perspective discrimination criterion. Step 4: Denoise the contour binary image obtained in Step 3. Step 5: Build an image pyramid for the denoised contour binary image, and select the best-resolution binary image in the image pyramid based on a preset comprehensive evaluation strategy. Step 6: Map the best-resolution binary image to a tactile image. Step 7: Determine whether the resolution of the tactile image is greater than the resolution of the tactile display. If so, display the tactile image in a window-sliding manner, otherwise display the tactile image centered on the tactile display. The tactile image generated by the present invention through a special image processing strategy is easy to recognize and can meet the requirements for display on a tactile display with a relatively small resolution.

Description

Technical Field

[0001] The present invention belongs to the technical field of tactile images, and specifically relates to a method for generating and displaying tactile images. Background Art

[0002] Touch is one of the important sensory modalities of humans. For visually impaired people, touch is even the main way to obtain information. With the progress of computer technology, there are already braille displays and tactile displays on the market, which help visually impaired people obtain text and graphic information more conveniently and quickly.

[0003] Nowadays, a large amount of digital media such as pictures, videos, and animations are included in Internet information, which expands the need for visually impaired people to understand graphic images. However, few existing technical methods convert visual images into tactile images and display them on tactile displays.

[0004] Currently, some scholars have conducted certain research on tactile images. For example, Yu Wenyuan et al. (Cognitive Mechanisms of Tactile Two-Dimensional Image Recognition, Advances in Psychological Science, 2019, 27(4), 611-622) summarized the cognitive mechanisms, influencing factors, and neural bases of tactile two-dimensional image recognition, and discussed the generation and application of tactile images, providing theoretical guidance for the design of tactile images for visually impaired people; Gong Jiangtao et al. (Quantitative Analysis of Factors Affecting Tactile Image Recognition, Journal of Computer-Aided Design & Computer Graphics, 2018, 30(8), 1438-1445) obtained important features affecting tactile recognition through experiments and gave suggestions for tactile image design. The experimental results show that the lines omitted due to occlusion, three-dimensional perspective, redundant lines expressing texture, operability, and picture symmetry have significant effects on the recognition of tactile images by visually impaired people. Among them, the first three are not conducive to recognition, and the latter two are helpful for recognition.

[0005] Chinese Patent with application number 201811160926.1 discloses a graphic display with a tactile guiding function, which generates tactile images in a certain order, and a slider guides the user to touch according to the tactile image generation order, helping the user understand the image semantics to a certain extent. However, the slow generation speed of tactile images will lead to a poor user experience.

[0006] At the same time, Chinese Patent with application number 201811298146.3 discloses a tactile touch screen, which obtains real-time scene images through a camera and reflects the gray value of the image pixels to the height of the touch screen contacts rising. Although this method can reproduce the scene in the form of relief, it hardly optimizes the images collected by the camera for visually impaired people. It contains many detailed textures and the object contours are not obvious, increasing the difficulty for visually impaired people to understand the graphics.

[0007] Generally speaking, the current tactile image generation and display technologies mainly have the following problems: 1) Except for the artificially made tactile images for teaching or demonstration, most tactile images are converted from visual two-dimensional images. However, in the conversion process, few corresponding treatments are made from the perspective of easy recognition, resulting in the generated tactile images being difficult for visually impaired people to understand; 2) The contradiction between the high resolution of tactile images and the limited size of tactile graphic displays: On the one hand, in order to retain more detailed information of the image, the resolution of tactile images is relatively high. On the other hand, considering cost, portability, and touch efficiency, the size of commercially available tactile image displays is not too large. Therefore, it is very likely that the graphics cannot be fully displayed on the display. Summary of the Invention

[0008] The purpose of the present invention is to provide a method for generating and displaying tactile images to solve the problems in the background art that the tactile images generated by existing methods are difficult for visually impaired people to understand, and the tactile images with higher resolution cannot be fully displayed on low-resolution tactile image displays.

[0009] To achieve the above purpose, the present invention provides the following technical solutions:

[0010] A method for generating and displaying tactile images, the method includes the following steps: Step 1, extract the target image from the acquired original visual two-dimensional digital image; Step 2, perform edge detection on the target image to obtain the contour binary map of the target image; Step 3, correct the contour binary map based on the perspective discrimination criterion; Step 4, denoise the contour binary map obtained in Step 3; Step 5, build an image pyramid for the denoised contour binary map, and select the best-resolution binary map in the image pyramid based on a preset comprehensive evaluation strategy; Step 6, map the best-resolution binary map to a tactile image; Step 7, determine whether the resolution of the tactile image is greater than the resolution of the tactile display. If so, display the tactile image in a window sliding manner, otherwise display the tactile image centered on the tactile display.

[0011] Preferably, Step 1 includes the following steps: Step 1.1, acquire the original visual two-dimensional digital image and perform target detection to obtain the target block diagram; Step 1.2, perform instance segmentation on the target block diagram to obtain the target image.

[0012] Preferably, the perspective discrimination criterion is: determine whether the contour binary map has a perspective effect. If so, perform an inverse perspective transformation on the contour binary map, otherwise retain the contour binary map.

[0013] Preferably, the method for determining whether the binary contour image has a perspective effect is as follows: extract the straight lines of the binary contour image through the Hough transform, and determine whether there are corner points of a cuboid-like oblique view in the straight line contour formed by all the straight lines. If so, it is determined that there is a perspective effect; otherwise, there is no perspective effect. The inverse perspective transformation is to find the polygon with the largest area formed by the straight lines and perform an affine transformation, and use the obtained affine transformation image as the binary contour image.

[0014] Preferably, the method for determining the corner points of the cuboid-like oblique view includes the following steps: obtain the intersection points of three lines based on the function expressions of the straight lines; determine whether there is a straight line connecting any two intersection points. If so, use the straight line as the connection line and the two intersection points connected by the connection line as the suspicious corner point pair; otherwise, end. Calculate the angle formed by the connection line and the straight line passing through the suspicious corner point, and determine whether the two included angles on the same side of the connection line are complementary. If so, the suspicious corner point is the corner point of the cuboid-like oblique view; otherwise, the suspicious corner point is not the corner point of the cuboid-like oblique view.

[0015] Preferably, the image denoising includes the following steps: Step 4.1, set the path length threshold; Step 4.2, traverse the binary contour image to obtain all the paths connected by the contour pixels M; Step 4.3, determine whether the path length is less than the path length threshold. If so, change each contour pixel in the path to a non-contour pixel; otherwise, keep it unchanged.

[0016] Preferably, in the image pyramid, each layer of pyramid image is generated by halving the width and height of the previous layer of pyramid image, and the first layer of pyramid image is the denoised binary contour image.

[0017] Preferably, the comprehensive evaluation strategy is to select the layer of pyramid image with the highest comprehensive score as the best display resolution image, and the calculation formula of the comprehensive score Sc is

[0018]

[0019] a, b, and c are all weight coefficients, and a + b + c = 1. S n is the image consistency score between the nth layer of pyramid image and the first layer of pyramid image. is the normalized image consistency score, and the calculation formula of the image consistency score is

[0020]

[0021] R n is the resolution consistency score between the nth layer of pyramid image and the resolution of the tactile image display. is the normalized resolution consistency score, and the calculation formula of the resolution consistency score is

[0022]

[0023] E n is the tactile reading efficiency score of the nth layer pyramid image, is the normalized tactile reading efficiency score, and the calculation formula of the tactile reading efficiency score is

[0024]

[0025] wherein, A 1 is the total number of pixels of the first layer pyramid image, F 1 is the number of pixels with pixel value 0 in the first layer pyramid image, A n is the total number of pixels of the nth layer pyramid image, F n is the number of pixels with pixel value 0 in the nth layer pyramid image, w d is the width of the tactile graphics display, h d is the height of the tactile graphics display, w n is the width of the nth layer pyramid image, h n is the height of the nth layer pyramid image.

[0026] Preferably, the mapping method is that the contour pixels in the binary image with the best resolution represent the raised contact points in the tactile image; the non - contour pixels in the binary image with the best resolution represent the lowered contact points in the tactile image.

[0027] Preferably, in step 7, the method of displaying the tactile image by window sliding is to move the window through a control device such as a button or a rocker, and select any local image in the tactile image for display, and the resolution of the local image is the same as the resolution of the tactile image display.

[0028] Compared with the prior art, the beneficial effects of the present invention are:

[0029] The present invention proposes a method for generating and displaying a dot - matrix tactile image. Through a special image - processing strategy, the generated tactile image is easy to recognize and can meet the requirements of being displayed on a tactile display with a relatively low resolution. Brief Description of the Drawings

[0030] Figure 1 is a flowchart of a method for generating and displaying a tactile image of the present invention.

[0031] Figure 2 is the target detection effect diagram in the embodiment of the present invention.

[0032] Figure 3 is the instance segmentation effect diagram in the embodiment of the present invention.

[0033] Figure 4 is the edge detection effect diagram in the embodiment of the present invention.

[0034] Figure 5 This is the denoising effect diagram in the embodiment of the present invention.

[0035] Figure 6 This is the schematic diagram of inverse perspective transformation in the embodiment of the present invention.

[0036] Figure 7 This is the image pyramid effect diagram in the embodiment of the present invention.

[0037] Figure 8 This is the denoising flowchart in the embodiment of the present invention.

[0038] Figure 9 This is the algorithm flowchart of the image pyramid in the embodiment of the present invention.

[0039] Figure 10 This is the operation schematic diagram of the window-sliding tactile image in the embodiment of the present invention.

[0040] Figure 11 This is the schematic diagram of the corner points of the cuboid-like oblique view in the embodiment of the present invention. Detailed implementation manners

[0041] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without making creative efforts shall fall within the protection scope of the present invention.

[0042] Refer to Figure 1 As shown, a method for generating and displaying a tactile image specifically includes 7 steps.

[0043] Step 1: Obtain an original visual two-dimensional digital image through a camera or the Internet, and extract the target image from the obtained original visual two-dimensional digital image.

[0044] Specifically, the extraction of the target image from the original visual two-dimensional digital image includes the following 2 sub-steps:

[0045] Step 1.1: Perform target detection on the original visual two-dimensional digital image to obtain N objects framed by bounding boxes, that is, target bounding boxes, as Figure 2 shown;

[0046] Step 1.2: Perform instance segmentation on each target bounding box to obtain the target image. This instance segmentation is also to remove background pixels and retain foreground (object) pixels, as Figure 3 shown.

[0047] In step 1.1 of the present invention, when the target block diagram cannot be detected by the target detection algorithm, step 1.2 cannot be executed.

[0048] In step 1.2 of the present invention, when there are multiple target block diagrams, instance segmentation needs to be performed on each target block diagram respectively. After instance segmentation of each target block diagram, a target image is obtained, and the number of target block diagrams is the same as the number of target images.

[0049] In step 1.1 of the present invention, the target detection algorithm can be RCNN, YOLO series, etc., and at the same time, the classification information of the object with the bounding box and the bounding box pillar is obtained. In step 1.2 of the present invention, for instance segmentation, traditional image processing algorithms can be used, such as the saliency target extraction method, or deep learning algorithms can be used. This is common knowledge in the art, and those skilled in the art can set it according to the actual situation.

[0050] In step 1 of the present invention, the Mask RCNN algorithm or other deep learning models can also be used to combine the target detection in step 1.1 and the instance segmentation in step 1.2 to directly extract the target image of the original visual two-dimensional digital image.

[0051] Through step 1 of the present invention, the objects in the original visual two-dimensional digital image are recognized and separated, and then displayed separately. Each time a visually impaired person touches and reads the tactile image, it is a single object, which reduces the memory requirement and recognition difficulty, and at the same time reduces the resolution requirement for the tactile image display.

[0052] Step 2: Perform edge detection on each target image to obtain a contour binary image of the target image.

[0053] The purpose of edge detection is to identify the points with obvious brightness changes in the digital image. The edge detection in step 2 of the present invention includes the following two sub-steps:

[0054] Step 2.1: Perform grayscale processing on the target image to obtain a grayscale image;

[0055] Step 2.2: Use the Canny operator to perform edge detection on the grayscale image to obtain a contour binary image of the target image. In this contour binary image, the pixel values of the object contour points are 0 (black), and the pixel values of the remaining points are 1 (white).

[0056] Figure 4 This is the edge detection effect diagram in the embodiment of the present invention. Refer to Figure 4 As shown, in step 2.2 of the present invention, when using the Canny operator for edge detection, weak edges can be suppressed by adjusting the first and second gradient thresholds, so as to more accurately represent the actual edges of the object contours in the target image.

[0057] Step 3: Correct the binary contour map based on the perspective discrimination criterion. Specifically, this Step 3 includes the following steps:

[0058] Step 3.1: Extract the straight lines of the binary contour map through the Hough transform, and determine whether there are corner points of a cuboid-like skew in the straight-line contour constructed by all the straight lines. If so, it is determined that there is a perspective effect, and Step 3.2 is executed; otherwise, there is no perspective effect, and the binary contour map of Step 2 is retained.

[0059] Step 3.2: Seek the polygon with the largest area formed by the straight lines and perform an affine transformation, and use the obtained affine transformation map as the binary contour map. Here, the polygon to be found is the one with the largest area and no lines inside.

[0060] In Step 3.1 of the present invention, in the two-dimensional image coordinate system, the Hough transform will draw all the straight lines exceeding the set length, and at the same time, the function expression of the straight line in the coordinate system can be obtained; in Step 3.1, determining whether there are corner points of a cuboid-like skew includes the following sub-steps:

[0061] Step 3.1.1: Determine the intersection points of all at least three straight lines through the function expressions of the straight lines.

[0062] Step 3.1.2: Determine whether there is a straight line connecting any two points through the function expression of the straight line. If so, the straight line connecting the two intersection points is used as the connection line, and the two intersection points connected by the connection line are used as the suspicious corner point pair. Here, the suspicious corner point pair includes two suspicious corner points; otherwise, it ends.

[0063] Step 3.1.3: Perform the following calculations for all the suspicious points respectively: Calculate the angles formed by the connection line and the other straight lines passing through the suspicious corner points, and determine whether the two angles on the same side of the connection line are complementary. If so, the suspicious corner point is a corner point of a cuboid-like skew; otherwise, it is not a corner point of a cuboid-like skew.

[0064] In the determination step of the corner points of the cuboid-like skew, a certain range of errors is allowed for calculating the intersection of points and the angles of the same-side angles.

[0065] In Step 3 of the present invention, it is determined whether the binary contour map has a perspective effect by setting a perspective discrimination criterion, and an inverse perspective transformation (affine transformation) is performed on the binary contour map with a perspective effect. The obtained affine transformation map is the corrected binary contour map.

[0066] Figure 6 It represents the perspective effect presented by the straight lines of the extracted binary contour map on the two-dimensional plane. Figure 6 2 in it represents Figure 6 The affine transformation map obtained after the affine transformation of 1 in it.

[0067] Step 4: Denoise the binary contour map obtained in Step 3. In Step 4 of the present invention, if the binary contour map obtained in Step 2 has a perspective effect, the binary contour map obtained in Step 3 is the affine transformation map. If the binary contour map obtained in Step 2 is determined to have no perspective effect according to the perspective discrimination criterion in Step 3, the binary contour map obtained in Step 3 is the binary contour map of Step 2.

[0068] Figure 8 is the denoising flowchart of the present invention. Referring to Figure 8 , the denoising of the binary contour map specifically includes the following 4 sub-steps:

[0069] Step 4.1: Set the path length threshold RL T , number the pixels of the binary contour map in the order from left to right and from top to bottom, and initialize the current pixel number α = 0;

[0070] Step 4.2: Increment the current pixel number value by 1, and determine whether the current pixel number is less than the maximum number of the binary contour map. If so, execute Step 4.3; otherwise, end.

[0071] Step 4.3: Determine whether the pixel value of the current pixel α is 0 (black). If so, execute Step 4.4; otherwise, return to Step 4.2.

[0072] Step 4.4: Determine all the paths connected to the current pixel M and the length L of each path, and determine whether the length of each path is less than the path length threshold RL T . If so, set the pixel values of all pixels in this path to 1 (white); otherwise, return to Step 4.2.

[0073] In Step 4.2 of the present invention, the maximum number of the binary contour map is the number of the last pixel in the binary contour map, and the pixel traversal condition of the binary contour map is restricted by comparing the current pixel number with the maximum number of the binary contour map.

[0074] In Step 4.4 of the present invention, "M-connected" is a common concept in digital image processing. Specifically, for pixels p and q with gray values in set V, if the following two conditions are met:

[0075] 1. q is in the 4-neighborhood of p, or

[0076] 2. q is in the D-neighborhood of p, and the intersection of the 4-neighborhood of p and the 4-neighborhood of q is empty (that is, there is no pixel point with a gray value in set V);

[0077] then these two pixels are said to be M-connected.

[0078] The denoising effect diagram obtained in Steps 4.1 - 4.4 of the present invention is shown in Figure 5As shown, by calculating the paths and path lengths connected to each object contour pixel (black pixel) M, and judging whether to update the black pixel to white based on the path length threshold; the present invention obtains a contour binary image after removing isolated points and short lines by performing digital image morphological processing on the contour binary image.

[0079] The present invention simplifies the target image through operations such as edge detection in step 2, perspective discrimination criterion in step 3, and denoising processing in step 4, removes features such as perspective and texture in the visual image, highlights the contour features, and is more helpful for visually impaired people to recognize and understand tactile images.

[0080] Step 5: Build an image pyramid for the denoised contour binary image, and select the best-resolution binary image in the image pyramid based on a preset comprehensive evaluation strategy.

[0081] In step 5 of the present invention, building an image pyramid means successively reducing the width and height to half of the original. When the width or height of the binary image is less than the set minimum limit, stop reducing. Accordingly, a set of binary images with the width and height reduced by 2 n times is obtained, where n = 1, 2, 3.... Refer to Figure 7 As shown, the resolutions of each layer of the image pyramid in the figure are 210×359, 105×179, 52×89, and 26×44 respectively.

[0082] Figure 9 is the algorithm flow chart for building the image pyramid of the present invention. Refer to Figure 9 , specifically, this step 5 specifically includes the following sub-steps:

[0083] Step 5.1: Take the denoised contour binary image as the first-layer pyramid image, set a width threshold or a height threshold, and execute step 5.2;

[0084] Step 5.2: Set the first-layer pyramid image as the current layer;

[0085] Step 5.3: According to the current-layer pyramid image, generate the next-layer pyramid image using the pyramid image generation formula. The pyramid image generation formula is

[0086]

[0087] where p n (i, j) is the pixel value at the i-th row and j-th column in the n-th layer pyramid image of the image pyramid, n = 2, 3,...;

[0088] Condition A is p n-1 (2i, 2j) = 0, p n-1 (2i, 2j + 1) = 0, p n-1 (2i + 1, 2j) = 0 or pn-1 (2i + 1, 2j + 1) = 0;

[0089] Step 5.4, determine whether the width or height of the newly generated pyramid image is less than the width threshold or the height threshold. If so, execute Step 5.5; otherwise, use the newly generated pyramid image as the current layer and return to Step 5.3.

[0090] Step 5.5, calculate the comprehensive score for each generated pyramid image using the comprehensive score calculation method, and select the pyramid image with the highest comprehensive score as the binary image with the best resolution.

[0091] The calculation of the comprehensive score mainly considers three aspects: the consistency with the image of the first layer of the pyramid, the consistency with the resolution of the tactile image display, and the tactile reading efficiency. The consistency with the image of the first layer of the pyramid reflects the similarity between the reduced image and the original image, ensuring the quality of the reduced image; the consistency with the resolution of the tactile image display is to ensure the utilization efficiency of the tactile image display and improve the display effect; the tactile reading efficiency takes into account that the memory of touch is much less than that of vision, so the tactile image should be as simple and small-sized as possible.

[0092] The calculation method of the comprehensive score is

[0093]

[0094] where a, b, and c are all weight coefficients, and a + b + c = 1, S n is the image consistency score between the nth layer pyramid image and the first layer pyramid image, is the normalized image consistency score, R n is the resolution consistency score between the nth layer pyramid image resolution and the tactile image display resolution, is the normalized resolution consistency score, E n is the tactile reading efficiency score of the nth layer pyramid image, is the normalized tactile reading efficiency score,

[0095] S n The calculation method is:

[0096]

[0097] In the formula, A 1 is the total number of pixels of the first layer pyramid image (denoised contour binary image), F 1 is the number of pixels with pixel value 0 in the first layer pyramid image (denoised contour binary image), A n is the total number of pixels of the nth layer pyramid image, F nis the number of pixels with pixel value 0 in the nth layer pyramid image.

[0098] R n is calculated as follows:

[0099]

[0100] In the formula, w d is the width of the tactile graphics display (i.e., the number of contacts), h d is the height of the tactile graphics display (i.e., the number of contacts), w n is the width of the nth layer pyramid image, h n is the height of the nth layer pyramid image.

[0101] E n is calculated as follows:

[0102]

[0103] In the formula, F n is the number of pixels with pixel value 0 in the nth layer pyramid image.

[0104] According to the calculation method of the comprehensive score, it is also necessary to calculate S n 、R n 、E n normalized values The calculation method is min-max normalization. Taking the calculation of as an example:

[0105]

[0106] Among them, S min is the minimum value in S n , and S max is the maximum value in S n .

[0107] In step 5 of the present invention, by constructing an image pyramid, the contour binary image is gradually reduced, and at the same time, a termination condition for reduction (width threshold or height threshold) is set to avoid excessive distortion of the pyramid image caused by excessive reduction, which can reduce the display resolution of the tactile image and display it completely on a tactile image display with a generally small resolution as much as possible; by selecting the pyramid graph with the highest comprehensive score as the binary image with the best resolution, the display effect of the mapped tactile image on the tactile image display is ensured.

[0108] Step 6, map the binary image with the best resolution to a dot matrix tactile image. The mapping method is: for the pixels with pixel value 1 in the binary image with the best resolution, it means that the contacts rise in the dot matrix tactile image; for the pixels with pixel value 0 in the binary image with the best resolution, it means that the contacts fall in the dot matrix tactile image.

[0109] This method is simple and clear, and visually impaired people generally have experience in using Braille dots, so it is easier for them to get started with dot matrix tactile images in a similar form.

[0110] Step 7: Determine whether the resolution of the dot matrix tactile image is greater than the resolution of the tactile display. If so, display the dot matrix tactile image in a window sliding manner; otherwise, center the dot matrix tactile image on the tactile display.

[0111] In the said step 7, the dot matrix tactile image is displayed in a window sliding manner as Figure 10 shown. In this Figure 10 , 1 is the dot matrix tactile image; Figure 10 , 2 is a window with the same resolution as the tactile image display; Figure 10 , 3 is the tactile image display; Figure 10 , 4 is a button. The window with the same resolution as the tactile image display aligns with a part of the dot matrix tactile image, and the dot matrix tactile image shown on the tactile display is the part within the window frame; the window is moved through a control device such as a button or a joystick to select which part of the dot matrix tactile image to display.

[0112] Through step 7, the present invention adopts a sliding window method to ensure the complete reading of the tactile image when the tactile image resolution is greater than the resolution of the tactile image display, and to a certain extent solves the contradiction between a tactile image with a large resolution and a tactile display with a small resolution.

[0113] The present invention comprehensively considers the problems existing in the generation and display process of tactile images. Based on the reality of the low resolution of tactile image displays, it can solve the problem that the current conversion from visual images to tactile images is too simple and rough, and achieve the goal of presenting tactile images that are easy to recognize and have a good tactile reading experience to visually impaired people.

Claims

1. A method for generating and displaying a tactile image, characterized in that, the method comprises the following steps: Step 1, extracting a target image from the acquired original visual two-dimensional digital image; Step 2, performing edge detection on the target image to obtain a binary contour map of the target image; Step 3, correcting the binary contour map based on a perspective discrimination criterion; Step 4, denoising the binary contour map obtained in Step 3; Step 5, building an image pyramid for the denoised binary contour map, and selecting the binary map with the best resolution in the image pyramid based on a preset comprehensive evaluation strategy; Step 6, mapping the binary map with the best resolution to a tactile image; Step 7, determining whether the resolution of the tactile image is greater than the resolution of the tactile display. If so, the tactile image is displayed in a window sliding manner, otherwise the tactile image is centered and displayed on the tactile display; In the image pyramid, each layer of pyramid image is generated by halving the width and height of the previous layer of pyramid image, and the first layer of pyramid image is the denoised binary contour map; The comprehensive evaluation strategy is to select the layer of pyramid image with the highest comprehensive score as the best display resolution image, and the calculation formula of the comprehensive score Sc is a, b, and c are all weight coefficients, and a + b + c = 1, S n is the image consistency score between the nth layer pyramid image and the first layer pyramid image, is the normalized image consistency score, R n is the resolution consistency score between the nth layer pyramid image and the resolution of the tactile image display, is the normalized resolution consistency score, E n is the tactile reading efficiency score of the nth layer pyramid image, is the normalized tactile reading efficiency score.

2. A method for generating and displaying a tactile image according to claim 1, characterized in that, Step 1 includes the following steps: Step 1.1, acquiring the original visual two-dimensional digital image and performing target detection to obtain a target block diagram; Step 1.2, performing instance segmentation on the target block diagram to obtain a target image.

3. A method for generating and displaying a tactile image according to claim 1, characterized in that, The perspective discrimination criterion is: determining whether the binary contour map has a perspective effect. If so, performing an inverse perspective transformation on the binary contour map, otherwise retaining the binary contour map.

4. A method for generating and displaying a tactile image according to claim 3, characterized in that, Determining whether the binary contour map has a perspective effect is: extracting the straight lines of the binary contour map through the Hough transform, and determining whether there are corner points of a cuboid-like skew in the straight line contour constructed by all the straight lines. If so, it is determined that there is a perspective effect, otherwise there is no perspective effect; the inverse perspective transformation is to find the polygon with the largest area formed by the straight lines and perform an affine transformation, and use the obtained affine transformation image as the binary contour map.

5. A method for generating and displaying a tactile image according to claim 4, characterized in that, The determination method of the corner points of the cuboid-like skew includes the following steps: Obtaining the intersection points of three lines based on the function expressions of the lines; Determining whether there is a straight line connecting any two intersection points. If so, taking the straight line as a connecting line and the two intersection points connected by the connecting line as a pair of suspicious corner points, otherwise ending; Calculating the angle formed by the connecting line and the straight line passing through the suspicious corner points, and determining whether the two included angles on the same side of the connecting line are complementary. If so, the suspicious corner points are the corner points of the cuboid-like skew, otherwise, the suspicious corner points are not the corner points of the cuboid-like skew.

6. A method for generating and displaying a tactile image according to claim 1, characterized in that, The denoising of the binary contour map includes the following steps: Step 4.1, setting a path length threshold; Step 4.2: Traverse the contour binary image to obtain all the paths connected by the contour pixels M. Step 4.3: Determine whether the path length is less than the path length threshold. If so, change each contour pixel in the path to a non - contour pixel; otherwise, keep it unchanged.

7. A method for generating and displaying a tactile image according to claim 1, characterized in that, in the calculation of the comprehensive score, the calculation formula for the image consistency score is the calculation formula for the resolution consistency score is the calculation formula for the touch - reading efficiency score is Among them, A 1 is the total number of pixels of the first-layer pyramid image, F 1 is the number of pixels with a pixel value of 0 in the first-layer pyramid image, A n is the total number of pixels of the nth-layer pyramid image, F n is the number of pixels with a pixel value of 0 in the nth-layer pyramid image, w d is the width of the tactile graphics display, h d is the height of the tactile graphics display, w n is the width of the nth-layer pyramid image, h n is the height of the nth-layer pyramid image.

8. A method for generating and displaying a tactile image according to claim 1, characterized in that, the mapping method is that the contour pixels in the optimal resolution binary image represent the rise of the contact points in the tactile image; the non - contour pixels in the optimal resolution binary image represent the fall of the contact points in the tactile image.

9. A method for generating and displaying a tactile image according to claim 1, characterized in that, in step 7, the method of displaying the tactile image by window sliding is to move the window through the control device to select any local image in the tactile image for display, and the resolution of the local image is the same as the resolution of the tactile image display.

Images