A high-precision method for segmentation and recognition of shopping mall floor plans

Abstract

The invention discloses a high-precision method for segmenting and identifying a shopping mall floor plan, comprising the following steps: A) obtaining a shopping mall floor plan including a shopping mall floor plan and a directory text corresponding to a room number and a room name; B) preprocessing the directory text to obtain each column of text images to be identified, and sending it to a text recognition system to obtain a directory dictionary corresponding to the room name and the room number; C) performing layout map preprocessing on the shopping mall floor plan, and then segmenting it based on a region growing algorithm to obtain multiple growth regions, and judging each growth region to be a room region or an aisle region; D) identifying the room number in the room region, inputting the room number into the directory dictionary, obtaining the room name corresponding to the room number, thereby obtaining the location information and room name information of the room, and replacing the room number in the room region with the corresponding room name, and marking each room position by color. The invention has high recognition accuracy and fast recognition speed.

Description

Technical Field

[0001] The invention relates to image digital processing, and in particular to a high-precision method for segmenting and identifying a shopping mall floor plan. Background Art

[0002] Navigation technology can accurately guide people or smart devices in unfamiliar places. Outdoor navigation technology has made amazing progress in the past decade, but consumers are more likely to get lost in large enclosed buildings such as stadiums, educational institutions and shopping malls, so indoor navigation is particularly important.

[0003] The floor plan represents the top view of a house or building and its dimensional relationships, while the segmentation and recognition of large indoor floor plans, as a preprocessing step for indoor navigation, determines the accuracy of the navigation system.

[0004] In the prior art, scanning data is mostly used to establish a three-dimensional structure diagram of the interior, or a two-dimensional wall distribution map, so that the plane structure of the building can be understood according to the arrangement of the walls. This method is difficult to implement, time-consuming, and difficult to accurately realize the identification of rooms and corridors, and automatically identify the name corresponding to each room. Therefore, it is difficult to apply to the identification of the plane structure of a shopping mall, and it is difficult to achieve accurate navigation in the shopping mall.

[0005] In view of this, it is necessary to provide a high-precision method for segmenting and identifying shopping mall floor plans. Summary of the invention

[0006] The technical problem to be solved by the present invention is to provide a high-precision method for segmenting and identifying a shopping mall floor plan, which has high recognition accuracy and fast recognition speed, and is convenient for realizing accurate navigation in the shopping mall.

[0007] In order to solve the above technical problems, the present invention provides a high-precision method for segmenting and identifying a shopping mall floor plan, comprising the following steps: A) obtaining a shopping mall floor plan including a shopping mall floor plan and a directory text corresponding to room numbers and room names; B) performing directory text preprocessing on the directory text to obtain a text image for each column to be identified, and sending the text image to a text recognition system to obtain a directory dictionary corresponding to the room name and the room number; C) performing layout map preprocessing on the shopping mall floor plan, and then segmenting the preprocessed shopping mall floor plan based on a region growing algorithm to obtain a plurality of growth regions, and determining whether each of the growth regions is a room region or an aisle region; D) performing post-processing on each of the growth regions, and then sending the post-processing to a text recognition system to identify the room number in the room region, and retrieving the room number in the directory dictionary to obtain the room name corresponding to the room number, thereby obtaining the location information and room name information of the room, and replacing the room number in the room region with the corresponding room name, and marking each room position by color.

[0008] Specifically, the shopping mall floor plan in step A) is a photographed picture or a digital image drawn by the shopping mall officials.

[0009] More specifically, the directory text preprocessing includes: correcting tilted text, and manually dividing the room number and the room name to obtain a room number image to be identified and a room name image to be identified, respectively.

[0010] More specifically, the text recognition system automatically identifies the room number image and the room name image respectively to obtain the position and semantic information of each character string, and sets an adaptive range window based on the center point coordinates of the room number character string position. When the center coordinates of the room name character string position are within the window, the room number can be matched with the room name in the same row to obtain the directory dictionary.

[0011] Specifically, the layout map preprocessing in step C) includes: correcting the tilted image; performing grayscale changes on the shopping mall floor plan to obtain a grayscale map, and using adaptive threshold segmentation to obtain a threshold segmentation binary map of the shopping mall floor plan; setting a grayscale threshold for the shopping mall floor plan, finding the exposure points in the shopping mall floor plan, and obtaining an exposure point binary map; performing an RGB separation operation on the shopping mall floor plan to separate the R, G, and B color channels, and performing a canny operator edge detection operation on each channel to obtain a binary map of each of the channels, and merging the binary maps of each channel to obtain an edge binary map; then merging the threshold segmentation binary map, the exposure point binary map, and the edge binary map to obtain a merged binary map, and finally performing a morphological dilation operation on the merged binary map to obtain a preprocessing result map.

[0012] More specifically, the region growing algorithm in step C) is to attach pixels with the same pixel value to each starting seed point based on the growth criterion for each pixel in each room in the floor plan of the shopping mall and continue to grow outward to obtain the room position; the starting seed point of the region growing algorithm is selected as follows: (1) the growth map records the grown area in the preprocessing result map, and the pixel points in the growth map corresponding to the grown area are assigned a value of 1, and the pixel points in the non-grown area are assigned a value of 0; (2) the preprocessing result map is globally scanned from top to bottom and from left to right, and the pixel point with a pixel value of 0 is selected. If the pixel values ​​in its adaptive window neighborhood are all 0, and the pixel value of the corresponding pixel point in the growth map is 0, then the pixel point is a valid starting seed point.

[0013] More specifically, the post-processing process of the growth area of ​​the room includes: taking the minimum rectangular frame of the growth area, performing regional growth on the edge of the growth area, and finally inverting it to obtain a post-processing result image of the character string pixels and the growth area, and then projecting the post-processing result image to an RGB color image.

[0014] More specifically, the text recognition system automatically recognizes the color image to be recognized, obtains the room number of each room, and matches it with the matching text. The room position is marked by color, and the successfully matched room name is marked on the left side of the corresponding room position.

[0015] More specifically, if there is still an unidentified room area after step D), the region growing algorithm is executed on the part corresponding to the room area on the preprocessing result image for secondary identification, and the room number in the room area is identified by adjusting the window threshold size to enable identification of the room area.

[0016] Through the above technical scheme, the high-precision method for segmentation and recognition of the shopping mall floor plan of the present invention can obtain the room number and the room name information corresponding to the room number according to the directory text corresponding to the room number and the room name in the shopping mall floor plan, and form a directory dictionary corresponding to the room name and the room number; because the color of the room in the shopping mall floor plan is usually different from the color of the wall, and the color of the aisle is usually different from the color of the room and the color of the wall, therefore, by segmenting the shopping mall floor plan based on the growth algorithm, the shopping mall floor plan can be divided according to the color difference, so that the aisle area and the room area can be obtained, and then according to the room number in the room area and combined with the directory dictionary, the room number in the room area is replaced with the corresponding room name, so that the location of each store or toilet stairwell in the shopping mall can be known. This method is implemented based on the shopping mall floor plan, so that the original data image is easy to obtain, and the processing steps are few and the purpose is strong, so that the recognition result is more accurate and the recognition speed is faster.

[0017] Other features and advantages of the present invention will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a principle diagram of the high-precision method for segmenting and identifying a shopping mall floor plan of the present invention;

[0019] Figure 2 It is a schematic diagram of catalog text preprocessing in the high-precision method for segmenting and identifying a shopping mall floor plan of the present invention;

[0020] Figure 3 It is a schematic diagram of the principle of determining the same row based on the adaptive range window Ω in the high-precision method for segmenting and identifying the shopping mall floor plan of the present invention;

[0021] Figure 4 It is a schematic diagram of layout diagram preprocessing in the high-precision method for segmenting and identifying a shopping mall floor plan of the present invention;

[0022] Figure 5 It is a schematic diagram of the determination principle of room areas and aisle areas of the high-precision method for segmenting and identifying a shopping mall floor plan of the present invention;

[0023] Figure 6 It is a schematic diagram of post-processing in the high-precision method for segmenting and identifying a shopping mall floor plan of the present invention;

[0024] Figure 7 It is a schematic diagram of the result of identifying marks in the high-precision method for segmenting and identifying a shopping mall floor plan of the present invention;

[0025] Figure 8 It is a schematic diagram of characters and edges connected in the high-precision method for segmenting and recognizing a shopping mall plan of the present invention;

[0026] Fig. 9 It is a schematic diagram of characters being clustered in the high-precision method for segmenting and recognizing a shopping mall floor plan of the present invention;

[0027] Fig.10 It is a preprocessing diagram for the second round of segmentation and recognition in the high-precision method for segmentation and recognition of a shopping mall floor plan of the present invention;

[0028] Fig.11 It is the original shopping mall floor plan layout diagram in the high-precision method for segmenting and identifying shopping mall floor plans of the present invention;

[0029] Fig.12 It is the final recognition and matching result diagram of the high-precision method for segmenting and recognizing the shopping mall floor plan of the present invention;

[0030] Fig.13 It is a digital image drawn in the high-precision method for segmenting and identifying a shopping mall floor plan of the present invention;

[0031] Fig.14 It is a recognition matching result diagram obtained from a drawn digital image in the high-precision method for segmenting and recognizing a shopping mall floor plan of the present invention. DETAILED DESCRIPTION

[0032] The specific implementation of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present invention, and is not used to limit the present invention.

[0033] like Figure 1 As shown, in an embodiment of the high-precision method for segmenting and identifying a shopping mall floor plan provided by the present invention, the method includes the following steps: A) obtaining a shopping mall floor plan including a shopping mall floor plan and a directory text corresponding to room numbers and room names; B) performing directory text preprocessing on the directory text to obtain a text image for each column to be identified, and sending the text image to a text recognition system to obtain a directory dictionary corresponding to room names and room numbers; C) performing layout map preprocessing on the shopping mall floor plan, and then segmenting the preprocessed shopping mall floor plan based on a region growing algorithm to obtain multiple growth regions, and determining whether each growth region is a room region or an aisle region; D) post-processing each growth region first, and then sending it to a text recognition system to identify the room number in the room region, and retrieving the room number in the directory dictionary to obtain the room name corresponding to the room number, thereby obtaining the location information and room name information of the room, and replacing the room number in the room region with the corresponding room name, and marking each room position with a color.

[0034] Through the above technical scheme, the high-precision method for segmentation and recognition of the shopping mall floor plan of the present invention can obtain the room number and the room name information corresponding to the room number according to the directory text corresponding to the room number and the room name in the shopping mall floor plan, and form a directory dictionary corresponding to the room name and the room number; because the color of the room in the shopping mall floor plan is usually different from the color of the wall, and the color of the aisle is usually different from the color of the room and the color of the wall, therefore, by segmenting the shopping mall floor plan based on the growth algorithm, the shopping mall floor plan can be divided according to the color difference, so that the aisle area and the room area can be obtained, and then according to the room number in the room area and combined with the directory dictionary, the room number in the room area is replaced with the corresponding room name, so that the location of each store or toilet stairwell in the shopping mall can be known. This method is implemented based on the shopping mall floor plan, so that the original data image is easy to obtain, and the processing steps are few and the purpose is strong, so that the recognition result is more accurate and the recognition speed is faster.

[0035] Specifically, in one embodiment of the high-precision method for segmenting and identifying a shopping mall floor plan provided by the present invention, the shopping mall floor plan in step A) is a photographed picture (i.e., obtained by photographing the map signs in the shopping mall) or a digital image drawn by the shopping mall officials (i.e., the original map signs drawn in a computer). Since there will be reflections when photographing map signs in a shopping mall environment, some lines on the shopping mall floor plan are blocked by reflective points, making it difficult to perform image processing and easily causing the phenomenon of incorrect identification. Therefore, the drawn digital image should be preferably used as the shopping mall floor plan.

[0036] However, since digital images are difficult to obtain, Figure 2 Taking the picture shown in (a) as an example of a shopping mall floor plan, the specific steps of a high-precision method for segmenting and recognizing a shopping mall floor plan are introduced in detail.

[0037] First, the catalog text is processed. Specifically, since the content in the captured image will inevitably have a certain degree of inclination, the catalog text preprocessing for the catalog text includes:

[0038] B1) Correcting tilted text. Specifically, perspective transformation can be used to correct the text. That is, firstly, the four corner coordinates of a column of the directory text to be recognized are manually provided as the source array pts of the perspective correction transformation matrix M, and then the minimum contour search is performed on pts to obtain the target array dst. The perspective correction transformation matrix M is calculated using pts and dst, and then the perspective correction transformation matrix M is used to obtain the perspective correction transformation result f. The corrected text is as follows: Figure 2 (b)

[0039] B2) Manually divide the room number and the room name to obtain the room number image to be identified and the room name image to be identified. Specifically, since the room name and the room number in the directory text are close to each other, the text recognition system will recognize them as one string by default, such as Figure 2 As shown in (c), it is necessary to draw a track between the room name and the room number and separate the perspective correction transformation result f into the room number image f to be identified according to this track. l and the room name image to be identified r , and send the two pictures to the text recognition system for recognition.

[0040] In the process of text recognition, Figure 2 As shown in (d), the text recognition system recognizes the room number image f l And room name picture f r Automatically identify the location and semantic information of each string and Figure 3 As shown, based on the center point coordinates (x l ,y l ), set the adaptive range window Ω to get the room number image f l Room name picture f r Specifically, since the room name and the room number are generally located in the same line in the directory text, they can be matched according to the horizontal relationship of their corresponding coordinates. However, due to the manual correction in the preprocessing stage, there is a certain error and they cannot be completely parallel. Therefore, the room number image f l The center coordinates of the string image of each room number character (x l ,y l ) as the reference point, and set the adaptive range window Ω to match the room name string whose center coordinates of the string image are within the adaptive range window Ω with the corresponding room number string, that is, when the y in the center coordinates of the room name string image is r satisfy Each text line is matched and returns a key-value pair with the room number as the key and the room name as the value. The key-value pairs returned by each line of all directories are collected into a directory dictionary as matching text to assist in the matching of shopping mall floor plans.

[0041] The pseudo code of the text recognition process is as follows:

[0042] Input: F0 (M×N×3);

[0043] Output:text;

[0044] 1:pts=selectCoordinate(F0);

[0045] 2: dst = boundingRect(pst);

[0046] 3:M=getPerspectiveTransform(pts,dst);

[0047] 4:f(m×n×3)=warpPerspective(F0,M);

[0048] 5:foreach f do;

[0049] 6: line = getLinePts(f);

[0050] 7: for j∈f do;

[0051] 8:f l ←i<linepts;

[0052] 9:f r ←i>linepts;

[0053] 10:test l , test r ←f l , f r do OCR;

[0054] 11:test=match(test l , test r ).

[0055] Subsequently, the plan layout diagram F0 of the shopping mall is processed. Since the content in the photographed image inevitably has a certain degree of inclination, the layout diagram preprocessing of the plan layout diagram F0 of the shopping mall includes: correcting the inclined image. The specific method is consistent with the method of correcting the inclined text mentioned above, so it will not be repeated here; subsequently, the plan layout diagram F0 of the shopping mall is gray-scaled to obtain a gray-scale image, and adaptive threshold segmentation is used to obtain the threshold segmentation binary image F0 of the plan layout diagram F0 of the shopping mall g1 , the adaptive threshold is to take each pixel of the image as the center to take the adaptive range window Ω, and then calculate the adaptive threshold of the adaptive range window Ω, where, When T is larger, the area involved in calculating the adaptive threshold is also larger, the detailed contour becomes less, and the overall boundary contour is thicker and more obvious, that is, the thickness of the boundary or font can be adjusted according to the adaptive threshold; the grayscale threshold is set for the mall floor plan, and the exposure point in the mall floor plan is found to obtain the exposure point binary map F g2 ,like Figure 4As shown in (a), the exposure point will form a breakpoint on the boundary. This kind of breakpoint has a large local range and the grayscale value at the exposure point is generally very high. Setting a suitable threshold δ for threshold segmentation can easily find the exposure point. In addition, due to the influence of light, the grayscale binary image F g1 The grayscale change is not obvious at some boundaries, and there are also small breakpoints, such as Figure 4 As shown in (b), these large and small breakpoints will cause overgrowth during region growth. Therefore, it is necessary to perform RGB separation on the mall floor plan F0 to separate F R 、F G and F B Three channels (i.e. R, G, B color channels), and then perform canny operator edge detection operation on the three channels respectively to obtain F Rc 、F Gc and F Bc Three binary images are taken, and the binary images of each channel are merged to obtain the edge binary image F g3 Finally, the threshold segmentation binary image F g1 , exposure point binary image F g2 and edge binary graph F g3 The preliminary preprocessing result graph F′1 is obtained by merging.

[0056] However, after the above processing, there will still be a small number of weak breakpoints in the preprocessing result image F′1. Therefore, the preliminary preprocessing result image F′1 can be further expanded using a 3x3 symmetrical structural element S to connect the breakpoints and obtain the final preprocessing result image F1. The above preprocessing can speed up the region growing algorithm through grayscale, and the processing of exposure points and breakpoints can improve the accuracy of segmentation.

[0057] The pseudo code of the above preprocessing process is as follows:

[0058] Input: F0 (m×n×3);

[0059] Output: F1 (m×n);

[0060] 1:F g ←F0do graying;

[0061] 2:for(i,j)∈F g1 =F g do;

[0062] 3:

[0063] 4:F g1 (i, j)do Adaptive Threshold inΩ;

[0064] 5:end for;

[0065] 6:for(i,j)∈F g1 =F g do;

[0066] 7:F g1 (i, j) = 1 if F g1 (i, j)>thresholdδ;

[0067] 8:F g1 (i, j) = 0 if F g1 (i, j) <thresholdδ;

[0068] 9:end for;

[0069] 10:F R , F G , F B ←F O do hierarchy;

[0070] 11:F Rc , F Gc , F Bc ←F R , F G , F B do canny operator;

[0071] 12:F g3 =F Rc +F Gc +F Bc ;

[0072] 13:F′1=F g1 +F g2 +F g3 ;

[0073] 14:

[0074] The preprocessing result image F1 is segmented using the region growing algorithm. The region growing algorithm is to attach similar pixels to each starting seed point based on the growth criterion and continue to grow outward. The starting seed point is selected by global scanning to automatically select the starting seed point. Specifically, first create a new growth image F growed (M×N) records the grown regions in the preprocessing result graph F1. To avoid the initial seed point being selected within the grown region R, each time a grown region R is grown, let F growed (pixel) = 1, pixel∈R. Then, each point (i, j) of the preprocessing result graph F1 (M×N) is globally scanned from top to bottom and from left to right: Fgrowed (i, j) = 0 and F1(pixel) = 1, pixel∈Ω, where Ω represents the neighborhood of point (i, j) in the adaptive window, for example, the 24-neighborhood of point (i, j), that is, the pixel points in the grown area are assigned 1, and the pixel points in the ungrown area are assigned 0; select the pixel point (i, j) assigned 0, if all the pixel points in the neighborhood of the corresponding adaptive window of point (i, j) are assigned 0, then the pixel point (i, j) is a valid starting seed point. Since the preprocessing result image F1 is a binary image, in order to speed up the algorithm, it is only necessary to determine whether the 8-neighborhood pixel of point (i, j) is the same as the seed point.

[0075] Since there are aisles that account for a large proportion of the mall floor plan, and the aisles are not the areas we need, it is necessary to determine whether the resulting growth area is a room area or an aisle area. The judgment process of the room area and the aisle area includes: taking the minimum rectangular frame of the growth area, and comparing the length and width of the minimum rectangular frame with the length and width of the preprocessing result map, and setting the ratio as a judgment threshold. If it is greater than the judgment threshold, the growth area is an aisle area. If it is less than the judgment threshold, the growth area is a room area. Specifically, Figure 5 As shown, after each region growing obtains a grown region R1, take its minimum rectangular box R1(m×n), determine the proportion of the minimum rectangular box R1(m×n) in the preprocessing result image F1(M×N), and then set the judgment threshold τ. When it satisfies: When the growth area is determined to be the aisle area, it is not sent to the text recognition system. Preferably, the determination threshold τ can be determined according to the actual situation. Generally, the value can be preferably taken as 0.6. If the aisle area is not identified when this value is taken, the value of the determination threshold τ needs to be reduced. When more room areas are identified as aisle areas when this value is taken, the value of the determination threshold τ needs to be increased.

[0076] This is followed by post-processing of the individual room areas to Figure 6 Taking the grown area shown in FIG. 1 as an example, the smallest rectangular frame of the grown area is first taken to form the grown area R1 (m×n) to be identified. The grown area R1 (m×n) is determined to be the room area (i.e. Figure 6 Medium R 1(RGB) gray area within the 1(RGB) The character pixels in the gray area within are not incorporated into the room area (i.e., the white area) of the grown area R1(m×n), which will cause the text recognition system to be unable to effectively recognize the characters after projection. Therefore, it is necessary to invert the grown area R1(m×n) to obtain Perform region growth on the edge of the growth area to get R2(m×n), and then invert it to get the growth area of ​​the merged character. R(m×n) is projected onto the color image F0(M×N×3) to obtain R (RGB) , R (RGB) Then send it to the text system for recognition to obtain the room number in the room area, and then perform the aforementioned step D), enter the room number into the directory dictionary to obtain the room name corresponding to the room number, and replace the room number in the room area with the corresponding room name to achieve the correspondence between the room and the room name, thereby obtaining the room name and room position of each room. The recognized room position is marked by color, and the successfully matched room name is marked on the left side of the corresponding room position, so that the location of each store or toilet stairwell in the mall can be known more directly, so that navigation can be facilitated. It should be noted that some room number characters are vertical. If there is no recognition result, they need to be rotated 90° counterclockwise or clockwise, as shown in the following example. Figure 7 shown.

[0077] The pseudo code of the above process of region growing, region recognition and room number recognition of room region is as follows:

[0078] Input:F1(M×N), F0(M×N×3);

[0079] Output:R(m×n),F growed (M×N);

[0080] 1:F growed (M×N)=0;

[0081] 2: for (i, j) ∈ F1do;

[0082] 3:Ω={i±Δx, j±Δy|Δx, Δy=2};

[0083] 4:if F growed (i,j)=0and F1(pixel)=1,pixel∈Ωthen;

[0084] 5: (i, j) as the seed;

[0085] 6: (i, j)do region-growing to get R1(m×n);

[0086] 7:

[0087] 8:F growed (pixel)=1, pixel∈R1;

[0088] 9: continue;

[0089] 10:end if;

[0090] 11: do region-growing to get R2;

[0091] 12: R = R2;

[0092] 13:F growed (pixel)=1, pixel∈R1;

[0093] 14:R RGB =AND(R BW , F0);

[0094] 15:R RGB to OCR system;

[0095] 16:end if;

[0096] 17:if the seed is none then;

[0097] 18:break;

[0098] 19:end if;

[0099] 20: end for.

[0100] In particular, after the above steps, if there is still an unrecognized room area, the region growing algorithm is executed on the part corresponding to the room area on the preprocessing result map for secondary recognition, and the room number in the room area is recognized by adjusting the window threshold size, so as to realize the recognition of the room area. Specifically, when the character pixels are close to the region boundary, the character and the region boundary will be connected in the preprocessing, that is, Figure 8 The character "7" in "C17" is shown, or the character forms a cluster of dots, as shown in Fig. 9As shown, the text recognition system cannot effectively recognize the character content at this time. The main reason is that in order to avoid the subsequent over-growth breakpoints in the preprocessing, the adaptive window Ω is large, and the result image is expanded, which makes the characters too thick and causes this situation. This type of tight frame accounts for a small proportion of the entire image to be recognized. After a full image recognition, the remaining unsuccessful recognition area is mostly this type of tight frame. To solve this problem, the unrecognized area can be segmented and recognized twice, or even multiple rounds of segmentation and recognition can be performed. The first round of segmentation and recognition uses the aforementioned preprocessing steps. If T=25 is taken in the adaptive window Ω in the first round of segmentation and recognition, and the threshold δ in the exposure point detection is 230, then the expansion process can be canceled in the second round of segmentation and recognition, and T in the adaptive window Ω can be adjusted to a smaller value, for example, adjusted to T=3, so that the characters of the room number become relatively thin, not easy to connect with the regional boundary or become clumped and unrecognizable. To prevent repeated segmentation and re-recognition of the same area in the second round, a new image F is created. detectde-mask = 1, the first round of successfully marked growth area and aisle information corresponds to F detectde-mask (i, j) = 0, using 5x5 symmetric structure element S to transform F detectde-mask Morphological dilation processing, such as Fig.10 (b) As shown. After the second round of preprocessing of the plane graph, the expanded F detectde-mask And the preprocessing result diagram Merge like Fig.10 As shown in (c), The final preprocessing result map is used for secondary segmentation and recognition. If there are still room areas that have not been successfully identified and matched, continue to repeat the above steps until the result does not change. The original shopping mall floor plan is as follows Fig.11 As shown in the figure, the final recognition and matching results are as follows Fig.12 shown.

[0101] Assume that the total number of effective areas to be identified in the mall floor plan is Z, then Z is expressed as:

[0102] Z=TP+FP0+FP1+FN

[0103]

[0104]

[0105]

[0106] Among them, DSR represents the proportion of correctly segmented and recognized areas, ISR represents the proportion of correctly identified areas among correctly segmented areas, SSR represents the proportion of correctly segmented areas, TP represents the number of correctly segmented and correctly recognized areas (i.e., the number of areas marked with marker colors and with correct matching text in the recognition result graph); FP0 represents the number of correctly segmented but incorrectly matched areas (i.e., the number of areas marked with marker colors but with incorrect matching text in the recognition result graph); FP1 represents the number of correctly segmented but incorrectly recognized areas (i.e., areas marked with marker colors in the recognition result graph but without room number recognition); FN represents the number of areas that were not successfully segmented (i.e., areas that were not marked with marker colors in the recognition result graph).

[0107] To ensure that the algorithm speed is not affected by the image resolution, all images to be processed can be scaled using bilinear interpolation. In particular, since the text recognition system limits the input image size to max(w,h)<4096, to ensure that the aspect ratio remains unchanged, the new length (h,) and width (w') of the scaled image are calculated using the following formula:

[0108]

[0109]

[0110] The use conditions of different text recognition systems vary. For example, Baidu's text recognition system only needs to be connected to the Internet to call the corresponding server, and there is no requirement for the client configuration; while the Easyocr text recognition system needs to download the application model for recognition and detection, and uses the cuda computing platform, so that recognition can be performed directly on the client without the need for an Internet connection. When a drawn digital image is used as a shopping mall floor plan, the recognition and matching effect will be better in theory because the influence of physical factors such as lighting and tilt is reduced. The selected drawn digital image is as follows: Fig.13 As shown, the results of its recognition and matching are as follows Fig.14 As shown in, to verify the robustness to different scenarios, we can add Fig.14 The drawn digital image shown is used as the experimental object. The following experiments discuss three cases respectively.

[0111] 1. Experiments using different text recognition systems

[0112] Different text recognition systems have different recognition accuracy in different scenarios. In order to select a better text recognition system for our application scenario, we first conduct a text recognition comparison experiment to determine which text recognition system to use in the subsequent ablation experiment. Table 1 shows the results of experiments using different text recognition systems, where Dataset represents the sum of 25 test set images, both means using both the EasyOCR text recognition system and the Baidu text recognition system, and the experimental data in Table 1 is the average value of all images after the experiment.

[0113]

[0114] Table 1

[0115] As can be seen from Table 1, when using only one text recognition system, the Baidu text recognition system has higher recognition accuracy than the easyoc text recognition system, but it takes longer time, because the Baidu text recognition system needs to be connected to the Internet to call the API server for recognition, and the speed is easily affected by the network speed, while the easyocr text recognition system only needs to download the detection model and recognition model in advance and can be used locally. The model is more lightweight, but the recognition accuracy is lower than that of the Baidu text recognition system. When we use two text recognition systems at the same time, the recognition accuracy will be higher than using any one of the text recognition systems alone, and the speed is only about 60s longer than using the Baidu text recognition system alone. Therefore, the combination of the two can improve the overall recognition accuracy. Therefore, the subsequent experiments use the text system that combines the easyocr text recognition system and the Baidu text recognition system as the recognition module. The detailed experimental results of the combination of the easyocr text recognition system and the Baidu text recognition system are shown in Table 2:

[0116]

[0117] Table 2

[0118] 2. Post-processing ablation experiment

[0119] Post-processing is to take the smallest rectangular frame for each segmented area, and use edge region growth to merge the character string pixels into the growth area, and finally project the growth area to the original image. The purpose of post-processing is to allow the character string to be displayed normally on the projection image and improve the recognition accuracy of the text recognition system. In order to verify the effectiveness of post-processing of region growing, the recognition results after post-processing and without post-processing can be compared. The comparison results are shown in Table 3:

[0120]

[0121] Table 3

[0122] Experimental data show that when the post-processing step is not used, the total recognition rate ISR is less than a quarter, and the total detection rate DSR is only 22.4%. After using post-processing, the recognition accuracy is improved to 91.5%, and the system speed is only about 30 seconds slower. Therefore, the post-processing step can significantly improve the recognition accuracy and total detection rate, so the post-processing step is indispensable.

[0123] 3. Different rounds of experiments

[0124] Since the expansion operation in the first round of preprocessing and the adaptive window being too large will cause the preprocessed string pixels to be connected to the contour edge, resulting in a decrease in segmentation accuracy, a multi-round segmentation and recognition method is proposed. Each round of detection obtains the detected area recorded in each round and is combined with the preprocessing result of the next round as the output of the image to be recognized in the next round. The final recognition result is obtained by combining the recognition results of each round. The main purpose of multi-round segmentation and recognition is to improve the segmentation accuracy and thus improve the recognition rate. To verify the effectiveness of multi-round segmentation and recognition, we take the round number to be 2. Table 4 shows the results after each round of segmentation and recognition:

[0125]

[0126] Table 4

[0127] In Table 4, when only one round of segmentation and recognition is performed, the average segmentation accuracy is 72.45%, which means that one quarter of the area is still not successfully segmented and extracted, making the final total detection rate DSR only 65.86%. When the second round of segmentation and recognition is performed, the segmentation accuracy is improved to 92.7%, and the total detection rate DSR is also improved to 84.8%. The second round of segmentation and recognition is essentially to check for omissions in the first round of segmentation and recognition, and the remaining unsegmented areas are re-preprocessed and segmented, and the idea of ​​merging the recognized areas of the previous round with the preprocessing results of the next round is reflected in Table 4 as the improvement of the second round of detection speed. The experimental results show that multiple rounds of segmentation and recognition can effectively improve the segmentation accuracy and total detection rate, and in the compromise between accuracy and speed, the number of rounds generally selected is 2.

[0128] The preferred embodiments of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited thereto. Within the technical concept of the present invention, the technical solution of the present invention can be subjected to a variety of simple modifications, including the combination of various specific technical features in any suitable manner. In order to avoid unnecessary repetition, the present invention will not further describe various possible combinations. However, these simple modifications and combinations should also be regarded as the contents disclosed by the present invention and belong to the protection scope of the present invention.

Claims

1. A high-precision method for segmenting and identifying a shopping mall floor plan, characterized in that: The steps include: A) obtaining a floor plan of a shopping mall including a floor plan of the shopping mall and a directory text corresponding to room numbers and room names; B) performing directory text preprocessing on the directory text to obtain a text image for each column to be recognized, and sending the text image to a text recognition system to obtain a directory dictionary corresponding to the room name and the room number; C) performing layout preprocessing on the floor plan of the shopping mall to obtain a preprocessing result map, and then segmenting the preprocessing result map based on a region growing algorithm to obtain a plurality of growth regions, and determining whether each of the growth regions is a room region or an aisle region; The judgment process of the room area and the aisle area includes: taking the minimum rectangular frame of the growth area, and comparing the length and width of the minimum rectangular frame with the length and width of the preprocessing result image respectively, and setting a judgment threshold for the ratio, if it is greater than the judgment threshold, the growth area is the aisle area, and if it is less than the judgment threshold, the growth area is the room area; The post-processing process of the growth area of ​​the room includes: taking the minimum rectangular frame of the growth area to form a grown area to be identified, performing region growth on the edge of the area obtained by inverting the grown area, inverting the area obtained after the region growth to obtain the growth area of ​​the merged character, and then projecting the growth area of ​​the post-merged character to the RGB color map to obtain the color map to be identified; D) each of the growth areas is first post-processed, and then sent to a text recognition system to identify the room number in the room area, and the room number is retrieved in the directory dictionary to obtain the room name corresponding to the room number, thereby obtaining the location information and room name information of the room, replacing the room number in the room area with the corresponding room name, and marking each room position with a color.

2. The high-precision method for segmenting and identifying a shopping mall floor plan according to claim 1, characterized in that: The shopping mall floor plan in step A) is a photographed picture or a digital image drawn by the shopping mall official.

3. The high-precision method for segmenting and identifying a shopping mall floor plan according to claim 1, characterized in that: In step B), the directory text preprocessing includes: correcting tilted text, and manually dividing the room number and the room name to obtain a room number image to be identified and a room name image to be identified, respectively.

4. The high-precision method for segmenting and identifying a shopping mall floor plan according to claim 3 is characterized in that: The text recognition system automatically recognizes the room number image and the room name image respectively to obtain the position and semantic information of each character string, and sets an adaptive range window based on the center point coordinates of the room number character string position. When the center coordinates of the room name character string position are within the window, the room number can be matched with the room name in the same row to obtain the directory dictionary.

5. The high-precision method for segmenting and identifying a shopping mall floor plan according to claim 1, characterized in that: The layout map preprocessing in step C) includes: correcting the tilted image; performing grayscale changes on the mall floor plan to obtain a grayscale map, and using adaptive threshold segmentation to obtain a threshold segmentation binary map of the mall floor plan; setting a grayscale threshold for the mall floor plan, finding the exposure points in the mall floor plan, and obtaining an exposure point binary map; performing RGB separation operation on the mall floor plan to separate the R, G, and B color channels, and performing a canny operator edge detection operation on each channel to obtain a binary map of each channel, and merging the binary maps of each channel to obtain an edge binary map; then merging the threshold segmentation binary map, the exposure point binary map and the edge binary map to obtain a merged binary map, and finally performing a morphological expansion operation on the merged binary map to obtain a preprocessing result map.

6. The high-precision method for segmenting and identifying a shopping mall floor plan according to claim 5, characterized in that: The region growing algorithm in step C) is to attach pixels with the same pixel value to each starting seed point based on the growth criterion for each pixel in each room in the floor plan of the shopping mall and continue to grow outward to obtain the room position; the starting seed point of the region growing algorithm is selected as follows: (1) the growth map records the grown area in the preprocessing result map, and the pixel points in the growth map corresponding to the grown area are assigned a value of 1, and the pixel points in the non-grown area are assigned a value of 0; (2) the preprocessing result map is globally scanned from top to bottom and from left to right, and the pixel point with a pixel value of 0 is selected. If the pixel values ​​in its adaptive window neighborhood are all 0, and the pixel value of the corresponding pixel point in the growth map is 0, then the pixel point is a valid starting seed point.

7. The high-precision method for segmenting and identifying a shopping mall floor plan according to claim 1, characterized in that: The text recognition system automatically recognizes the color image to be recognized, obtains the room number of each room, and matches it with the matching text. The room position is marked by color, and the successfully matched room name is marked on the left side of the corresponding room position.

8. The high-precision method for segmenting and identifying a shopping mall floor plan according to claim 7, characterized in that: If there is still an unidentified room area after step D), the region growing algorithm is executed on the portion corresponding to the room area on the preprocessing result image for secondary identification, and the room number in the room area is identified by adjusting the window threshold size to enable identification of the room area.

Images