A signboard character recognition method and device

Abstract

The application relates to the technical field of image recognition, in particular to a street sign character recognition method and device, which comprises the following steps: cutting out a first sub-image containing to-be-recognized street sign characters from a target image; processing the first sub-image into a second sub-image containing only black and white; judging whether the black and white distribution area in the second sub-image meets a preset area distribution condition; if yes, recognizing the street sign characters in the second sub-image to generate a recognition result; if not, expanding the cutting area and returning to the step of cutting out the first sub-image containing to-be-recognized street sign characters from the target image; in this way, the problem of incomplete street sign character extraction can be effectively improved, so that the accuracy of street sign character extraction is improved.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and more specifically to a method and apparatus for recognizing road sign characters. Background Technology

[0002] Currently, there is a need for image recognition in many fields, such as autonomous driving, street view mapping, and dynamic perception, especially for recognizing road sign characters in images.

[0003] In existing technologies, neural learning networks are typically used to directly recognize images. However, due to factors such as shooting distance, shooting angle, and occlusion, existing methods often fail to fully recognize road sign characters, thus reducing the accuracy of recognition. Summary of the Invention

[0004] In view of this, the present invention provides a method and apparatus for recognizing road sign characters to improve the accuracy of road sign recognition.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] The first aspect of this application provides a method for recognizing road sign characters, including:

[0007] Crops out the first sub-image containing the road sign characters to be recognized from the target image;

[0008] The first sub-image is processed into a second sub-image containing only black and white;

[0009] Determine whether the black and white distribution area in the second sub-image meets the preset area distribution conditions; if yes, identify the road sign characters in the second sub-image and generate a recognition result; if no, expand the cropping area and return to the step of cropping the first sub-image containing the road sign characters to be identified in the target image.

[0010] Optional, also includes:

[0011] The first sub-image is first subjected to noise filtering, and then magnified.

[0012] Accordingly, processing the first sub-image into a second sub-image containing only black and white includes:

[0013] The first sub-image, after being magnified, is processed into a second sub-image containing only black and white.

[0014] Optionally, processing the first sub-image into a second sub-image containing only black and white includes: processing the foreground of the first sub-image to black as a foreground color region and processing the background of the first sub-image to white as a background color region; or, processing the foreground of the first sub-image to white as a foreground color region and processing the background of the first sub-image to black as a background color region.

[0015] Accordingly, determining whether the black and white distribution area in the second sub-image meets the preset region distribution conditions includes:

[0016] Determine whether the area of ​​the foreground color region in the second sub-image reaches a preset threshold, and / or determine whether the position of the foreground color region in the second sub-image is at a specified position.

[0017] Optionally, if the area of ​​the foreground color region in the second sub-image does not reach a preset threshold, before expanding the cropping area, the method further includes:

[0018] The cropping ratio is determined based on the proportion of the foreground color region in the second sub-image, and the cropping region is determined based on the cropping ratio.

[0019] Optionally, after generating the recognition result, the method further includes:

[0020] If the recognition result does not conform to the road sign marking rules, the recognition result shall be corrected.

[0021] Optionally, cropping the first sub-image containing the road sign characters to be recognized from the target image includes:

[0022] Identify a specified graphic in the target image; the specified graphic is a graphic containing the road sign characters to be identified.

[0023] A first sub-image associated with the specified graphic is cropped from the target image.

[0024] A second aspect of this application provides a road sign character recognition device, comprising:

[0025] The first cropping unit is used to crop out the first sub-image containing the road sign characters to be recognized from the target image;

[0026] The first processing unit is configured to process the first sub-image into a second sub-image containing only black and white;

[0027] The first judgment unit is used to determine whether the black and white distribution area in the second sub-image meets the preset area distribution conditions;

[0028] The first recognition unit is used to recognize road sign characters in the second sub-image and generate a recognition result when the black and white distribution area in the second sub-image meets the preset regional distribution conditions.

[0029] The first enlargement unit is used to enlarge the cropping area when the black and white distribution area in the second sub-image does not meet the preset area distribution conditions, and to control the first cropping unit to re-crop with the enlarged cropping area.

[0030] Optional, also includes:

[0031] The second processing unit is used to first perform noise filtering on the first sub-image, and then perform magnification processing.

[0032] Accordingly, the first processing unit is specifically used to process the magnified first sub-image into a second sub-image containing only black and white.

[0033] Optionally, the first processing unit includes:

[0034] The first processing module is used to process the foreground of the first sub-image into black as the foreground color area and process the background of the first sub-image into white as the background color area.

[0035] Alternatively, the second processing module is used to process the foreground in the first sub-image to white as the foreground color area and process the background of the first sub-image to black as the background color area.

[0036] Accordingly, the first determination unit includes:

[0037] The first judgment module is used to determine whether the area of ​​the foreground color region in the second sub-image reaches a preset threshold, and / or the second judgment module is used to determine whether the position of the foreground color region in the second sub-image is at a specified position.

[0038] Optionally, if the first determining module determines that the area of ​​the foreground color region in the second sub-image does not reach a preset threshold, the method further includes:

[0039] The first determining unit is used to determine the cropping ratio based on the proportion of the foreground color region in the second sub-image, and to determine the cropping region based on the cropping ratio.

[0040] Optional, also includes:

[0041] The first correction unit is used to correct the recognition result if the recognition result does not conform to the road sign marking rules.

[0042] Optionally, the first trimming unit includes:

[0043] An image determination module is used to determine a specified graphic in a target image; the specified graphic is a graphic containing road sign characters to be identified.

[0044] The first cropping module is used to crop out a first sub-image associated with the specified graphic from the target image.

[0045] As can be seen from the above technical solution, compared with the prior art, this application provides a method for recognizing road sign characters. By processing a first sub-image cropped from the target image into a second sub-image containing only black and white, it is determined whether the black and white distribution area in the second sub-image meets the preset regional distribution conditions. If it does not meet the requirements, the cropping area is expanded, and the target image is cropped again. The processing and judgment steps are then performed again on the re-cropped first sub-image. Thus, this application effectively improves the problem of incomplete road sign character extraction and increases the accuracy of road sign character extraction by processing the first sub-image and adding a judgment step, so that the cropping area can be expanded and re-identified when the requirements are not met.

[0046] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0047] The above and other objects, features and advantages of this application will become more apparent from the more detailed description of exemplary embodiments thereof in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments thereof.

[0048] Figure 1 A flowchart illustrating a method for recognizing road sign characters provided in one embodiment of this application;

[0049] Figure 2 This is an example diagram of a road sign provided in one method embodiment of this application;

[0050] Figure 3a A schematic diagram illustrating the division of a second sub-image into a four-grid layout, provided as an embodiment of a method of this application;

[0051] Figure 3b This is a schematic diagram illustrating the division of a second sub-image into a nine-square grid, as provided in one embodiment of the method of this application.

[0052] Figure 4 A flowchart illustrating a method for recognizing road sign characters provided in another embodiment of this application;

[0053] Figure 5 A flowchart illustrating a method for recognizing road sign characters, provided as another embodiment of this application;

[0054] Figure 6 A schematic diagram of the target image and the second sub-image provided for one example of this application;

[0055] Figure 7 A schematic diagram of the structure of a road sign character device provided in one embodiment of this application;

[0056] Figure 8 A schematic diagram of the structure of a road sign character device provided for another embodiment of this application;

[0057] Figure 9 This is a schematic diagram of the structure of a road sign character device provided in another embodiment of this application. Detailed Implementation

[0058] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] One embodiment of the present invention discloses a method for recognizing road sign characters, such as... Figure 1 As shown, it may include the following steps:

[0060] Step 101: Crop out the first sub-image containing the road sign characters to be recognized from the target image;

[0061] The target image is the image to be identified, which is an image containing a road sign with road sign characters to be identified. The target image can be an image captured by a camera on the road. For example, it can be an image captured by a camera mounted on a vehicle while the vehicle is in motion; more specifically, it can be an image of the road captured by a camera mounted on the windshield of a vehicle. Of course, this application is not limited to images captured by cameras mounted on vehicles; all other images containing road sign characters to be identified are within the scope of protection of this application.

[0062] The aforementioned road sign characters to be identified can refer to text or numerical markings on the road sign. Figure 2 Taking a road sign as an example, this road sign contains various types of road sign markings, namely, private car only, driving lane, direction of travel, and speed limit 100; assuming the target image contains Figure 2Regarding road signs, it should be noted that the target image may also include other features, such as vehicles and people traveling on the road, with the road sign serving as a partial feature in the target image. The first sub-image cropped in this application can be an image containing one or more road sign characters to be recognized, and the specific cropping area varies according to the recognition requirements.

[0063] It should be noted that neural network learning algorithms can be used to determine the road sign characters to be recognized in the target image. For example, the YOLO (You Only Look Once) neural network learning algorithm can be used to determine the road sign characters to be recognized in the target image and crop out the first sub-image containing the road sign characters to be recognized.

[0064] In one implementation, cropping a first sub-image containing the road sign characters to be recognized from the target image may include the following steps:

[0065] (1) Identify the specified graphic in the target image; the specified graphic is a graphic containing the road sign characters to be recognized;

[0066] (2) Cropping out the first sub-image associated with the specified graphic from the target image.

[0067] The specified graphic is related to the characters on the road sign to be recognized. For example, to recognize the speed limit size, the speed limit sign is represented by a circle containing the speed limit number, so the specified graphic is a circle. As another example, to recognize a specific vehicle type, the vehicle type is represented by a rectangle containing the vehicle type, so the specified graphic is a rectangle.

[0068] The region of the first sub-image is associated with a specified graphic. This association can mean that the cropped region of the first sub-image includes at least a portion of the specified graphic. For example, for a speed limit sign, the cropped region of the first sub-image could only contain the inner part of the circle. The shape of the cropped region of the first sub-image can be the same as the specified graphic, or it can be other pre-defined graphics, such as a rectangle or a square.

[0069] Step 102: Process the first sub-image into a second sub-image containing only black and white;

[0070] During the processing, the first sub-image can be converted into a grayscale image, and then the first sub-image converted into a grayscale image can be processed into a second sub-image containing only black and white through a binarization algorithm.

[0071] Step 103: Determine whether the black and white distribution area in the second sub-image meets the preset area distribution conditions. If yes, proceed to step 104; if no, proceed to step 105.

[0072] It should be noted that in the process of processing the first sub-image into a second sub-image containing only black and white, there are two methods: First, the foreground of the first sub-image is processed into black as the foreground color region, and the background is processed into white as the background color region. Second, the foreground of the first sub-image is processed into white as the foreground color region, and the background is processed into black as the background color region. Since the road sign characters to be recognized belong to the foreground image in the first image, and black provides better recognition results, the first method is the preferred method.

[0073] Accordingly, determining whether the black and white distribution area in the second sub-image meets the preset area distribution conditions may include: determining whether the size of the foreground color area in the second sub-image reaches a preset threshold, and / or determining whether the position of the foreground color area in the second sub-image is at a specified position.

[0074] The determination of whether the size of the foreground color region in the second sub-image reaches a preset threshold can include the following implementation methods:

[0075] Determine whether the ratio of the foreground color region to the total region in the second sub-image is higher than a preset first threshold.

[0076] Alternatively, determine whether the ratio of the background color region to the total region in the second sub-image is lower than the first threshold.

[0077] Alternatively, determine whether the ratio of the foreground color region to the background color region in the second sub-image is higher than a preset second threshold.

[0078] Alternatively, determine whether the ratio of the background color region to the foreground color region in the second sub-image is lower than a preset second threshold.

[0079] The first and second thresholds mentioned above can be set according to the actual situation. Generally speaking, it is preferable for the foreground color area to account for 50% of the second sub-image.

[0080] The specific implementation of determining whether the foreground color region in the second sub-image is at a specified position can be as follows:

[0081] Using the center of the second sub-image as a reference, the second sub-image is divided into multiple (at least two) regions of equal area, and it is determined whether the foreground color regions in the second sub-image all belong to the multiple regions of equal area pre-divided by the second sub-image.

[0082] For example, such as Figure 3a As shown, the second sub-image is divided into a four-grid layout, and it is determined whether each of the four grids contains a black area; for example... Figure 3bAs shown, the second sub-image is divided into a nine-square grid, and it is determined whether there is a foreground color area in each of the nine squares.

[0083] Step 104: Identify the road sign characters in the second sub-image and generate the recognition result;

[0084] Specifically, the road sign characters in the second sub-image can be identified using the SAR (Show Attend and Read) network algorithm to generate the recognition results.

[0085] Step 105: Expand the cropping area and return to the step described in step 101 of cropping out the first sub-image of the target image containing the road sign characters to be identified.

[0086] If it is determined that the black and white distribution area in the second sub-image does not meet the preset regional distribution conditions, the cropping area is expanded so that when step 101 is executed again, the first sub-image is cropped from the target image based on the expanded cropping area.

[0087] It should be noted that the shape of the expanded cutting area is consistent with the shape of the previous cutting area, and the expansion range is also proportionally expanded based on the previous cutting area.

[0088] In one implementation, this application can expand the cropping area based on a pre-set expansion ratio. For example, the expansion ratio is set to increase the previous cropping area by 10%.

[0089] Another preferred implementation can also be based on the proportion of the foreground color region in the second sub-image. Specifically, in another embodiment of this application, if the area of ​​the foreground color region in the second sub-image does not reach a preset threshold, before expanding the cropping area, the following steps are also included:

[0090] The cropping ratio is determined based on the proportion of the foreground color region in the second sub-image, and the cropping region is determined based on the cropping ratio.

[0091] Different percentages correspond to different cropping ratios. Specifically, the smaller the percentage, the larger the cropping ratio, thereby further improving the accuracy of recognition. This cropping ratio is the proportion of the cropped area to the total area of ​​the target image.

[0092] Therefore, this application processes the first sub-image cropped from the target image into a second sub-image containing only black and white, thereby determining whether the black and white distribution area in the second sub-image meets the preset regional distribution conditions. If it does not meet the requirements, the cropping area is expanded, and the target image is cropped again. The processing and judgment steps are then repeated on the re-cropped first sub-image. In this way, the problem of incomplete road sign character extraction can be effectively improved, thereby increasing the accuracy of road sign character extraction.

[0093] To further improve the accuracy of recognition, another embodiment of this application provides a method for recognizing road sign characters, such as... Figure 4 As shown, the method includes the following steps:

[0094] Step 401: Crop out the first sub-image containing the road sign characters to be recognized from the target image;

[0095] The target image is the image to be recognized, which contains a road sign with characters to be recognized. These characters can refer to text or numbers on the road sign.

[0096] It should be noted that neural network learning algorithms can be used to determine the road sign characters to be recognized in the target image. For example, the YOLO neural network learning algorithm can be used to determine the road sign characters to be recognized in the target image and crop out the first sub-image containing the road sign characters to be recognized.

[0097] Step 402: First, perform noise filtering on the first sub-image, and then perform magnification processing;

[0098] To further improve the accuracy of recognition, this application can first perform noise filtering on the first sub-image before performing black and white color processing, thereby reducing noise in the image. The specific noise filtering algorithm is not limited in this application. Then, magnification processing is performed to magnify the noise-filtered first sub-image. Specifically, a bilinear interpolation algorithm can be used to magnify the noise-filtered first sub-image.

[0099] Step 403: Process the first sub-image after magnification into a second sub-image containing only black and white;

[0100] For the first sub-image after noise filtering and magnification, it can be converted into a grayscale image. Then, the first sub-image converted into a grayscale image can be processed into a second sub-image containing only black and white through a binarization algorithm.

[0101] Step 404: Determine whether the black and white distribution area in the second sub-image meets the preset area distribution conditions; if yes, proceed to step 405; if no, proceed to step 406.

[0102] It should be noted that in the process of processing the first sub-image into a second sub-image containing only black and white, there are two methods: First, the foreground of the first sub-image is processed into black as the foreground color region, and the background is processed into white as the background color region. Second, the foreground of the first sub-image is processed into white as the foreground color region, and the background is processed into black as the background color region. Since the road sign characters to be recognized belong to the foreground image in the first image, and black provides better recognition results, the first method is the preferred method.

[0103] Accordingly, determining whether the black and white distribution area in the second sub-image meets the preset area distribution conditions may include: determining whether the size of the foreground color area in the second sub-image reaches a preset threshold, and / or determining whether the position of the foreground color area in the second sub-image is at a specified position.

[0104] Determining whether the size of the foreground color region in the second sub-image reaches a preset threshold can include the following implementation methods:

[0105] Determine whether the ratio of the foreground color region to the total region in the second sub-image is higher than a preset first threshold.

[0106] Alternatively, determine whether the ratio of the background color region to the total region in the second sub-image is lower than the first threshold.

[0107] Alternatively, determine whether the ratio of the foreground color region to the background color region in the second sub-image is higher than a preset second threshold.

[0108] Alternatively, determine whether the ratio of the background color region to the foreground color region in the second sub-image is lower than a preset second threshold.

[0109] The first and second thresholds mentioned above can be set according to the actual situation. Generally speaking, it is preferable for the foreground color area to account for 50% of the second sub-image.

[0110] The specific implementation of determining whether the foreground color region in the second sub-image is at a specified position can be as follows:

[0111] Using the center of the second sub-image as a reference, the second sub-image is divided into multiple (at least two) regions of equal area, and it is determined whether the foreground color regions in the second sub-image all belong to the multiple regions of equal area pre-divided by the second sub-image.

[0112] Step 405: Identify the road sign characters in the second sub-image and generate the recognition result;

[0113] Step 406: Expand the cropping area and return to step 401.

[0114] If it is determined that the black and white distribution area in the second sub-image does not meet the preset area distribution conditions, the cropping area is expanded so that when step 401 is executed again, the first sub-image is cropped from the target image based on the expanded cropping area.

[0115] It should be noted that the shape of the expanded cutting area is consistent with the shape of the previous cutting area, and the expansion range is also proportionally expanded based on the previous cutting area.

[0116] This application can expand the cutting area based on a pre-set expansion ratio. For example, the expansion ratio can be set to increase the previous cutting area by 10%.

[0117] Of course, as a preferred implementation, the foreground color region can also be expanded based on its proportion in the second sub-image. Specifically, in another embodiment of this application, if the area of ​​the foreground color region in the second sub-image does not reach a preset threshold, before expanding the cropping area, the following steps are also included:

[0118] The cropping ratio is determined based on the proportion of the foreground color region in the second sub-image, and the cropping region is determined based on the cropping ratio.

[0119] Different percentages correspond to different cropping ratios. Specifically, the smaller the percentage, the larger the cropping ratio, thereby further improving the accuracy of recognition. This cropping ratio is the proportion of the cropped area to the total area of ​​the target image.

[0120] Therefore, in this embodiment, the first sub-image is first subjected to noise filtering, thereby reducing the impact of noise on the image. Furthermore, the image after noise filtering is magnified again, which makes subsequent recognition more accurate. Thus, this embodiment can further improve the accuracy of road sign character extraction.

[0121] Another embodiment of this application provides a method for recognizing road sign characters, such as... Figure 5 As shown, the method includes the following steps:

[0122] Step 501: Crop out the first sub-image of the target image that contains the road sign characters to be recognized;

[0123] The target image is the image to be recognized, which contains a road sign with characters to be recognized. These characters can refer to text or numbers on the road sign.

[0124] It should be noted that neural network learning algorithms can be used to determine the road sign characters to be recognized in the target image. For example, the YOLO neural network learning algorithm can be used to determine the road sign characters to be recognized in the target image and crop out the first sub-image containing the road sign characters to be recognized.

[0125] Step 502: Process the first sub-image into a second sub-image containing only black and white;

[0126] Step 503: Determine whether the black and white distribution area in the second sub-image meets the preset area distribution conditions; if yes, proceed to step 504; if no, proceed to step 506.

[0127] It should be noted that in the process of processing the first sub-image into a second sub-image containing only black and white, there are two methods: First, the foreground of the first sub-image is processed into black as the foreground color region, and the background is processed into white as the background color region. Second, the foreground of the first sub-image is processed into white as the foreground color region, and the background is processed into black as the background color region. Since the road sign characters to be recognized belong to the foreground image in the first image, and black provides better recognition results, the first method is the preferred method.

[0128] Accordingly, determining whether the black and white distribution area in the second sub-image meets the preset area distribution conditions may include: determining whether the size of the foreground color area in the second sub-image reaches a preset threshold, and / or determining whether the position of the foreground color area in the second sub-image is at a specified position.

[0129] Determining whether the size of the foreground color region in the second sub-image reaches a preset threshold can include the following implementation methods:

[0130] Determine whether the ratio of the foreground color region to the total region in the second sub-image is higher than a preset first threshold.

[0131] Alternatively, determine whether the ratio of the background color region to the total region in the second sub-image is lower than the first threshold.

[0132] Alternatively, determine whether the ratio of the foreground color region to the background color region in the second sub-image is higher than a preset second threshold.

[0133] Alternatively, determine whether the ratio of the background color region to the foreground color region in the second sub-image is lower than a preset second threshold.

[0134] The first and second thresholds mentioned above can be set according to the actual situation. Generally speaking, it is preferable for the foreground color area to account for 50% of the second sub-image.

[0135] The specific implementation of determining whether the foreground color region in the second sub-image is at a specified position is as follows:

[0136] Using the center of the second sub-image as a reference, the second sub-image is divided into multiple (at least two) regions of equal area, and it is determined whether the foreground color regions in the second sub-image all belong to the multiple regions of equal area pre-divided by the second sub-image.

[0137] Step 504: Identify the road sign characters in the second sub-image and generate the recognition result;

[0138] Step 505: If the recognition result does not conform to the road sign marking rules, then the recognition result is corrected;

[0139] To further improve the accuracy of recognition, this application can also correct the recognition results if they do not conform to the road sign marking rules. For example, if the road sign characters in a speed limit sign are all numbers, and a letter is recognized, it can be corrected to the corresponding number based on the shape of the letter. For example, if the letter 'b' is recognized, it can be corrected to the number 6; if the letter 'o' is recognized, it can be corrected to the number 0.

[0140] Step 506: Expand the cropping area and return to step 501.

[0141] If it is determined that the black and white distribution area in the second sub-image does not meet the preset area distribution conditions, the cropping area is expanded so that when step 501 is executed again, the first sub-image is cropped from the target image based on the expanded cropping area.

[0142] It should be noted that the shape of the expanded cutting area is consistent with the shape of the previous cutting area, and the expansion range is also proportionally expanded based on the previous cutting area.

[0143] Therefore, in this embodiment, if the recognition result does not conform to the road sign marking rules, the recognition result can be corrected, thereby further improving the accuracy of recognition.

[0144] To facilitate understanding, this application provides a simple illustration using a specific example. Figure 6Taking the target image as an example, the purpose of this example is to identify the speed limit sign "100" on a road sign in the target image. After cropping, processing, and judgment, it is determined that the black area in the second sub-image does not reach 50% of the total area of ​​the second sub-image, so the cropping area needs to be expanded. For example, the size of the first sub-image obtained by the first cropping is 70*70, and the black area is 10, which does not reach 50% of the total area. After expanding the cropping area, the size of the first sub-image obtained by the second cropping is 80*80, and the black area is 100, reaching 50% of the total area. Thus, the character "100" can be identified in the image, which obviously improves the accuracy of the recognition.

[0145] Corresponding to the above-mentioned method for recognizing road sign characters, this application also provides a device for recognizing road sign characters. The following describes several device embodiments, and the specific implementation can be referred to the method embodiments.

[0146] One embodiment of this application provides a road sign character recognition device, such as... Figure 7 As shown, the device includes: a first cutting unit 100, a first processing unit 200, a first judgment unit 300, a first identification unit 400, and a first enlarging unit 500;

[0147] The first cropping unit 100 is used to crop out a first sub-image containing the road sign characters to be recognized from the target image.

[0148] The target image is the image to be identified, which is an image containing a road sign with road sign characters to be identified. The target image can be an image captured by a camera on the road. For example, it can be an image captured by a camera mounted on a vehicle while the vehicle is in motion; more specifically, it can be an image of the road captured by a camera mounted on the windshield of a vehicle. Of course, this application is not limited to images captured by cameras mounted on vehicles; all other images containing road sign characters to be identified are within the scope of protection of this application.

[0149] The aforementioned road sign characters to be identified can refer to text or numerical markings on the road sign. The first sub-image cropped in this application can be an image containing one or more road sign characters to be identified, and the specific cropping area varies according to the recognition requirements.

[0150] It should be noted that the first cropping unit can use a neural network learning algorithm to determine the road sign characters to be recognized in the target image. For example, the YOLO neural network learning algorithm can be used to determine the road sign characters to be recognized in the target image and crop out the first sub-image containing the road sign characters to be recognized.

[0151] In one implementation, the first cropping unit 100 may include: an image determination module and a first cropping module, specifically:

[0152] An image determination module is used to determine a specified graphic in a target image; the specified graphic is a graphic containing road sign characters to be identified.

[0153] The first cropping module is used to crop out a first sub-image associated with the specified graphic from the target image.

[0154] The specified graphic is related to the characters on the road sign to be recognized. For example, to recognize the speed limit size, the speed limit sign is represented by a circle containing the speed limit number, so the specified graphic is a circle. As another example, to recognize a specific vehicle type, the vehicle type is represented by a rectangle containing the vehicle type, so the specified graphic is a rectangle.

[0155] The region of the first sub-image is associated with a specified graphic. This association can mean that the cropped region of the first sub-image includes at least a portion of the specified graphic. For example, for a speed limit sign, the cropped region of the first sub-image could only contain the inner part of the circle. The shape of the cropped region of the first sub-image can be the same as the specified graphic, or it can be other pre-defined graphics, such as a rectangle or a square.

[0156] The first processing unit 200 is used to process the first sub-image into a second sub-image containing only black and white.

[0157] During the processing, the first processing unit 200 can first convert the first sub-image into a grayscale image, and then process the first sub-image converted into a grayscale image into a second sub-image containing only black and white through a binarization algorithm.

[0158] The first judgment unit 300 is used to determine whether the black and white distribution area in the second sub-image meets the preset area distribution conditions.

[0159] Optionally, the first processing unit 200 includes:

[0160] The first processing module is used to process the foreground of the first sub-image into black as the foreground color area and process the background of the first sub-image into white as the background color area.

[0161] Alternatively, the second processing module is used to process the foreground in the first sub-image to white as the foreground color area, and to process the background of the first sub-image to black as the background color area.

[0162] Accordingly, the first judgment unit 300 includes:

[0163] The first judgment module is used to determine whether the area of ​​the foreground color region in the second sub-image reaches a preset threshold, and / or the second judgment module is used to determine whether the position of the foreground color region in the second sub-image is at a specified position.

[0164] Specifically, the first judgment module can be used to determine whether the ratio of the foreground color region to the total region in the second sub-image is higher than a preset first threshold, or to determine whether the ratio of the background color region to the total region in the second sub-image is lower than the first threshold, or to determine whether the ratio of the foreground color region to the background color region in the second sub-image is higher than a preset second threshold, or to determine whether the ratio of the background color region to the foreground color region in the second sub-image is lower than a preset second threshold.

[0165] The first and second thresholds mentioned above can be set according to the actual situation. Generally speaking, it is preferable for the foreground color area to account for 50% of the second sub-image.

[0166] The second judgment module can be implemented as follows:

[0167] Using the center of the second sub-image as a reference, the second sub-image is divided into multiple (at least two) regions of equal area, and it is determined whether the foreground color regions in the second sub-image all belong to the multiple regions of equal area pre-divided by the second sub-image.

[0168] The first recognition unit 400 is used to recognize road sign characters in the second sub-image and generate a recognition result when the black and white distribution area in the second sub-image meets the preset regional distribution conditions.

[0169] The first enlargement unit 500 is used to enlarge the cropping area when the black and white distribution area in the second sub-image does not meet the preset area distribution conditions, and to control the first cropping unit to re-crop with the enlarged cropping area.

[0170] Correspondingly, the first processing unit 200, the first judgment unit 300, and the first identification unit 400 also perform their respective functions after the first cropping unit 100 re-crops the material.

[0171] It should be noted that the shape of the expanded cutting area is consistent with the shape of the previous cutting area, and the expansion range is also proportionally expanded based on the previous cutting area.

[0172] In one implementation, the first enlarging unit 300 is used to enlarge the cutting area based on a pre-set enlargement cutting ratio. For example, the enlargement ratio is set to increase by 10% based on the previous cutting area.

[0173] Another preferred implementation further includes, when the first judgment module determines that the area of ​​the foreground color region in the second sub-image does not reach a preset threshold:

[0174] The first determining unit is configured to determine a cropping ratio based on the proportion of the foreground color region in the second sub-image, and to determine a cropping region based on the cropping ratio. Correspondingly, the first expanding unit 300 is configured to expand the cropping region based on the cropping region determined by the first determining unit.

[0175] Different percentages correspond to different cropping ratios. Specifically, the smaller the percentage, the larger the cropping ratio, thereby further improving the accuracy of recognition. This cropping ratio is the proportion of the cropped area to the total area of ​​the target image.

[0176] Therefore, this application processes the first sub-image cropped from the target image into a second sub-image containing only black and white, thereby determining whether the black and white distribution area in the second sub-image meets the preset regional distribution conditions. If it does not meet the requirements, the cropping area is expanded, and the target image is cropped again. The processing and judgment steps are then repeated on the re-cropped first sub-image. In this way, the problem of incomplete road sign character extraction can be effectively improved, thereby increasing the accuracy of road sign character extraction.

[0177] Another embodiment of this application also provides a road sign character recognition device, such as... Figure 8 As shown, the device includes: a first cutting unit 100, a second processing unit 600, a first processing unit 200, a first judgment unit 300, a first identification unit 400, and a first enlargement unit 500; specifically:

[0178] The first cropping unit 100 is used to crop out a first sub-image containing the road sign characters to be recognized from the target image.

[0179] The second processing unit 600 is used to first perform noise filtering on the first sub-image, and then perform magnification processing.

[0180] To further improve the accuracy of recognition, before processing the first sub-image in black and white, the second processing unit 600 can first perform noise filtering on the first sub-image to reduce noise in the image. The specific noise filtering algorithm is not limited in this application. Then, the first sub-image is magnified to enlarge it. Specifically, a bilinear interpolation algorithm can be used to enlarge the first sub-image after noise filtering.

[0181] The first processing unit 200 is used to process the magnified first sub-image into a second sub-image containing only black and white.

[0182] For the first sub-image after noise filtering and magnification, the second processing unit first converts it into a grayscale image, and then processes the first sub-image converted into a grayscale image into a second sub-image containing only black and white through a binarization algorithm.

[0183] The first judgment unit 300 is used to determine whether the black and white distribution area in the second sub-image meets the preset area distribution conditions;

[0184] The first recognition unit 400 is used to recognize road sign characters in the second sub-image and generate a recognition result when the black and white distribution area in the second sub-image meets the preset regional distribution conditions.

[0185] The first enlargement unit 500 is used to enlarge the cropping area when the black and white distribution area in the second sub-image does not meet the preset area distribution conditions, and to control the first cropping unit to re-crop with the enlarged cropping area.

[0186] Therefore, in this embodiment, the first sub-image is first subjected to noise filtering, thereby reducing the impact of noise on the image. Furthermore, the image after noise filtering is magnified again, which makes subsequent recognition more accurate. Thus, this embodiment can further improve the accuracy of road sign character extraction.

[0187] Another embodiment of this application provides a road sign character recognition device, such as... Figure 9 As shown, the device includes: a first cutting unit 100, a first processing unit 200, a first judgment unit 300, a first identification unit 400, a first correction unit 700, and a first enlargement unit 500; specifically:

[0188] The first cropping unit 100 is used to crop out a first sub-image containing the road sign characters to be recognized from the target image.

[0189] The first processing unit 200 is used to process the first sub-image into a second sub-image containing only black and white.

[0190] The first judgment unit 300 is used to determine whether the black and white distribution area in the second sub-image meets the preset area distribution conditions;

[0191] The first recognition unit 400 is used to recognize road sign characters in the second sub-image and generate a recognition result when the black and white distribution area in the second sub-image meets the preset regional distribution conditions.

[0192] The first correction unit 700 is used to correct the recognition result if the recognition result does not conform to the road sign marking rules.

[0193] To further improve the accuracy of recognition, this application can also correct the recognition results if they do not conform to the road sign marking rules. For example, if the road sign characters in a speed limit sign are all numbers, and a letter is recognized, it can be corrected to the corresponding number based on the shape of the letter. For example, if the letter 'b' is recognized, it can be corrected to the number 6; if the letter 'o' is recognized, it can be corrected to the number 0.

[0194] The first enlargement unit 500 is used to enlarge the cropping area when the black and white distribution area in the second sub-image does not meet the preset area distribution conditions, and to control the first cropping unit to re-crop with the enlarged cropping area.

[0195] Therefore, in this embodiment, if the recognition result does not conform to the road sign marking rules, the recognition result can be corrected, thereby further improving the accuracy of recognition.

[0196] The solution of this application has been described in detail above with reference to the accompanying drawings. In the above embodiments, the descriptions of each embodiment have different emphases; parts not described in detail in a certain embodiment can be referred to in the relevant descriptions of other embodiments. Those skilled in the art should also understand that the actions and modules involved in the specification are not necessarily essential to this application. Furthermore, it is understood that the steps in the method of this application embodiment can be adjusted, combined, and deleted according to actual needs, and the modules in the device of this application embodiment can be combined, divided, and deleted according to actual needs.

[0197] Furthermore, the method according to this application can also be implemented as a computer program or computer program product, which includes computer program code instructions for performing some or all of the steps in the method described above.

[0198] Alternatively, this application may be implemented as a non-transitory machine-readable storage medium (or computer-readable storage medium, or machine-readable storage medium) storing executable code (or computer program, or computer instruction code) that, when executed by a processor of an electronic device (or electronic device, server, etc.), causes the processor to perform some or all of the steps of the methods described above according to this application.

[0199] Those skilled in the art will also understand that the various exemplary logic blocks, modules, circuits, and algorithm steps described in connection with the present application can be implemented as electronic hardware, computer software, or a combination of both.

[0200] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems and methods according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0201] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for recognizing road sign characters, characterized in that, include: Crops out the first sub-image containing the road sign characters to be identified from the target image; The first sub-image is processed into a second sub-image containing only black and white; Determine whether the black and white distribution area in the second sub-image meets the preset area distribution conditions; if yes, identify the road sign characters in the second sub-image and generate a recognition result; if no, expand the cropping area and return to the step of cropping the first sub-image containing the road sign characters to be identified in the target image. The step of processing the first sub-image into a second sub-image containing only black and white includes: processing the foreground of the first sub-image to black as a foreground color region and processing the background of the first sub-image to white as a background color region; or, processing the foreground of the first sub-image to white as a foreground color region and processing the background of the first sub-image to black as a background color region. Accordingly, determining whether the black and white distribution area in the second sub-image meets the preset region distribution conditions includes: Determine whether the area of ​​the foreground color region in the second sub-image reaches a preset threshold, and determine whether the position of the foreground color region in the second sub-image is at a specified position; If the area of ​​the foreground color region in the second sub-image does not reach a preset threshold, before expanding the cropping area, the method further includes: The cropping ratio is determined based on the proportion of the foreground color region in the second sub-image, and the cropping region is determined based on the cropping ratio.

2. The method according to claim 1, characterized in that, Also includes: The first sub-image is first subjected to noise filtering, and then magnified. Accordingly, processing the first sub-image into a second sub-image containing only black and white includes: The first sub-image, after being magnified, is processed into a second sub-image containing only black and white.

3. The method according to claim 1, characterized in that, After generating the recognition result, the process also includes: If the recognition result does not conform to the road sign marking rules, the recognition result shall be corrected.

4. The method according to claim 1, characterized in that, The step of cropping out the first sub-image containing the road sign characters to be recognized from the target image includes: Identify a specified graphic in the target image; the specified graphic is a graphic containing the road sign characters to be identified. A first sub-image associated with the specified graphic is cropped from the target image.

5. A road sign character recognition device, characterized in that, include: The first cropping unit is used to crop out the first sub-image containing the road sign characters to be recognized from the target image; The first processing unit is configured to process the first sub-image into a second sub-image containing only black and white; The first judgment unit is used to determine whether the black and white distribution area in the second sub-image meets the preset area distribution conditions; The first recognition unit is used to recognize road sign characters in the second sub-image and generate a recognition result when the black and white distribution area in the second sub-image meets the preset regional distribution conditions. The first enlargement unit is used to enlarge the cropping area when the black and white distribution area in the second sub-image does not meet the preset area distribution conditions, and to control the first cropping unit to re-crop with the enlarged cropping area. The first processing unit includes: The first processing module is used to process the foreground of the first sub-image into black as the foreground color area and process the background of the first sub-image into white as the background color area. Alternatively, the second processing module is used to process the foreground in the first sub-image to white as the foreground color area and process the background of the first sub-image to black as the background color area. Accordingly, the first determination unit includes: The first judgment module is used to determine whether the area of ​​the foreground color region in the second sub-image reaches a preset threshold, and the second judgment module is used to determine whether the position of the foreground color region in the second sub-image is at a specified position. If the first judgment module determines that the area of ​​the foreground color region in the second sub-image does not reach a preset threshold, the method further includes: The first determining unit is used to determine the cropping ratio based on the proportion of the foreground color region in the second sub-image, and to determine the cropping region based on the cropping ratio.

6. The apparatus according to claim 5, characterized in that, Also includes: The second processing unit is used to first perform noise filtering on the first sub-image, and then perform magnification processing. Accordingly, the first processing unit is specifically used to process the magnified first sub-image into a second sub-image containing only black and white.

7. The apparatus according to claim 5, characterized in that, Also includes: The first correction unit is used to correct the recognition result if the recognition result does not conform to the road sign marking rules.

8. The apparatus according to claim 5, characterized in that, The first trimming unit includes: An image determination module is used to determine a specified graphic in a target image; the specified graphic is a graphic containing road sign characters to be identified. The first cropping module is used to crop out a first sub-image associated with the specified graphic from the target image.

Images