Method, device, electronic device and storage medium for identifying traffic signs

Abstract

The present disclosure provides a method, device, electronic device, storage medium and computer program product for identifying traffic signs, which relate to the field of image processing technology, and in particular to the field of intelligent traffic technology. The specific implementation scheme is: using a first detection model to perform detection on an original image for a first traffic sign, and obtain a first sign area that potentially contains the first traffic sign; using a second detection model to perform detection on the original image for a second traffic sign, and obtain a second sign area that potentially contains the second traffic sign; determining the degree of overlap between the first sign area and the second sign area; and identifying the content of the first traffic sign when it is determined that the original image contains the first traffic sign according to the degree of overlap.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, in particular to the field of intelligent transportation technology, and specifically to a method, device, electronic device, storage medium and computer program product for identifying traffic signs. Background Art

[0002] With the popularization of mobile Internet, navigation electronic products are combined with multi-platform devices such as mobile phones, driving recorders, and car computers, and are widely used in driving.

[0003] Traditional technologies usually use data collected by on-board cameras to automatically identify traffic signs to assist navigation systems in providing users with forecasts. However, traffic sign recognition is currently based on a single feature, which may lead to misidentification and low recognition accuracy. The recognition results must be reviewed manually, resulting in low recognition efficiency. Summary of the invention

[0004] The present disclosure provides a method, an apparatus, an electronic device, a storage medium, and a computer program product for recognizing a traffic sign.

[0005] According to one aspect of the present disclosure, there is provided a method for identifying a traffic sign, comprising:

[0006] Using the first detection model to perform detection on the original image for the first traffic sign, and obtaining a first sign area that potentially contains the first traffic sign;

[0007] Using a second detection model to detect a second traffic sign on the original image, and obtaining a second sign area that potentially contains the second traffic sign;

[0008] determining a degree of overlap between the first mark area and the second mark area;

[0009] When it is determined according to the overlap degree that the original image contains the first traffic sign, the content of the first traffic sign is identified.

[0010] According to another aspect of the present disclosure, there is provided a device for recognizing a traffic sign, comprising:

[0011] A first detection module, configured to detect a first traffic sign on an original image using a first detection model, and obtain a first sign area that potentially contains the first traffic sign;

[0012] A second detection module, configured to use a second detection model to detect a second traffic sign on the original image, and obtain a second sign area that potentially contains the second traffic sign;

[0013] an overlap calculation module, used to determine the overlap degree between the first mark area and the second mark area;

[0014] The content recognition module is used to recognize the content of the first traffic sign when it is determined that the original image contains the first traffic sign according to the overlap degree.

[0015] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0016] at least one processor; and

[0017] a memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor so that the at least one processor can implement the method described above.

[0019] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method as described above.

[0020] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program implements the method described above when executed by a processor.

[0021] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0023] Figure 1 is a flow chart of a method for identifying a traffic sign according to an embodiment of the present disclosure;

[0024] Figure 2 is a flow chart of a method for identifying a traffic sign according to another embodiment of the present disclosure;

[0025] Figure 3 is a schematic diagram of a method for determining that an original image contains a first traffic sign according to an embodiment of the present disclosure;

[0026] Figure 4 is a flow chart of a method for determining text content of a first traffic sign according to an embodiment of the present disclosure;

[0027] Figure 5is a schematic diagram of a method for identifying the content of a first traffic sign according to an embodiment of the present disclosure;

[0028] Figure 6 is a flow chart of a method for determining text content located in an overlapping area between a first logo area and an image area according to an embodiment of the present disclosure;

[0029] Figure 7 is a flow chart of a method for determining text content located in an overlapping area between a first logo area and an image area according to another embodiment of the present disclosure;

[0030] Figure 8 is a block diagram of a device for recognizing traffic signs according to an embodiment of the present disclosure;

[0031] Fig. 9 The block diagram of an electronic device used to implement the method for recognizing traffic signs according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0033] Embodiments of the present disclosure provide a method, device, electronic device, storage medium, and computer program product for identifying traffic signs. The above method includes: using a first detection model to perform detection on an original image for a first traffic sign to obtain a first sign area that potentially contains the first traffic sign; using a second detection model to perform detection on the original image for a second traffic sign to obtain a second sign area that potentially contains the second traffic sign; determining the degree of overlap between the first sign area and the second sign area; and identifying the content of the first traffic sign when it is determined that the original image contains the first traffic sign based on the degree of overlap. The method for identifying traffic signs in the present disclosure detects the first traffic sign and the second traffic sign on the original image based on two target detection models respectively, and determines whether the original image contains the first traffic sign based on the two results, thereby reducing or even avoiding the possibility of misidentifying the second traffic sign as the first traffic sign, thereby improving the accuracy and efficiency of traffic sign recognition.

[0034] Figure 1 is a flowchart of a method for recognizing traffic signs according to an embodiment of the present disclosure.

[0035] like Figure 1As shown, the method for recognizing a traffic sign includes operations S110 to S240.

[0036] In operation S110 , a first traffic sign is detected on an original image using a first detection model to obtain a first sign region that potentially contains the first traffic sign.

[0037] The original image may be a road image captured by a vehicle-mounted camera device, such as a road image captured by a vehicle-mounted camera, a driving recorder, etc. The server may obtain the original image from the vehicle for identifying traffic signs. In some embodiments, an area to be detected may be determined in the original image, and a first detection model may be used to perform detection of a first traffic sign on the area to be detected of the original image, thereby obtaining a first sign area that potentially contains the first traffic sign, so as to determine whether the area to be detected contains the first traffic sign. The so-called first traffic sign here may be understood as a traffic sign to be determined.

[0038] In the embodiment of the present disclosure, the first traffic sign may be any type of traffic sign, for example, various traffic signs containing text, such as speed limit signs, height limit signs, width limit signs, etc., which are not limited here.

[0039] The first detection model may include, for example, an RCNN series model or a Yolo series model. The RCNN series model is a two-stage algorithm model, that is, first generating candidate regions and then performing CNN classification. The Yolo series model is an end-to-end target detection algorithm that does not require pre-extraction of candidate regions. It directly applies the algorithm to the input image and outputs the coordinate positions, categories, and corresponding confidence levels of all objects in the image.

[0040] In the disclosed embodiment, the first detection model may include, for example, a fast regional convolutional neural network (FasterRCNN) model or a Yolo V3 model, wherein the Yolo V3 model is the third version of the Yolo series target detection algorithm model. The Yolo V3 model is mainly composed of convolutional layers, and does not use fully connected layers and pooling layers. The Yolo V3 model can use feature maps of multiple different scales to perform object detection and can detect more fine-grained features. Compared with previous algorithms, especially for small targets, the detection accuracy of the Yolo V3 model is significantly improved.

[0041] In this operation, the first detection model is used to perform detection on the original image for the first traffic sign, and a first sign area that potentially contains the first traffic sign can be obtained. For example, when it is necessary to determine whether the area to be detected in the original image contains a speed limit sign, the speed limit sign is used as the first traffic sign, and the first detection model, such as the Yolo V3 model, is used to perform detection on the area to be detected in the original image for the speed limit sign, and a sign area that potentially contains the speed limit sign can be obtained. Subsequently, whether the area to be detected in the original image contains the first traffic sign (i.e., the speed limit sign) will be determined based on the sign area that potentially contains the speed limit sign and the second sign area that potentially contains the second traffic sign.

[0042] In operation S120, a second traffic sign is detected on the original image using a second detection model to obtain a second sign region that potentially contains the second traffic sign.

[0043] Specifically, the second detection model is used to detect the second traffic sign on the original image, which specifically refers to the second detection model being used to detect the second traffic sign on the area to be detected in the original image, thereby obtaining a second sign area that potentially contains the second traffic sign, so as to identify whether the first traffic sign in the area to be detected is misidentified. In other words, the so-called second traffic sign here can be understood as a standard or reference for identifying whether the first traffic sign is misidentified.

[0044] In the disclosed embodiment, the second traffic sign may be any one or more traffic signs similar to the first traffic sign, and the number of the second traffic signs may not be limited to one. For example, when the first traffic sign is a speed limit sign, the second traffic sign may be, for example, any other traffic sign containing text other than the speed limit sign, such as at least one of an axle load limit sign, a height limit sign, and a width limit sign, and the number of each traffic sign may also be selected according to actual conditions, and is not limited here.

[0045] In the embodiment of the present disclosure, the second detection model has the same or similar definition as the first detection model, which will not be repeated here. In the embodiment of the present disclosure, the second detection model can be the same as the first detection model or different, and can be selected according to actual conditions, which is not limited here.

[0046] In this operation S120, the second detection model is used to perform detection on the original image for the second traffic sign, and the process of obtaining the second sign area potentially containing the second traffic sign is the same or similar to the process of obtaining the first sign area potentially containing the first traffic sign in operation S110, which is not repeated here.

[0047] In addition, the second logo area and the first logo area can have the same size or different sizes. When the sizes of the first logo area and the second logo area are the same or similar, the result of the subsequent calculation of the overlap degree between the two will be more accurate. The sizes of the first logo area and the second logo area can be set according to actual conditions and are not limited here.

[0048] In operation S130, an overlap degree between the first mark region and the second mark region is determined.

[0049] In this operation, when determining the degree of overlap between the first sign area and the second sign area, if the second sign area potentially containing the second traffic sign involves multiple second traffic signs, or involves multiple second traffic signs of the same type, the degree of overlap between the corresponding second sign area and the first sign area can be calculated separately.

[0050] In operation S140, when it is determined that the original image includes the first traffic sign according to the degree of overlap, the content of the first traffic sign is recognized.

[0051] For example, the content of the first traffic sign may be identified based on semantic segmentation and text recognition technology.

[0052] Compared with the method of combining single feature recognition with manual review, the method of identifying traffic signs in the disclosed embodiment detects the first traffic sign and the second traffic sign in the original image based on two target detection models respectively, and determines whether the original image contains the first traffic sign based on the two results, which can reduce or even avoid the possibility of misidentifying the second traffic sign as the first traffic sign, thereby improving the accuracy of traffic sign recognition. Since manual review is not required, recognition efficiency can be improved and labor costs can be reduced.

[0053] Figure 2 is a flowchart of a method for recognizing a traffic sign according to another embodiment of the present disclosure. Figure 3 is a schematic diagram of a method for determining whether an original image contains a first traffic sign according to an embodiment of the present disclosure. Figure 2 and Figure 3 An example implementation of the above operations S130 to S140 is described below.

[0054] like Figure 2 As shown, the method for identifying traffic signs includes operations S210 to S280. Operations S210 to S220 can be implemented in the same manner as operations S110 to S120, respectively, and the repeated parts will not be described in detail.

[0055] In operation S210 , a first traffic sign is detected on an original image using a first detection model to obtain a first sign region that potentially contains the first traffic sign.

[0056] like Figure 3 As shown, the area to be detected is determined in the original image R ( Figure 3 The first detection model is used to detect the first traffic sign (e.g., speed limit sign) in the area to be detected in the original image R, and a first sign area R1 that potentially contains the first traffic sign is obtained. The first detection model is the same or similar to the definition described above, and will not be repeated here.

[0057] In operation S220, a second traffic sign is detected on the original image using a second detection model to obtain a second sign region that potentially contains the second traffic sign.

[0058] like Figure 3 As shown, the second detection model is used to detect the second traffic sign (such as a weight limit sign) on the original image R, and a second sign area R2 that potentially contains the second traffic sign is obtained. The second detection model is the same or similar to the definition described above, and will not be repeated here.

[0059] In operation S230, an intersection-over-union ratio between the first mark region and the second mark region is calculated.

[0060] Specifically, the intersection-over-union (IOU) of the first mark region and the second mark region is calculated, which satisfies the following relationship:

[0061]

[0062] In the formula, A1 represents the first marking area, and A2 represents the second marking area.

[0063] In operation S240 , it is determined whether the intersection-over-union ratio is lower than a preset threshold. If so, operation S250 is performed next; if not, operation S280 is performed next.

[0064] The calculated intersection-over-union (IOU) of the first sign area R1 and the second sign area R2 is compared with a preset threshold value r. If the intersection-over-union (IOU) is lower than the preset threshold value r, it is determined that the area to be detected in the original image contains the first traffic sign, and operation S250 is performed next. If the intersection-over-union (IOU) is not lower than the preset threshold value r, it is determined that the area to be detected in the original image does not contain the first traffic sign. In this case, the second traffic sign may be mistakenly identified as the first traffic sign. The mistakenly identified area needs to be eliminated later to avoid misjudgment.

[0065] It should be noted that, in the present disclosure, the above-mentioned preset threshold r can be a preset value or a preset range, which can be set according to actual conditions and is not limited here.

[0066] In operation S250 , it is determined that the original image contains a first traffic sign.

[0067] like Figure 3 As shown, when the intersection-over-union ratio IOU of the first sign region R1 and the second sign region R2 is lower than the preset threshold f, it can be determined that the area to be detected in the original image includes the first traffic sign (eg, a speed limit sign).

[0068] In the embodiment of the present disclosure, whether the area to be detected in the original image contains the first traffic sign is determined based on the intersection-and-union ratio of the first sign area and the second sign area. That is, whether there is a possibility of misidentifying the second traffic sign as the first traffic sign is determined based on the intersection-and-union ratio of the two. The above method realizes the identification of misidentification in a simple and efficient manner, thereby further improving the accuracy of traffic sign recognition.

[0069] In operation S260, semantic segmentation is performed on the original image to obtain a semantic segmentation map.

[0070] Since there may be detection errors in model recognition, when it is determined that the original image contains the first traffic sign, the first traffic sign is recognized more finely by recognizing the content of the first traffic sign to further ensure the accuracy and reliability of the recognition.

[0071] In this operation, an image semantic segmentation model is used to perform semantic segmentation processing on the original image to generate a semantic segmentation map, and based on the semantic segmentation map, segmentation elements belonging to the traffic sign class are extracted, so as to determine the semantic segmentation area for the traffic sign in the semantic segmentation map, that is, the connected area representing the traffic sign.

[0072] The so-called image semantic segmentation model here may include, for example, but is not limited to, a Fully Convolutional Networks (FCN) model, a DeepLab series model, or a Pyramid Scene Parsing Network (PSPNET) model, etc. The specific image semantic segmentation model may be selected based on actual needs and is not limited here.

[0073] In operation S270 , text content of the first traffic sign is determined based on the semantic segmentation map and the first sign area.

[0074] Since the generated semantic segmentation map includes semantic elements corresponding to the first sign area, the segmentation elements in the obtained semantic segmentation map can be compared with the elements in the first sign area to determine the text content of the first traffic sign, and then further determine whether the recognition of the first traffic sign is incorrect based on the determined text content of the first traffic sign, thereby further ensuring the reliability and accuracy of traffic sign recognition.

[0075] In some embodiments of the present disclosure, the above operation S270 also includes operations S471 to S473. Figure 4 is a flow chart of a method for determining the text content of a first traffic sign according to an embodiment of the present disclosure. Figure 4 and Figure 5 An example implementation of the above operation S270 is described below.

[0076] like Figure 4 As shown, the method for determining the text content of the first traffic sign includes operations S471 to S473.

[0077] In operation S471 , a semantic segmentation region for a traffic sign is determined in the semantic segmentation map.

[0078] For example, the above-described method can be used to perform semantic segmentation on the original image to obtain a semantic segmentation map. The semantic segmentation map contains segmentation elements with different semantics, such as segmentation elements representing vehicles, segmentation elements representing traffic signs, etc. The segmentation elements belonging to traffic signs in the semantic segmentation map can be extracted and the connected domain can be calculated to determine the semantic segmentation area for the traffic sign.

[0079] In operation S472, an image region corresponding to the semantic segmentation region is determined in the original image.

[0080] For example, for a semantic segmentation region for a traffic sign in a semantic segmentation map, an image region corresponding to the semantic segmentation region is found in the original image, and the position of the image region in the original image is consistent with the position of the semantic segmentation region in the semantic segmentation map. The image region will be used to compare with the first sign region containing the first traffic sign in the subsequent step.

[0081] In operation S473, text content located in an overlapping area between the first sign area and the image area is determined as text content of the first traffic sign.

[0082] For example, the first sign area can be compared with the image area, and the text content in the overlapping area between the two can be extracted as the text content of the first traffic sign. By determining the overlapping area between the first sign area and the image area corresponding to the semantic segmentation area, the area where the first traffic sign is located can be more accurately determined, and by extracting the text content located in the overlapping area, the accuracy of recognizing the text content of the first traffic sign can be further improved.

[0083] Figure 5 is a schematic diagram of a method for identifying the content of a first traffic sign according to an embodiment of the present disclosure.

[0084] like Figure 5 As shown, the original image 510 is semantically segmented using the method described above to obtain a semantic segmentation map 520. The semantic segmentation map 520 contains elements with different semantics, for example, segmentation element ① representing a vehicle, segmentation element ② representing a traffic sign, etc. In this operation, the segmentation element ② representing the traffic sign is extracted from the semantic segmentation map 520 and the connected domain is calculated to determine the semantic segmentation area for the traffic sign.

[0085] like Figure 5 As shown, for example, for the semantic segmentation region in the semantic segmentation map 520, the semantic segmentation region is mapped to the original image to determine the image region 530 corresponding to the semantic segmentation region in the original image. Figure 5 As shown, the image area corresponding to the semantic segmentation area ② contains a speed limit sign, a direction sign, and a road sign, and the corresponding text content includes "80", "North", and "G104". This image area will be used to compare with the first sign area containing the first traffic sign (such as a speed limit sign) later.

[0086] like Figure 5 As shown, the overlapping area between the image area 510 and the first sign area 540 that has been determined to contain the speed limit sign is determined. Figure 5 The middle image area 510 and the first sign area 540 both contain speed limit signs, so the speed limit sign is located in the overlapping area of ​​the two. Among the text contents "80", "North" and "G104" in the first sign area 510, the text content "80" located in the overlapping area is extracted as the text content of the first traffic sign (i.e., the speed limit sign). In this way, the refined recognition of the first traffic sign is achieved, and the reliability and accuracy of the recognition are guaranteed.

[0087] Figure 6 is a flowchart of a method for determining text content located in an overlapping area between a first logo area and an image area according to an embodiment of the present disclosure, Figure 7is a flow chart of a method for determining text content located in an overlapping area between a first logo area and an image area according to another embodiment of the present disclosure. Figure 6 and Figure 7 To illustrate an example implementation of the above operation S473.

[0088] like Figure 6 As shown, the above operation S473 may include operations S6731 to S6733, for example.

[0089] In operation S6731, text recognition is performed on the semantically segmented area to obtain a plurality of characters.

[0090] Specifically, text recognition can be performed on each semantic segmentation area to obtain multiple characters related to each semantic segmentation area. In this embodiment, text recognition can be implemented by optical character recognition (OCR) technology, for example. In some other embodiments, it can also be implemented by other suitable methods, which are not limited here.

[0091] In operation S6732, an overlapping area between the first mark area and the image area is determined.

[0092] Specifically, the first mark area is compared with the image area, and the overlapping area between the two is extracted.

[0093] In operation S6733, characters located in the overlapping area are extracted from the obtained multiple characters as text content of the first traffic sign.

[0094] Specifically, the characters located in the overlapping area are extracted from the obtained multiple characters as the text content of the first traffic sign, and then it can be determined whether the text content of the first traffic sign matches the type of the traffic sign. For example, assuming that the characters extracted from the first sign area 510 include "80", "North" and "G104", where the character "80" is located in the overlapping area, the character "80" located in the overlapping area can be extracted from the characters "80", "North" and "G104" as the text content of the first traffic sign (i.e., the speed limit sign).

[0095] Figure 7 A flow chart of another method for determining text content in the overlapping area between the first mark area and the image area is shown. Figure 7 As shown, the above operation S473 may also include operations S7731 to S7732, for example.

[0096] In operation S7731, an overlapping area between the first mark area and the image area is determined.

[0097] In this operation, the overlapping area between the first mark area and the image area is first determined, and then text recognition is performed on the overlapping area.

[0098] In operation S7732, text recognition is performed on the overlapping area to obtain at least one character as the text content of the first traffic sign.

[0099] Specifically, text recognition is performed on the overlapping area to obtain at least one character as the text content of the first traffic sign. For example, assuming that the overlapping area includes a speed limit sign, and the speed limit sign includes two characters "80", through this operation, the two characters "80" can be extracted from the overlapping area as the text content of the speed limit sign.

[0100] Figure 8 is a block diagram of an apparatus for recognizing traffic signs according to an embodiment of the present disclosure.

[0101] like Figure 8 As shown, the device 800 for identifying a traffic sign includes a first detection module 810 , a second detection module 820 , an overlap calculation module 830 and a content recognition module 840 .

[0102] The first detection module 810 is used to perform detection on the first traffic sign on the original image using the first detection model to obtain a first sign area that potentially contains the first traffic sign.

[0103] The second detection module 820 is used to perform detection on the second traffic sign on the original image using the second detection model to obtain a second sign area that potentially contains the second traffic sign.

[0104] The overlap calculation module 830 is used to determine the overlap degree between the first mark area and the second mark area.

[0105] The content recognition module 840 is configured to recognize the content of the first traffic sign when it is determined according to the degree of overlap that the original image contains the first traffic sign.

[0106] In some embodiments of the present disclosure, the overlap calculation module 830 includes an overlap calculation submodule, which is used to calculate the intersection-and-union ratio of the first sign area and the second sign area. The content recognition module 840 is used to determine that the original image contains the first traffic sign in response to the intersection-and-union ratio being lower than a preset threshold.

[0107] In some embodiments of the present disclosure, the content recognition module 840 includes a semantic segmentation submodule and a text recognition submodule. The semantic segmentation submodule is used to perform semantic segmentation on the original image to obtain a semantic segmentation map when it is determined that the original image contains the first traffic sign according to the degree of overlap. The text recognition submodule is used to determine the text content of the first traffic sign based on the semantic segmentation map and the first sign area.

[0108] In some embodiments of the present disclosure, the text recognition submodule includes: a first determination unit, a second determination unit, and a third determination unit. The first determination unit is used to determine a semantic segmentation region for a traffic sign in a semantic segmentation map. The second determination unit is used to determine an image region corresponding to the semantic segmentation region in the original image. The third determination unit is used to determine text content located in an overlapping region between the first sign region and the image region as the text content of the first traffic sign.

[0109] In some embodiments of the present disclosure, the third determination unit includes: a first text recognition subunit, a first overlap determination subunit, and a character extraction subunit. The first text recognition subunit is used to perform text recognition on the semantic segmentation area to obtain multiple characters. The first overlap determination subunit is used to determine the overlapping area between the first sign area and the image area. The character extraction subunit is used to extract the characters located in the overlapping area from the obtained multiple characters as the text content of the first traffic sign.

[0110] In some embodiments of the present disclosure, the third determination unit includes: a second overlap determination subunit and a second text recognition subunit. The second overlap determination subunit is used to determine the overlap area between the first sign area and the image area. The second text recognition subunit is used to perform text recognition on the overlap area to obtain at least one character as the text content of the first traffic sign.

[0111] In some embodiments, the text recognition includes optical character recognition (OCR).

[0112] In some embodiments, the first traffic sign includes but is not limited to a speed limit sign, and the second traffic sign includes but is not limited to at least one of a load limit sign, an axle weight limit sign, a height limit sign and a width limit sign.

[0113] In some embodiments, each of the first detection model and the second detection model includes a Faster Regional Convolutional Neural Network (Faster RCNN) model or a Yolo V3 model.

[0114] It should be noted that the implementation methods, technical problems solved, functions realized, and technical effects achieved of each module / unit / sub-unit in the device part embodiment are the same or similar to the implementation methods, technical problems solved, functions realized, and technical effects achieved of each corresponding step in the method part embodiment, and will not be repeated here.

[0115] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0116] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product. The embodiment of the present disclosure detects the first traffic sign and the second traffic sign on the original image based on two target detection models respectively, and determines whether the original image contains the first traffic sign based on the two results, thereby reducing or even avoiding the possibility of misidentifying the second traffic sign as the first traffic sign, thereby improving the accuracy and efficiency of traffic sign recognition.

[0117] Fig. 9 A schematic block diagram of an example electronic device 900 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0118] like Fig. 9 As shown, the device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0119] A number of components in the device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0120] The computing unit 901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 901 performs the various methods and processes described above, such as a method for identifying a traffic sign. For example, in some embodiments, the method for identifying a traffic sign may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the method for identifying a traffic sign described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform a method for identifying a traffic sign in any other appropriate manner (e.g., by means of firmware).

[0121] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0122] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0123] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0124] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0125] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0126] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0127] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0128] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for identifying a traffic sign, comprising: Using the first detection model to perform detection on the original image for the first traffic sign, and obtaining a first sign area that potentially contains the first traffic sign; Using a second detection model to detect a second traffic sign on the original image, and obtaining a second sign area that potentially contains the second traffic sign; determining a degree of overlap between the first mark area and the second mark area; In a case where it is determined according to the overlap degree that the original image contains the first traffic sign, identifying content of the first traffic sign; Wherein, determining, according to the overlap degree, that the original image contains the first traffic sign comprises: In response to the overlap degree being lower than a preset threshold, it is determined that the original image contains the first traffic sign.

2. The method according to claim 1, wherein: Determining the overlap degree between the first mark area and the second mark area includes: An intersection-over-union ratio of the first mark region and the second mark region is calculated.

3. The method according to claim 1, wherein: The content of identifying the first traffic sign includes: Performing semantic segmentation on the original image to obtain a semantic segmentation map; and Based on the semantic segmentation map and the first sign area, text content of the first traffic sign is determined.

4. The method according to claim 3, wherein: The determining, based on the semantic segmentation map and the first sign area, the text content of the first traffic sign includes: Determining a semantic segmentation region for a traffic sign in the semantic segmentation map; Determining an image region corresponding to the semantic segmentation region in the original image; Determine text content located in an overlapping area between the first sign area and the image area as text content of the first traffic sign.

5. The method according to claim 4, wherein: The step of determining the text content located in the overlapping area between the first logo area and the image area comprises: Performing text recognition on the semantic segmentation area to obtain a plurality of characters; Determine an overlapping area between the first mark area and the image area; Characters located in the overlapping area are extracted from the obtained multiple characters as text content of the first traffic sign.

6. The method according to claim 4, wherein: The step of determining the text content located in the overlapping area between the first logo area and the image area comprises: Determine an overlapping area between the first mark area and the image area; Perform text recognition on the overlapping area to obtain at least one character as text content of the first traffic sign.

7. The method according to claim 5, wherein: The text recognition includes optical character recognition (OCR).

8. The method according to any one of claims 1 to 7, wherein: The first traffic sign includes a speed limit sign, and the second traffic sign includes at least one of a load limit sign, an axle weight limit sign, a height limit sign, and a width limit sign.

9. The method according to any one of claims 1 to 7, wherein: Each of the first detection model and the second detection model includes a fast regional convolutional neural network Faster RCNN model or a Yolo V3 model.

10. A device for identifying traffic signs, comprising: A first detection module, configured to detect a first traffic sign on an original image using a first detection model, and obtain a first sign area that potentially contains the first traffic sign; A second detection module, configured to use a second detection model to detect a second traffic sign on the original image, and obtain a second sign area that potentially contains the second traffic sign; an overlap calculation module, used to determine the overlap degree between the first mark area and the second mark area; A content recognition module, configured to recognize the content of the first traffic sign when it is determined that the original image contains the first traffic sign according to the overlap degree; The content recognition module is used for determining that the original image contains the first traffic sign in response to the overlap degree being lower than a preset threshold.

11. The device according to claim 10, wherein: The overlap calculation module comprises: The overlap calculation submodule is used to calculate the intersection-and-union ratio of the first mark area and the second mark area.

12. The device according to claim 10, wherein: The content identification module comprises: a semantic segmentation submodule, configured to, when it is determined according to the overlap degree that the original image contains the first traffic sign, perform semantic segmentation on the original image to obtain a semantic segmentation map; and The text recognition submodule is used to determine the text content of the first traffic sign based on the semantic segmentation map and the first sign area.

13. The device according to claim 12, wherein: The text recognition submodule includes: A first determining unit, configured to determine a semantic segmentation region for a traffic sign in the semantic segmentation map; A second determining unit, configured to determine an image region corresponding to the semantic segmentation region in the original image; The third determining unit is used to determine text content located in an overlapping area between the first sign area and the image area as the text content of the first traffic sign.

14. The device according to claim 13, wherein: The third determining unit includes: A first character recognition subunit is used to perform character recognition on the semantic segmentation area to obtain a plurality of characters; A first overlap determination subunit, used to determine an overlap area between the first mark area and the image area; The character extraction subunit is used to extract characters located in the overlapping area from the obtained multiple characters as the text content of the first traffic sign.

15. The device according to claim 13, wherein: The third determining unit includes: A second overlap determination subunit, used to determine an overlap area between the first mark area and the image area; The second text recognition subunit is used to perform text recognition on the overlapping area to obtain at least one character as the text content of the first traffic sign.

16. The device according to claim 14, wherein: The text recognition includes optical character recognition.

17. The device according to any one of claims 10 to 16, wherein: The first traffic sign includes a speed limit sign, and the second traffic sign includes at least one of a load limit sign, an axle weight limit sign, a height limit sign, and a width limit sign.

18. The device according to any one of claims 10 to 16, wherein: Each of the first detection model and the second detection model includes a Faster RCNN model or a Yolo V3 model.

19. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

20. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-9.

21. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 9.

Images