Road sign detection method, device, equipment and computer storage medium

Abstract

The embodiments of the present invention provide a road sign detection method, device, equipment and computer storage medium. The method includes: obtaining a first fused image to be identified, the fused image includes the color RGB information of the road sign and the infrared light intensity information of the road sign; determining at least one target detection area of ​​the first fused image to be identified based on the RGB information and the infrared light intensity information, and obtaining a second fused image to be identified; inputting the second fused image to be identified into a road sign detection model to obtain a road sign detection result, and the detection result includes the color information of the road sign, the shape information of the road sign and the integrity information of the road sign. The road sign detection method according to the embodiments of the present invention can effectively improve the processing efficiency of road sign inspection, and improve real-time performance and accuracy.

Description

Technical Field

[0001] The present invention belongs to the field of business support, and in particular relates to a road sign detection method, device, equipment and computer storage medium. Background Art

[0002] Accurately and effectively setting up road signs is one of the ways to solve traffic safety and congestion problems.

[0003] However, since road signs are usually located in outdoor environments, they are affected by the natural environment and human factors, and are inevitably prone to fading, deformation, and even falling off and disappearing. Therefore, road signs need to be inspected and maintained regularly.

[0004] At present, road signs are regularly inspected and maintained by assigning dedicated personnel to specific posts. However, the workload is huge and it is difficult to ensure the timeliness and accuracy of the road sign inspection. Summary of the invention

[0005] The embodiments of the present invention provide a road sign detection method, device, equipment and computer storage medium, which can effectively improve the processing efficiency of road sign inspection and improve the real-time performance and accuracy.

[0006] In a first aspect, an embodiment of the present invention provides a road sign detection method, the method comprising:

[0007] Acquire a first fused image to be identified, the fused image including color RGB information of the road sign and infrared light intensity information of the road sign;

[0008] Determine at least one target to-be-detected area of ​​the first to-be-identified fused image according to the RGB information and the infrared light intensity information, and obtain a second to-be-identified fused image;

[0009] The second fused image to be identified is input into the road sign detection model to obtain a road sign detection result, which includes color information of the road sign, shape information of the road sign and integrity information of the road sign.

[0010] In some implementations of the first aspect, obtaining a first fused image to be recognized includes:

[0011] Acquire at least one first visible light image of a road sign and at least one first infrared image of the road sign, wherein the first visible light image includes RGB information;

[0012] Determine a second visible light image and a second infrared image belonging to the same time and the same position according to the first time information and the first position information of the first visible light image and the second time information and the second position information of the first infrared image;

[0013] A first fused image to be identified is generated according to the second visible light image and the second infrared image.

[0014] In some implementations of the first aspect, obtaining a first fused image to be recognized includes:

[0015] The first fused image to be identified is subjected to noise reduction processing to obtain the first fused image to be identified after noise reduction processing.

[0016] In some implementations of the first aspect, determining at least one target to-be-detected area of ​​the first to-be-identified fused image according to the RGB information and the infrared light intensity information to obtain the second to-be-identified fused image includes:

[0017] Determine a first to-be-detected area of ​​the first to-be-identified fused image according to at least one preset RGB value and RGB information; and

[0018] Determine a second to-be-detected area of ​​the first to-be-identified fused image according to at least one preset temperature difference value and infrared light intensity information;

[0019] At least one target region to be detected of the first fused image to be identified is determined according to the first region to be detected and the second region to be detected, so as to obtain the second fused image to be identified.

[0020] In some implementations of the first aspect, determining at least one target to-be-detected area of ​​the first to-be-recognized fused image includes:

[0021] A first screening is performed on at least one target to-be-detected region of the first to-be-identified fused image according to a preset graphic attribute screening condition to obtain a first screened target to-be-detected region.

[0022] In some implementations of the first aspect, the graphic attribute screening condition includes: a graphic scale condition and / or a graphic area condition.

[0023] In some implementations of the first aspect, performing a first screening on at least one target to-be-detected area of ​​the first to-be-identified fused image according to a preset graphic attribute screening condition includes:

[0024] Performing image enhancement processing on at least one target to-be-detected region of the first to-be-identified fused image to obtain at least one target to-be-detected region after image enhancement processing;

[0025] A first screening is performed on at least one target area to be detected after image enhancement processing according to a preset graphic attribute screening condition to obtain a target area to be detected after the first screening.

[0026] In some implementations of the first aspect, after obtaining the target area to be detected after the first screening, the method further includes:

[0027] The target area to be detected after the first screening is subjected to a second screening according to a preset shape screening condition to obtain the target area to be detected after the second screening.

[0028] In some implementations of the first aspect, performing a second screening on the target area to be detected after the first screening according to a preset shape screening condition to obtain the target area to be detected after the second screening includes:

[0029] Performing edge enhancement processing on the target area to be detected after the first screening to obtain an edge-enhanced target area to be detected;

[0030] The edge-enhanced target detection area is secondly screened according to a preset shape screening condition to obtain a target detection area after the second screening.

[0031] In a second aspect, an embodiment of the present invention provides a road sign detection device, the device comprising:

[0032] An image acquisition module is used to acquire a first fused image to be identified, wherein the fused image includes RGB color information of the road sign and infrared light intensity information of the road sign;

[0033] A detection area acquisition module, used to determine at least one target detection area of ​​the first fused image to be identified according to the RGB information and the infrared light intensity information, and obtain a second fused image to be identified;

[0034] The recognition module is used to input the second fused image to be recognized into the road sign detection model to obtain a road sign detection result, which includes color information of the road sign, shape information of the road sign and integrity information of the road sign.

[0035] In a third aspect, the present invention provides a road sign detection device, comprising: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, it implements the road sign detection method described in the first aspect or any implementable method of the first aspect.

[0036] In a fourth aspect, the present invention provides a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, implement the road sign detection method described in the first aspect or any implementable manner of the first aspect.

[0037] An embodiment of the present invention provides a road sign detection method, which uses a fused image of a road sign including color information and infrared light intensity information to determine the road sign in the fused image, that is, the target area to be detected. Without excessive manual intervention, the relevant information of the road sign in the image can also be quickly identified in a complex and changeable road environment, such as the color information of the road sign, the shape information of the road sign, and the integrity information of the road sign, thereby significantly improving the processing efficiency, real-time performance and accuracy of road sign inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the following is a brief introduction to the drawings required for use in the embodiment of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0039] Figure 1 is a schematic flow chart of a road sign detection method provided by an embodiment of the present invention;

[0040] Figure 2 is a schematic structural diagram of another road sign detection device provided by an embodiment of the present invention;

[0041] Figure 3 is a schematic structural diagram of a road sign detection device provided by an embodiment of the present invention;

[0042] Figure 4 This is a schematic diagram of the deployment of a vehicle-mounted mobile data collection device provided by an embodiment of the present invention;

[0043] Figure 5 This is a schematic diagram of the deployment of another vehicle-mounted mobile data collection device provided by an embodiment of the present invention;

[0044] Figure 6 This is a schematic diagram of the layout of another vehicle-mounted mobile data collection device provided by an embodiment of the present invention;

[0045] Figure 7 It is a structural schematic diagram of a road sign detection device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The features and exemplary embodiments of various aspects of the present invention will be described in detail below. In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only configured to explain the present invention and are not configured to limit the present invention. For those skilled in the art, the present invention can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present invention by illustrating examples of the present invention.

[0047] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0048] The term "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0049] Accurately and effectively setting up road signs is one of the ways to solve traffic safety and congestion problems. Common traffic signs include signs, signboards, markings and other facilities. Traffic signs can provide drivers with a wealth of prohibitions, warnings, instructions and other information, thereby reducing the occurrence of traffic accidents to a certain extent and improving traffic efficiency.

[0050] However, since road signs are usually located in outdoor environments, they are affected by the natural environment and human factors, and are inevitably prone to fading, deformation, and even falling off and disappearing. Therefore, road signs need to be inspected and maintained regularly.

[0051] At present, in order to maintain traffic signs in a timely manner, on the one hand, regular inspection and maintenance of road signs are carried out by assigning dedicated personnel to specific posts, but the workload of this work is huge and it is difficult to ensure the real-time and accuracy of road sign inspection. At the same time, there is a lack of fusion and analysis of multi-source data, which is not suitable for large-scale and full-life road sign management needs.

[0052] On the other hand, by using computer vision processing methods, by establishing image segmentation models, such as one-dimensional histograms, two-dimensional histograms, etc., the collected images are processed. In order to obtain more accurate processing results, the more information needs to be processed, and the amount of calculation will be too large. Moreover, in the process of image acquisition, the appearance of weather conditions has a great influence on the image acquisition results. For example, in foggy weather conditions, road signs will become brighter and lighter than normal; in dusty weather, road signs will become yellower than normal; if there are tree shades, rain and snow particles, etc., the signs will be blocked. Therefore, the images collected in the current solution are very prone to incomplete data and missing signs. As a result, it is impossible to fully realize the detection of all road signs, and it is impossible to reasonably guide the maintenance and repair of further signs, which makes the cost of later manual review huge, and is not conducive to providing recognition references for recognition terminals in future autonomous driving scenarios.

[0053] In response to one or more of the above-mentioned problems, an embodiment of the present invention provides a road sign detection method, which obtains a fused image including road signs, processes the fused image based on the color (RGB) information and infrared light intensity information in the fused image, analyzes and identifies the road signs in the processed fused image, and obtains the detection result of the road signs. It does not require excessive manual intervention, saves human and material resources, is applicable to complex and changeable road environments, and improves the effectiveness of data.

[0054] The following first introduces the road sign detection method provided by the embodiment of the present invention. Figure 1 FIG. 2 is a flow chart of a road sign detection method provided by an embodiment of the present invention. Figure 1 As shown, the method may include the following steps:

[0055] S110: Acquire a first fused image to be identified, where the fused image includes RGB color information of the road sign and infrared light intensity information of the road sign.

[0056] Among them, obtaining the first fused image to be identified can be carried out through the following steps: first, obtaining at least one first visible light image of the road sign and at least one first infrared image of the road sign, the first visible light image including RGB information; then, according to the first time information and first position information of the first visible light image, and the second time information and second position information of the first infrared image, determining the second visible light image and the second infrared image belonging to the same time and the same position; finally, generating the first fused image to be identified based on the second visible light image and the second infrared image.

[0057] In some embodiments, the visible light image acquisition device of the first visible light image may be an industrial camera. The image acquisition device of the first infrared image may be an infrared thermal imager. Since the infrared thermal imager converts the invisible infrared energy emitted by an object into a visible thermal image, different colors on the thermal image may represent different temperatures of the measured object.

[0058] In some embodiments, when using a visible light image acquisition device and an infrared image acquisition device to acquire images, the timestamp of each infrared image and each visible light image acquired is recorded, and the longitude and latitude position information of each infrared image and each visible light image acquired can be determined in combination with the Global Positioning System (GPS). When the first fused image to be identified is obtained, the second visible light image and the second infrared image belonging to the same time and the same position can be obtained through the unique GPS longitude and latitude position corresponding to each time point, and finally, the first fused image to be identified is obtained based on the fusion of the second visible light image and the second infrared image.

[0059] The first fused image to be identified may include RGB information of the visible light image and infrared light intensity information.

[0060] In some embodiments, due to the presence of unavoidable obstructions such as tree shades and buildings, the images collected including road signs will also include these obstructions; in addition, common severe weather conditions mainly include fog, haze, sand and dust, rain and snow, among which impurities with larger and sparse particles are similar to leaves and buildings, and their impact on road signs is also mainly manifested as obstruction. Small and dense particles will produce more obvious refraction and scattering effects after the light passes through, causing the reflection of the road signs to change. Due to these environmental factors, the quality of the collected images will deteriorate, and the images will be unclear.

[0061] Therefore, in some embodiments, in order to obtain a clear image so that subsequent road sign detection and analysis are more accurate, the first fused image to be identified may be subjected to noise reduction processing to obtain the first fused image to be identified after noise reduction processing.

[0062] In some embodiments, a denoising model may be generated based on severe weather conditions, and denoising processing such as inverse transformation may be performed on the input first fused image to be identified to obtain the first fused image to be identified after denoising processing.

[0063] In the embodiment of the present invention, after the first fused image to be recognized is obtained, S120 and S130 may be performed.

[0064] S120 . Determine at least one target to-be-detected area of ​​the first to-be-recognized fused image according to the RGB information and the infrared light intensity information, and obtain a second to-be-recognized fused image.

[0065] Road signs are usually public signs that are a combination of graphics, text, or graphics and text, and are usually yellow, red, green, blue, and white. The shapes of road signs generally include triangles, circles, rectangles, and squares, and the graphic styles of signs mainly include numbers, arrows, and characters. Compared with the surrounding environment, road signs are usually in prominent positions and have easily recognizable colors.

[0066] In some embodiments, to obtain the second fused image to be identified, S121 and S122 may be performed:

[0067] S121. Determine a first area to be detected of the first fused image to be identified according to at least one preset RGB value and RGB information; and determine a second area to be detected of the first fused image to be identified according to at least one preset temperature difference value and infrared light intensity information.

[0068] As a specific example, to obtain the first area to be detected, the pixel value v of the first fused image to be identified can be calculated according to formula (1) = [v R ,v G ,v B ] to facilitate subsequent processing.

[0069]

[0070] Among them, S is the sum of the pixel values ​​of the three components R, G, and B, S = v R +v G +v B .

[0071] Then, the transformed image data is threshold segmented using formula (2) according to the preset RGB value and the RGB information of the transformed image, so as to obtain the first to-be-detected area of ​​the first to-be-identified fused image.

[0072]

[0073] Among them, (x, y) is the pixel coordinate, c represents the preset color, Threshold c Indicates the extraction threshold of the preset color. Road signs are usually red, blue, yellow, white, and green.

[0074] Exemplarily, the preset colors may include red, blue, yellow, white, and green.

[0075] When c is red, Threshold c You can take 0.12;

[0076] When c is blue, Threshold c You can take 0.15;

[0077] When c is yellow, Threshold c It can be 0.1;

[0078] When c is white, Threshold c You can take 0.12;

[0079] When c is green, Threshold c You can take 0.12.

[0080] According to the preset RGB value and the RGB information of the transformed image, the preset color areas in the first fused image to be identified are highlighted respectively to obtain the area information of interest of the road sign, i.e., the first area to be detected. In some embodiments, after the image segmentation work is completed at the corresponding threshold, a binary image corresponding to the first fused image to be identified can be obtained.

[0081] In some embodiments, in order to obtain the second area to be detected, identification detection based on the infrared light intensity information of the first fused image to be identified can be performed. It can be understood that the infrared light intensity information in the first fused image to be identified is obtained based on the conversion of invisible infrared energy into visible thermal image information. Due to the different materials of the road sign area and other parts of the image, a temperature difference will occur between the road sign and the surrounding area. Optionally, the temperature difference point of the first fused image to be identified can be obtained based on at least one preset temperature difference value and the infrared light intensity information of the first fused image to be identified.

[0082] Exemplarily, the infrared light intensity information f(x, y) of the first fused image to be identified is segmented by at least one preset temperature difference value to obtain the second area to be detected g(x, y). The segmentation method according to at least one preset temperature difference value can be:

[0083] When T k <f(x,y)≤T k+1 , when k=0, 1, 2, …, n, g(x, y)=k, where n is a positive integer and k is the label of each different area in the segmented image.

[0084] After the first area to be detected and the second area to be detected are obtained, S122 is executed.

[0085] S122: Determine at least one target region to be detected of the first fused image to be identified according to the first region to be detected and the second region to be detected, and obtain a second fused image to be identified.

[0086] Optionally, according to the first to-be-detected area and the second to-be-detected area, determining at least one target to-be-detected area of ​​the first to-be-identified fused image may retain the detection result according to the following judgment method to obtain at least one target to-be-detected area:

[0087] (1) When the detection results of the first area to be detected and the second area to be detected are the same, the detection results are retained.

[0088] (2) When the detection results of the first area to be detected and the second area to be detected are inconsistent, the detection result of the second area to be detected can be used.

[0089] (3) When the detection results of the first area to be detected and the second area to be detected are inconsistent, and there is no detection result of the second area to be detected, the detection result of the first area to be detected can be retained.

[0090] Optionally, if there is no detection result of the road sign in the first area to be detected and the second area to be detected, it may be that an error occurs in the image acquisition system, and a maintenance prompt message can be sent.

[0091] After the above judgment on the first area to be detected and the second area to be detected, at least one target area to be detected can be directly used as a second fused image to be identified, and the second fused image to be identified is input into the road sign detection model to execute S130.

[0092] S130: Input the second fused image to be identified into a road sign detection model to obtain a road sign detection result, where the detection result includes color information of the road sign, shape information of the road sign, and integrity information of the road sign.

[0093] In some embodiments, when acquiring the original image of the traffic sign, the original image may include interference from some non-traffic signs. In order to eliminate the interference from these non-traffic signs, before executing S130, at least one target detection area of ​​the first fused image to be identified can be determined, and S123 can also be optionally executed.

[0094] S123, performing a first screening on at least one target to be detected region of the first fused image to be identified according to a preset graphic attribute screening condition, to obtain a first screened target to be detected region. The graphic attribute screening condition includes: a graphic ratio condition and / or a graphic area condition.

[0095] Exemplarily, according to step S122, at least one target area to be detected is obtained, and at least one target area to be detected corresponding to one of the preset colors c is recorded as BIc, and each target area to be detected of the preset color c is recorded as Regioni,c, i=1, 2, ..., N, where N is the number of target areas to be detected.

[0096] Combined with the preset graphic ratio condition and the preset graphic area condition, wherein the preset graphic ratio condition is, for example, the aspect ratio, two Boolean operators (4) and (5) are defined according to actual needs, for example:

[0097]

[0098] Among them, Rwh i,c is the aspect ratio of the target area to be detected; R L The lower limit of the preset aspect ratio threshold; R H The preset lower limit of the aspect ratio threshold. For example, R L =0.5, R H =2.

[0099]

[0100] Among them, Area i,c is the graphic area of ​​the target area to be detected; T LA is the lower limit of the preset graphic area threshold; T HA is the upper limit of the preset graph area threshold; for example, T LA =20000, T HA =400.

[0101] The above example is a screening of at least one target area to be detected corresponding to a preset color c among all target areas to be detected. If the preset color c includes multiple colors, a preset graphic attribute screening condition can be set for each color.

[0102] By filtering the target detection area of ​​the first fused image to be identified according to the preset graphic attribute screening condition, some impurity areas can be filtered to exclude the situation where the aspect ratio is too small or too large. Optionally, in order to eliminate the interference factors of some non-traffic signs, at least one target detection area corresponding to one of the preset colors c can be used as an example, and the morphological filter shown in formula (6) can also be combined for screening.

[0103]

[0104] After morphological filtering, the area retained in the first fused image to be identified is used as the target area to be detected, that is, the target region of interest (ROI), to obtain a second fused image to be identified. Optionally, the second fused image to be identified obtained at this time can be input into the road sign detection model to execute S130.

[0105] In some embodiments, due to the influence of weather, light intensity and similar colors, impurities and noise will inevitably exist in the first fused image to be identified, especially for non-traffic signs, such as occlusion of plants such as leaves, which will cause great interference to the detection of S123 or subsequent road signs. Therefore, in order to reduce interference and improve the accuracy of the first screening, optionally, before executing S123, image enhancement processing can be performed on at least one target area to be detected in the first fused image to be identified obtained in S122. After obtaining at least one target area to be detected after image enhancement processing, a first screening is performed on the at least one target area to be detected after image enhancement processing according to preset graphic attribute screening conditions to obtain the target area to be detected after the first screening.

[0106] For the convenience of description, at least one target to-be-detected region of the first to-be-recognized fused image obtained in S122 is referred to as a third to-be-recognized fused image.

[0107] Optionally, the image enhancement processing can be to first perform an erosion operation on the third fused image to be identified to remove and reduce the influence of noise spots. After the dilation operation, the edge data on a larger area can be effectively restored, thereby eliminating the area reduction problem caused by the erosion operation.

[0108] Exemplarily, taking the third fused image to be identified as a binary image as an example, the erosion and dilation processing of the binary image satisfies formula (7).

[0109] (fΘb)(x,y)=max {f(xx′,yy′)-b(x′,y′)|(x′,y′)∈D b} (7)

[0110] The data of each translation position is processed, the sum of the pixel size of the binary image and the value of the rotated structural element is calculated, and the maximum value data is solved to satisfy formula (8).

[0111]

[0112] After processing the data of each translation position, the value of its structural element will reduce the pixel value of the binary image, and the obtained value is the minimum value data. The next step is filling, which is mainly to fill the hole area in the binary image. Through the above image enhancement processing, the reconstruction of the road sign area is realized, and the accuracy of the first screening is improved.

[0113] In some embodiments, optionally, the target area to be detected obtained after the first screening in S123 can be directly used as the second fused image to be identified, and the second fused image to be identified is input into the road sign detection model to execute S130.

[0114] In some embodiments, optionally, in combination with common shapes of road signs, such as triangles, circles, squares, hexagons, and rectangles, in order to improve the accuracy and efficiency of road sign detection, before the target area to be detected after the first screening obtained in S123 is input into the road sign detection model as the second fused image to be identified, the target area to be detected after the first screening can also be subjected to a second screening according to preset shape screening conditions to obtain a target area to be detected after the second screening.

[0115] By performing a second screening on the target area to be detected after the first screening, interference elimination and shape classification processing of the target area to be detected can be achieved, thereby improving the accuracy and efficiency of road sign detection.

[0116] In some embodiments, in order to enhance the gap between the target area to be detected and the background and improve the accuracy of the second screening, S124 is optionally performed.

[0117] S124, first perform edge enhancement processing on the target area to be detected after the first screening to obtain the edge-enhanced target area to be detected; then perform a second screening on the edge-enhanced target area to be detected according to a preset shape screening condition to obtain the target area to be detected after the second screening.

[0118] For the convenience of description, the target area to be detected after the first screening is called the fourth fused image to be identified.

[0119] Exemplarily, an edge detection matrix template may be used to perform a matrix convolution operation on the fourth fused image to be identified, so as to obtain an edge-enhanced fourth fused image to be identified.

[0120] The convolution operation multiplies each pixel in the image area used by each element of the edge detection matrix template (weight matrix), and the sum of all products is the new center pixel value of the area.

[0121] For example, the fourth fused image to be identified is R:

[0122]

[0123] Matrix template G for edge detection:

[0124]

[0125] The new center pixel value R5 can be expressed as:

[0126] R5=R1G1+R2G2+R3G3+R4G4+R5G5+R6G6+R7G7+R8G8+R9G9

[0127] After performing matrix convolution operation on the fourth fused image to be identified, the Sobel operator composed of gradient and difference principles is used to highlight the edge by weighting to the center point. At the same time, the image segmentation algorithm with adjustable threshold is combined to distinguish cracks and background, for example, irrelevant areas and impurities are converted to black, and the marked area is converted to white, to achieve edge enhancement and extraction, effectively improving the accuracy of the second screening.

[0128] Therefore, in the embodiment of the present invention, by inputting the second screened target area to be detected obtained in S124 as the second fused image to be identified into the road sign detection model, the accuracy and efficiency of road sign detection can be effectively improved.

[0129] In some embodiments, the road sign detection model may be a classification convolutional neural network model (Visual Geometry Group Network, VGGNet) obtained by pre-training with a sufficient number of sign image samples for distinguishing complete signs from incomplete signs.

[0130] The specific training process of the VGGNet network model may be: preprocessing the labeled road sign training sample images, inputting the preprocessed road sign training sample images into the VGGNet network model, and adding known road attribute category labels.

[0131] Cross entropy is used as the loss function and stochastic gradient descent (SGD) is used to calculate the back propagation gradient. The network parameters are optimized according to the gradient. When the loss function gradually decreases and finally approaches 0, the trained VGGNet network model, i.e., the road sign detection model, is obtained.

[0132] By classifying and identifying the extracted images through the road sign detection model, the integrity attributes of the road signs can be accurately and efficiently determined. At the same time, the GPS location and attribute information of the road signs corresponding to the recognition results can be recorded in combination with the time and location information corresponding to the fused image. The attribute information can include the color information, shape information and related information on whether the road signs are complete, as well as integrity-related information such as whether the road signs are missing or damaged.

[0133] The road sign detection method provided by the embodiment of the present invention adopts a fused image of the road sign including color information and infrared light intensity information to determine the road sign in the fused image, that is, the target area to be detected. Without excessive manual intervention, the relevant information of the road sign in the image, such as the color information of the road sign, the shape information of the road sign and the integrity information of the road sign, can be quickly identified even in a complex and changeable road environment, thereby significantly improving the processing efficiency, real-timeness and accuracy of road sign inspection.

[0134] In order to describe the road sign detection method more clearly, Figure 2 is a flow chart of a road sign detection method according to another embodiment of the present invention.

[0135] S210: Acquire a visible light image and an infrared image.

[0136] In some embodiments, a visible light image can be obtained by a visible light image acquisition device, and an infrared image can be obtained by an infrared thermal imager. In order to obtain a visible light image with high stability and little interference, the visible light image acquisition device can be an industrial camera, and the infrared image acquisition device can be an infrared thermal imager.

[0137] In some embodiments, when collecting visible light images and infrared images, time information of each visible light image and each infrared image may be determined, and location information of each visible light image and each infrared image may be determined.

[0138] S220: Obtain a first fused image to be identified.

[0139] In some embodiments, the visible light image and the infrared image at the same time and the same location are acquired based on the time information and the location information of each visible light image and each infrared image.

[0140] The visible light image and the infrared image at the same time and the same position are fused to obtain a first fused image to be identified. In an example, the first fused image to be identified may include RGB information of the visible light image and infrared light intensity information.

[0141] In some embodiments, whether to execute S230 may be determined first based on the acquisition environment of the visible light image and the infrared image.

[0142] For example, due to environmental factors such as tree shades, buildings and other unavoidable obstructions and adverse weather conditions, the quality of the collected image may deteriorate or the image may be unclear. Therefore, in order to obtain a clear image so that subsequent road sign detection and analysis are more accurate, S230 may be performed first, and S240 may be performed based on the first fused image to be identified after noise reduction processing.

[0143] S230 , performing noise reduction processing on the first fused image to be identified to obtain the first fused image to be identified after the noise reduction processing.

[0144] In some embodiments, the first fused image to be identified may be subjected to noise reduction processing by methods such as median filtering and wavelet denoising to reduce the interference of environmental factors on the image. The method may be selected according to the actual environment.

[0145] S240: Acquire at least one target to-be-detected region of the first to-be-recognized fused image.

[0146] In embodiment S240 of the present invention, the first area to be detected can be determined according to the RGB information, and the second area to be detected can be determined according to the infrared light intensity information. Then, the target area to be detected (target ROI) of the first fused image to be identified can be determined according to the first area to be detected and the second area to be detected.

[0147] In some embodiments, the first area to be detected is determined from the first fused image to be identified. First, the first fused image to be identified can be converted into RGB to obtain RGB information of the first fused image to be identified. The first fused image to be identified is converted into RGB according to formula (1) to obtain RGB information of the first fused image to be identified, wherein the RGB information of the first fused image to be identified may include fused image data of a preset color. The preset color may be red, blue, yellow, white, or green.

[0148] Next, the preset color threshold is segmented according to the RGB information of the first fused image to be identified and the extraction threshold of the preset color.

[0149] In one example, threshold segmentation is performed on the fused image data of the preset color according to the extraction threshold of the preset color and formula (2) to obtain the first area to be detected.

[0150] Since road signs are usually red, blue, yellow, white and green, in some embodiments, when the preset colors include red, blue, yellow, white and green, the area of ​​interest of the road signs is located for each color respectively, and the area of ​​interest of the road signs determined by each preset color is integrated to obtain the first area to be detected in the first fused image to be identified.

[0151] Due to the different materials of the road sign area and other parts of the image, a temperature difference will be generated between the road sign and the surrounding area. In some embodiments, the infrared light intensity information f(x, y) of the first fused image to be identified is segmented by at least one preset temperature difference value to obtain the second area to be detected g(x, y). The method of segmenting according to at least one preset temperature difference value can be:

[0152] When T k <f(x,y)≤T k+1 , when k=0, 1, 2, …, n, g(x, y)=k, where n is a positive integer and k is the label of each different area in the segmented image.

[0153] In one example, a one-dimensional histogram or a two-dimensional histogram can be established to obtain a preset temperature difference threshold value based on the environmental information when the infrared image is collected. For example, in order to reduce the amount of calculation and increase the processing speed, a one-dimensional histogram can be established to obtain the preset temperature difference threshold value.

[0154] For example, in order to improve the processing accuracy of the image, a two-dimensional histogram can be established to obtain a preset temperature difference threshold. Next, based on the established histogram, a method such as the maximum entropy method or the maximum inter-class variance method can be selected to determine the criteria for obtaining the preset temperature difference threshold. When obtaining the preset temperature difference threshold according to the criteria, if the actual established histogram is not complicated, the exhaustive method can be used to obtain the preset temperature difference threshold. If the actual established histogram is relatively complex, a swarm intelligence algorithm can be used to obtain the threshold, such as a particle swarm algorithm, which is not specifically limited here.

[0155] In the embodiment of the present invention, next, the target area to be detected of the first fused image to be identified, that is, the second fused image to be identified, may be determined according to the first area to be detected and the second area to be detected.

[0156] In one example, when the detection results of the first area to be detected and the second area to be detected are the same, the detection results are retained. When the detection results of the first area to be detected and the second area to be detected are inconsistent, the detection results of the second area to be detected can be used. When the detection results of the first area to be detected and the second area to be detected are inconsistent, and there is no detection result in the second area to be detected, the detection results of the first area to be detected can be retained.

[0157] Optionally, if there is no detection result of the road sign in the first area to be detected and the second area to be detected, it may be that an error occurs in the image acquisition system, and a maintenance prompt message can be sent.

[0158] After the above judgment on the first area to be detected and the second area to be detected, at least one target area to be detected can be directly used as a second fused image to be identified, and the second fused image to be identified is input into the road sign detection model to directly execute S270.

[0159] In order to more accurately determine the target to-be-detected area of ​​the first to-be-identified fused image, S250 and S260 may be performed.

[0160] S250: Obtain the target detection area after the first screening.

[0161] In some embodiments, at least one target to be detected region of the first fused image to be identified may be subjected to image enhancement processing to obtain at least one target to be detected region after image enhancement processing, thereby completing the reconstruction of the road sign region. Next, at least one target to be detected region after image enhancement processing may be subjected to a first screening according to a preset graphic attribute screening condition to obtain a target to be detected region after the first screening.

[0162] For specific image enhancement methods, please refer to the present invention Figure 1 The description of image enhancement processing in will not be repeated here.

[0163] In some embodiments, in combination with common shapes of road signs, such as triangles, circles, squares, hexagons, and rectangles, etc., in order to improve the accuracy and efficiency of road sign detection, S260 may be selected for execution.

[0164] S260: Obtain the target detection area after the second screening.

[0165] In one example, a second screening is performed on the target area to be detected after the first screening according to a preset shape screening condition, so as to obtain the target area to be detected after the second screening.

[0166] By performing a second screening on the target area to be detected after the first screening, interference elimination and shape classification processing of the target area to be detected can be achieved, so as to improve the accuracy and efficiency of road sign detection.

[0167] The target detection area obtained after the second screening in S260 is input into the road sign detection model as the second fused image to be identified, and S270 is executed, which can effectively improve the accuracy and efficiency of road sign detection.

[0168] S270: Input the second fused image to be identified into a road sign detection model to obtain a road sign detection result.

[0169] The road sign detection method described in the embodiment of the present invention uses a fused image of road signs including color information and infrared light intensity information to determine the road signs in the fused image, that is, the target area to be detected. Without excessive manual intervention, the relevant information of the road signs in the image, such as the color information of the road signs, the shape information of the road signs, and the integrity information of the road signs, can be quickly identified even in complex and changeable road environments, thereby significantly improving the processing efficiency, real-time performance and accuracy of road sign inspection.

[0170] Figure 3 is a schematic diagram of the structure of a road sign detection device provided by an embodiment of the present invention. Figure 3 As shown, the road sign detection device may include: an image acquisition module 310 , a detection area acquisition module 320 , and a recognition module 330 .

[0171] The image acquisition module 310 is used to acquire a first fused image to be identified, where the fused image includes RGB color information of the road sign and infrared light intensity information of the road sign.

[0172] The detection area acquisition module 320 is used to determine at least one target detection area of ​​the first fused image to be identified according to the RGB information and the infrared light intensity information, and obtain the second fused image to be identified.

[0173] The recognition module 330 is used to input the second fused image to be recognized into the road sign detection model to obtain a road sign detection result, which includes color information of the road sign, shape information of the road sign and integrity information of the road sign.

[0174] In some embodiments, the image acquisition module 310 is also used to acquire at least one first visible light image of a road sign and at least one first infrared image of a road sign, the first visible light image including RGB information; determine a second visible light image and a second infrared image belonging to the same time and the same position based on the first time information and the first position information of the first visible light image, and the second time information and the second position information of the first infrared image; and generate a first fused image to be identified based on the second visible light image and the second infrared image.

[0175] In some embodiments, the image acquisition module 310 is further configured to perform noise reduction processing on the first fused image to be identified, so as to obtain the first fused image to be identified after the noise reduction processing.

[0176] In some embodiments, the detection area acquisition module 320 is also used to determine a first area to be detected of the first fused image to be identified based on at least one preset RGB value and RGB information; and to determine a second area to be detected of the first fused image to be identified based on at least one preset temperature difference value and infrared light intensity information; and to determine at least one target area to be detected of the first fused image to be identified based on the first area to be detected and the second area to be detected, to obtain a second fused image to be identified.

[0177] In some embodiments, the detection region acquisition module 320 is further used to perform a first screening on at least one target detection region of the first fused image to be identified according to a preset graphic attribute screening condition to obtain the target detection region after the first screening.

[0178] In some embodiments, the graphic attribute screening condition may include: a graphic scale condition and / or a graphic area condition.

[0179] In some embodiments, the detection area acquisition module 320 is also used to perform image enhancement processing on at least one target area to be detected in the first fused image to be identified, so as to obtain at least one target area to be detected after image enhancement processing; and perform a first screening on at least one target area to be detected after image enhancement processing according to preset graphic attribute screening conditions, so as to obtain the target area to be detected after the first screening.

[0180] In some embodiments, the detection region acquisition module 320 is further used to perform a second screening on the target region to be detected after the first screening according to a preset shape screening condition to obtain the target region to be detected after the second screening.

[0181] In some embodiments, the detection area acquisition module 320 is also used to perform edge enhancement processing on the target area to be detected after the first screening to obtain an edge-enhanced target area to be detected; and perform a second screening on the edge-enhanced target detection area according to a preset shape screening condition to obtain the target area to be detected after the second screening.

[0182] In some embodiments, the road sign detection device provided by the embodiments of the present invention can be applied to a vehicle-mounted mobile acquisition device, wherein the vehicle-mounted mobile acquisition device may include a visible light image acquisition device, an infrared image acquisition device and a GPS positioning system device.

[0183] Among them, the visible light image acquisition device is used to acquire visible light images including timestamps, and the infrared image acquisition device can be used to acquire infrared images including timestamps. Combined with the GPS positioning system and timestamps, the longitude and latitude position information of the infrared image and the visible light image corresponding to the timestamp can be obtained. Finally, according to the timestamp and longitude and latitude position information, the two visible light and infrared images at the same time and the same position can be fused into a fused image to be identified. When realizing image fusion, multiple frames of images can be extracted from the video collected by the mobile inspection device based on the Matlab program, and the two functions split and merge can be used to realize the fusion of the two images.

[0184] In some embodiments, the infrared image acquisition device and the visible light image acquisition device can be set to the same sampling frequency to record the entire process to meet the coverage of the entire road width data. In some embodiments, an uncooled focal plane temperature measurement type infrared thermal imager can be used. For example, among the main imaging parameters of the DM60-S type online infrared thermal imager, the image resolution can be set to 384×288; when the temperature is 30°C, the thermal sensitivity is set to 0.06°C; and the shooting frame rate is set to 50Hz.

[0185] As a specific example, according to the timestamps of the images collected by the infrared image acquisition device and the visible light image acquisition device, a program is used to read one frame from each of the infrared image acquisition device and the visible light image acquisition device, and two frames are saved, corresponding to a time point, and then this cycle is repeated for different time points, and then the unique GPS longitude and latitude are corresponded according to the timestamp. Finally, each time point corresponds to a unique GPS longitude and latitude position, and a fused image is obtained, which contains the RGB information of the visible light image and the infrared light intensity information.

[0186] In some embodiments, in order to facilitate the fusion processing of the collected infrared image and visible light image, Figure 4 , Figure 5 and Figure 6 The various deployment modes of the visible light image acquisition device, infrared image acquisition device and GPS positioning system device included in the vehicle-mounted mobile acquisition device are shown respectively, for example, Figure 4 The three-point tandem arrangement shown in the figure, Figure 5 The three-point horizontal arrangement shown in the figure, Figure 6 The two-to-one double-row arrangement shown.

[0187] Understandably, Figure 4 , Figure 5 and Figure 6The arrangement of the visible light image acquisition device, infrared image acquisition device and GPS positioning system device included in the vehicle-mounted mobile acquisition device shown is only an optional arrangement provided by the embodiment of the present invention and is not applicable to limit the present invention.

[0188] In some embodiments, the vehicle-mounted mobile acquisition device provided by the embodiments of the present invention may also include an acquisition card and a communication device, wherein the acquisition card is used to assist in storing the acquired images in the computer hard disk for convenient retrieval and processing at any time; the communication device may be, for example, a fourth generation mobile communication network (4th Generation of mobile networks, 4G) or 5G communication device, which is used to realize real-time remote transmission of data information.

[0189] It can be understood that the road sign detection device of the embodiment of the present invention can correspond to the executor of the road sign detection method described in the embodiment of the present invention. The specific details of the operation and / or function of each module / unit of the road sign detection device can refer to the description of the corresponding parts in the road sign detection described in the above embodiment of the present invention. For the sake of brevity, they will not be repeated here.

[0190] The road sign detection device provided by the embodiment of the present invention uses a fused image of the road sign including color information and infrared light intensity information to determine the road sign in the fused image, that is, the target area to be detected. Without excessive manual intervention, the relevant information of the road sign in the image, such as the color information of the road sign, the shape information of the road sign and the integrity information of the road sign, can be quickly identified even in a road environment with complex and changeable conditions, thereby significantly improving the processing efficiency, real-time performance and accuracy of road sign inspection.

[0191] Figure 7 It is a schematic diagram of the hardware structure of a road sign detection device provided by an embodiment of the present invention.

[0192] like Figure 7 As shown, the road sign detection device in this embodiment includes an input device 701, an input interface 702, a central processing unit 703, a memory 704, an output interface 705, and an output device 706. The input interface 702, the central processing unit 703, the memory 704, and the output interface 705 are interconnected via a bus 710, and the input device 701 and the output device 706 are connected to the bus 710 via the input interface 702 and the output interface 705, respectively, and then connected to other components of the road sign detection device.

[0193] Specifically, the input device 701 receives input information from the outside and transmits the input information to the central processing unit 703 through the input interface 702; the central processing unit 703 processes the input information based on the computer executable instructions stored in the memory 704 to generate output information, temporarily or permanently stores the output information in the memory 704, and then transmits the output information to the output device 706 through the output interface 705; the output device 706 outputs the output information to the outside of the road sign detection device for user use.

[0194] That is to say, Figure 7 The road sign detection device shown may also be implemented to include: a memory storing computer executable instructions; and a processor, which can implement the road sign detection method described in conjunction with the example of the present invention when executing the computer executable instructions.

[0195] In one embodiment, Figure 7 The road sign detection device shown includes: a memory 704 for storing programs; and a processor 703 for running the programs stored in the memory to execute the road sign detection method described in the embodiment of the present invention.

[0196] An embodiment of the present invention further provides a computer-readable storage medium having computer program instructions stored thereon; when the computer program instructions are executed by a processor, the road sign detection method described in the embodiment of the present invention is implemented.

[0197] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0198] The functional blocks shown in the structural block diagram described above can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), appropriate firmware, plug-in, function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or communication link by a data signal carried in a carrier. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, read-only memories (Read-Only Memory, ROM), flash memory, erasable ROM (Erasable Read Only Memory, EROM), floppy disks, compact discs (Compact Disc Read-Only Memory, CD-ROM), optical discs, hard disks, optical fiber media, radio frequency (Radio Frequency, RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0199] It should also be noted that the exemplary embodiments mentioned in the present invention describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiments, or in a different order from the embodiments, or several steps can be performed simultaneously.

[0200] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0201] The above is only a specific implementation of the present invention. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the system, module and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. It should be understood that the protection scope of the present invention is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be covered within the protection scope of the present invention.

Claims

1. A road sign detection method, characterized in that: include: Acquire a first fused image to be identified, wherein the fused image includes RGB color information of the road sign and infrared light intensity information of the road sign; Determine at least one target to-be-detected area of ​​the first to-be-identified fused image according to the RGB information and the infrared light intensity information, and obtain a second to-be-identified fused image; Inputting the second fused image to be identified into a road sign detection model to obtain a road sign detection result, wherein the detection result includes color information of the road sign, shape information of the road sign, and integrity information of the road sign; The step of determining at least one target to-be-detected area of ​​the first to-be-identified fused image according to the RGB information and the infrared light intensity information to obtain a second to-be-identified fused image includes: Determine a first to-be-detected area of ​​the first to-be-identified fused image according to at least one preset RGB value and the RGB information; and Determine a second to-be-detected area of ​​the first to-be-identified fused image according to at least one preset temperature difference value and the infrared light intensity information; Determine at least one target region to be detected of the first fused image to be identified according to the first region to be detected and the second region to be detected, and obtain a second fused image to be identified; The determining of at least one target to-be-detected area of ​​the first to-be-identified fused image includes: When the detection results of the first to-be-detected area and the second to-be-detected area are the same, determining the detection result as the second to-be-identified fused image; When the detection results of the first to-be-detected area and the second to-be-detected area are different, determining the detection result of the second to-be-detected area as a second to-be-identified fused image; When the detection results of the first to-be-detected area and the second to-be-detected area are different and there is no detection result of the second detection area, the detection result of the first to-be-detected area is determined to be the second to-be-identified fused image.

2. The method according to claim 1, characterized in that The step of obtaining a first fused image to be identified includes: Acquire at least one first visible light image of the road sign and at least one first infrared image of the road sign, wherein the first visible light image includes RGB information; Determine a second visible light image and a second infrared image belonging to the same time and the same position according to the first time information and the first position information of the first visible light image and the second time information and the second position information of the first infrared image; The first to-be-recognized fused image is generated according to the second visible light image and the second infrared image.

3. The method according to claim 1, characterized in that The step of obtaining a first fused image to be identified includes: The first fused image to be identified is subjected to noise reduction processing to obtain the first fused image to be identified after the noise reduction processing.

4. The method according to claim 1, characterized in that: The determining of at least one target to-be-detected area of ​​the first to-be-identified fused image includes: A first screening is performed on at least one target to-be-detected region of the first to-be-identified fused image according to a preset graphic attribute screening condition to obtain the target to-be-detected region after the first screening.

5. The method according to claim 4, characterized in that The graphic attribute screening condition includes: a graphic scale condition and / or a graphic area condition.

6. The method according to claim 4, characterized in that The first screening of at least one target to-be-detected area of ​​the first to-be-identified fused image according to a preset graphic attribute screening condition includes: Performing image enhancement processing on at least one target to-be-detected region of the first to-be-identified fused image to obtain at least one target to-be-detected region after image enhancement processing; A first screening is performed on the target area to be detected after the at least one image enhancement process according to a preset graphic attribute screening condition to obtain the target area to be detected after the first screening.

7. The method according to any one of claims 4 to 6, characterized in that: After obtaining the target area to be detected after the first screening, the method further includes: A second screening is performed on the target area to be detected after the first screening according to a preset shape screening condition to obtain the target area to be detected after the second screening.

8. The method according to claim 7, characterized in that The step of performing a second screening on the target area to be detected after the first screening according to a preset shape screening condition to obtain the target area to be detected after the second screening includes: Performing edge enhancement processing on the target area to be detected after the first screening to obtain an edge-enhanced target area to be detected; The edge-enhanced target detection area is subjected to a second screening according to a preset shape screening condition to obtain a target detection area after the second screening.

9. A road sign detection device, characterized in that: The device comprises: An image acquisition module, used to acquire a first fused image to be identified, wherein the fused image includes RGB color information of the road sign and infrared light intensity information of the road sign; A detection area acquisition module, used to determine at least one target detection area of ​​the first fused image to be identified according to the RGB information and the infrared light intensity information, and obtain a second fused image to be identified; a recognition module, configured to input the second fused image to be recognized into a road sign detection model to obtain a road sign detection result, wherein the detection result includes color information of the road sign, shape information of the road sign, and integrity information of the road sign; The step of determining at least one target to-be-detected area of ​​the first to-be-identified fused image according to the RGB information and the infrared light intensity information to obtain a second to-be-identified fused image includes: Determine a first to-be-detected area of ​​the first to-be-identified fused image according to at least one preset RGB value and the RGB information; and Determine a second to-be-detected area of ​​the first to-be-identified fused image according to at least one preset temperature difference value and the infrared light intensity information; Determine at least one target region to be detected of the first fused image to be identified according to the first region to be detected and the second region to be detected, and obtain a second fused image to be identified; The determining of at least one target to-be-detected area of ​​the first to-be-identified fused image includes: When the detection results of the first to-be-detected area and the second to-be-detected area are the same, determining the detection result as the second to-be-identified fused image; When the detection results of the first to-be-detected area and the second to-be-detected area are different, determining the detection result of the second to-be-detected area as a second to-be-identified fused image; When the detection results of the first to-be-detected area and the second to-be-detected area are different and there is no detection result of the second detection area, the detection result of the first to-be-detected area is determined to be the second to-be-identified fused image.

10. A road sign detection device, characterized in that: The device comprises: a processor, and a memory storing computer program instructions; The processor reads and executes the computer program instructions to implement the road sign detection method as described in any one of claims 1-8.

11. A computer storage medium, characterized in that: The computer storage medium stores computer program instructions, and when the computer program instructions are executed by the processor, the road sign detection method according to any one of claims 1 to 8 is implemented.

Images