Road sign damage detection method and device and storage medium

Abstract

This application discloses a method, apparatus, and storage medium for detecting road sign damage, belonging to the field of intelligent transportation technology. In this application embodiment, multiple sign image sets are pre-stored. Based on the sign type and location information of the sign to be detected, a sign image set matching the sign type and location information of the sign to be detected is obtained from the multiple sign image sets. This increases the probability that the obtained sign image set contains the sign to be detected. Furthermore, since each sign image set includes multiple pre-collected images of road signs that have already been installed, using the obtained sign image set as a reference, the similarity between the image of the sign to be detected and the sign images in the obtained sign image set is used to determine whether the sign to be detected is damaged. This allows for a more accurate determination of whether the sign to be detected is damaged, thus providing data support for timely replacement of road signs.

Description

Technical Field

[0001] This application relates to the field of traffic management, and in particular to a method, device and storage medium for detecting damage to road signs. Background Technology

[0002] In road traffic, road signs are among the most basic and important safety facilities. They use specific colors, graphics, symbols, lines, and text to convey relevant road information to motor vehicles, non-motorized vehicles, and pedestrians, providing a safe and efficient road environment. However, this all depends on the road signs being in good condition. When road signs develop cracks, rust, or other damage, they cannot effectively guide traffic and may even lead to accidents. Therefore, regular damage inspection and maintenance of road signs are crucial. Summary of the Invention

[0003] This application provides a method, apparatus, and storage medium for detecting road sign damage, which can more accurately detect the damage to road signs, thereby providing data support for timely replacement of road signs. The technical solution is as follows:

[0004] On the one hand, a method for detecting damage to road signs is provided, the method comprising:

[0005] Obtain the sign type and location information of the sign to be detected contained in the first sign image;

[0006] Based on the sign type and location information of the sign to be detected, a first sign image set is obtained from multiple stored sign image sets, each sign image set including multiple sign images of a road sign that have been pre-captured;

[0007] The sign to be detected is damaged based on the similarity between the first sign image and the sign images in the first sign image set.

[0008] Optionally, each of the multiple sign image sets corresponds to a sign type and location information;

[0009] The step of obtaining a first sign image set from multiple stored sign image sets based on the sign type and location information of the sign to be detected includes:

[0010] Candidate sign image sets are obtained from the plurality of sign image sets, and the distance between the location information corresponding to each candidate sign image set and the location information of the sign to be detected is within a reference threshold range;

[0011] A second sign image set is obtained from the acquired candidate sign image set, wherein the sign type corresponding to each second sign image set is the same as the sign type of the sign to be detected;

[0012] Select the first sign image set from the second sign image set.

[0013] Optionally, selecting the first sign image set from the second sign image set includes:

[0014] Determine the similarity between the first sign image and each sign image in each of the second sign image sets;

[0015] Based on the similarity between the first sign image and each sign image in each second sign image set, determine the average similarity between the first sign image and the sign images in the corresponding second sign image set;

[0016] The first sign image set is selected from the plurality of second sign image sets, based on the second sign image set with the highest average similarity.

[0017] Optionally, the step of performing damage detection on the sign to be detected based on the similarity between the first sign image and sign images in the first sign image set includes:

[0018] Obtain the similarity reference value of the first set of sign images;

[0019] Damage detection is performed on the sign to be detected based on the similarity between the first sign image and each sign image in the first sign image set and the similarity reference value.

[0020] Optionally, the step of performing damage detection on the sign to be detected based on the similarity between the first sign image and each sign image in the first sign image set and the similarity reference value includes:

[0021] Based on the similarity between the first sign image and each sign image in the first sign image set, determine the average similarity between the first sign image and the first sign image set.

[0022] If the average similarity between the first sign image and the first set of sign images is less than the similarity reference value, then the sign to be detected is determined to be damaged.

[0023] Optionally, the step of performing damage detection on the sign to be detected based on the similarity between the first sign image and each sign image in the first sign image set and the similarity reference value further includes:

[0024] Determine the absolute value of the difference between the average similarity between the first sign image and the first set of sign images and the similarity reference value;

[0025] The degree of damage to the sign to be inspected is determined based on the reference difference range in which the absolute value of the difference falls.

[0026] Optionally, the method further includes:

[0027] Acquire multiple pre-collected sign images and the sign type and location information of the road signs in each sign image;

[0028] Based on the sign type and location information of the road signs in each sign image, the multiple sign images are classified to obtain multiple sign image sets, each sign image set corresponding to a sign type and location information;

[0029] Determine the similarity reference value for each set of sign images.

[0030] Optionally, determining the similarity reference value for each set of sign images includes:

[0031] Obtain the image quality score for each sign image in the third sign image set, where the third sign image set is the set of sign images for any road sign;

[0032] Based on the image quality score of each sign image in the third sign image set and the similarity between each sign image and each other sign image in the other sign images, a similarity reference value for the third sign image set is determined.

[0033] Optionally, determining the similarity reference value of the third sign image set based on the image quality score of each sign image in the third sign image set and the similarity between each sign image and each other sign image in the other sign images (excluding itself) includes:

[0034] Based on the image quality score of each sign image in the third sign image set and the similarity between the corresponding sign image and each sign image in other sign images excluding itself, the weighted average of the similarity of the corresponding sign image is determined.

[0035] Based on the weighted average similarity and image quality score of each sign image in the third sign image set, the weighted average similarity of the third sign image set is determined.

[0036] Delete the sign images in the third sign image set whose weighted similarity average is less than the weighted similarity average of the third sign image set, and determine the similarity reference value of the third sign image set based on the weighted similarity average and image quality score of each remaining sign image.

[0037] On the other hand, a road sign damage detection device is provided, the device comprising:

[0038] The first acquisition module is used to acquire the sign type and location information of the sign to be detected contained in the first sign image;

[0039] The second acquisition module is used to acquire a first sign image set from multiple stored sign image sets according to the sign type and location information of the sign to be detected. Each sign image set includes multiple sign images of a road sign that have been pre-captured.

[0040] The detection module is used to perform damage detection on the sign to be detected based on the similarity between the first sign image and the sign images in the first sign image set.

[0041] Optionally, each of the multiple sign image sets corresponds to a sign type and location information;

[0042] The second acquisition module includes:

[0043] The first acquisition submodule is used to acquire candidate sign image sets from the plurality of sign image sets, wherein the distance between the location information corresponding to each candidate sign image set and the location information of the sign to be detected is within a reference threshold range;

[0044] The second acquisition submodule is used to acquire a second sign image set from the acquired candidate sign image set, wherein the sign type corresponding to each second sign image set is the same as the sign type of the sign to be detected;

[0045] The selection submodule is used to select the first sign image set from the second sign image set.

[0046] Optionally, the selection submodule is used for:

[0047] Determine the similarity between the first sign image and each sign image in each of the second sign image sets;

[0048] Based on the similarity between the first sign image and each sign image in each second sign image set, determine the average similarity between the first sign image and the sign images in the corresponding second sign image set;

[0049] The first sign image set is selected from the plurality of second sign image sets, based on the second sign image set with the highest average similarity.

[0050] Optionally, the detection module includes:

[0051] The third acquisition submodule is used to acquire similarity reference values ​​of the first sign image set;

[0052] The detection submodule is used to perform damage detection on the sign to be detected based on the similarity between the first sign image and each sign image in the first sign image set and the similarity reference value.

[0053] Optionally, the detection submodule is used for:

[0054] Based on the similarity between the first sign image and each sign image in the first sign image set, determine the average similarity between the first sign image and the first sign image set.

[0055] If the average similarity between the first sign image and the first set of sign images is less than the similarity reference value, then the sign to be detected is determined to be damaged.

[0056] Optionally, the detection submodule is further configured to:

[0057] Determine the absolute value of the difference between the average similarity between the first sign image and the first set of sign images and the similarity reference value;

[0058] The degree of damage to the sign to be inspected is determined based on the reference difference range in which the absolute value of the difference falls.

[0059] Optionally, the device further includes:

[0060] The third acquisition module is used to acquire multiple pre-collected sign images and the sign type and location information of the road signs in each sign image;

[0061] The classification module is used to classify the multiple sign images according to the sign type and location information of the road signs in each sign image, so as to obtain multiple sign image sets, each sign image set corresponding to a sign type and location information;

[0062] The determination module is used to determine the similarity reference value for each set of sign images.

[0063] Optionally, the determining module includes:

[0064] The fourth acquisition submodule is used to acquire the image quality score of each sign image in the third sign image set, wherein the third sign image set is the set of sign images of any road sign;

[0065] The determination submodule is used to determine the similarity reference value of the third sign image set based on the image quality score of each sign image in the third sign image set and the similarity between each sign image and each other sign image in other sign images excluding itself.

[0066] Optionally, the determining submodule is used for:

[0067] Based on the image quality score of each sign image in the third sign image set and the similarity between the corresponding sign image and each sign image in other sign images excluding itself, the weighted average of the similarity of the corresponding sign image is determined.

[0068] Based on the weighted average similarity and image quality score of each sign image in the third sign image set, the weighted average similarity of the third sign image set is determined.

[0069] Delete the sign images in the third sign image set whose weighted similarity average is less than the weighted similarity average of the third sign image set, and determine the similarity reference value of the third sign image set based on the weighted similarity average and image quality score of each remaining sign image.

[0070] On the other hand, a computer device is provided, the computer device including a processor and a memory, the memory for storing computer programs, and the processor for executing the programs stored in the memory to implement the steps of the road sign damage detection method described above.

[0071] On the other hand, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when the computer program is executed by a computer, the steps of the road sign damage detection method described above are implemented.

[0072] On the other hand, a computer program product containing instructions is provided, which, when run on a computer, causes the computer to perform the steps of the road sign damage detection method described above.

[0073] The beneficial effects of the technical solutions provided in this application include at least the following:

[0074] In this embodiment, multiple sign image sets are pre-stored. A first sign image set is obtained from these pre-stored sets based on the sign type and location information of the sign to be detected. This increases the probability that the obtained image set contains the sign to be detected. Furthermore, since each of the multiple sign image sets includes pre-collected images of already installed road signs, using the obtained image sets as a reference and determining whether the sign to be detected is damaged based on the similarity between the image of the sign to be detected and the images in the obtained image sets allows for a more accurate assessment. This provides data support for timely replacement of road signs. Attached Figure Description

[0075] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0076] Figure 1 This is a system architecture diagram of a road sign damage detection method provided in an embodiment of this application;

[0077] Figure 2 This is a flowchart illustrating a method for establishing a road sign database according to an embodiment of this application;

[0078] Figure 3 This is a flowchart of a road sign damage detection method provided in an embodiment of this application;

[0079] Figure 4 This is a schematic diagram of a road sign damage detection device provided in an embodiment of this application;

[0080] Figure 5 This is a schematic diagram of the structure of a server for detecting damage to road signs provided in an embodiment of this application. Detailed Implementation

[0081] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0082] Before providing a detailed explanation of the embodiments of this application, the system architecture involved in the embodiments of this application will be introduced first.

[0083] Figure 1 This is a system architecture diagram of a road sign damage detection method provided in an embodiment of this application. Figure 1 As shown, the system includes an image acquisition device 101 and a server 102. The image acquisition device 101 can communicate with the server 102 via a wireless network.

[0084] The image acquisition device 101 is used to acquire an image of the sign to be inspected and obtain the location information at the time of acquisition, and then send the sign image and location information to the server 102. The sign image is an image containing the sign to be inspected with its background removed. For example, the image acquisition device 101 first acquires an original image containing the sign to be inspected and other background information, and then performs background removal processing on the original image to obtain the sign image of the sign to be inspected. Optionally, the image acquisition device 101 can also identify the sign type of the sign to be inspected based on the sign image, and then send the identified sign type to the server 102. After receiving the sign image, sign type, and location information of the sign to be inspected, the server 102 can perform damage detection on the sign to be inspected based on multiple sign image sets pre-stored in the road sign database, using the method provided in this embodiment.

[0085] It should be noted that the image acquisition device 101 can be a vehicle-mounted camera, and the vehicle-mounted camera is equipped with a GPS (Global Positioning System) device. Thus, when the vehicle-mounted camera 101 acquires an image of a sign to be detected, it can also use the GPS device to obtain the location information corresponding to the sign. Optionally, the vehicle-mounted camera is also equipped with an AI (Artificial Intelligence) algorithm chip. Based on this, after acquiring the image of the sign to be detected, the vehicle-mounted camera 101 can also process the sign image to identify the sign type.

[0086] In addition, in this embodiment, before performing damage detection on the road signs to be inspected, the image acquisition device 101 can also acquire multiple images of each road sign installed on the road and obtain the location information at each acquisition time, thereby obtaining multiple sign images of each road sign and the location information of the road signs contained in each sign image. Optionally, the sign type of the road signs in each sign image can also be identified. The acquired multiple sign images and the location information and sign type corresponding to each sign image are sent to the server 102. The server 102 can establish a road sign database based on the received multiple sign images and the location information and sign type corresponding to each sign image, using the method provided in this embodiment, so that damage detection of the road signs to be inspected can be performed subsequently based on the road sign database.

[0087] The server 102 can be a single server, a server cluster, or a cloud platform; this embodiment does not limit the specific type of server.

[0088] In this embodiment of the application, the server can perform damage detection on the sign to be detected based on the sign images in multiple sign image sets stored in the road sign database. Based on this, the process of establishing the road sign database is first introduced. Figure 2 This is a flowchart illustrating a method for establishing a road sign database according to an embodiment of this application. Figure 2 As shown, the method includes the following steps:

[0089] Step 201: Obtain multiple pre-collected sign images and the sign type and location information of the road signs in each sign image.

[0090] In this embodiment, the image acquisition device performs multiple image acquisitions on various road signs installed on the road, thereby obtaining multiple sign images. When acquiring each sign image, the image acquisition device can obtain the location information at the time of acquisition as the location information of the road sign contained in that sign image, and after acquiring the sign image, it identifies the sign type of the road sign in the sign image. Then, the image acquisition device can send the acquired sign images, along with the location information and sign type of the road signs contained in the sign images, to a server. Correspondingly, the server receives the multiple sign images sent by the image acquisition device, along with the sign type and location information of the road signs in each sign image.

[0091] It should be noted that the image acquisition device can be a vehicle-mounted camera. In this case, when a vehicle equipped with a vehicle-mounted camera patrols multiple road signs installed on the road, it can acquire images of multiple road signs. A single patrol can acquire images of multiple road signs once, and multiple patrols can acquire images of multiple road signs multiple times. Thus, even for the same road sign, the images acquired in each patrol may differ due to factors such as shooting angle, field of view, and lighting intensity. Furthermore, the vehicle-mounted camera can be equipped with a GPS device, allowing it to locate itself and obtain corresponding location information each time it acquires an image. Subsequently, the vehicle-mounted camera can use an AI model to identify the sign type within the acquired sign images, thereby determining the sign type of the road sign in the image.

[0092] Optionally, the image acquisition device can also be other forms of intelligent devices with image acquisition capabilities, such as intelligent robots or drones used for road patrols. In this case, the image acquisition device can also obtain multiple sign images and the location information and sign type of the road signs contained in each sign image in the manner described above.

[0093] Alternatively, in one possible implementation, after the image acquisition device acquires the image of the road sign and obtains its location information, it may not identify the sign type in the image, but instead directly send the sign image and corresponding location information to the server. Upon receiving the sign image and location information, the server can then use an AI model to identify the sign type of the road sign in the image.

[0094] In addition, in this embodiment of the application, when the image acquisition device acquires images of road signs, in order to facilitate subsequent image processing, it can extract the image region containing the road signs from the acquired images and use the extracted image region as the sign image of the road signs.

[0095] Step 202: Based on the sign type and location information of the road signs in each sign image, classify the multiple sign images to obtain sign image sets corresponding to multiple road signs.

[0096] After obtaining multiple sign images and the sign type and location information of the road signs in each sign image, the server can classify the multiple sign images according to the sign type and location information of the road signs in each sign image, thereby obtaining multiple sign image sets corresponding to different road signs. Each sign image set corresponds to a sign type and location information.

[0097] In this embodiment, the server can first classify the multiple sign image set according to the location information of the road signs in each sign image, resulting in multiple classification sets. Since the sign types of the multiple sign images contained in each classification set may be different, the server can further classify the multiple sign images according to the sign types of the road signs in the multiple sign images contained in each classification set, thereby obtaining multiple sign image sets corresponding to multiple road signs.

[0098] For example, since each sign image in multiple sign images corresponds to a location information, the server can label the location information corresponding to each sign image in the multiple sign images on a map. Then, based on the multiple locations labeled on the map, the server can determine multiple center location points with high location cluster density. A circular region is defined with each center location point as the center and a preset radius. Next, the sign images corresponding to the multiple labeled locations within each circular region are classified into a category set, thus obtaining multiple category sets. At this point, the location information of the center location point corresponding to each category set can be used as the location information of the corresponding category set.

[0099] After obtaining multiple classification sets, since the sign types of road signs in the multiple sign images within each classification set may differ, for any classification set, such as the first classification set, the server treats multiple sign images with the same sign type included in the first classification set as the sign image of the same road sign, thereby obtaining one or more sign image sets for road signs. At this point, the sign type corresponding to each sign image set is the sign type of the road sign in the included sign images. In addition, the server can also use the location information of the first classification set, i.e., the location information of the center point corresponding to the first classification set, as the location information corresponding to these sign image sets for road signs.

[0100] Using the above method, the server further classifies the sign images in each category set according to the sign type, thereby obtaining multiple road sign image sets.

[0101] Optionally, in some possible implementations, after the server obtains a set of sign images for multiple road signs, it can also manually check the sign images in the set of sign images for multiple road signs to avoid mixing sign images of other road signs into the set of sign images for each road sign.

[0102] For example, taking any set of road sign images as an example, the server can provide a display interface, and then display multiple sign images contained in the sign image set on the display interface, and then the user can delete sign images that are not road signs from the multiple sign images displayed on the display interface.

[0103] Optionally, the server can first classify the multiple sign images according to the sign type of the road sign in each sign image to obtain multiple classification sets. Then, the server can further classify the multiple sign images in the corresponding classification set according to the location information of the road signs in the multiple sign images contained in each classification set, thereby obtaining a sign image set corresponding to multiple road signs.

[0104] For example, the server can compare the sign types of road signs contained in each of multiple sign images, and then classify multiple sign images with the same sign type into one category, thereby obtaining multiple classification sets corresponding one-to-one with multiple sign types. Since each classification set may include sign images of road signs of the same sign type at different locations, after obtaining multiple classification sets, the server reclassifies the multiple sign images in the corresponding classification set according to the location information of the road signs contained in the multiple sign images in each classification set, and takes each image set obtained from the reclassification as a sign image set of a road sign. The implementation method of reclassifying according to location information is the same as the implementation method of classifying signs according to the location information of signs described above, and will not be repeated in this embodiment. At this time, the location information corresponding to each sign image set of road signs obtained by classification is the location information of the center point corresponding to the classification.

[0105] Step 203: Determine the similarity reference value of the sign image set for each road sign.

[0106] After obtaining the sign image sets corresponding to multiple road signs, the server can determine the similarity reference value of the sign image set for each road sign in the multiple road sign sets. This similarity reference value characterizes the degree of similarity between the individual sign images included in the sign image set.

[0107] In one possible implementation, the server can obtain the similarity between each sign image in each sign image set and every other sign image in the same sign image set except itself, and then determine a similarity reference value for each road sign image set based on the similarity between each sign image and each sign image in the other sign images.

[0108] For example, taking any road sign image set from multiple road sign image sets as an example, for ease of explanation, this set is referred to as the third road sign image set. Taking any one of the multiple road sign images contained in this third road sign image set as an example, for ease of explanation, this set is referred to as road sign image A. The server compares road sign image A with every other road sign image contained in the third road sign image set, and then determines the similarity between road sign image A and all other road sign images contained in the third road sign image set, excluding road sign image A.

[0109] For example, the similarity scores of the sign image A with other sign images are P1, P2, P3...P... n-1 n is the number of sign images contained in the third sign image set. The server can calculate the average similarity D1 corresponding to sign image A using the following formula (1).

[0110]

[0111] Using the method described above, the server can obtain the average similarity of each sign image in the third sign image set, which can be denoted as D1, D2, D3, ... D n Since the average similarity of any sign image in the sign image set is the average similarity between that sign image and other sign images in the set, the average similarity of that sign image can indicate the degree of similarity between that sign image and other sign images in the set.

[0112] After calculating the average similarity of each sign image in the third sign image set, the server determines the average similarity of the third sign image set using the following formula (2) based on the average similarity of each sign image in the third sign image set.

[0113]

[0114] D1~D n Let W be the average similarity value of each sign image in the third sign image set. Since the average similarity value of the third sign image set is the average of the average similarity values ​​of all sign images in the set, it reflects the average similarity of the sign images in the set.

[0115] After calculating the average similarity value corresponding to the third set of sign images, the server compares the average similarity value of each sign image in the third set with the average similarity value of the entire third set of sign images. That is, it compares the average similarity values ​​D1, D2, D3, ... D of each sign image in the third set with the average similarity values ​​D1, D2, D3, ... D2. n The similarity average W of the third sign image set is compared with the average similarity of the third sign image set. Sign images with an average similarity lower than the average similarity of the third sign image set are deleted. Then, based on the average similarity and image quality score of each remaining sign image, a similarity reference value for the third sign image set is determined.

[0116] For example, suppose the average similarity of each sign image in the third sign image set is D1, D2, D3, ... D 10If D2 and D3 are less than the average similarity W of the third sign image set, then the sign images corresponding to D2 and D3 are deleted. Then, based on the average similarity of the remaining 8 sign images, the similarity reference value C of the third sign image set is calculated using the following formula (3).

[0117]

[0118] Where C is the calculated similarity reference value of the third set of sign images.

[0119] It should be noted that if the average similarity of a sign image in the sign image set is less than the average similarity of the entire set, it means that the similarity between that sign image and other sign images is lower than the average similarity of the sign images in the set. In other words, the similarity between that sign image and other sign images is low. In this case, after removing that sign image, the average of the average similarity of the remaining sign images is calculated and used as a similarity reference value, which can more accurately reflect the similarity of the sign images in the set.

[0120] Alternatively, in another possible implementation, the server can obtain the image quality score of each sign image in each of multiple sign image sets, and determine a similarity reference value for each sign image set based on the image quality score of each sign image in each sign image set and the similarity between each sign image and each sign image in other sign images besides itself.

[0121] For example, still taking the third sign image set as an example, the server first obtains the image quality score of each sign image in the third sign image set; based on the image quality score of each sign image in the third sign image set and the similarity between the corresponding sign image and each sign image in other sign images besides itself, the server determines the weighted average similarity of the corresponding sign image; based on the weighted average similarity of each sign image in the third sign image set and the image quality score, the server determines the weighted average similarity of the third sign image set; the server deletes the sign images whose weighted average similarity of the third sign image set is less than the weighted average similarity of the third sign image set, and determines the similarity reference value of the third sign image set based on the weighted average similarity of each remaining sign image and the image quality score.

[0122] As described above, when image acquisition equipment captures images of road signs, it is affected by factors such as shooting angle, field of view, and light intensity, resulting in varying image quality among multiple captured road sign images. Based on this, the server can obtain image quality scores for the sign images and then combine these scores with the similarity scores between the sign images in the image set to determine a similarity reference value. This similarity reference value takes into account the impact of image quality on image similarity, thus better reflecting the degree of similarity among the images in the sign image set.

[0123] The server can display each sign image from the third sign image set on the display interface and receive image quality scores for each sign image displayed on the interface submitted by the user. It should be noted that the user's image quality score for the road sign image can be within the range of (0, 100), where a higher image quality score indicates that the road sign in the image is more intact and clear.

[0124] After obtaining the image quality score of each sign image in the third sign image set, the server can determine the weighted average similarity of each sign image in the third sign image set based on the image quality score of each sign image in the third sign image set and the similarity between the corresponding sign image and each other sign image in the other sign images. This weighted average similarity is used to indicate the degree of similarity between the corresponding sign image and other sign images.

[0125] For example, taking sign image A from the third sign image set as an example, suppose the image quality score of each sign image in the third sign image set is S1, S2, S3...S n As described above, the similarity between sign image A and each sign image in the third sign image set (excluding sign image A) is P1, P2, P3...P... n-1 Then the server can calculate the weighted average similarity D1 corresponding to the sign image A using the following formula (4).

[0126]

[0127] Using the method described above, the server can obtain the weighted average similarity of each sign image in the third sign image set, which can be denoted as D1, D2, D3, ... D n .

[0128] It should be noted that, as can be seen from the above formula (4), the weighted average similarity of any sign image A in the third sign image set is obtained by weighting the similarity of sign image A with each other sign image. In the weighting, the weight of the similarity between sign image A and other sign images is the image quality score of the other sign images. In this way, the higher the image quality of the sign image, the greater the influence of the similarity between the sign image A and the sign image A on the weighted average similarity of the sign image A. Thus, the weighted average similarity of the sign image A calculated in this way can more accurately reflect the degree of similarity between the sign image A and other sign images in the sign image set.

[0129] After calculating the weighted average similarity of each sign image in the third sign image set, the server determines the weighted average similarity of the third sign image set according to the weighted average similarity of each sign image in the third sign image set and the image quality score, using the following formula (5). The weighted average similarity of the third sign image set is used to indicate the degree of similarity between each sign image in the third sign image set.

[0130]

[0131] Among them, D1~D n S1 to S2 are the weighted average similarity values ​​of each sign image in the third sign image set. n The image quality score is given for each sign image in the third sign image set, and W is the weighted average of the similarity scores corresponding to the third sign image set.

[0132] It should be noted that, as can be seen from the above formula (5), the weighted average similarity of the third sign image set is obtained by weighting the weighted average similarity of each sign image in the third sign image set again. When weighting, the weight of the weighted average similarity of each sign image is its own image quality score. In this way, the higher the image quality of the sign image, the greater the influence of the weighted average similarity of the sign image set on the weighted average similarity of the sign image set. Thus, the weighted average similarity of the third sign image set calculated in the end can more accurately reflect the average similarity of each sign image.

[0133] After calculating the weighted average similarity for the third set of sign images, the server compares the weighted average similarity for each sign image in the third set with the weighted average similarity for the entire third set. Sign images with a weighted average similarity lower than the weighted average similarity for the entire third set are deleted. Then, based on the weighted average similarity for each remaining sign image and its image quality score, a similarity reference value for the third set of sign images is determined.

[0134] Similarly, assuming that the weighted average similarity of D2 and D3 of each sign image in the third sign image set is less than the weighted average similarity of W of the third sign image set, the sign images corresponding to D2 and D3 are deleted. Then, based on the weighted average similarity of the remaining 8 sign images and the image quality score, the similarity reference value C of the third sign image set is calculated by the following formula (6).

[0135]

[0136] Where C is the calculated similarity reference value of the third set of sign images.

[0137] Using the method described above, the server can calculate a similarity reference value for the image set of each road sign.

[0138] In summary, this implementation method uses the image quality score of each sign image as a weight in the process of calculating the similarity reference value of the sign image set. The higher the image quality of the sign image, the greater its influence on the magnitude of the similarity reference value of the sign image set. In this way, the calculated similarity reference value can more accurately reflect the degree of similarity between the sign images in the sign image set and is more reliable.

[0139] As described above, each road sign in the multiple road signs has a corresponding sign image set with its sign type and location information. Therefore, after calculating the similarity reference value of each road sign's sign image set, the server can also store the sign image sets of each road sign, the corresponding similarity reference values, and the sign type and location information of the corresponding road signs to obtain a road sign database, providing a reference standard for subsequent damage identification of road signs.

[0140] In this embodiment, multiple sign images are pre-captured using an image acquisition device. The sign type and location information of each road sign in the image are obtained. The multiple sign images are then classified based on the sign type and location information of each road sign, resulting in multiple sign image sets corresponding to different road signs. Since each road sign image set contains different images captured for the actual installed road sign, the similarity reference value determined based on the similarity of the sign images in the set comprehensively considers the presentation effect of the road sign under different shooting conditions. Establishing a road sign database using these multiple sign image sets and each sign image set provides a more accurate reference standard for subsequently determining whether a sign to be detected is damaged, thereby improving the accuracy of sign damage detection and providing data support for timely replacement of road signs.

[0141] In addition, in this embodiment of the application, when determining the similarity reference value corresponding to the sign image set, the similarity can be weighted and averaged based on the image quality score of the sign images in the image set. Then, the sign images in the sign image set can be screened based on the weighted average of the similarity, so as to ensure that the sign images in the sign image set have good image quality and high similarity, thereby providing a better reference standard for subsequent judgment on whether the sign to be detected is damaged.

[0142] The following section describes the road sign damage detection method provided in the embodiments of this application.

[0143] Figure 3 This application provides a method for detecting damage to road signs. For example... Figure 3 As shown, the method includes the following steps:

[0144] Step 301: Obtain the sign type and location information of the sign to be detected contained in the first sign image.

[0145] In this embodiment, the image acquisition device acquires an image of the sign to be detected to obtain a first sign image. Simultaneously with image acquisition, the image acquisition device can obtain location information via a GPS device, using this location information as the location information of the sign to be detected. Then, the image acquisition device uses an AI model to identify the sign type of the sign to be detected in the first sign image. The first sign image, the location information of the sign to be detected, and the identified sign type of the sign to be detected are sent to a server. Correspondingly, the server receives the first sign image and the sign type and location information of the sign to be detected contained within it.

[0146] Optionally, in some possible implementations, after the image acquisition device acquires the first sign image and obtains the location information of the sign to be detected, it may not identify the sign type of the sign to be detected in the first sign image, but instead directly send the first sign image and location information to the server. After receiving the first sign image and the corresponding location information, the server uses an AI model to identify the sign type of the sign to be detected in the first sign image.

[0147] Step 302: Based on the sign type and location information of the sign to be detected, obtain the first sign image set from multiple stored sign image sets. Each sign image set includes multiple sign images of a road sign that have been pre-collected.

[0148] After obtaining the sign type and location information of the sign to be detected, the server can obtain candidate sign image sets from multiple stored sign image sets. The distance between the location information of each candidate sign image set and the location information of the sign to be detected is within a reference threshold range. Then, a second sign image set is obtained from the obtained candidate sign image sets. The sign type corresponding to each second sign image set is the same as the sign type of the sign to be detected. Then, a first sign image set is selected from the second sign image set. Each sign image set in the multiple sign image sets corresponds to a sign type and location information. The location information corresponding to each sign image set is used to indicate the location of the road sign among the multiple sign images included in the corresponding sign image set.

[0149] As described above, the road sign database stores multiple sign image sets, and each sign image set corresponds to a sign type and location information. Based on this, the server can calculate the distance between the sign to be detected and the road signs contained in each sign image set, according to the location information of the sign to be detected and the location information corresponding to each sign image set. Then, the sign image sets corresponding to distances within a reference threshold range are selected as candidate sign image sets.

[0150] Optionally, the server can also determine the location of the sign to be detected and the location of the road sign corresponding to each sign image set in the multiple sign image sets on the map based on the location information of the sign to be detected and the location information of each sign image set in the multiple sign image sets. Then, a circular area is determined with the location of the sign to be detected on the map as the center and the upper limit of the reference threshold range as the radius. The sign image sets whose locations are within the circular area are selected as candidate sign image sets.

[0151] For example, if the upper limit of the reference threshold range is 10m and the lower limit is 0, then a circular area is determined with the location of the sign to be detected in the map as the center and a radius of 10m. The set of sign images corresponding to the road signs located in the circular area is used as the candidate sign image set. Here, 10m is only an exemplary value and does not limit the embodiments of this application.

[0152] It should be noted that there may be one or more candidate sign image sets obtained based on the location information. When there is only one candidate sign image set, the server can determine whether the sign type corresponding to the candidate sign image set is the same as the sign type of the sign to be detected. If they are the same, the candidate sign image set is used as the first sign image set.

[0153] When there are multiple candidate sign image sets, the server can compare the sign type of the sign to be detected with the sign types corresponding to each candidate sign image set, and then obtain the sign image set whose sign type is the same as that of the sign to be detected from the multiple candidate sign image sets as the second sign image set. At this time, there may be one or more second sign image sets.

[0154] Optionally, the server can first select a set of candidate image sets based on the sign type of the sign to be detected from multiple image sets stored in the road sign database. Then, based on the location information of the candidate image sets and the location information of the sign to be detected, a second set of image sets is selected from the candidate image sets whose distance from the location information of the candidate image set to the location information of the sign to be detected is within a reference threshold range. At this point, there may be one or more second image sets.

[0155] The process by which the server obtains the second sign image set from the acquired candidate sign image set can refer to the aforementioned implementation method of determining the candidate sign image set based on the sign image set and the location information of the sign to be detected. This embodiment will not be repeated here.

[0156] After obtaining the second set of sign images, the server can select the sign image set that has the highest matching degree with the first sign image set from the second set of sign images as the first set of sign images.

[0157] As described above, the server may obtain one or more second sign image sets that match the sign to be detected. Therefore, selecting the sign image set with the highest matching degree from the second sign image set as the first sign image set can be implemented in the following two ways.

[0158] The first scenario: When there are multiple sets of second signage images, the server can select the signage image set with the highest matching degree with the first signage image from the multiple sets of second signage images as the first signage image set.

[0159] In this embodiment of the application, the server can determine the similarity between the first sign image and each sign image in each second sign image set; then, based on the similarity between the first sign image and each sign image in each second sign image set, determine the average similarity between the first sign image and the sign images in the corresponding second sign image set; and then select the second sign image set with the largest average similarity from multiple second sign image sets as the first sign image set.

[0160] For example, taking any one of multiple second sign image sets as an example, if the second sign image set contains 10 sign images, the server can perform similarity comparisons between the first sign image and each of the 10 sign images in the second sign image set, thereby obtaining the similarity between the first sign image and each of the 10 sign images. The server can denote the determined similarity between the first sign image and the 10 sign images as p1, p2, p3...p 10 Using the same method, the similarity between the first sign image and each sign image in each of the multiple second sign image sets can be determined.

[0161] After calculating the similarity between the first sign image and each sign image in each of the multiple second sign image sets, for any second sign image set, the server calculates the average similarity between the first sign image and each sign image in that second sign image set, thereby obtaining the average similarity between the first sign image and the sign images in the corresponding second sign image set.

[0162] Continuing with the previous example, when the similarity between the first sign image and the 10 sign images contained in the second sign image set are p1, p2, p3...p... 10 At that time, the server can calculate the similarity p1, p2, p3...p between the first sign image and the 10 sign images contained in the second sign image set. 10 The average similarity between the first sign image and the sign images in the corresponding second sign image set is obtained by taking the average value of the similarity values. This average similarity value can be denoted as E. The formula for calculating this average similarity value can be found in the following formula (7).

[0163]

[0164] Where n is the number of sign images in the sign image set.

[0165] Using the same calculation method, the average similarity between the first sign image and the sign images in each of the multiple second sign image sets can be obtained. As explained above, the average similarity between the first sign image and the sign images in each of the second sign image sets is actually the average similarity between the first sign image and each sign image in the corresponding sign image set. Therefore, this average similarity effectively represents the degree of similarity between the first sign image and the sign images in that sign image set; in other words, it represents the degree of similarity between the sign to be detected in the first sign image and the road signs in that sign image set.

[0166] Since the average similarity between the first sign image and the sign images in the sign image set actually represents the degree of similarity between the first sign image and the sign images in that set, a higher average similarity indicates a higher degree of similarity between the first sign image and the sign images in that set, that is, a higher degree of matching between the first sign image and the sign image set. Based on this, after obtaining the average similarity between the first sign image and the sign images in each of the multiple second sign image sets, the maximum average similarity can be determined from the average similarity values ​​corresponding to the multiple second sign image sets, and the second sign image set corresponding to the maximum average similarity can be taken as the first sign image set with the highest degree of matching with the first sign image.

[0167] The second scenario: When the number of second sign image sets obtained is 1, the server can use this second sign image set as the first sign image set. In this case, the server can further determine the similarity between the first sign image and each sign image in the first sign image set, and determine the average similarity between the first sign image and the sign images in the first sign image set based on the similarity between the first sign image and each sign image in the first sign image set.

[0168] The method for calculating the average similarity between the first sign image and the sign images in the first sign image set refers to the method for calculating the average similarity described above, and will not be repeated in this embodiment.

[0169] Step 303: Based on the similarity between the first sign image and the sign images in the first sign image set, perform damage detection on the sign to be detected.

[0170] As described above, each sign image set corresponds to a similarity reference value, which characterizes the degree of similarity between the sign images contained in the corresponding sign image set. Based on this, after determining the first sign image set, the server can obtain the similarity reference value corresponding to the first sign image set, and perform damage detection on the signs to be detected contained in the first sign image set according to the similarity and similarity reference values ​​between the first sign image and each sign image in the first sign image set.

[0171] For example, the server compares the average similarity between the first sign image and the first sign image set determined in step 302 above with the similarity reference value corresponding to the first sign image set. If the average similarity is greater than or equal to the similarity reference value, it is determined that the sign to be detected contained in the first sign image is not damaged. If the average similarity is less than the similarity reference value, it is determined that the sign to be detected contained in the first sign image is damaged.

[0172] Optionally, in some possible implementations, if the average similarity is less than the similarity reference value corresponding to the first sign image set, the server can also calculate the difference between the similarity reference value and the average similarity. If the difference is within a preset threshold range, then the sign to be detected contained in the first sign image is determined to be damaged.

[0173] Optionally, after determining that the sign to be detected is damaged, the server can further determine the degree of damage to the sign to be detected based on the average similarity between the first sign image and the sign images in the first sign image set and the similarity reference value corresponding to the first sign image set.

[0174] For example, the server can determine the absolute value of the difference between the average similarity between the first sign image and sign images in the first sign image set, and the similarity reference value corresponding to the first sign image set. Then, based on the reference difference range in which the absolute value of the difference falls, the degree of damage to the sign to be detected is determined.

[0175] It should be noted that the server can store different mapping relationships between reference difference ranges and corresponding damage levels. Based on this, the server determines the reference difference range in which the absolute value of the calculated difference falls, and uses the damage level corresponding to the determined reference difference range as the damage level of the sign to be inspected. The damage level of the sign can be characterized by percentages or other damage levels, and this embodiment does not limit this.

[0176] For example, the server stores a mapping relationship between three different reference difference intervals and their corresponding degrees of damage. These three different reference difference intervals can be (1, 3), (4, 7), and (8, 10), where the reference difference interval (1, 3) represents minor damage, the reference difference interval (4, 7) represents moderate damage, and the reference difference interval (8, 10) represents severe damage. Based on this, when the absolute value of the difference calculated by the server is 2, the sign to be inspected is considered to be slightly damaged; when the absolute value of the difference calculated by the server is 5, the sign to be inspected is considered to be moderately damaged; and when the absolute value of the difference calculated by the server is 9, the sign to be inspected is considered to be severely damaged. It should be noted that the above reference difference intervals and the absolute values ​​of the differences are exemplary values ​​and do not limit the embodiments of this application.

[0177] In this embodiment, multiple sign image sets are pre-stored. Based on the sign type and location information of the sign to be detected, a sign image set matching the sign type and location information of the sign to be detected is obtained from the pre-stored multiple sign image sets. This increases the probability that the obtained sign image set contains the sign to be detected. Furthermore, since each of the multiple sign image sets includes multiple different sign images of pre-collected, already installed road signs, using the obtained sign image sets as a reference, the similarity between the image of the sign to be detected and the sign images in the obtained sign image sets can be used to determine whether the sign to be detected is damaged. This allows for a more accurate determination of whether the sign to be detected is damaged, thereby providing data support for timely replacement of road signs.

[0178] Furthermore, since each road sign's image set contains different images captured for the actual installed road sign, the similarity reference value determined based on the similarity of the sign images in this image set effectively considers the road sign's appearance under different shooting conditions. Therefore, using this similarity reference value as a more accurate standard for judging whether the sign to be detected in the first captured image is damaged is more precise.

[0179] Finally, this embodiment of the application determines the degree of damage to the sign to be inspected by setting a mapping relationship between a reference difference range and the corresponding degree of damage. In this way, the degree of damage can be quantified, thereby providing better data support for replacing road signs.

[0180] Next, the road sign damage detection device provided in the embodiments of this application will be introduced.

[0181] See Figure 4This application provides a road sign damage detection device 400, which includes: a first acquisition module 401, a second acquisition module 402, and a detection module 403.

[0182] The first acquisition module 401 is used to acquire the sign type and location information of the sign to be detected contained in the first sign image;

[0183] The second acquisition module 402 is used to acquire a first sign image set from multiple stored sign image sets according to the sign type and location information of the sign to be detected. Each sign image set includes multiple sign images of a road sign that have been pre-captured.

[0184] The detection module 403 is used to perform damage detection on the sign to be detected based on the similarity between the first sign image and the sign images in the first sign image set.

[0185] Optionally, each of the multiple sign image sets corresponds to a sign type and location information;

[0186] The second acquisition module 402 includes:

[0187] The first acquisition submodule is used to acquire candidate sign image sets from multiple sign image sets, wherein the distance between the location information corresponding to each candidate sign image set and the location information of the sign to be detected is within a reference threshold range.

[0188] The second acquisition submodule is used to acquire a second sign image set from the acquired candidate sign image set, wherein the sign type corresponding to each second sign image set is the same as the sign type of the sign to be detected;

[0189] The selection submodule is used to select the first sign image set from the second sign image set.

[0190] Optionally, select a submodule for:

[0191] Determine the similarity between the first sign image and each sign image in each of the second sign image sets;

[0192] Based on the similarity between the first sign image and each sign image in each second sign image set, determine the average similarity between the first sign image and the sign images in the corresponding second sign image set;

[0193] The second sign image set with the highest average similarity among multiple second sign image sets is selected as the first sign image set.

[0194] Optionally, the detection module 403 includes:

[0195] The third acquisition submodule is used to acquire similarity reference values ​​of the first set of sign images;

[0196] The detection submodule is used to perform damage detection on the sign to be detected based on the similarity and similarity reference values ​​between the first sign image and each sign image in the first sign image set.

[0197] Optionally, the detection submodule is used for:

[0198] Based on the similarity between the first sign image and each sign image in the first sign image set, determine the average similarity between the first sign image and the first sign image set;

[0199] If the average similarity between the first sign image and the first set of sign images is less than the similarity reference value, then the sign to be detected is determined to be damaged.

[0200] Optionally, the detection submodule is also used for:

[0201] Determine the absolute value of the difference between the average similarity between the first sign image and the first set of sign images and the similarity reference value;

[0202] The degree of damage to the sign to be inspected is determined based on the reference difference range in which the absolute value of the difference falls.

[0203] Optionally, the device 400 further includes:

[0204] The third acquisition module 404 is used to acquire multiple pre-collected sign images and the sign type and location information of the road signs in each sign image;

[0205] The classification module 405 is used to classify multiple sign images according to the sign type and location information of the road signs in each sign image, so as to obtain multiple sign image sets, each sign image set corresponding to a sign type and location information;

[0206] The determination module 406 is used to determine the similarity reference value for each set of sign images.

[0207] Optionally, module 406 is defined, including:

[0208] The fourth acquisition submodule is used to acquire the image quality score of each sign image in the third sign image set, which is the set of sign images for any road sign;

[0209] The determination submodule is used to determine the similarity reference value of the third sign image set based on the image quality score of each sign image in the third sign image set and the similarity between each sign image and each other sign image in other sign images excluding itself.

[0210] Optionally, a submodule is defined for:

[0211] Based on the image quality score of each sign image in the third sign image set and the similarity between the corresponding sign image and each other sign image in the other sign images excluding itself, determine the weighted average of the similarity of the corresponding sign image;

[0212] Based on the weighted average similarity and image quality score of each sign image in the third sign image set, the weighted average similarity of the third sign image set is determined.

[0213] Delete the sign images in the third sign image set whose weighted similarity average is less than the weighted similarity average of the third sign image set. Then, determine the similarity reference value of the third sign image set based on the weighted similarity average and image quality score of each remaining sign image.

[0214] In summary, in this embodiment, multiple sign image sets are pre-stored. A first sign image set is obtained from these pre-stored sets based on the sign type and location information of the sign to be detected. This increases the probability that the obtained sign image set contains the sign to be detected. Furthermore, since each of the multiple sign image sets includes multiple pre-collected images of already installed road signs, using the obtained sign image set as a reference and determining whether the sign to be detected is damaged based on the similarity between the image of the sign to be detected and the sign images in the obtained set can more accurately determine whether the sign to be detected is damaged, thus providing data support for timely replacement of road signs.

[0215] It should be noted that the road sign damage detection device provided in the above embodiments is only illustrated by the division of the above functional modules when detecting road signs. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the road sign damage detection device and the road sign damage detection method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0216] Figure 5 This is a schematic diagram of a server structure according to an exemplary embodiment. The road sign damage detection function in the above embodiment can be achieved through... Figure 5 The server shown is used to implement this. This server can be a server in a backend server cluster. Specifically:

[0217] Server 500 includes a Central Processing Unit (CPU) 501, a system memory 504 including Random Access Memory (RAM) 502 and Read-Only Memory (ROM) 503, and a system bus 505 connecting the system memory 504 and the CPU 501. Server 500 also includes a basic input / output system (I / O system) 506 that facilitates the transfer of information between various devices within the computer, and a mass storage device 507 for storing the operating system 513, application programs 514, and other program modules 515.

[0218] The basic input / output system 506 includes a display 508 for displaying information and an input device 509 for user input, such as a mouse or keyboard. Both the display 508 and the input device 509 are connected to the central processing unit 501 via an input / output controller 510 connected to the system bus 505. The basic input / output system 506 may also include the input / output controller 510 for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 510 also provides output to a display screen, printer, or other types of output devices.

[0219] Mass storage device 507 is connected to central processing unit 501 via a mass storage controller (not shown) connected to system bus 505. Mass storage device 507 and its associated computer-readable media provide non-volatile storage for server 500. That is, mass storage device 507 may include computer-readable media (not shown) such as hard disk or CD-ROM (CompactDisc Read-Only Memory) drive.

[0220] Without loss of generality, computer-readable media can include computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented using any method or technique for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes RAM, ROM, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), flash memory or other solid-state storage devices, CD-ROM, DVD (Digital Versatile Disc) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that computer storage media are not limited to the above-mentioned types. The system memory 504 and mass storage device 507 described above can be collectively referred to as memory.

[0221] According to various embodiments of this application, server 500 can also be connected to a remote computer on a network, such as the Internet. That is, server 500 can be connected to network 512 via network interface unit 511 connected to system bus 505, or it can use network interface unit 511 to connect to other types of networks or remote computer systems (not shown).

[0222] The aforementioned memory also includes one or more programs, which are stored in the memory and configured to be executed by the CPU. The one or more programs contain instructions for performing the road sign damage detection method provided in the embodiments of this application.

[0223] This application also provides a computer-readable storage medium that, when executed by a server's processor, enables the server to perform the road sign damage detection method provided in the above embodiments. For example, the computer-readable storage medium may be a ROM, RAM, CD-ROM, magnetic tape, floppy disk, or optical data storage device. It is worth noting that the computer-readable storage medium mentioned in this application embodiment may be a non-volatile storage medium; in other words, it may be a non-transient storage medium.

[0224] It should be understood that all or part of the steps of the above embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented wholly or partially in the form of a computer program product. The computer program product includes one or more computer instructions. The computer instructions can be stored in the above-described computer-readable storage medium.

[0225] That is, in some embodiments, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute the road sign damage detection method provided in the above embodiments.

[0226] The above description is not intended to limit the embodiments of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of the embodiments of this application.

Claims

1. A method for detecting damage to road signs, characterized in that, The method includes: Obtain the sign type and location information of the sign to be detected contained in the first sign image; Based on the sign type and location information of the sign to be detected, a first sign image set is obtained from multiple stored sign image sets, each sign image set including multiple sign images of a road sign that have been pre-captured; Based on the similarity between the first sign image and the sign images in the first sign image set, damage detection is performed on the sign to be detected; The method further includes: Acquire multiple pre-collected sign images and the sign type and location information of the road signs in each sign image; Based on the sign type and location information of the road signs in each sign image, the multiple sign images are classified to obtain multiple sign image sets, each sign image set corresponding to a sign type and location information; Obtain the image quality score for each sign image in the third sign image set, where the third sign image set is the set of sign images for any road sign; Based on the image quality score of each sign image in the third sign image set and the similarity between the corresponding sign image and each sign image in other sign images excluding itself, the weighted average of the similarity of the corresponding sign image is determined. Based on the weighted average similarity and image quality score of each sign image in the third sign image set, the weighted average similarity of the third sign image set is determined. Delete the sign images in the third sign image set whose weighted similarity average is less than the weighted similarity average of the third sign image set, and determine the similarity reference value of the third sign image set based on the weighted similarity average and image quality score of each remaining sign image. The step of performing damage detection on the sign to be detected based on the similarity between the first sign image and sign images in the first sign image set includes: Obtain the similarity reference value of the first set of sign images; Damage detection is performed on the sign to be detected based on the similarity between the first sign image and each sign image in the first sign image set, and the similarity reference value of the first sign image set.

2. The method according to claim 1, characterized in that, Each of the multiple sign image sets corresponds to a sign type and location information; The step of obtaining a first sign image set from multiple stored sign image sets based on the sign type and location information of the sign to be detected includes: Candidate sign image sets are obtained from the plurality of sign image sets, and the distance between the location information corresponding to each candidate sign image set and the location information of the sign to be detected is within a reference threshold range; A second sign image set is obtained from the acquired candidate sign image set, wherein the sign type corresponding to each second sign image set is the same as the sign type of the sign to be detected; Select the first sign image set from the second sign image set.

3. The method according to claim 2, characterized in that, The step of selecting the first sign image set from the second sign image set includes: Determine the similarity between the first sign image and each sign image in each of the second sign image sets; Based on the similarity between the first sign image and each sign image in each second sign image set, determine the average similarity between the first sign image and the sign images in the corresponding second sign image set; The first sign image set is selected from the plurality of second sign image sets, based on the second sign image set with the highest average similarity.

4. The method according to claim 1, characterized in that, The step of performing damage detection on the sign to be detected based on the similarity scores between the first sign image and each sign image in the first sign image set, and the similarity reference value of the first sign image set, includes: Based on the similarity between the first sign image and each sign image in the first sign image set, determine the average similarity between the first sign image and the first sign image set. If the average similarity between the first sign image and the first set of sign images is less than the similarity reference value of the first set of sign images, then the sign to be detected is determined to be damaged.

5. The method according to claim 4, characterized in that, The step of performing damage detection on the sign to be detected based on the similarity scores between the first sign image and each sign image in the first sign image set, and the similarity reference value of the first sign image set, further includes: Determine the absolute value of the difference between the average similarity between the first sign image and the first set of sign images and the reference value of the similarity between the first set of sign images; The degree of damage to the sign to be inspected is determined based on the reference difference range in which the absolute value of the difference falls.

6. A road sign damage detection device, characterized in that, The device includes: The first acquisition module is used to acquire the sign type and location information of the sign to be detected contained in the first sign image; The second acquisition module is used to acquire a first sign image set from multiple stored sign image sets according to the sign type and location information of the sign to be detected. Each sign image set includes multiple sign images of a road sign that have been pre-captured. The detection module is used to perform damage detection on the sign to be detected based on the similarity between the first sign image and the sign images in the first sign image set; The device further includes: The third acquisition module is used to acquire multiple pre-collected sign images and the sign type and location information of the road signs in each sign image; The classification module is used to classify the multiple sign images according to the sign type and location information of the road signs in each sign image, so as to obtain multiple sign image sets, each sign image set corresponding to a sign type and location information; The determination module is used to determine the similarity reference value for each set of sign images; The determining module includes: The fourth acquisition submodule is used to acquire the image quality score of each sign image in the third sign image set, wherein the third sign image set is the set of sign images of any road sign; The determination submodule is used to determine the similarity reference value of the third sign image set based on the image quality score of each sign image in the third sign image set and the similarity between each sign image and each other sign image in other sign images excluding itself. The determining submodule is mainly used for: Based on the image quality score of each sign image in the third sign image set and the similarity between the corresponding sign image and each sign image in other sign images excluding itself, the weighted average of the similarity of the corresponding sign image is determined. Based on the weighted average similarity and image quality score of each sign image in the third sign image set, the weighted average similarity of the third sign image set is determined. Delete the sign images in the third sign image set whose weighted similarity average is less than the weighted similarity average of the third sign image set, and determine the similarity reference value of the third sign image set based on the weighted similarity average and image quality score of each remaining sign image. The detection module includes: The third acquisition submodule is used to acquire similarity reference values ​​of the first sign image set; The detection submodule is used to perform damage detection on the sign to be detected based on the similarity between the first sign image and each sign image in the first sign image set and the similarity reference value of the first sign image set.

7. The apparatus according to claim 6, characterized in that, Each of the multiple sign image sets corresponds to a sign type and location information; The second acquisition module includes: The first acquisition submodule is used to acquire candidate sign image sets from the plurality of sign image sets, wherein the distance between the location information corresponding to each candidate sign image set and the location information of the sign to be detected is within a reference threshold range; The second acquisition submodule is used to acquire a second sign image set from the acquired candidate sign image set, wherein the sign type corresponding to each second sign image set is the same as the sign type of the sign to be detected; The selection submodule is used to select the first sign image set from the second sign image set; The selection submodule is mainly used for: Determine the similarity between the first sign image and each sign image in each of the second sign image sets; Based on the similarity between the first sign image and each sign image in each second sign image set, determine the average similarity between the first sign image and the sign images in the corresponding second sign image set; Select the second sign image set with the largest average similarity from the plurality of second sign image sets as the first sign image set; The detection submodule is mainly used for: Based on the similarity between the first sign image and each sign image in the first sign image set, determine the average similarity between the first sign image and the first sign image set. If the average similarity between the first sign image and the first set of sign images is less than the similarity reference value of the first set of sign images, then the sign to be detected is determined to be damaged. The detection submodule is further used for: Determine the absolute value of the difference between the average similarity between the first sign image and the first set of sign images and the similarity reference value; The degree of damage to the sign to be inspected is determined based on the reference difference range in which the absolute value of the difference falls.

8. A computer device, characterized in that, The computer device includes a processor and a memory, the memory being used to store computer programs, and the processor being used to execute the computer programs stored in the memory to implement the road sign damage detection method according to any one of claims 1-5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a computer, implements the road sign damage detection method according to any one of claims 1-5.

Images