A road disease detection method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a road disease detection method and device, electronic equipment and storage medium. The method comprises: obtaining target position information and a target image corresponding to a to-be-processed road disease, extracting image features of the target image as target features. Based on the target features, the target position information and the disease information stored in the pre-established road disease library, it is determined whether there is an existing road disease in the road disease library that is repeated with the to-be-processed road disease, wherein the disease information comprises a correspondence relationship between pre-acquired road disease identifiers, image features and position information. If there is, the to-be-processed road disease is determined to be a repeated disease. If there is not, the identifier of the to-be-processed road disease is stored in the road disease library in correspondence with the target features and the target position information. The scheme provided by the embodiments of the present application can realize the deduplication processing of road disease data.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation technology, and in particular to a method, apparatus, electronic device, and storage medium for detecting road defects. Background Technology

[0002] Road defects refer to various types of damage, deformation, and other defects that occur on roads. Common road defects include cracks, potholes, loosening, and subsidence. The presence of road defects not only shortens the service life of roads but also increases the risk of traffic accidents. Therefore, road condition surveys and analyses are extremely important.

[0003] During road condition surveys and analyses, patrol vehicles travel on the roads and collect road surface images. These images are then analyzed to determine if any road defects are present. If defects are found, the relevant information can be reported to the appropriate authorities for appropriate action.

[0004] However, during the aforementioned road defect inspections, the road defects included in the continuous multi-frame road surface images are repetitive. Furthermore, when the inspection vehicle travels on the aforementioned roads again, it will also collect a large number of images of already identified road defects, resulting in a large amount of duplicate road defect data. Summary of the Invention

[0005] The purpose of this invention is to provide a method, apparatus, electronic device, and storage medium for detecting road defects, so as to achieve deduplication of road defect data. The specific technical solution is as follows:

[0006] In a first aspect, embodiments of the present invention provide a method for detecting road defects, the method comprising:

[0007] Acquire the target location information and target image corresponding to the road defects to be treated;

[0008] Extract the image features of the target image as target features;

[0009] Based on the target features, the target location information, and the disease information stored in the pre-established road disease database, it is determined whether there are existing road diseases in the road disease database that are duplicates of the road disease to be processed. The disease information includes the correspondence between each road disease identifier and image features and location information obtained in advance.

[0010] If present, the road defect to be treated is determined to be a recurring defect;

[0011] If it does not exist, the identifier of the road defect to be processed, the target feature, and the target location information are stored in the road defect database.

[0012] Optionally, the defect information may also include the type corresponding to each road defect identifier;

[0013] Before the step of determining whether there are existing road defects in the road defect database that duplicate the road defect to be processed, based on the target features, the target location information, and defect information stored in a pre-established road defect database, the method further includes:

[0014] Determine the target type of the road defects to be treated;

[0015] The step of determining whether there are existing road defects in the road defect database that overlap with the road defect to be processed, based on the target features, the target location information, and defect information stored in a pre-established road defect database, includes:

[0016] Based on the target features, the target location information, the target type, and the disease information stored in the pre-established road disease database, it is determined whether there are existing road diseases in the road disease database that are duplicates of the road disease to be processed.

[0017] Optionally, the step of determining whether there are existing road defects in the road defect database that overlap with the road defect to be processed, based on the target features, the target location information, the target type, and defect information stored in a pre-established road defect database, includes:

[0018] Based on the location information stored in the pre-established road defect database, determine whether there are existing road defects whose distance from the target location information meets the preset distance condition;

[0019] If present, determine whether the type of the existing road defect is the same as the target type;

[0020] If they are the same, calculate the similarity between the image features corresponding to the existing road defects and the target features;

[0021] If the similarity reaches a preset threshold, the road defect to be processed is determined to be a duplicate defect.

[0022] Optionally, the method further includes:

[0023] If the location information stored in the pre-established road defect database does not contain any existing road defects whose distance from the target location information meets the preset distance condition, the identifier of the road defect to be processed is stored in the road defect database along with the target feature, the target location information, and the target type; or,

[0024] If the type of the existing road defect is different from the target type, the identifier of the road defect to be processed is stored in the road defect database along with the target features, the target location information, and the target type; or,

[0025] If the similarity does not reach a preset threshold, the identifier of the road defect to be processed is stored in the road defect database along with the target feature, the target location information, and the target type.

[0026] Optionally, the target image is acquired by an image acquisition device installed on the patrol vehicle; the step of acquiring the target location information and target image corresponding to the road defects to be processed includes:

[0027] Each time a road image is acquired by the image acquisition device, the road defects to be processed included in the road image are identified until the road defects to be processed are no longer present in the road images currently acquired by the image acquisition device. Then, the road image including the road defects to be processed is taken as the target image.

[0028] The vehicle position is obtained when the image acquisition device acquires each frame of the target image;

[0029] For each frame of the target image, the location information of the road defect to be processed is calculated based on the calibration information of the image acquisition device and the vehicle position corresponding to that frame of the target image.

[0030] The target location information of the road defects to be processed is determined based on the location information corresponding to each frame of the target image.

[0031] Optionally, the target image consists of multiple frames;

[0032] The step of extracting image features from the target image as target features includes:

[0033] Image features are extracted from each frame of the target image, and the image features of a preset number of frames of the target image are combined into an image feature set, which is used as the target feature corresponding to the road defect to be processed.

[0034] Secondly, embodiments of the present invention provide a road defect detection device, the device comprising:

[0035] The road disease information acquisition module is used to acquire the target location information and target image of the road disease to be treated;

[0036] The image feature extraction module is used to extract image features from the target image as target features;

[0037] The duplicate road disease determination module is used to determine whether there are existing road diseases that duplicate the road disease to be processed in the road disease database based on the target features, the target location information and the disease information stored in the pre-established road disease database. The disease information includes the correspondence between each road disease identifier and image features and location information obtained in advance.

[0038] The road defect deduplication module is used to determine that the road defect to be processed is a duplicate defect if there is an existing road defect in the road defect database that is duplicated with the road defect to be processed.

[0039] The road defect storage module is used to store the identifier of the road defect to be processed, the target feature, and the target location information in the road defect database if there is no existing road defect in the database that duplicates the road defect to be processed.

[0040] Optionally, the defect information may also include the type corresponding to each road defect identifier;

[0041] The device further includes:

[0042] The target type determination module is used to determine the target type of the road defect to be processed before determining whether there are existing road defects in the road defect database that are duplicates of the road defect to be processed, based on the target features, the target location information and the defect information stored in the pre-established road defect database.

[0043] The recurring disease identification module includes:

[0044] The duplicate road defect determination unit is used to determine, based on the target features, the target location information, the target type, and the defect information stored in the pre-established road defect database, whether there are existing road defects in the road defect database that duplicate the road defect to be processed.

[0045] Optionally, the recurring disease determination unit includes:

[0046] The first determining subunit is used to determine, based on the location information stored in the pre-established road defect database, whether there are existing road defects whose distance from the target location information meets the preset distance condition.

[0047] The second determining subunit is used to determine whether the type of the existing road defects is the same as the target type if there are existing road defects whose distance from the target location information meets a preset distance condition.

[0048] The third determining subunit is used to calculate the similarity between the image features corresponding to the existing road defects and the target features if the type of the existing road defects is the same as the target type.

[0049] The fourth determining subunit is used to determine that the road defect to be processed is a repeating defect if the similarity reaches a preset threshold.

[0050] Optionally, the recurring disease determination unit further includes:

[0051] A storage subunit is configured to, if the location information stored in the pre-established road defect database does not contain any existing road defects whose distance from the target location information meets a preset distance condition, store the identifier of the road defect to be processed in the road defect database, corresponding to the target feature, the target location information, and the target type; or,

[0052] This is used to store the identifier of the road defect to be processed in the road defect database, corresponding to the target feature, the target location information, and the target type, if the type of the existing road defect is different from the target type; or,

[0053] If the similarity does not reach a preset threshold, the identifier of the road defect to be processed is stored in the road defect database along with the target feature, the target location information, and the target type.

[0054] Optionally, the target image is acquired by an image acquisition device installed on the patrol vehicle; the disease information acquisition module includes:

[0055] The target image acquisition unit is used to acquire road images acquired by the image acquisition device, identify the road defects to be processed included in the road images, until the road images currently acquired by the image acquisition device do not contain the road defects to be processed, and then take the road images containing the road defects to be processed as target images.

[0056] The vehicle position acquisition unit is used to acquire the vehicle position when the image acquisition device acquires each frame of the target image;

[0057] The location information calculation unit is used to calculate the location information of the road defect to be processed for each frame of target image, based on the calibration information of the image acquisition device and the vehicle position corresponding to the frame of target image;

[0058] The target location information determination unit is used to determine the target location information of the road defect to be processed based on the location information corresponding to each frame of the target image.

[0059] Optionally, the target image consists of multiple frames; the image feature extraction module includes:

[0060] The image feature extraction unit is used to extract the image features of each frame of the target image, and to form an image feature set by combining the image features of a preset number of frames of the target image, which serves as the target feature corresponding to the road defect to be processed.

[0061] Thirdly, embodiments of the present invention provide an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0062] Memory, used to store computer programs;

[0063] When a processor executes a program stored in memory, it implements any of the steps described in the first aspect above.

[0064] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the steps described in the first aspect above.

[0065] Beneficial effects of the embodiments of the present invention:

[0066] In the solution provided by this invention, an electronic device can acquire the target location information and target image corresponding to the road defect to be processed, and extract image features from the target image as target features. Then, based on the target features, target location information, and defect information stored in a pre-established road defect database, it determines whether there are existing road defects in the database that duplicate the road defect to be processed. The defect information includes the pre-acquired correspondence between each road defect identifier and its image features and location information. If such a defect exists, the road defect to be processed is determined to be a duplicate; if not, the identifier of the road defect to be processed is stored in the road defect database along with its target features and target location information. This allows for deduplication when duplicate road defects occur, preventing the storage of duplicate road defect data in the road defect database and avoiding duplicate storage of existing road defects. Of course, implementing any product or method of this invention does not necessarily require achieving all of the above advantages simultaneously. Attached Figure Description

[0067] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0068] Figure 1 A flowchart of a road defect detection method provided in an embodiment of the present invention;

[0069] Figure 2 Based on Figure 1 A flowchart illustrating a target type determination method in the illustrated embodiment;

[0070] Figure 3 for Figure 1 A specific flowchart of S103 in the illustrated embodiment;

[0071] Figure 4 for Figure 1 A specific flowchart of S101 in the illustrated embodiment;

[0072] Figure 5 This is a schematic diagram of the structure of a road defect detection device provided in an embodiment of the present invention;

[0073] Figure 6 This is another structural schematic diagram of the road defect detection device provided in an embodiment of the present invention;

[0074] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0075] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art based on the present invention are within the scope of protection of the present invention.

[0076] To achieve deduplication of road defect data, this invention provides a road defect detection method, apparatus, electronic device, computer-readable storage medium, and computer program product. The following section first describes a road defect detection method provided by an embodiment of this invention.

[0077] The road defect detection method provided in this embodiment of the invention can be applied to any electronic device that needs to detect road defects, such as a server or terminal. For the sake of clarity, it will be referred to as an electronic device.

[0078] like Figure 1 As shown in the figure, an embodiment of the present invention provides a method for detecting road defects, which may include:

[0079] S101, Obtain the target location information and target image corresponding to the road defects to be processed;

[0080] S102, Extract the image features of the target image as target features;

[0081] S103, based on the target features, the target location information, and the disease information stored in the pre-established road disease database, determine whether there are existing road diseases in the road disease database that are duplicates of the road disease to be processed; if there are, proceed to step S104; if not, proceed to step S105.

[0082] The disease information includes the correspondence between pre-acquired road disease identifiers, image features, and location information.

[0083] S104, determine that the road defect to be treated is a recurring defect;

[0084] S105, the identifier of the road defect to be processed is stored in the road defect database in correspondence with the target feature and the target location information.

[0085] As can be seen, in the solution provided by this embodiment of the invention, the electronic device can acquire the target location information and target image corresponding to the road defect to be processed, and extract image features from the target image as target features. Then, based on the target features, target location information, and defect information stored in a pre-established road defect database, it determines whether there are existing road defects in the database that duplicate the road defect to be processed. The defect information includes the pre-acquired correspondence between each road defect identifier and its image features and location information. If such a correspondence exists, the road defect to be processed is determined to be a duplicate defect; if not, the identifier of the road defect to be processed is stored in the road defect database along with its target features and target location information. In this way, when duplicate road defects occur, deduplication can be performed, preventing the duplicate road defect data from being stored in the road defect database. This avoids repeatedly storing existing road defects in the road defect database, thus achieving deduplication of road defect data.

[0086] During road condition inspections, road surface images can be collected to determine if road defects have been detected. For example, intelligent inspection vehicles traveling on the road can capture images of the road surface using installed intelligent recognition cameras and identify road defects, i.e., road defects to be addressed. In step S101 above, the electronic device can acquire the target location information and target image of the road defect to be addressed. The target location information reflects the specific geographical location of the road defect, and the target image is the image containing the identified road defect, which can reflect the overall or partial appearance of the road defect to be addressed.

[0087] In one implementation, the target location information of the road defect to be treated can be calculated based on positioning data collected by the positioning module in the intelligent patrol vehicle. The target location information can be recorded in the form of geographic coordinates, such as latitude and longitude.

[0088] The target image can be an image sent in real time to the electronic device by the aforementioned intelligent recognition camera during the patrol, or it can be an image of road defects already stored in the electronic device or other devices. The target image can be a picture captured by the image acquisition device, or a video frame from the video recorded by the image acquisition device; both are reasonable.

[0089] After acquiring the target location information and target image of the road defects to be processed, the electronic device can perform the above step S102, that is, extract the image features of the target image as target features. The target features can characterize the features of the road defects to be processed in the target image. The image features can be extracted from the target image through convolution operations or other methods, which are not specifically limited here.

[0090] Then, the electronic device can perform the above-mentioned step S103, that is, based on the target features, target location information, and the defect information stored in the pre-established road defect database, determine whether there are existing road defects in the road defect database that duplicate the road defect to be processed. The road defect database is a pre-established database used to store defect information for road defects, and the defect information may include the pre-acquired correspondence between each road defect identifier and image features and location information.

[0091] To facilitate the determination of whether a road defect is a duplicate after obtaining relevant information about the defects to be processed, and to perform deduplication of road defect data, a road defect database can be established in advance. In one embodiment, the road defect database can be established as follows: an intelligent patrol vehicle travels on the road surface, takes images of the road surface through an installed intelligent recognition camera, and identifies the road defects. For each road defect identified, the electronic device can extract features from the road surface image to obtain the image features of the road defect and obtain the location information of the road defect.

[0092] Furthermore, electronic devices can store the road defect sign along with its corresponding image features and location information, thereby establishing a road defect database. Of course, it is also reasonable to store related information such as the road surface image corresponding to the road defect sign; this is not specifically limited here.

[0093] For example, the road defect information stored in the road defect database can be shown in the following table:

[0094] Road defect signs Image features Location information Disease 1 Image Feature 1 Position 1 Disease 2 Image Feature 2 Position 2 … … … Disease n Image features n Position n

[0095] Since the target feature can characterize the characteristics of the road defect to be processed in the target image, and the image features stored in the road defect database can characterize the characteristics of the corresponding road defect in the road surface image, by comparing the target feature with each image feature, it can be determined whether the road defect to be processed is similar in appearance to the road defect corresponding to each road defect identifier stored in the road defect database.

[0096] The target location information can identify the geographical location of the road defect to be treated, while the location information stored in the road defect database can represent the geographical location of the corresponding road defect. Therefore, by comparing the target location information with the location information stored in the road defect database, it can be determined whether the road defect to be treated is geographically close to the road defect corresponding to each road defect identifier stored in the road defect database.

[0097] Therefore, the electronic device can determine whether there are existing road defects in the road defect database that are duplicates of the road defect to be processed, based on the aforementioned target features, target location information, and defect information stored in the road defect database. If the electronic device determines that there are already existing road defects in the road defect database that are duplicates of the road defect to be processed, it means that the road defect to be processed has already been identified and stored during the process of establishing the road defect database. Consequently, the electronic device can identify the road defect to be processed as a duplicate defect, that is, the electronic device executes step 104 without storing the location information, image features, etc. of the road defect to be processed into the road defect database, thus avoiding redundant information in the road defect database and achieving deduplication of road defect data.

[0098] If the electronic device determines that there is no existing road defect in the road defect database that duplicates the defect to be treated, it means that the road defect to be treated was not identified during the database creation process and may be a newly generated defect. In this case, the electronic device can proceed to step 105, which involves storing the identifier, target features, and target location information of the road defect to be treated in the road defect database. Of course, to ensure that relevant personnel can handle the road defect promptly, the electronic device can also send a notification message to the relevant personnel, i.e., issue an alarm.

[0099] For example, the road defects to be treated have the following defect information:

[0100] Road defect signs Image features Location information Disease X Image feature X Position X

[0101] If it is determined that the road defect to be processed is not a recurring defect, the electronic device should store the corresponding "defect X" with "image feature X" and "location X" in the road defect database. The defect information stored in the updated road defect database can be shown in the table below:

[0102] Road defect signs Image features Location information Disease 1 Image Feature 1 Position 1 Disease 2 Image Feature 2 Position 2 … … … Disease n Image features n Position n Disease X Image feature X Position X

[0103] In this embodiment, when duplicate road defects are found, deduplication can be performed instead of storing duplicate road defect data in the road defect database. This avoids storing existing road defects repeatedly in the database, thus achieving deduplication of road defect data. As a result, relevant personnel will not receive numerous duplicate alarms due to a large number of duplicate road defects, effectively improving their work convenience and efficiency.

[0104] As one embodiment of the present invention, in order to further improve the accuracy of deduplication of road defects, before the step of determining whether there are existing road defects in the road defect database that duplicate the road defect to be processed, based on target features, target location information, and defect information stored in a pre-established road defect database, the method may further include:

[0105] Determine the target type of the road defects to be treated.

[0106] The aforementioned road defect information may also include the type corresponding to each road defect sign. The type corresponding to a road defect sign reflects the type of road defect associated with that sign. Common types of defects may include cracks, potholes, ruts, loosening, and subsidence. Depending on the road material, for example, for common asphalt roads, cracks can be further categorized into transverse cracks, longitudinal cracks, and alligator cracks. The type of road defect can characterize its features to a certain extent, and electronic equipment can determine the target type of road defect to be treated.

[0107] As one embodiment of the present invention, the target type of road defects to be treated can be determined by measuring the depth of the defects. For example... Figure 2 As shown, the steps for determining the target type of road defects to be treated may include:

[0108] S201, Obtain the depth information of the road defects to be treated.

[0109] The depth information of the road defects to be treated can be obtained by a detection device with depth measurement function. In one embodiment, a depth image of the road defects to be treated can be acquired by a depth camera. The pixel value of each pixel in the depth image is the distance between each point in the road defects to be treated and the depth camera. Therefore, the depth information of the road defects to be treated can be determined based on the image features of the depth image.

[0110] In another implementation, the electronic device can also use two images acquired from different acquisition angles corresponding to the road defects to be treated, and obtain the depth information of the road defects to be treated through steps such as feature point calibration, feature point matching, and three-dimensional reconstruction. This is all reasonable and will not be specifically limited here.

[0111] S202, Based on the depth information and the target image, determine the target type of the road defect to be processed.

[0112] After acquiring the depth information of the road defects to be treated, the electronic equipment can determine the target type of the road defects based on the depth information and the target image. Based on the three-dimensional depth information and two-dimensional image of the road defects to be treated, both the surface morphology and the depth of the road defects can be determined. Therefore, the electronic equipment can determine the target type of the road defects to be treated based on the depth information and the target image.

[0113] Taking cracks and subsidence as examples of road defects, two-dimensional images can characterize the planar morphology of road defects, such as their direction, length, width, and offset, while depth information can characterize the three-dimensional morphology of the road defects, such as their depth and depth variations. Electronic devices can then determine the type of road defect based on this information. For example, cracks and subsidence can include: through cracks, deep cracks, surface cracks, uniform subsidence, non-uniform subsidence, and localized subsidence, etc., without specific limitations here.

[0114] The method for determining the type of each road defect identifier stored in the road defect database is the same as the method for determining the target type of the road defect to be treated. Please refer to the above description of the method for determining the target type of the road defect to be treated, which will not be repeated here.

[0115] Accordingly, the step of determining whether there are existing road defects in the road defect database that overlap with the road defect to be treated, based on target features, target location information, and defect information stored in a pre-established road defect database, may include:

[0116] Based on the target features, the target location information, the target type, and the disease information stored in the pre-established road disease database, it is determined whether there are existing road diseases in the road disease database that are duplicates of the road disease to be processed.

[0117] Since the target type can characterize the features of the road defects to be processed to a certain extent, in order to improve the accuracy of deduplication of road defect data, electronic devices can not only compare the target features and target location information of the road defects to be processed with the image features and location information stored in the road defect database, but also compare the target type of the road defects to be processed with the type corresponding to each road defect identifier stored in the road defect database, thereby more accurately determining whether there are existing road defects in the road defect database that are duplicates of the road defects to be processed.

[0118] For example, the road defect database stores defect information as shown in the table below:

[0119] Road defect signs Location information type Image features Disease 1 Position 1 Surface cracks Image Feature 1 Disease 2 Position 2 Local subsidence Image Feature 2 Disease 3 Position 3 Deformation Image Feature 3 Disease 4 Position 3 Surface cracks Image Feature 4 … … … … Disease n Position n pit Image features n

[0120] If the target type of the road defect to be treated is local subsidence, and the target location information of the road defect to be treated is similar to that of location 2, and the target image features are similar to those of image feature 2, then the road defect to be treated and the road defect identified as defect 2 are likely to be the same road defect. In this case, the electronic device can determine that there is an existing road defect in the road defect database that is the same as the road defect to be treated, i.e., the road defect identified as defect 2.

[0121] As can be seen, in this embodiment, the disease information stored in the road disease database may also include the type corresponding to each road disease identifier. In this case, during the process of determining whether the road disease to be processed is a duplicate disease, the electronic device can determine the target type of the road disease to be processed based on the depth information and target image of the road disease to be processed, and then determine whether the road disease to be processed is a duplicate disease based on the target location information, target type, target features, and disease information stored in the road disease database. This can improve the accuracy of the judgment result and thus improve the deduplication accuracy of the road disease data.

[0122] As one embodiment of the present invention, such as Figure 3 As shown, the step of determining whether there are existing road defects in the road defect database that overlap with the road defect to be processed, based on target features, target location information, target type, and defect information stored in a pre-established road defect database, may include:

[0123] S301, Based on the location information stored in the pre-established road defect database, determine whether there are existing road defects whose distance from the target location information meets the preset distance condition; if so, proceed to step S302;

[0124] If the location of an existing road defect is far from the location of the road defect to be treated, the probability that they are the same road defect is extremely low. Road defects that are close to each other are more likely to be the same road defect. Therefore, a distance condition that needs to be met between the two can be preset, i.e., a preset distance condition. In this way, the electronic device can determine whether there is an existing road defect whose distance from the target location meets the preset distance condition based on the location information stored in the pre-established road defect database.

[0125] The preset distance condition can be a distance less than a certain value, such as less than 50 meters, less than 30 meters, less than 60 meters, etc., without specific limitations. If there are existing road defects in the pre-established road defect database that meet the preset distance condition from the target location information, it indicates that there are existing road defects that are close to the location of the road defect to be treated. This existing road defect is very likely to be the same road defect as the road defect to be treated. In this case, step S302 can be executed to further determine whether the two are the same road defect.

[0126] Taking the two road defects in the table below as examples, defect A is located at location 1, and defect B is located at location 99. If location 1 and location 99 are far apart, for example, hundreds of kilometers apart, then defect A and defect B cannot be the same road defect. If location 1 and location 99 are very close, for example, 20 meters apart, then defect A and defect B may be the same road defect.

[0127] Road defect signs Location information type Image features Disease A Position 1 Surface cracks Image Feature 1 Disease B Position 99 Surface cracks Image Feature 1

[0128] Based on the location information stored in the pre-established road defect database, it is determined whether there are existing road defects whose distance from the target location information meets the preset distance condition. This is equivalent to performing location clustering on the location information stored in the road defect database. Existing road defects whose distance from the target location information meets the preset distance condition can be clustered into a class of candidate existing road defects, thereby reducing the amount of calculation in subsequent processes.

[0129] S302, determine whether the type of the existing road defects is the same as the target type; if they are the same, proceed to step S303;

[0130] If there are existing road defects in the road defect database that meet the preset distance conditions between the target location information and the existing road defect, in order to further determine whether the existing road defect and the defect to be treated are the same road defect, the electronic device can determine whether the type of the existing road defect is the same as the target type of the road defect to be treated.

[0131] Even if the locations are very close, if the type of existing road defects is different from the target type of road defects to be treated, they may not be the same road defects. Therefore, electronic equipment can determine whether the type of existing road defects is the same as the target type of road defects to be treated.

[0132] For example, if the locations of defects 1 and 2 in the table below are both close to the location of defect y to be treated, and the target type of road defect y to be treated is pothole, then the electronic device can determine whether the types of defects 1 and 2 are the same as the target type of road defect y to be treated. The electronic device can determine that the type of defect 1 is different from the target type of road defect y to be treated, while the type of defect 2 is the same as the target type of road defect y to be treated. Therefore, defect 2 may be the same road defect as road defect y to be treated, and step S303 can be continued.

[0133] Road defect signs Location information type Image features Disease 1 Position 15 Surface cracks Image Feature 19 Disease 2 Position 18 pit Image Features 21 Untreated disease y Position 20 pit Target image features

[0134] S303, calculate the similarity between the image features corresponding to the existing road defects and the target features; if the similarity reaches a preset threshold, proceed to step S304;

[0135] S304, the road defect to be treated is determined to be a recurring defect.

[0136] In some cases, new road defects of the same type may appear in a location close to an existing road defect. Therefore, even if the type of the existing road defect is the same as the target type of the road defect to be treated, they may still not be duplicate road defects. Therefore, in order to ensure the accuracy of deduplication, electronic equipment can calculate the similarity between the image features of the existing road defect and the target features.

[0137] The similarity between the image features of existing road defects and the target features can be calculated using methods such as cosine distance and Euclidean distance, without specific limitations or explanations here.

[0138] If the similarity between the image features corresponding to existing road defects and the target features reaches a preset threshold, it means that the existing road defects and the defects to be processed are very similar in appearance. Therefore, the two are likely to be the same road defects, and the electronic device can determine that the road defects to be processed are duplicate defects, that is, execute step S304.

[0139] The preset threshold can be set according to factors such as the actual requirements for the repetition rate of the road defect database. For example, it can be 80%, 85%, 92%, etc., without specific limitations here.

[0140] For example, if the preset threshold is 80%, and the similarity between the image feature 21 corresponding to the above-mentioned disease 2 and the target image feature is 95%, then since 95% is greater than 80%, it means that the morphology of disease 2 and the disease to be treated y are very similar, and the electronic device can determine that the two are the same road disease, that is, the road disease to be treated is a duplicate disease.

[0141] As can be seen, in this embodiment, the electronic device can determine whether there are existing road defects whose distance from the target location meets a preset distance condition based on the location information stored in a pre-established road defect database. If such defects exist, it determines whether the type of the existing road defect is the same as the target type. If they are the same, it further calculates the similarity between the image features corresponding to the existing road defect and the target features. If the similarity reaches a preset threshold, the road defect to be processed is determined to be a duplicate defect. In this way, it is possible to quickly and accurately determine whether the road defect to be processed is a duplicate defect, further improving the accuracy and efficiency of deduplication of road defect data.

[0142] As one embodiment of the present invention, the above method may further include:

[0143] When any of the following three conditions are met, the identifier of the road defect to be processed is stored in the road defect database in correspondence with the target feature, the target location information, and the target type.

[0144] In the first scenario, if the location information stored in the pre-established road defect database does not contain any existing road defects whose distance from the target location information meets the preset distance condition.

[0145] If the location information stored in the pre-established road defect database does not contain any existing road defects whose distance from the target location meets the preset distance condition, it means that the location of the road defect to be processed is far from the location of the existing road defects. Therefore, the road defect to be processed is likely a new road defect. Thus, the electronic device can store the identifier of the road defect to be processed, along with the target features, target location information, and target type, in the road defect database to update the road defect database.

[0146] For example, the target location information of the road defect to be processed is: 156°23′17″E, 69°54′27″N, with a preset distance condition of within 150 meters centered on the location of the road defect to be processed. If the pre-established road defect database does not contain any stored road defects within this 150-meter range, the electronic device can determine that the road defect to be processed is not a duplicate, and can then store its identifier in the road defect database, corresponding to the target characteristics, target location information, and target type.

[0147] The second scenario is if the type of existing road defects is different from the target type.

[0148] If the distance between the location of an existing road defect and the target location meets the preset distance condition, but the type of the existing road defect is different from the target type, and they are not the same road defect, the electronic device can also store the identifier of the road defect to be processed, the target features, the target location information, and the target type in the road defect database to update the road defect database.

[0149] For example, if the target type of the road defect to be processed is deformation, the existing road defects whose location information stored in the road defect database meets the preset distance condition between the target location information and the location information are shown in the table below:

[0150] Road defect signs Location information type Image features Disease x Position 15 Surface cracks Image Feature 19 Disease y Position 20 pit Image feature 26

[0151] At this point, since the target type of the road defect to be processed is different from the types of existing road defects in the table above, the electronic device can determine that the road defect to be processed is not a duplicate road defect, and can then store the identifier of the road defect to be processed in the road defect database along with the target features, target location information and target type.

[0152] The third scenario is if the similarity does not reach the preset threshold.

[0153] If the distance between the location of an existing road defect and the target location meets a preset distance condition, and the types are the same as the target type, but since a new road defect of the same type may appear in a location close to an existing road defect, if the similarity between the corresponding image features does not reach a preset threshold, it also indicates that the two are not the same road defect. In this case, the electronic device can also store the identifier of the road defect to be processed, along with the target features, target location information, and target type, in the road defect database to update the road defect database.

[0154] For example, if the distance between the target image features corresponding to the road defect to be processed and the location information of the target location meets the preset distance condition, and the similarity between the image features corresponding to existing road defects of the same type and target type does not reach the preset threshold, then the electronic device can determine that the road defect to be processed is not a duplicate road defect. Then, the identifier of the road defect to be processed can be stored in the road defect database along with the target features, target location information and target type.

[0155] As one implementation method, when storing the identification of the road defects to be treated in the road defect database along with the target features, target location information and target type, the target image corresponding to the road defects to be treated can also be stored so that relevant personnel can view the target image corresponding to the road defects to be treated when needed.

[0156] As can be seen, in this embodiment, for the above three situations, the electronic device can store the identifier of the road defect to be processed, the target features, the target location information, and the target type in the road defect database when the road defect to be processed is a new road defect, thereby updating the road defect database.

[0157] As one embodiment of the present invention, the target image described above can be acquired by an image acquisition device installed on a patrol vehicle.

[0158] During road patrols, image acquisition can be performed by patrol vehicles equipped with image acquisition devices. In one embodiment, the image acquisition device can be an intelligent recognition camera. When the patrol vehicle is driving on the road, the intelligent recognition camera acquires road images and identifies them. When a road defect is identified, the road image including the defect is the target image.

[0159] Correspondingly, such as Figure 4 As shown, the steps described above for obtaining the target location information and target image corresponding to the road defects to be processed may include:

[0160] S401, for each road image acquired by the image acquisition device, identify the road defects to be processed included in the road image, until the road image currently acquired by the image acquisition device does not contain the road defects to be processed, and take the road image containing the road defects to be processed as the target image.

[0161] Image acquisition equipment can capture road images in real time while the patrol vehicle is in motion, and identify whether the road defects to be treated are included in the images. If they are, the electronic equipment can obtain the vehicle's position corresponding to that frame of the target image. The image acquisition equipment then continues to acquire the next frame of the target image and obtain the vehicle's position corresponding to that frame. This continues until no road defects to be treated can be identified from the acquired road images, meaning that the road defects to be treated do not exist in the road images. At this point, the road image containing the road defects to be treated can be used as the target image. For example, all road images containing the road defects to be treated can be used as the target image; alternatively, from the road images containing the road defects to be treated, a clear image with a well-defined outline of the road defects to be treated can be selected as the target image.

[0162] S402, Obtain the vehicle position when the image acquisition device acquires each frame of the target image;

[0163] Since the location information of the road defects to be treated is generally not directly obtainable, electronic equipment can calculate the location information of the road defects based on the position of the patrol vehicle and the positional relationship between the target image acquired by the image acquisition device and the patrol vehicle. Specifically, the patrol vehicle can have a positioning module that can locate the vehicle's position. The electronic equipment can obtain the vehicle position reported by the positioning module when the image acquisition device acquires the target image. Alternatively, the image acquisition device can have a positioning function, in which case, while acquiring the target image, the image acquisition device can obtain the current vehicle position and send the vehicle position and the target image to the electronic equipment.

[0164] Furthermore, the image acquisition device collects road images in real time during the patrol vehicle's journey. When each frame of image is acquired, the electronic device can obtain the vehicle position corresponding to that frame of target image. After the step of using the road image including the road defects to be processed as the target image, the vehicle position when the image acquisition device acquires each frame of target image can be obtained.

[0165] S403, for each frame of the target image, calculate the location information of the road defect to be processed based on the calibration information of the image acquisition device and the vehicle position corresponding to the frame of the target image.

[0166] The target image can consist of multiple frames. For each frame, the electronic device can acquire a corresponding vehicle position. To improve the accuracy of the target position information for the road defects to be treated, for each frame, the electronic device can calculate the position information of the road defects based on the calibration information of the image acquisition device and the vehicle position corresponding to that frame. The calibration information of the image acquisition device can include its extrinsic parameters, i.e., its mounting data on the inspection vehicle, such as height and angle, as well as its intrinsic parameters, such as focus and optical center coordinates.

[0167] Based on the calibration information of the image acquisition device, the mapping relationship between the world coordinate system and the image coordinate system can be determined. Then, for each frame of the target image, the electronic device can calculate the position of the road defect to be processed in the world coordinate system based on the vehicle position and the pixel coordinates of the road defect to be processed in the target image, i.e., the position information of the road defect to be processed.

[0168] S404, determine the target location information of the road defect to be processed based on the location information corresponding to each frame of the target image.

[0169] Through the above steps, the electronic device can calculate the location information corresponding to multiple target images. Based on the location information corresponding to each target image, the electronic device can determine the target location information of the road defects to be processed.

[0170] In one implementation, the location information corresponding to multiple target images can form a set of location information corresponding to the road defects to be processed. The electronic device can calculate the average value of the location information corresponding to all or a portion of the target images in this set to obtain the target location information of the road defects to be processed. The average value can be calculated as an arithmetic mean, a weighted average, etc., and is not specifically limited here. For example, if the target images corresponding to the road defects to be processed include target image 1 to target image 15, and their corresponding location information are location information 1 to location information 15, then the electronic device can calculate the average value of location information 1 to location information 15 as the target location information of the road defects to be processed. Of course, the electronic device can also use the location information corresponding to any single frame of the target image as the target location information of the road defects to be processed.

[0171] In another implementation, the electronic device may use some or all of the location information corresponding to multiple frames of target images as the target location information of the road defect to be processed. In this case, when determining whether there is an existing road defect whose distance from the target location information meets the preset distance condition, the electronic device can calculate the distance between the target location information of the road defect to be processed and the location information stored in the pre-established road defect database. If there is a location information in the target location information that does not meet the preset distance condition with the location information stored in the pre-established road defect database, the electronic device can determine that the road defect to be processed is not an existing road defect, and then store the identifier of the road defect to be processed, the target features, and the target location information in the road defect database.

[0172] As can be seen, in this embodiment, each time a road image is acquired by the image acquisition device, the electronic device can identify the road defects to be processed included in the road image until no road defects to be processed are found in the currently acquired road image. At this point, the road image containing the road defects to be processed can be used as the target image. The electronic device can calculate the location information of the road defects to be processed based on the calibration information of the image acquisition device and the vehicle position corresponding to the acquisition of the target image frame. Then, based on the location information corresponding to each target image frame, the target location information of the road defects to be processed can be determined. In this way, the location information of the road defects to be processed can be accurately calculated, thereby improving the accuracy of the target location information of the road defects to be processed, and consequently improving the accuracy of subsequent deduplication of road defect data.

[0173] As one embodiment of the present invention, the target image described above can be multiple frames.

[0174] During the image acquisition process for road defects, the patrol vehicle travels on the road and can acquire multiple frames of images from far to near for a specific road defect to be treated. In other words, the target image can be multiple frames. These multiple target images are acquired from far to near for the same road defect. Although these multiple target images are for the same road defect to be treated, the acquisition angle from far to near helps to more comprehensively characterize the morphological features of the road defect to be treated, which is helpful for subsequent determination of target type and extraction of image features.

[0175] Therefore, in this case, the step of extracting image features of the target image as target features as described above can include:

[0176] Extract the image features of each target image frame, and combine the image features of a preset number of target images frames to form an image feature set, which serves as the target features corresponding to the road defects to be processed.

[0177] For multiple frames of target images of road defects to be processed, the electronic device can extract image features from each frame and save the extracted features. This allows the image features of a preset number of frames to be combined into an image feature set, which can then be used as the target features corresponding to the road defects. This helps improve the accuracy of subsequent image feature similarity comparison processes. The preset number of target frames can be set based on the accuracy requirements of the actual application and the computing power of the electronic device. For example, it could be one or more frames from the target images, or even all the target images; these are all reasonable options and are not specifically limited here.

[0178] Correspondingly, when establishing the aforementioned road defect database, multiple road surface images can be collected for each road defect, and then image features can be extracted from the multiple road surface images corresponding to each road defect. The image features of a preset number of target images are combined into an image feature set, and the image feature set is stored in relation to the road defect identifier.

[0179] In one embodiment, when calculating the similarity between image features corresponding to existing road defects and target features, the electronic device, for each existing road defect, can calculate the similarity between each image feature in its corresponding image feature set and the corresponding image features among the multiple image features corresponding to the road defect to be processed. The average of these multiple similarities is then used as the similarity between the image features corresponding to the existing road defect and the target features. The average value can be an arithmetic mean, a weighted average, etc., and is not specifically limited here.

[0180] For example, the existing image feature set corresponding to road defect P includes image features A to J corresponding to road surface images 1 to 10, where road surface images 1 to 10 were acquired by the image acquisition device from far to near. The image feature set corresponding to the road defect to be processed includes image features a to j corresponding to target images 1 to 10, where target images 1 to 10 were acquired by the image acquisition device from far to near.

[0181] The electronic device can then calculate the similarity between image feature A and image feature a, image feature B and image feature b, image feature C and image feature c, ..., image feature J and image feature j, respectively, obtaining similarity scores from 1 to 10. Furthermore, the electronic device can calculate the arithmetic mean of similarity scores from 1 to 10, and use this arithmetic mean as the similarity score between the image features corresponding to the existing road defect P and the target image features corresponding to the road defect to be processed.

[0182] In another implementation, when the electronic device calculates the similarity between the image features corresponding to existing road defects and the target features, for each existing road defect, the similarity between its corresponding image features and the image features corresponding to each target image included in the image feature set corresponding to the road defect to be processed can be calculated separately. If the similarity of any frame of target image reaches a preset threshold, the road defect to be processed can be determined to be a duplicate defect.

[0183] Of course, in another implementation, for each existing road defect, its corresponding image features can be compared with each image feature in the image feature set corresponding to the road defect to be processed. If a target number of similarities reaches a preset threshold, the road defect to be processed can be determined to be a duplicate defect. For example, if the image feature set includes image features corresponding to 5 target images, 5 similarities can be calculated. Assuming the target number is 3, if 3 or more of these 5 similarities reach the preset threshold, the road defect to be processed can be determined to be a duplicate defect.

[0184] As can be seen, in this embodiment, the target image can be multiple frames. In this case, the electronic device can extract the image features of each frame of the target image and combine the image features of a preset number of frames of target images into an image feature set, which serves as the target feature corresponding to the road defect to be processed. Since the multiple target images are collected from far to near on the road defect to be processed, although the multiple target images are for the same road defect to be processed, the collection angle from far to near helps to more comprehensively characterize the morphological features of the road defect to be processed. This is helpful for subsequent determination of target type, image feature extraction, etc., and can improve the accuracy of subsequent image feature similarity comparison process, thereby improving the accuracy of deduplication of road defect data.

[0185] Corresponding to the above-described road defect detection method, this invention also provides a road defect detection device. The following is a description of the road defect detection device provided by this invention.

[0186] like Figure 5 As shown, a road defect detection device includes:

[0187] The disease information acquisition module 510 is used to acquire the target location information and target image corresponding to the road disease to be processed.

[0188] The image feature extraction module 520 is used to extract the image features of the target image as target features.

[0189] The duplicate road disease determination module 530 is used to determine whether there are existing road diseases that duplicate the road disease to be processed in the road disease database based on the target features, the target location information and the disease information stored in the pre-established road disease database.

[0190] The disease information includes the correspondence between pre-acquired road disease identifiers, image features, and location information.

[0191] The road defect deduplication module 540 is used to determine that the road defect to be processed is a duplicate defect if there is an existing road defect in the road defect database that is duplicated with the road defect to be processed.

[0192] The road defect storage module 550 is used to store the identifier of the road defect to be processed, the target feature, and the target location information in the road defect database if there is no existing road defect in the road defect database that is duplicated with the road defect to be processed.

[0193] As can be seen, in the solution provided by this embodiment of the invention, the electronic device can acquire the target location information and target image corresponding to the road defect to be processed, and extract image features from the target image as target features. Then, based on the target features, target location information, and defect information stored in a pre-established road defect database, it determines whether there are existing road defects in the database that duplicate the road defect to be processed. The defect information includes the pre-acquired correspondence between each road defect identifier and its image features and location information. If such a correspondence exists, the road defect to be processed is determined to be a duplicate defect; if not, the identifier of the road defect to be processed is stored in the road defect database along with its target features and target location information. In this way, when duplicate road defects occur, deduplication can be performed, preventing the duplicate road defect data from being stored in the road defect database. This avoids repeatedly storing existing road defects in the road defect database, thus achieving deduplication of road defect data.

[0194] As one embodiment of the present invention, the above-mentioned disease information may further include the type corresponding to each road disease identifier;

[0195] like Figure 6 As shown, the above-mentioned device may further include:

[0196] The target type determination module 610 is used to determine the target type of the road defect to be processed before determining whether there is an existing road defect in the road defect database that is duplicated with the road defect to be processed, based on the target features, the target location information and the defect information stored in the pre-established road defect database.

[0197] The aforementioned recurring disease identification module 530 may include:

[0198] The duplicate road defect determination unit is used to determine, based on the target features, the target location information, the target type, and the defect information stored in the pre-established road defect database, whether there are existing road defects in the road defect database that duplicate the road defect to be processed.

[0199] As one embodiment of the present invention, the above-mentioned recurring disease determination unit may include:

[0200] The first determining subunit is used to determine, based on the location information stored in the pre-established road defect database, whether there are existing road defects whose distance from the target location information meets the preset distance condition.

[0201] The second determining subunit is used to determine whether the type of the existing road defects is the same as the target type if there are existing road defects whose distance from the target location information meets a preset distance condition.

[0202] The third determining subunit is used to calculate the similarity between the image features corresponding to the existing road defects and the target features if the type of the existing road defects is the same as the target type.

[0203] The fourth determining subunit is used to determine that the road defect to be processed is a repeating defect if the similarity reaches a preset threshold.

[0204] As one embodiment of the present invention, the above-mentioned recurring disease determination unit may further include:

[0205] A storage subunit is configured to, if the location information stored in the pre-established road defect database does not contain any existing road defects whose distance from the target location information meets a preset distance condition, store the identifier of the road defect to be processed in the road defect database, corresponding to the target feature, the target location information, and the target type; or,

[0206] This is used to store the identifier of the road defect to be processed in the road defect database, corresponding to the target feature, the target location information, and the target type, if the type of the existing road defect is different from the target type; or,

[0207] If the similarity does not reach a preset threshold, the identifier of the road defect to be processed is stored in the road defect database along with the target feature, the target location information, and the target type.

[0208] As one embodiment of the present invention, the target image described above can be acquired by an image acquisition device installed on a patrol vehicle;

[0209] The aforementioned disease information acquisition module 510 may include:

[0210] The target image acquisition unit is used to acquire road images acquired by the image acquisition device, identify the road defects to be processed included in the road images, until the road images currently acquired by the image acquisition device do not contain the road defects to be processed, and then take the road images containing the road defects to be processed as target images.

[0211] The vehicle position acquisition unit is used to acquire the vehicle position when the image acquisition device acquires each frame of the target image;

[0212] The location information calculation unit is used to calculate the target location information of the road defect to be processed for each frame of target image, based on the calibration information of the image acquisition device and the vehicle position corresponding to the frame of target image.

[0213] The target location information determination unit is used to determine the target location information of the road defect to be processed based on the location information corresponding to each frame of the target image.

[0214] As one embodiment of the present invention, the target image can be multiple frames; the image feature extraction module 520 can include:

[0215] The image feature extraction unit is used to extract the image features of each frame of the target image, and to form an image feature set by combining the image features of a preset number of frames of the target image, which serves as the target feature corresponding to the road defect to be processed.

[0216] This invention also provides an electronic device, such as... Figure 7 As shown, it includes a processor 701, a communication interface 702, a memory 703, and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other through the communication bus 704.

[0217] Memory 703 is used to store computer programs;

[0218] When the processor 701 executes the program stored in the memory 803, it implements the steps of the road defect detection method described in any of the above embodiments.

[0219] As can be seen, in the solution provided by this embodiment of the invention, the electronic device can acquire the target location information and target image corresponding to the road defect to be processed, and extract image features from the target image as target features. Then, based on the target features, target location information, and defect information stored in a pre-established road defect database, it determines whether there are existing road defects in the database that duplicate the road defect to be processed. The defect information includes the pre-acquired correspondence between each road defect identifier and its image features and location information. If such a correspondence exists, the road defect to be processed is determined to be a duplicate defect; if not, the identifier of the road defect to be processed is stored in the road defect database along with its target features and target location information. In this way, when duplicate road defects occur, deduplication can be performed, preventing the duplicate road defect data from being stored in the road defect database. This avoids repeatedly storing existing road defects in the road defect database, thus achieving deduplication of road defect data.

[0220] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0221] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0222] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0223] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0224] In another embodiment of the present invention, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of the road defect detection method described in any of the above embodiments.

[0225] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute the road defect detection method described in any of the above embodiments.

[0226] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of the present invention is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)).

[0227] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0228] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for devices, electronic devices, computer-readable storage media, and computer program products, since they are basically similar to the method embodiments, the descriptions are relatively simple, and relevant parts can be referred to the descriptions of the method embodiments.

[0229] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A method for detecting road defects, characterized in that, The method includes: Acquire the target location information and target image corresponding to the road defects to be treated; Extract the image features of the target image as target features; Based on the target features, the target location information, and the disease information stored in the pre-established road disease database, it is determined whether there are existing road diseases in the road disease database that duplicate the road disease to be processed. The disease information includes the correspondence between each road disease identifier and image features and location information obtained in advance. The target location information identifies the geographical location of the road disease to be processed, and the location information stored in the road disease database represents the geographical location of the corresponding road disease. If it exists, the road defect to be treated is determined to be a duplicate defect, wherein the distance between the geographical location corresponding to the road defect to be treated and the geographical location corresponding to the duplicate existing road defect meets a preset distance condition. If it does not exist, the identifier of the road defect to be processed, the target feature, and the target location information are stored in the road defect database.

2. The method according to claim 1, characterized in that, The disease information also includes the type corresponding to each road disease marker; Before the step of determining whether there are existing road defects in the road defect database that duplicate the road defect to be processed, based on the target features, the target location information, and defect information stored in a pre-established road defect database, the method further includes: Determine the target type of the road defects to be treated; The step of determining whether there are existing road defects in the road defect database that overlap with the road defect to be processed, based on the target features, the target location information, and defect information stored in a pre-established road defect database, includes: Based on the target features, the target location information, the target type, and the disease information stored in the pre-established road disease database, it is determined whether there are existing road diseases in the road disease database that are duplicates of the road disease to be processed.

3. The method according to claim 2, characterized in that, The step of determining whether there are existing road defects in the road defect database that duplicate the road defect to be processed, based on the target features, the target location information, the target type, and defect information stored in a pre-established road defect database, includes: Based on the location information stored in the pre-established road defect database, determine whether there are existing road defects whose distance from the target location information meets the preset distance condition; If present, determine whether the type of the existing road defect is the same as the target type; If they are the same, calculate the similarity between the image features corresponding to the existing road defects and the target features; If the similarity reaches a preset threshold, the road defect to be processed is determined to be a duplicate defect.

4. The method according to claim 3, characterized in that, The method further includes: If the location information stored in the pre-established road defect database does not contain any existing road defects whose distance from the target location information meets the preset distance condition, the identifier of the road defect to be processed is stored in the road defect database along with the target feature, the target location information, and the target type; or, If the type of the existing road defect is different from the target type, the identifier of the road defect to be processed is stored in the road defect database along with the target features, the target location information, and the target type; or, If the similarity does not reach a preset threshold, the identifier of the road defect to be processed is stored in the road defect database along with the target feature, the target location information, and the target type.

5. The method according to any one of claims 1-4, characterized in that, The target image is acquired by an image acquisition device installed on the patrol vehicle; the step of acquiring the target location information and target image corresponding to the road defects to be processed includes: Each time a road image is acquired by the image acquisition device, the road defects to be processed included in the road image are identified until the road defects to be processed are no longer present in the road images currently acquired by the image acquisition device. Then, the road image including the road defects to be processed is taken as the target image. The vehicle position is obtained when the image acquisition device acquires each frame of the target image; For each frame of the target image, the location information of the road defect to be processed is calculated based on the calibration information of the image acquisition device and the vehicle position corresponding to that frame of the target image. The target location information of the road defects to be processed is determined based on the location information corresponding to each frame of the target image.

6. The method according to any one of claims 1-4, characterized in that, The target image consists of multiple frames; The step of extracting image features from the target image as target features includes: Image features are extracted from each frame of the target image, and the image features of a preset number of frames of the target image are combined into an image feature set, which is used as the target feature corresponding to the road defect to be processed.

7. A road defect detection device, characterized in that, The device includes: The road disease information acquisition module is used to acquire the target location information and target image of the road disease to be treated; The image feature extraction module is used to extract image features from the target image as target features; The duplicate road defect determination module is used to determine whether there are existing road defects in the road defect database that duplicate the road defect to be processed, based on the target features, the target location information, and the defect information stored in the pre-established road defect database. The defect information includes the correspondence between each road defect identifier and image features and location information obtained in advance. The target location information identifies the geographical location of the road defect to be processed, and the location information stored in the road defect database represents the geographical location of the corresponding road defect. The road defect deduplication module is used to determine that the road defect to be processed is a duplicate defect if there is an existing road defect in the road defect database that is duplicated with the road defect to be processed. The distance between the geographical location of the road defect to be processed and the geographical location of the duplicate existing road defect meets a preset distance condition. The road defect storage module is used to store the identifier of the road defect to be processed, the target feature, and the target location information in the road defect database if there is no existing road defect in the database that duplicates the road defect to be processed.

8. The apparatus according to claim 7, characterized in that, The disease information also includes the type corresponding to each road disease marker; The device further includes: The target type determination module is used to determine the target type of the road defect to be processed before determining whether there are existing road defects in the road defect database that are duplicates of the road defect to be processed, based on the target features, the target location information and the defect information stored in the pre-established road defect database. The recurring disease identification module includes: The duplicate road disease determination unit is used to determine, based on the target features, the target location information, the target type, and the disease information stored in the pre-established road disease database, whether there are existing road diseases in the road disease database that duplicate the road disease to be processed. The recurring disease identification unit includes: The first determining subunit is used to determine, based on the location information stored in the pre-established road defect database, whether there are existing road defects whose distance from the target location information meets the preset distance condition. The second determining subunit is used to determine whether the type of the existing road defects is the same as the target type if there are existing road defects whose distance from the target location information meets a preset distance condition. The third determining subunit is used to calculate the similarity between the image features corresponding to the existing road defects and the target features if the type of the existing road defects is the same as the target type. The fourth determining subunit is used to determine that the road defect to be processed is a repeating defect if the similarity reaches a preset threshold. The recurring disease identification unit also includes: A storage subunit is configured to, if the location information stored in the pre-established road defect database does not contain any existing road defects whose distance from the target location information meets a preset distance condition, store the identifier of the road defect to be processed in the road defect database, corresponding to the target feature, the target location information, and the target type; or, This is used to store the identifier of the road defect to be processed in the road defect database, corresponding to the target feature, the target location information, and the target type, if the type of the existing road defect is different from the target type; or, If the similarity does not reach a preset threshold, the identifier of the road defect to be processed is stored in the road defect database along with the target feature, the target location information, and the target type. The target image was acquired by an image acquisition device installed on the patrol vehicle; the disease information acquisition module includes: The target image acquisition unit is used to acquire road images acquired by the image acquisition device, identify the road defects to be processed included in the road images, until the road images currently acquired by the image acquisition device do not contain the road defects to be processed, and then take the road images containing the road defects to be processed as target images. The vehicle position acquisition unit is used to acquire the vehicle position when the image acquisition device acquires each frame of the target image; The location information calculation unit is used to calculate the location information of the road defect to be processed for each frame of target image, based on the calibration information of the image acquisition device and the vehicle position corresponding to the frame of target image; The target location information determination unit is used to determine the target location information of the road defect to be processed based on the location information corresponding to each frame of the target image. The target image consists of multiple frames; the image feature extraction module includes: The image feature extraction unit is used to extract the image features of each frame of the target image, and to form an image feature set by combining the image features of a preset number of frames of the target image, which serves as the target feature corresponding to the road defect to be processed.

9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method described in any one of claims 1-6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-6.

Images