Solving method for deduplication of road disease patrol work order based on computer vision technology

Abstract

The invention discloses a computer vision technology-based road disease patrol work order deduplication solving method, which comprises the following steps of: S1, road patrol: acquiring road disease information by adopting a road patrol identification terminal, and uploading the disease information to a work order platform; s2, the platform processes and distributes a work order: when the work order platform receives the disease information of the road inspection identification terminal, the work order service platform carries out screening according to the disease grade degree and judges whether the disease is the same disease, and if the disease is a new disease, the work order is distributed to the mobile terminal; and S3, the mobile terminal processes the work order: after receiving the work order, the mobile terminal performs construction maintenance and returns information to the work order platform. According to the method, the problem of repeated work orders is solved, and the road detection efficiency and accuracy are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of road pavement disease detection, and in particular relates to a solution and system for deduplication of road disease inspection work orders based on computer vision technology. Background Art

[0002] After decades of construction, the national highway mileage has reached several million kilometers. However, a large part of the highway construction has been completed for more than ten years. Whether it is a concrete road or an asphalt road, the road surface may be damaged and aged due to factors such as climate and temperature; or due to the increased pressure on the road surface caused by overloaded vehicles, improper construction process leading to unstable road structure, excessive traffic flow leading to wear and damage of the road surface; or due to the poor quality or unreasonable matching of road materials, the road surface structure is unstable. Due to the above reasons, the road surface continues to breed diseases, including cracks, subsidence, potholes, rutting and other problems. Road maintenance personnel need to continuously inspect the roads and maintain and repair the diseases found to ensure the normal traffic of the roads. In the traditional way, when inspectors drive on the road for inspection, they stop to take pictures when they find diseases, which has problems such as unsafe parking, low efficiency and high cost. In recent years, with the development of artificial intelligence technology, inspectors have used computer vision technology to identify diseases and capture photos of diseases through portable vehicle-mounted binocular cameras, AI recognition terminals, GPS and other equipment when inspecting roads. They also record the GPS location of the disease, automatically upload it to the platform and form a disease work order.

[0003] However, in actual vehicle inspections, a certain section of the same road may be inspected multiple times due to route planning and other reasons, or when driving in a two-way lane, the same defect point may be identified when driving in both the forward and reverse directions, resulting in multiple defect problem work orders for the same defect point, that is, duplicate work orders. When duplicate work orders are issued for the same defect point, maintenance workers may have to go to the site multiple times, which will lead to repeated calculation of maintenance workload, waste of maintenance resources, and inability to carry out refined management of road maintenance.

[0004] Chinese patent document CN116070883A discloses a method and system for automatic dispatching of daily road maintenance, which includes: obtaining a picture of a defect and its location based on a preset intelligent sensing device; identifying the picture of the defect to determine the size and type of the defect; searching for road information in a preset filing library according to the location of the defect; determining the maintenance score of the defect according to the road information, the size of the defect and the type of the defect; removing duplicates from the defect, integrating the maintenance scores of the duplicated defects, and obtaining the maintenance score within a 100-meter section. However, this technical solution cannot effectively solve the problem of duplicate work orders.

[0005] Chinese patent document CN116070883A discloses a method and system for automatic dispatching management of daily road maintenance, the method comprising obtaining a disease image and a disease location based on a preset intelligent sensing device; identifying the disease image, determining the disease size and the disease type; querying the road information in a preset filing library according to the disease location; determining the maintenance score of the disease according to the road information, the disease size and the disease type; deduplicating the disease, integrating the maintenance score of the deduplicated disease, and obtaining the maintenance score within a 100-meter section. The present invention utilizes road disease data collected at a high frequency, adopts a method of multiple comparisons to obtain the optimal solution, and dynamically corrects and periodically adjusts the relevant parameters of the pavement disease deduplication model, which can greatly improve the accuracy of pavement disease identification, and improve the accuracy and efficiency of road maintenance. The deduplication execution unit in the technical solution is used to determine that one of the diseases is a duplicate disease and delete it when the size difference is less than a preset threshold. According to the results described in the patent, the deduplication process only describes the comparison based on the size difference and the preset threshold. This method is rather one-sided and cannot relatively scientifically determine whether the disease point is a repeated disease.

[0006] Chinese patent document CN117893985A discloses a method for deduplicating road inspection results based on similarity measurement, including the following steps: 1. Collect road inspection results; 2. Preprocess the data after the road inspection results are converted; 3. Divide the preprocessed data set into a training set and a test set; 4. Use the twin network model to extract image features and obtain the feature vector of the input image; 5. Calculate the distance between the feature vectors of the input image group to obtain the similarity; 6. Divide the threshold according to the distribution of the similarity in the test set, and determine whether the input image group is the same disease according to the threshold; 7. Collect multiple inspection results and cluster them; 8. Deduplication of the inspection results that have completed clustering. In this technical solution, the similarity of the inspection results within the clustered class is calculated using the trained model, and only one copy is retained for the inspection results determined to be the same disease, so as to complete the deduplication of road inspection results. According to the results described in the technical solution, it can be seen that if we only rely on similarity to judge whether the disease points are the same disease, the accuracy of the judgment will be affected because the images will be affected by various noise factors such as different angles, different lighting, different backgrounds, and different distances. At the same time, the disease points are also affected by multiple factors such as GPS location, disease size, and disease type. If we only rely on image similarity to remove duplicates, it is relatively one-sided. Moreover, the technical solution only mentions the method of deduplication, and does not describe the detailed steps for deduplication of duplicate work orders.

[0007] Therefore, it is necessary to provide a solution for deduplicating road defect inspection work orders based on computer vision technology to solve the problem of duplicate work orders and improve the efficiency and accuracy of road detection. Summary of the invention

[0008] The technical problem to be solved by the present invention is to provide a solution for deduplicating road disease inspection work orders based on computer vision technology, thereby solving the problem of duplicate work orders and improving the efficiency and accuracy of road detection.

[0009] In order to solve the above technical problems, the technical solution adopted by the present invention is: the solution for deduplication of road disease inspection work orders based on computer vision technology specifically includes the following steps:

[0010] The specific steps include:

[0011] S1 Road Inspection: Use road inspection identification terminal to collect road damage information and upload the damage information to the work order platform;

[0012] The S2 platform processes and dispatches work orders: When the work order platform receives the disease information from the road inspection and identification terminal, the work order service platform screens the disease levels according to the mild, general and severe levels, and determines whether it is the same disease. If it is a new disease, the work order is dispatched to the mobile terminal;

[0013] S3 mobile terminal processes work orders: After receiving the work order, the mobile terminal performs construction and maintenance and sends the information back to the work order platform.

[0014] Preferably, the road inspection identification terminal in step S1 comprises an image acquisition module, an identification terminal and a positioning module, wherein the positioning acquisition module and the image acquisition module are integrated into one and connected to the identification terminal; wherein the image acquisition module is arranged in front of the identification terminal.

[0015] Preferably, the specific steps of step S1 are:

[0016] S11: The road inspection and recognition terminal pulls the video stream of the image acquisition module and extracts frames from the video stream;

[0017] S12: The recognition terminal recognizes the extracted image, records the recognized disease data, records the disease type (including pits, subsidence, cracks, etc.), captures the disease image and measures the size of the disease; at the same time, the recognition terminal matches the disease image with the real-time recorded positioning information one by one;

[0018] S13: The identification terminal puts the data corresponding to the disease image and the positioning information into the cache queue; then the program uploads the disease data (disease image and GPS information) in the queue to the work order platform in sequence in a single-threaded manner through the 4G network.

[0019] Preferably, in step S2, the work order platform first compares the new disease with the diseases already existing in the work order platform system. If the new disease already exists in the work order platform, the new disease is discarded, and no duplicate work order will be generated after discarding; if the new disease does not exist in the existing work order platform, it is a newly generated work order and is entered into the work order platform; the work order platform dispatches the newly generated work order to the mobile terminal.

[0020] Preferably, the specific steps of step S2 are:

[0021] S21 Correlation factor identification: First, the characteristic values ​​of the disease points are analyzed through correlation factors; the correlation factors include the GPS location information of the disease point (whether the distance is very close), the disease type (whether the disease type is consistent), the disease size (whether the length, width, and area are the same), and the image similarity (whether they are similar);

[0022] S22 training model: select the training set and perform correlation and weight training of the recognition model;

[0023] S23: Create a test set and test the model: Create a test set to test the relevance and weight effect of the recognition model, and retain the weight value verified by the test as the final weight value;

[0024] S24 Deduplication on the work order platform: Calculate the relevance weight value of the new disease. If the relevance weight value of the new disease is greater than or equal to the final weight value after traversing all the existing disease points on the work order platform, it is judged as a duplicate disease and discarded; otherwise, it is judged as a new disease, written into the work order platform, and recorded as a new disease;

[0025] S25: Repeat step S24 for each received disease data to make a judgment.

[0026] Preferably, the specific steps of step S22 are:

[0027] S221 makes a training set: select several roads for intelligent driving inspection to find diseases and collect samples, generating n diseases, recorded as set G train-A , G train-A ={A train-1 ,A train-2 ,A train-3 ......A train-n}; After the first round of inspection, these roads are inspected again, resulting in m defects, which are recorded as a set, G train-B , G train-B = {B train-1 ,B train-2 ,B train-3 ......B train-m}; Compare the disease data of the two inspections by manual review, find out the same diseases in the two inspections and make a record, a total of t pairs are generated, recorded as set G train-T ={T train-1 ,T train-2 ,T train- 3......T train-t}, and G train-T As the standard answer of the training set; therefore, the correlation coefficients of the influencing factors of judging whether two disease points are the same disease, the type of disease, the size of the disease (length, width, area) and the specific data in the image similarity are trained respectively, and the weight of each influencing factor is trained by the Monte Carlo algorithm of the simulated statistical method;

[0028] Correlation coefficient R of different deviation distances of S222 training GPS GPS :The irradiation range of the image acquisition module is usually 0 to 50m. Because the GPS positioning of the same disease point is different when it is inspected at different times, but it is actually the same disease point after manual judgment, it is necessary to use the random forest correlation analysis algorithm to train the correlation coefficient R based on the GPS deviation distance GPS , assuming that the deviation distance between the two disease points is δD, specifically:

[0029] Note δD GPS-1 =[0m, 10m], that is, when the GPS position deviation is [0m, 10m], the correlation coefficient of the same disease point is R GPS-1 ;

[0030] δD GPS-2 =(10m, 20m], that is, when the GPS position deviation is (10m, 20m], the correlation coefficient of the same disease point is R GPS-2 ;

[0031] δD GPS-3 =(20m, 30m], that is, when the GPS position deviation is (20m, 30m], the correlation coefficient of the same disease point is R GPS-3 ;

[0032] δD GPS-4 =(30m, 40m], that is, when the GPS position deviation is (30m, 40m], the correlation coefficient of the same disease point is R GPS-4 ;

[0033] δD GPS-5 =(40m, 50m], that is, when the GPS position deviation is (40m, 50m], the correlation coefficient of the same disease point is R GPS-5 ;

[0034] δD GPS-6=(50m, +∞m], that is, when the GPS position deviation is (50m, +∞m], the correlation coefficient of the same disease point is R GPS-6 ;

[0035] Therefore, the correlation is trained based on the GPS deviation distance between different disease points. After the training, the correlation coefficient R GPS is a specific value. Assuming that the distance between two disease points is δD, the correlation between the two disease points is r GPS will belong to the set R GPS A value in the set R GPS It is expressed as:

[0036] r GPS ∈{R GPS-1 , R GPS-2 , R GPS-3 , R GPS-4 , R GPS-5 , R GPS-6};

[0037] S223 Training disease type correlation R tpye :If two disease data are the same disease point, then the two disease types are also the same; conversely, disease points with inconsistent disease types cannot be the same disease point; assuming that the weight of the disease type is R type , if the two disease points are of the same type, the correlation coefficient of the disease type = 1; if the two disease points are of different types, the correlation coefficient of the disease type = 0;

[0038] S224 training disease size correlation coefficient R size :When the same disease point is inspected twice, the distance or angle at which the camera shoots the disease point is different, so the disease size reflected by the two inspections will be biased. However, in fact, it is still the same disease point after manual judgment. Therefore, it is necessary to use the random forest correlation analysis algorithm to train the correlation coefficient R based on the deviation amplitude of the disease size. size , assuming that the correlation of disease size is R size , the area (area = length * width) size deviation is δS, specifically:

[0039] δS size-1 =[0%, 10%], that is, the area size deviation is [0%, 10%], and the correlation coefficient of the same disease point is R size-1 ;

[0040] δS size-2 =(10%, 20%], that is, the area size deviation is (10%, 20%], and the correlation coefficient of the same disease point is R size-2 ;

[0041] δS size-3 =(20%, 30%], that is, the area size deviation is (20%, 30%], and the correlation coefficient of the same disease point is R size-3 ;

[0042] δS size-4 =(30%, 40%], that is, the area size deviation is (30%, 40%], and the correlation coefficient of the same disease point is R size-4 ;

[0043] δS size-5 =(40%, 50%], that is, the area size deviation is (40%, 50%], and the correlation coefficient of the same disease point is R size-5 ;

[0044] δS size-6 =(50%, 100%], that is, the area size deviation is (50%, 100%], and the correlation coefficient of the same disease point is R size-6 ;

[0045] Therefore, the correlation is trained based on the deviation amplitude of the area size between different disease points. After the training, the correlation coefficient R size is a specific value. Assuming that the deviation amplitude of two disease points is δS, the correlation r between the two disease points is size will belong to the set R size A value in the set R size Expressed as

[0046] r size ∈{R size-1 , R size-2 , R size-3 , R size-4 , R size-5 , R size-6};

[0047] S225 correlation coefficient R of training image similarity sim : Perform image similarity algorithm calculation on two disease points to obtain image variance, and compare the image generation variance to determine the similarity of the two disease points;

[0048] S226 Monte Carlo algorithm comprehensive training: The training factors, namely distance deviation GPS, disease type, disease size (length, width, area) and image similarity, are put together for Monte Carlo algorithm comprehensive training, among which the weight ratios of the training factors distance deviation GPS, disease type, disease size (length, width, area) and image similarity, and the weight of each influencing factor is multiplied by the correlation coefficient of the factor to obtain the specific value of the correlation between the two disease points.

[0049] Preferably, the specific steps of step S225 are:

[0050] S2251 cropping pictures: crop the area framed by the diseased points in the image, perform image similarity comparison on the area framed by the diseased points, and scale the image to be processed to the specified size; specifically: if the images of two diseased points are directly compared, the external interference factors (such as leaves, green belts, curbs, etc.) are very different and the effect is not good, so only the image similarity comparison is performed on the area framed by the diseased points, as shown in the following figure Image Similarity Comparison - Diseased Points A and B, crop the area framed by the diseased points, compare only the cropped images, and scale the image to be processed to the specified size. Since the complexity of the diseased points is relatively low, scaling the image to a size of 8*8 is sufficient, with a total of 64 pixels;

[0051] S2252 Grayscale processing: Usually, the similarity and color relationship of the compared images are not very large, so they are processed into grayscale images and converted to 64 levels of grayscale, that is, all pixels have a total of 64 colors;

[0052] S2253 calculates the average value: calculates the average value of each row of pixels in the image in turn, and records the average value of each row of pixels; each average value corresponds to the characteristics of a row, and calculates the grayscale average value of all 64 pixels.

[0053] S2254 calculates variance: calculates variance of all average values, and the obtained variance is the characteristic value of the image; the variance can well reflect the fluctuation of the pixel characteristics of each row, that is, it records the information of the image;

[0054] S2255 Comparison of variance: Based on the variance of the obtained image, that is, each image will generate a variance eigenvalue, so the image similarity is compared, that is, the degree of closeness of the variance generated by the image is compared, and the image with smaller variance is divided by the image with larger variance to obtain the variance comparison value; suppose the variance of the defect point A is The variance of the disease point B is The calculation formula for the variance comparison value is:

[0055]

[0056] S2256 makes a judgment: based on the variance comparison value R sim The size of R determines the similarity between two disease points. sim The closer to 1, the more similar the two disease points are. sim The closer it is to 0, the greater the difference between the two disease points.

[0057] Preferably, the formula for calculating the specific value of the correlation between the two defect points in step S226 is:

[0058] R A-B=r GPS ·W GPS +r size ·W size +r type ·W type +r sim ·W sim ;

[0059] Among them, r GPS is the correlation coefficient based on the GPS distance deviation of the two disease point images, r GPS ∈

[0060] {R GPS-1 , R GPS-2 , R GPS-3 , R GPS-4 , R GPS-5 , R GPS-6};

[0061] Among them, r size is the correlation coefficient based on the size deviation of the two diseased point images, r size ∈{R size-1 , R size-2 , R size-3 , R size-4 , R size-5 , R size-6};

[0062] Among them, r type is the correlation coefficient of disease types based on the two disease point images; r sim is the correlation coefficient based on the similarity of the two disease point images,

[0063] Substitute the values ​​of steps S222 to S225 into the correlation R A-B In the formula, the final formula is:

[0064] R A-B = {R GPS-1 , R GPS-2 , R GPS-3 , R GPS-4 , R GPS-5 , R GPS-6}·W GPS +{R size-1 , R size-2 , R size-3 , R size-4 , R size-5 , R size-6}·W size +r type ·W type +r sim ·W sim .

[0065] Preferably, the specific steps of step S23 are:

[0066] S231 Make a test set: Test the correlation and weight effect, which is different from the training set. Then select several roads for intelligent driving inspection to obtain data as the test set, and generate a diseases, which are recorded as set G test-A , G test-A ={A test-1 ,A test-2 ,A test-3 ......A test-a}; After the first round of inspection, these roads are inspected again, resulting in b diseases, recorded as set G test-B ,

[0067] G test-B = {B test-1 ,B test-2 ,B test-3 ......B test-b}; The diseases of the two inspections were compared by manual review and recorded in pairs, resulting in a total of p pairs, recorded as set

[0068] G test-P ={T test-1 ,T test-2 ,T test-3 ......T test-p}, and G test-P As the standard answer for the test set;

[0069] The formula based on the training set training completed R = {R GPS-1 , R GPS-2 , R GPS-3 , R GPS-4 , R GPS-5 , R GPS-6}·W GPS +{R size-1 , R size-2 , R size-3 , R size-4 , R size-5 , R size-6}·W size +r type ·W type +r sim ·W sim , verify the test set effect;

[0070] S232: Confirm the final weight: The weight value R0 that has been tested and verified to be qualified is as follows:

[0071] R0={R GPS-1 , R GPS-2 , RGPS-3 , R GPS-4 , R GPS-5 , R GPS-6}·W GPS +{R size-1 , R size-2 , R size-3 , R size-4 , R size-5 , R size-6}·W size +r type ·W type +r sim ·W sim .

[0072] Preferably, the step S24 is specifically as follows: based on the formula of weight value R0, the weight value R of the diseased point is calculated. x , determine whether the new disease point and the existing disease point are the same disease; if the calculated correlation R x ≥R0, it is judged as a repeated disease and the disease point is discarded; if the calculated correlation R x If it is less than R0, it is judged as a new defect, written into the work order platform system, and recorded as a new defect.

[0073] Preferably, the specific steps of step S3 are: the construction personnel receive the work order through the mobile terminal, take photos and record before, during and after the construction, and upload the record receipt to the work order platform. After receiving the work order platform, the construction effect is evaluated. If the evaluation passes, the work order is terminated; if the evaluation fails, it returns to step S2 and re-assigns the work order to the mobile terminal. The construction personnel perform secondary repairs and then upload the secondary receipt until the evaluation passes.

[0074] Compared with the prior art, the present invention has the following beneficial effects:

[0075] (1) It solves the problem of generating multiple duplicate work orders for the same defect point, and it is difficult, slow, or even impossible to remove duplicate work orders manually by the naked eye, thus playing a role in reducing costs and increasing efficiency;

[0076] (2) It can ensure that work orders for defective points are not issued repeatedly, thus solving the problem of maintenance workers receiving a large number of duplicate work orders and having to go to the same defective point for multiple repairs.

[0077] (3) Solved the problem of a large number of duplicate work orders making it difficult for maintenance units to settle accounts. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 A flowchart of a solution for deduplication of road disease inspection work orders based on computer vision technology of the present invention;

[0079] Figure 2A schematic diagram of a road maintenance object showing a solution for deduplication of road disease inspection work orders based on computer vision technology of the present invention;

[0080] Figure 3 A schematic diagram of nine types of road damage in the solution to deduplication of road damage inspection work orders based on computer vision technology of the present invention;

[0081] Figure 4 It is a schematic diagram of image similarity comparison of a defect point A of the solution for deduplication of road defect inspection work orders based on computer vision technology of the present invention;

[0082] Figure 5 This is a schematic diagram of image similarity comparison of the defect point B of the solution for deduplication of road defect inspection work orders based on computer vision technology of the present invention. DETAILED DESCRIPTION

[0083] The embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the protection scope of the present invention.

[0084] Example: Figure 1 As shown in the figure, the solution for deduplication of road disease inspection work orders based on computer vision technology specifically includes the following steps:

[0085] S1 Road Inspection: Use a road inspection identification terminal to collect road disease information and upload the disease information to the work order platform; the road inspection identification terminal in step S1 includes an image acquisition module, an identification terminal and a positioning module, the positioning acquisition module and the image acquisition module are integrated into one and connected to the identification terminal; wherein the image acquisition module is arranged in front of the identification terminal; the identification terminal in this embodiment is a vehicle, the image acquisition module is a binocular camera, and the positioning module is a GPS positioning module;

[0086] like Figure 2 As shown, the specific steps of step S1 are:

[0087] S11: The road inspection and recognition terminal pulls the video stream of the image acquisition module and extracts frames from the video stream;

[0088] S12: The recognition terminal recognizes the extracted image, records the recognized disease data, records the disease type (including pits, subsidence, cracks, etc.), captures the disease image and measures the size of the disease; at the same time, the recognition terminal matches the disease image with the real-time recorded positioning information one by one;

[0089] S13: The identification terminal puts the data corresponding to the disease image and the positioning information into the cache queue; then the program uploads the disease data (disease image and GPS information) in the queue to the work order platform in sequence through the 4G network in a single-threaded manner;

[0090] The S2 platform processes and dispatches work orders: When the work order platform receives the damage information from the road inspection and identification terminal, the work order service platform screens it according to the damage level, such as mild, general, and severe, and determines whether it is the same damage. If it is a new damage, the work order is dispatched to the mobile terminal; the work order management personnel weigh factors such as construction progress and maintenance costs, select some damages, and dispatch work orders to on-site construction personnel; however, since some damage points are driven multiple times, the same damage point may generate multiple work orders. When multiple duplicate work orders are dispatched, construction personnel will receive multiple orders and go to the site for repairs. At the same time, duplicate work orders will also affect the settlement of construction workload; since the damage points uploaded to the work order platform are not related, the work order platform needs to deduplicate the uploaded damage points to ensure the uniqueness of the damage points;

[0091] In the step S2, the work order platform first compares the new disease with the diseases already existing in the work order platform system. If the new disease already exists in the work order platform, the new disease is discarded, and no duplicate work order will be generated after the discarding; if the new disease does not exist in the existing work order platform, it is a newly generated work order and is entered into the work order platform; the work order platform dispatches the newly generated work order to the mobile terminal; the work order platform determines whether the two diseases are the same disease process. When the identification terminal identifies a new disease and uploads it to the work order platform, the system compares the new disease with the diseases already existing in the work order platform system. If the new disease already exists in the work order platform system, the new disease is discarded. In other words, no duplicate work order will be generated after the discarding; if the new disease does not exist in the existing work order platform system, it is a newly generated work order and is entered into the work order platform system;

[0092] The specific steps of step S2 are:

[0093] S21 Correlation factor identification: First, the characteristic values ​​of the disease points are analyzed through correlation factors; the correlation factors include the GPS location information of the disease point (whether the distance is very close), the disease type (whether the disease type is consistent), the disease size (whether the length, width, and area are the same), and the image similarity (whether they are similar);

[0094] S22 training model: select the training set and perform correlation and weight training of the recognition model;

[0095] The specific steps of step S22 are:

[0096] S221 makes a training set: select several roads for intelligent driving inspection to find diseases and collect samples, generating n diseases, recorded as set G train-A , G train-A ={A train-1 ,A train-2 ,A train-3 ......A train-n}; After the first round of inspection, these roads are inspected again, resulting in m defects, which are recorded as a set, G train-B , G train-B = {B train-1 ,B train-2 ,B train-3 ......B train-m}; Compare the disease data of the two inspections by manual review, find out the same diseases in the two inspections and make a record, a total of t pairs are generated, recorded as set G train-T ={T train-1 ,T train-2 ,T train- 3......T train-t}, and G train-T As the standard answer of the training set; therefore, the correlation coefficients of the influencing factors of judging whether two disease points are the same disease, the type of disease, the size of the disease (length, width, area) and the specific data in the image similarity are trained respectively, and the weight of each influencing factor is trained by the Monte Carlo algorithm of the simulated statistical method;

[0097] Correlation coefficient R of different deviation distances of S222 training GPS GPS :The irradiation range of the image acquisition module is usually 0 to 50m. Because the GPS positioning of the same disease point is different when it is inspected at different times, but it is actually the same disease point after manual judgment, it is necessary to use the random forest correlation analysis algorithm to train the correlation coefficient R based on the GPS deviation distance GPS , assuming that the deviation distance between the two disease points is δD, specifically:

[0098] Note δD GPS-1 =[0m, 10m], that is, when the GPS position deviation is [0m, 10m], the correlation coefficient of the same disease point is R GPS-1 ;

[0099] δD GPS-2 =(10m, 20m], that is, when the GPS position deviation is (10m, 20m], the correlation coefficient of the same disease point is R GPS-2 ;

[0100] δD GPS-3=(20m, 30m], that is, when the GPS position deviation is (20m, 30m], the correlation coefficient of the same disease point is R GPS-3 ;

[0101] δD GPS-4 =(30m, 40m], that is, when the GPS position deviation is (30m, 40m], the correlation coefficient of the same disease point is R GPS-4 ;

[0102] δD GPS-5 =(40m, 50m], that is, when the GPS position deviation is (40m, 50m], the correlation coefficient of the same disease point is R GPS-5 ;

[0103] δD GPS-6 =(50m, +∞m], that is, when the GPS position deviation is (50m, +∞m], the correlation coefficient of the same disease point is R GPS-6 ;

[0104] Therefore, the correlation is trained based on the GPS deviation distance between different disease points. After the training, the correlation coefficient R GPS is a specific value. Assuming that the distance between two disease points is δD, the correlation between the two disease points is r GPS will belong to the set R GPS A value in the set R GPS It is expressed as:

[0105] r GPS ∈{R GPS-1 , R GPS-2 , R GPS-3 , R GPS-4 , R GPS-5 , R GPS-6};

[0106] S223 Training disease type correlation R tpye As shown in the figure, there are 9 types of damage, including potholes, bumps, joint damage, strip cracks, subsidence, mesh cracks, road surface damage, rutting, and pockmarks. Figure 3 As shown in the figure, from the business analysis, if two disease data are the same disease point, then the two disease types are also the same; conversely, disease points with inconsistent disease types cannot be the same disease point; assuming that the weight of the disease type is R type , if the two disease points are of the same type, the correlation coefficient of the disease type = 1; if the two disease points are of different types, the correlation coefficient of the disease type = 0;

[0107] S224 training disease size correlation coefficient R size:When the same disease point is inspected twice, the distance or angle at which the camera shoots the disease point is different, so the disease size reflected by the two inspections will be biased. However, in fact, it is still the same disease point after manual judgment. Therefore, it is necessary to use the random forest correlation analysis algorithm to train the correlation coefficient R based on the deviation amplitude of the disease size. size , assuming that the correlation of disease size is R size , the area (area = length * width) size deviation is δS, specifically:

[0108] Note δS size-1 =[0%, 10%], that is, the area size deviation is [0%, 10%], and the correlation coefficient of the same disease point is R size-1 ;

[0109] δS size-2 =(10%, 20%], that is, the area size deviation is (10%, 20%], and the correlation coefficient of the same disease point is R size-2 ;

[0110] δS size-3 =(20%, 30%], that is, the area size deviation is (20%, 30%], and the correlation coefficient of the same disease point is R size-3 ;

[0111] δS size-4 =(30%, 40%], that is, the area size deviation is (30%, 40%], and the correlation coefficient of the same disease point is R size-4 ;

[0112] δS size-5 =(40%, 50%], that is, the area size deviation is (40%, 50%], and the correlation coefficient of the same disease point is R size-5 ;

[0113] δS size-6 =(50%, 100%], that is, the area size deviation is (50%, 100%], and the correlation coefficient of the same disease point is R size-6 ;

[0114] Therefore, the correlation is trained based on the deviation amplitude of the area size between different disease points. After the training, the correlation coefficient R size is a specific value. Assuming that the deviation amplitude of two disease points is δS, the correlation r between the two disease points is size will belong to the set R size A value in the set R size Expressed as

[0115] r size ∈{R size-1, R size-2 , R size-3 , R size-4 , R size-5 , R size-6};

[0116] S225 correlation coefficient R of training image similarity sim : Perform image similarity algorithm calculation on two disease points to obtain image variance, and compare the image generation variance to determine the similarity of the two disease points;

[0117] S226 Monte Carlo algorithm comprehensive training: The training factors, namely distance deviation GPS, disease type, disease size (length, width, area) and image similarity, are put together for Monte Carlo algorithm comprehensive training, among which the weight ratios of the training factors distance deviation GPS, disease type, disease size (length, width, area) and image similarity, and the weight of each influencing factor is multiplied by the correlation coefficient of the factor to obtain the specific value of the correlation between the two disease points.

[0118] The specific steps of step S225 are:

[0119] S2251 Crop the image: crop the area framed by the diseased point in the image, perform image similarity comparison on the framed area of ​​the diseased point, and scale the image to be processed to the specified size; specifically: if the images of two diseased points are directly compared, the external interference factors (such as leaves, green belts, curbs, etc.) are very different and the effect is not good, so only the image similarity comparison is performed on the framed area of ​​the diseased point, such as Figures 4-5 The image similarity comparison shown - the damaged points A and B, the damaged points A and B are framed and cropped out, only the cropped images are compared, and the images to be processed are scaled to the specified size. Since the complexity of the damaged points is relatively low, it is sufficient to scale the images to 8*8, a total of 64 pixels;

[0120] S2252 Grayscale processing: Usually, the similarity and color relationship of the compared images are not very large, so they are processed into grayscale images and converted to 64 levels of grayscale, that is, all pixels have a total of 64 colors;

[0121] S2253 calculates the average value: calculates the average value of each row of pixels in the image in turn, and records the average value of each row of pixels; each average value corresponds to the feature of a row, and calculates the grayscale average value of all 64 pixels;

[0122] S2254 calculates variance: calculates variance of all average values, and the obtained variance is the characteristic value of the image; the variance can well reflect the fluctuation of the pixel characteristics of each row, that is, it records the information of the image. Suppose the variance of the defect point A is The variance of the diseased point B is

[0123] S2255 Comparison of variance: According to the variance of the obtained image, a variance eigenvalue will be generated for each image; the image similarity is compared, that is, the degree of closeness of the variance generated by the image is compared, and the image with smaller variance is divided by the image with larger variance to obtain the variance comparison value; suppose the variance of the defect point A is and the variance of the defect point B is; the calculation formula is: The calculation formula for the variance comparison value is:

[0124]

[0125] S2256 makes a judgment: based on the variance comparison value R sim The value determines the similarity between two disease points. sim The closer it is to 1, the more similar the two disease points are. sim The closer it is to 0, the greater the difference between the two disease points;

[0126] The formula for calculating the specific value of the correlation between the two defect points in step S226 is:

[0127] R A-B =r GPS ·W GPS +r size ·W size +r type ·W type +r sim ·W sim ;

[0128] Among them, r GPS is the correlation coefficient based on the GPS distance deviation of the two disease point images, r GPS ∈

[0129] {R GPS-1 , R GPS-2 , R GPS-3 , R GPS-4 , R GPS-5 , R GPS-6};

[0130] Among them, r size is the correlation coefficient based on the size deviation of the two diseased point images, r size ∈{R size-1 , R size-2 , R size-3 , R size-4 , R size-5 , R size-6};

[0131] Among them, r type is the correlation coefficient of disease types based on the two disease point images; r simis the correlation coefficient based on the similarity of the two disease point images,

[0132] Substitute the values ​​of steps S222 to S225 into the correlation R A-B In the formula, the final formula is:

[0133] R A-B = {R GPS-1 , R GPS-2 , R GPS-3 , R GPS-4 , R GPS-5 , R GPS-6}·W GPS +{R size-1 , R size-2 , R size-3 , R size-4 , R size-5 , R size-6}·W size +r type ·W type +r sim ·W sim ;

[0134] S23: Create a test set and test the model: Create a test set to test the relevance and weight effect of the recognition model, and retain the weight value verified by the test as the final weight value;

[0135] The specific steps of step S23 are:

[0136] S231 Make a test set: Test the correlation and weight effect, which is different from the training set. Then select several roads for intelligent driving inspection to obtain data as the test set, and generate a diseases, which are recorded as set G test-A , G test-A ={A test-1 ,A test-2 ,A test-3 ......A test-a}; After the first round of inspection, these roads are inspected again, resulting in b diseases, recorded as set G test-B ,

[0137] G test-B = {B test-1 ,B test-2 ,B test-3 ......B test-b}; The diseases of the two inspections are compared by manual review and recorded in pairs, resulting in a total of p pairs, recorded as set G test-P ={T test-1 ,T test-2 ,T test- 3......Ttest-p}, and G test-P As the standard answer for the test set;

[0138] The formula based on the training set training completed R = {R GPS-1 , R GPS-2 , R GPS-3 , R GPS-4 , R GPS-5 , R GPS-6}·W GPS +{R size-1 , R size-2 , R size-3 , R size-4 , R size-5 , R size-6}·W size +r type ·W type +r sim ·W sim , verify the test set effect;

[0139] S232: Confirm the final weight: The weight value R0 that has been tested and verified to be qualified is as follows:

[0140] R0={R GPS-1 , R GPS-2 , R GPS-3 , R GPS-4 , R GPS-5 , R GPS-6}·W GPS +{R size-1 , R size-2 , R size-3 , R size-4 , R size-5 , R size-6}·W size +r type ·W type +r sim ·W sim ;

[0141] S24 Deduplication of work order platform: Calculate the relevance weight value of the new disease. If the relevance weight value of the new disease is greater than or equal to the final weight value after traversing all the existing disease points in the work order platform, it is judged as a duplicate disease and discarded; otherwise, it is judged as a new disease, written into the work order platform, and recorded as a new disease. The specific step S24 is: calculate the weight value R of the disease point based on the formula of the weight value R0 x , determine whether the new disease point and the existing disease point are the same disease; if the calculated correlation R x ≥R0, it is judged as a repeated disease and the disease point is discarded; if the calculated correlation R xIf the value is less than R0, it is considered as a new defect and written into the work order platform system as a new defect.

[0142] S25: Repeat step S24 for each received disease data to make a judgment;

[0143] S3 mobile terminal processes the work order: after receiving the work order, the mobile terminal performs construction and maintenance and transmits information back to the work order platform; the specific steps of step S3 are: the construction personnel receive the work order through the mobile terminal, take photos and records before, during and after the construction, and upload the record receipt to the work order platform, and the work order platform evaluates the construction effect after receiving it. If the evaluation is passed, the work order is terminated; if the evaluation is not passed, it returns to step S2 to re-dispatch the work order to the mobile terminal, and the construction personnel upload the secondary receipt after performing secondary maintenance until the evaluation is passed; in this embodiment, the mobile terminal is a mobile phone, and the work order platform dispatches the work order to the app or WeChat applet;

[0144] For ordinary technicians in this field, the specific embodiments are only illustrative descriptions of the present invention. It is obvious that the specific implementation of the present invention is not limited to the above-mentioned methods. As long as various non-substantial improvements are made using the method concepts and technical solutions of the present invention, or the concepts and technical solutions of the present invention are directly applied to other occasions without improvement, they are all within the protection scope of the present invention.

Claims

1. A solution for deduplication of road disease inspection work orders based on computer vision technology, characterized in that: The specific steps include: S1 Road Inspection: Use road inspection identification terminal to collect road damage information and upload the damage information to the work order platform; The S2 platform processes and dispatches work orders: When the work order platform receives the disease information from the road inspection and identification terminal, the work order service platform screens it according to the disease level and determines whether it is the same disease. If it is a new disease, the work order is dispatched to the mobile terminal; S3 mobile terminal processes work orders: After receiving the work order, the mobile terminal performs construction and maintenance and sends the information back to the work order platform.

2. The method for removing duplicate road damage inspection work orders based on computer vision technology according to claim 1 is characterized in that: The road inspection identification terminal in step S1 includes an image acquisition module, an identification terminal and a positioning module, wherein the positioning acquisition module and the image acquisition module are integrated into one and connected to the identification terminal; wherein the image acquisition module is arranged in front of the identification terminal.

3. The solution for deduplication of road damage inspection work orders based on computer vision technology according to claim 2 is characterized in that: The specific steps of step S1 are: S11: The road inspection and recognition terminal pulls the video stream of the image acquisition module and extracts frames from the video stream; S12: The recognition terminal recognizes the extracted image and records the recognized disease data. At the same time, the recognition terminal makes a one-to-one correspondence between the disease image and the real-time recorded positioning information; S13: The identification terminal puts the data corresponding to the disease image and the positioning information into a cache queue; and then uploads the disease data in the queue to the work order platform in sequence.

4. The method for removing duplicate road damage inspection work orders based on computer vision technology according to claim 3 is characterized in that: In step S2, the work order platform first compares the new defect with the defects already existing in the work order platform system. If the new defect already exists in the work order platform, the new defect will be discarded, and no duplicate work order will be generated after discarding. If the new defect does not exist in the existing work order platform, it is a newly generated work order and is entered into the work order platform. The work order platform dispatches the newly generated work order to the mobile terminal.

5. The solution for deduplication of road damage inspection work orders based on computer vision technology according to claim 3 is characterized in that: The specific steps of step S2 are: S21 Correlation factor identification: First, the characteristic values ​​of the diseased points are analyzed through correlation factors; the correlation factors include the location information of the diseased points, the disease type, the disease size and the image similarity; S22 training model: select the training set and perform correlation and weight training of the recognition model; S23: Create a test set and test the model: Create a test set to test the relevance and weight effect of the recognition model, and retain the weight value verified by the test as the final weight value; S24 Deduplication of work order platform: Calculate the relevance weight value of the new disease. If the relevance weight value of the new disease is greater than or equal to the final weight value after traversing all the existing disease points in the work order platform, it is judged as a duplicate disease and discarded; Otherwise, it is judged as a new defect, written into the work order platform, and recorded as a new defect; S25: Repeat step S24 for each received disease data to make a judgment.

6. The method for removing duplicate road damage inspection work orders based on computer vision technology according to claim 5 is characterized in that: The specific steps of step S22 are: S221 makes a training set: select several roads for intelligent driving inspection to find diseases and collect samples, generating n diseases, recorded as set G train-A , G train-A ={A train-1 ,A train-2 ,A train-3 ......A train-n }; After the first round of inspection, these roads are inspected again, resulting in m defects, recorded as a set, G train-B , G train-B = {B train-1 ,B train-2 ,B train-3 ......B train-m }; Compare the disease data of the two inspections by manual review, find out the same diseases in the two inspections and make a record, a total of t pairs are generated, recorded as set G train-T ={T train-1 ,T train-2 ,T train- 3......T train-t }, and G train-T As the standard answer of the training set; therefore, the correlation coefficients of the influencing factors of judging whether two disease points are the same disease, the type of disease, the size of the disease and the specific data of the image similarity are trained respectively, and the weight of each influencing factor is trained by the Monte Carlo algorithm of the simulated statistical method; Correlation coefficient R of different deviation distances of S222 training GPS GPS :The irradiation range of the image acquisition module is 0 to 50m, and the correlation coefficient R based on the GPS deviation distance is trained by the random forest correlation analysis algorithm GPS , assuming that the deviation distance between the two disease points is δD, specifically: Note δD GPS-1 =[0m, 10m], that is, when the GPS position deviation is [0m, 10m], the correlation coefficient of the same disease point is R GPS-1 ; δD GPS-2 =(10m, 20m], that is, when the GPS position deviation is (10m, 20m], the correlation coefficient of the same disease point is R GPS-2 ; δD GPS-3 =(20m, 30m], that is, when the GPS position deviation is (20m, 30m], the correlation coefficient of the same disease point is R GPS-3 ; δD GPS-4 =(30m, 40m], that is, when the GPS position deviation is (30m, 40m], the correlation coefficient of the same disease point is R GPS-4 ; δD GPS-5 =(40m, 50m], that is, when the GPS position deviation is (40m, 50m], the correlation coefficient of the same disease point is R GPS-5 ; δD GPS-6 =(50m, +∞m], that is, when the GPS position deviation is (50m, +∞m], the correlation coefficient of the same disease point is R GPS-6 ; Therefore, the correlation is trained based on the GPS deviation distance between different disease points. After the training, the correlation coefficient R GPS is a specific value. Assuming that the distance between two disease points is δD, the correlation between the two disease points is r GPS will belong to the set R GPS A value in the set R GPS It is expressed as: r GPS ∈{R GPS-1 ,R GPS-2 ,R GPS-3 ,R GPS-4 ,R GPS-5 ,R GPS-6 }; S223 Training disease type correlation R tpye :If two disease data are the same disease point, then the two disease types are also the same; conversely, disease points with inconsistent disease types cannot be the same disease point; assuming that the weight of the disease type is R type , if the two disease points are of the same type, the correlation coefficient of the disease type = 1; If the types of two disease points are different, the correlation coefficient of disease type = 0; S224 training disease size correlation coefficient R size :If the same disease point has size deviation after two inspections, the correlation coefficient R based on the deviation of disease size is trained by random forest correlation analysis algorithm size , assuming that the correlation of disease size is R size , the area size deviation is δS, specifically: δS size-1 =[0%, 10%], that is, the area size deviation is [0%, 10%], and the correlation coefficient of the same disease point is R size-1 ; δS size-2 =(10%, 20%], that is, the area size deviation is (10%, 20%], and the correlation coefficient of the same disease point is R size-2 ; δS size-3 =(20%, 30%], that is, the area size deviation is (20%, 30%], and the correlation coefficient of the same disease point is R size-3 ; δS size-4 =(30%, 40%], that is, the area size deviation is (30%, 40%], and the correlation coefficient of the same disease point is R size-4 ; δS size-5 =(40%, 50%], that is, the area size deviation is (40%, 50%], and the correlation coefficient of the same disease point is R size-5 ; δS size-6 =(50%, 100%], that is, the area size deviation is (50%, 100%], and the correlation coefficient of the same disease point is R size-6 ; Therefore, the correlation is trained based on the deviation amplitude of the area size between different disease points. After the training, the correlation coefficient R size is a specific value. Assuming that the deviation amplitude of two disease points is δS, the correlation r between the two disease points is size will belong to the set R size A value in the set R size It is expressed as: r size ∈{R size-1 ,R size-2 ,R size-3 ,R size-4 ,R size-5 ,R size-6 }; S225 correlation coefficient R of training image similarity sim : Perform image similarity algorithm calculation on two disease points to obtain image variance, and compare the image generation variance to determine the similarity of the two disease points; S226 Monte Carlo algorithm comprehensive training: Put all training factors, namely distance deviation GPS, disease type, disease size and image similarity, together for Monte Carlo algorithm comprehensive training, where the weight ratios of the training factors distance deviation GPS, disease type, disease size and image similarity are multiplied. The weight of each influencing factor is multiplied by the correlation coefficient of the factor to obtain the specific value of the correlation between the two disease points.

7. The method for removing duplicate road damage inspection work orders based on computer vision technology according to claim 6 is characterized in that: The specific steps of step S225 are: S2251 Crop the image: crop the area framed by the diseased points in the image, perform image similarity comparison on the framed area of ​​the diseased points, and scale the image to be processed to a specified size; S2252 Grayscale processing: process the image into a grayscale image; S2253 calculates the average value: calculates the average value of each row of pixels in the image in turn, and records the average value of each row of pixels; S2254 calculates variance: calculates variance of all the average values ​​obtained, and the obtained variance is the characteristic value of the image; S2255 Compare variances: Based on the variances of the obtained images, compare the image similarity, that is, compare the degree of closeness of the variances generated by the images. Then divide the image with a smaller variance by the image with a larger variance to obtain the variance comparison value. Suppose the variance of the defect point A is The variance of the diseased point B is The formula for calculating the variance comparison value is: S2256 makes a judgment: based on the variance comparison value R sim The size of R determines the similarity between two disease points. sim The closer it is to 1, the more similar the two disease points are. sim The closer it is to 0, the greater the difference between the two disease points.

8. The method for removing duplicate road damage inspection work orders based on computer vision technology according to claim 7 is characterized in that: The formula for calculating the specific value of the correlation between the two defect points in step S226 is: R A-B =r GPS ·W GPS +r size ·W size +r type ·W type +r sim ·W sim ; Among them, r GPS is the correlation coefficient based on the GPS distance deviation of the two disease point images, r GPS ∈ {R GPS-1 ,R GPS-2 ,R GPS-3 ,R GPS-4 ,R GPS-5 ,R GPS-6 }; Among them, r size is the correlation coefficient based on the size deviation of the two diseased point images, r size ∈{R size-1 , R size-2 , R size-3 , R size-4 , R size-5 , R size-6 }; Among them, r type is the correlation coefficient of disease types based on the two disease point images; r sim is the correlation coefficient based on the similarity of the two disease point images, Substitute the values ​​of steps S222 to S225 into the correlation R A-B In the formula, the final formula is: R A-B ={R GPS-1 ,R GPS-2 ,R GPS-3 ,R GPS-4 ,R GPS-5 ,R GPS-6 }·W GPS +{R size-1 , R size-2 ,R size-3 ,R size-4 ,R size-5 ,R size-6 }·W size +r type ·W type +r sim ·W sim 。 9. The method for removing duplicate road damage inspection work orders based on computer vision technology according to claim 6 is characterized in that: The specific steps of step S23 are: S231 Make a test set: Test the correlation and weight effect, reselect several roads for driving intelligent inspection and obtain data as the test set, generate a diseases, recorded as set G test-A , G test-A ={A test-1 ,A test-2 ,A test-3 ......A test-a }; After the first round of inspection, these roads are inspected again, resulting in b diseases, recorded as set G test-B , G test-B = {B test-1 ,B test-2 ,B test-3 ......B test-b }; Compare the diseases from the two inspections and make pairs of records, generating a total of p pairs, recorded as set G test-P ={T test-1 ,T test-2 ,T test-3 ......T test-p }, and G test-P As the standard answer for the test set; The formula based on the training set training completed R = {R GPS-1 , R GPS-2 , R GPS-3 , R GPS-4 , R GPS-5 , R GPS-6 }·W GPS +{R size-1 , R size-2 , R size-3 , R size-4 , R size-5 , R size-6 }·W size +r type ·W type +r sim ·W sim , verify the test set effect; S232: Confirm the final weight: The weight value R0 that has been tested and verified to be qualified is as follows: R0={R GPS-1 ,R GPS-2 ,R GPS-3 ,R GPS-4 ,R GPS-5 ,R GPS-6 }·W GPS +{R size-1 , R size-2 ,R size-3 ,R size-4 ,R size-5 ,R size-6 }·W size +r type ·W type +r sim ·W sim 。 10. The method for removing duplicate road damage inspection work orders based on computer vision technology according to claim 6, characterized in that: The specific steps of step S3 are as follows: the construction personnel receive the work order through the mobile terminal, take photos and record before, during and after the construction, and upload the record receipt to the work order platform. After receiving the work order platform, the construction effect is evaluated. If the evaluation is passed, the work order is terminated; If the evaluation fails, return to step S2 to reassign the work order to the mobile terminal.

Images