Road de-weighting method and system based on comprehensive consideration of GPS positioning, disease type and disease size

Abstract

The invention discloses a road de-duplication method based on GPS positioning and comprehensive consideration of disease types and disease sizes, and the method comprises the specific steps: S1, data collection and processing: collecting road image data through an edge detection device, carrying out the road disease recognition of the road image data, obtaining disease data, and processing and storing the disease data; and S2, disease duplicate removal: matching the positioning information, the disease type and the size information of the processed disease data, if the positioning information, the disease type and the size information are all within a preset matching threshold, determining that the disease data are duplicated items, removing the duplicated items, and then storing the disease data after duplicate removal. According to the method, duplicate removal is performed by integrating multiple data dimensions, so that the accuracy of road disease data can be remarkably improved, and errors and redundant information are reduced; the intelligent and automatic level of road maintenance is improved, technical support is provided for sustainable development of road maintenance, and the intelligent road maintenance management system becomes an effective road maintenance management tool.

Description

Technical Field

[0001] The present invention belongs to the technical field of road management, and in particular relates to a road deduplication method and system based on comprehensive consideration of GPS positioning, disease type and disease size. Background Art

[0002] Judging from the current development of highway maintenance, the identification of highway pavement defects has been carried out by digital and intelligent means. In the entire continuous process of road defect discovery, repair, evaluation and decision-making, the location of road defects is an important link, such as in the deduplication of road defects, file creation, repair, and evaluation of the effect after repair.

[0003] After a road defect is discovered, it needs to be surveyed and evaluated, and priorities are arranged according to the actual situation, and corresponding resources are allocated for maintenance and repair. This process takes a certain period of time. During this period, the same defect may be recorded multiple times, resulting in data redundancy. At the same time, during the process of road defect detection and management, due to multiple tests or data collection from different detection equipment, repeated records of the same defect may be generated, resulting in data redundancy.

[0004] Currently, the commonly used ones include road damage detection systems based on image analysis: using road images taken by vehicle-mounted cameras or drones, automatically identifying and classifying road damage and estimating the size of the damage through computer vision and image processing technology. Or road damage automatic reporting and notification systems: through mobile applications or online platforms, the public and maintenance personnel are allowed to report road damage, and the system automatically compares the reports with existing data to avoid duplicate records. Or cloud road damage data analysis platforms: using cloud computing resources to store and process large amounts of road damage data, providing powerful data analysis and deduplication capabilities, while supporting large-scale data sharing and collaboration.

[0005] The existing technology has the following main defects: 1. Data collected manually may be subjective, and different people may have different identification and records of diseases. 2. High-precision automated testing equipment is expensive. 3. The collection and processing of disease data lacks a unified standardized process.

[0006] Therefore, it is necessary to provide a road deduplication method based on comprehensive considerations of GPS positioning, disease type and disease size to reduce data redundancy, improve maintenance efficiency, reduce maintenance costs and optimize resource allocation. Summary of the invention

[0007] The technical problem to be solved by the present invention is to provide a road deduplication method based on comprehensive consideration of GPS positioning, disease type and disease size, thereby reducing data redundancy, improving maintenance efficiency, reducing maintenance costs and optimizing resource allocation, thereby improving the positioning accuracy of road diseases.

[0008] In order to solve the above technical problems, the technical solution adopted by the present invention is: the road deduplication method based on comprehensive consideration of GPS positioning, disease type and disease size specifically includes the following steps:

[0009] S1 Data collection and processing: Use edge detection equipment to collect road image data, identify road damage on the road image data, obtain damage data, process it and then store it;

[0010] S2 Disease deduplication: The processed disease data is matched with the location information, disease type and size information respectively. If the location information, disease type and size information are all within the preset matching threshold, they are determined to be duplicates, the duplicates are removed, and the deduplicated disease data is stored.

[0011] By adopting the above technical scheme, patrol personnel drive patrol vehicles to conduct normal inspections on the road. Edge recognition and edge detection equipment are installed on the inspection vehicles. The edge recognition and edge detection equipment is externally connected to a USB binocular depth camera on the front of the vehicle and a GPS positioning antenna on the roof. A road disease detection program is installed on the equipment. The program uses a road disease algorithm model to analyze the video stream captured by the camera frame by frame in real time and identify each frame of the image. That is, by using different dimensions such as GPS positioning, disease type and disease size, a large number of recorded disease samples are screened and deduplicated, thereby reducing data redundancy, reducing the waste of data storage space, and improving the accuracy of data analysis. It reduces unnecessary repeated investigation work, enables the maintenance team to respond to and deal with diseases more quickly, and improves maintenance efficiency. This is of great significance in improving efficiency and accuracy, reducing maintenance costs, and optimizing resource allocation in the actual management of road diseases.

[0012] Preferably, the specific steps of step S1 are:

[0013] S11 data collection: First, the intelligent road inspection system collects data through edge detection equipment and stores it in the database;

[0014] S12 Disease Identification: Identify diseases in road image data through image recognition models, determine the disease type, and calculate the disease size;

[0015] S13 Data preprocessing: Data formatting is performed on the corresponding disease location coordinates, disease types and disease sizes in the collected road image data to form a unified data structure, which is then stored in the database again.

[0016] The above technical solution is used to integrate data from different sources (such as GPS data, disease type, and disease size) to form a unified disease database. Through a unified data collection and processing method, data from different sources and times are comparable, which is convenient for long-term analysis and trend research. A deep neural network algorithm is used to quickly and frequently detect and identify road diseases, and obtain the pixel rectangular area, disease category, and GPS positioning information of the disease. The image depth information is obtained through a binocular camera, and the distance between the spatial vectors of the pixel coordinate frame is calculated based on the depth information, and the disease area is finally obtained; the inspection results of different rounds are clustered using the disease type and longitude and latitude information. Finally, the similarity between diseases is determined based on GPS positioning, disease type, and disease area.

[0017] Preferably, the method further includes step S3 of result display: storing the deduplicated road damage data in a database and displaying it on a map for road maintenance personnel to make maintenance decisions.

[0018] Preferably, in the step S11, the road image data collected by the binocular camera in the edge detection device includes GPS positioning information, disease type, original disease image, disease depth map and disease annotated pixel coordinate information; wherein the disease depth map and the disease annotated pixel coordinate information will be processed twice to the disease size after being saved, and saved again in the database; in the step S12, the disease depth map and the disease annotated pixel coordinate information are calculated by the disease size calculation service module, and the disease size and center point distance are obtained and stored in the database again.

[0019] With the above technical solution, when the edge detection device detects a disease, it will not upload the relevant information immediately, but will cache the relevant data. If the same type of disease is photographed multiple times in a very short period of time, and the change in longitude and latitude coordinates is less than the set threshold, it will be determined that this belongs to the same disease point. At this time, the confidence value of the recognition result will be checked in all cached data. The confidence value indicates the certainty or trust level of the algorithm model in estimating the disease point. The record with the highest confidence will be selected, and the detected road disease type, screenshot, marked identification box, recording time, corresponding depth and GPS positioning information will be uploaded to the cloud platform through the 5G network, and the entire set of equipment and programs will be started with the vehicle.

[0020] Preferably, the specific steps of calculating the disease size in step S12 are:

[0021] S121 Four-corner pixel coordinates: The defect size calculation service module searches for coordinate points in the set range in reverse according to the specific positions of the defect-annotated pixel coordinate points in the image (upper left, lower left, lower right, upper right). If the depth values ​​of the current round of search are all still 0, the search round is expanded;

[0022] S122 spatial coordinates: combine the four-corner pixel coordinates with the binocular camera internal reference to obtain the spatial coordinate points of the disease-marked pixels;

[0023] S123 Center point image coordinates: Obtain the pixel coordinates of the center point of the disease according to the four-corner pixel coordinates, and calculate the distance between the center point of the disease and the camera;

[0024] S124 calculates the size of the disease: divides the spatial coordinates into two triangles, and uses vector cross product to calculate the area of ​​the two triangles to obtain the size of the disease.

[0025] Preferably, in step S121, assuming that the coordinates of the defect-annotated pixel coordinate point are (x, y), the initial search range is m, and the search round is n, the search ranges of the four points corresponding to the specific positions are:

[0026] Upper left: (x~x+m*n, ym*n~y);

[0027] Lower left: (x~x+m*n, y~y+m*n);

[0028] Lower right: (xm*n~x, y~y+m*n);

[0029] Upper right: (xm*n~x, ym*n~y).

[0030] Preferably, the specific steps of step S122 are:

[0031] S1221: The camera's intrinsic parameter matrix K describes the camera's internal characteristics, including focal length and principal point (reference point on the image plane); the intrinsic parameter matrix K is expressed as:

[0032]

[0033] Among them, f x and f y is the focal length along the x-axis and y-axis; c x and c y are the coordinates of the principal point;

[0034] S1222: When the upper left original coordinate point (x, y) is used for calculation, the coordinate set range obtained in the first round is (x~x+m, ym~y) with a total of (m+1) 2 coordinate points, the depth data of each cross position is expressed as:

[0035]

[0036] Take the non-zero values ​​to calculate the depth average d;

[0037] Similarly, the same method is used to calculate the lower left, upper right, and lower right;

[0038] S1223: Calculate the three-dimensional Z coordinate value of the plane pixel coordinate in the camera coordinate system according to the focal length f of the binocular camera and the baseline parameter b, and then use the normalization formula and the principal point coordinates of the binocular camera to calculate the X, Y values ​​and the distance D from the space coordinate to the camera;

[0039] Z = 8*f*b / d;

[0040] X=(xc x ) / f*Z;

[0041] X=(yc y ) / f*Z;

[0042]

[0043] After obtaining the spatial coordinates and distance sets corresponding to all pixel coordinates, the middle value is taken according to D sorting to obtain the final spatial coordinate point (X, Y, Z) of the original annotated pixel coordinates.

[0044] Preferably, in step S124, the spatial coordinates are divided into two triangles, that is, for the four points A (x1, y1, z1), B (x2, y2, z2), C (x3, y3, z3), D (x4, y4, z4) in the space, they are divided into two triangles ABC and ACD, and the areas are calculated respectively. Then, the areas of triangles ABC and ACD are calculated by vector cross multiplication to obtain the size of the defect. The formula for calculating the size of the defect is:

[0045]

[0046] Among them, A, B, and C are the coordinates of the three vertices of the triangle, AB and AC are the vectors from point A to B and C respectively, x1, y1, and z1 are the spatial coordinates of A, x2, y2, and z2 are the spatial coordinates of B, and x3, y3, and z3 are the spatial coordinates of C. i, j, and k are unit vectors in three-dimensional space, which point to the positive directions of the x-axis, y-axis, and z-axis respectively. These unit vectors have a length of 1 and are perpendicular to each other.

[0047] For the four points A(x1,y1,z1), B(x2,y2,z2), C(x3,y3,z3), and D(x4,y4,z4) in space, their areas cannot be simply calculated using AB*BC, so they are divided into two triangles ABC and ACD and their areas are calculated separately; for one of the triangles ABC, its area can be calculated by vector cross multiplication, that is, 0.5*||AB×AC||, to obtain the size of the disease.

[0048] Preferably, the specific steps of step S2 are:

[0049] S21: For a road R, the starting point R0 and the end point R0 on the road will be marked when the file is created. n And the middle segmentation points R1, R2...R n-1 , using the Haversine formula to calculate any segment point R m With the previous segment point R m-1 The distance between each segment point and the starting point R0 can be accumulated in sequence, with the starting point being 0 and the end point being D n And the middle segmentation points are D1, D2.....D n-1 ;

[0050] S22: Based on the longitude Lon and latitude Lat of the disease information GPS positioning, the disease recording point, the corresponding matching road R and the inspection direction R w , also use the Haversine formula to find the disease record point R r The two points closest to each other, select the segment point R closest to the starting point R0 k , by the segmentation point R k Distance to starting point D k and disease record point R r To segment point R k The distance D rk Adding them together gives the distance D from the disease record point to the starting point of the road. r ;

[0051] S23: When the inspection direction is from the starting point R0 to the end point R n When , the fuzzy calculation of the disease center point D c The distance to the starting point is D r +D j , the opposite direction is D r -D j , where D j The distance between the center of the disease and the camera obtained by the disease size calculation service;

[0052] S24: When a new disease record is recorded in the database, N disease records of the same type on the road are initially screened according to the disease type T and the road R matched by GPS positioning. When N>0, the disease records are further screened according to the distance D from the disease center to the starting point. c Compare with the diseased area S to remove the duplicates. c The difference is within the preset distance threshold ±E d If the difference in the disease area S is within the preset area threshold ±Es, they are judged to be the same disease, otherwise they are saved as new records.

[0053] Preferably, the Haversine formula in step S21 and step S22 is based on the spherical model of the earth, and uses the relationship between spherical trigonometry and longitude and latitude to calculate the spherical distance between two points. The specific formula is:

[0054]

[0055] d = R·c;

[0056] in, are the latitudes of the two points in radians; is the latitude difference between the two points in radians.

[0057] Compared with the prior art, the present invention has the following beneficial effects: the method not only improves the intelligence and automation level of road maintenance, but also provides technical support for the sustainable development of road maintenance, becoming an effective road maintenance management tool; specifically, it includes:

[0058] (1) Reduce data redundancy: By integrating multiple data dimensions to remove duplicates, i.e. identifying and eliminating duplicate damage records, the waste of data storage space is reduced and the accuracy of data analysis is improved; this can significantly improve the accuracy of road damage data;

[0059] (2) Improve operational and maintenance efficiency: By using high-precision positioning and deep neural networks to establish a unique information for each road defect, each defect discovered during an inspection is first matched with the existing information. If there is no match, it is a new defect and maintenance personnel need to be notified to carry out repairs. Otherwise, it is a repeated defect and maintenance personnel do not need to be notified, which greatly improves the operational efficiency of road maintenance. At the same time, it reduces unnecessary repeated surveys, allowing the maintenance team to respond to and handle defects more quickly. With accurate defect data, road maintenance work can be planned and scheduled more effectively, improving the efficiency of maintenance work. With fast and accurate high-precision positioning and labeling information of road defects, and using it to judge the same defects during daily inspections, the work efficiency of maintenance personnel is improved.

[0060] (3) Reduce maintenance costs: Through accurate disease identification and deduplication, unnecessary maintenance work is reduced, reducing long-term maintenance costs; at the same time, waste of resources is avoided. Accurate data helps avoid multiple trips to the site for the same disease, thus saving time and resources;

[0061] (4) Data standardization and integration: Through unified data collection and processing methods, ensure that data from different sources and over time are comparable, facilitating long-term analysis and trend research;

[0062] (5) Enhanced decision support: Provides more accurate and detailed disease data to support data-based decision making. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 It is a flow chart of the road deduplication method based on comprehensive consideration of GPS positioning, disease type and disease size of the present invention;

[0064] Figure 2 It is a flow chart of step S1 of the road deduplication method based on comprehensive consideration of GPS positioning, disease type and disease size of the present invention;

[0065] Figure 3 It is a schematic diagram of step S11 in the road deduplication method based on comprehensive consideration of GPS positioning, disease type and disease size of the present invention;

[0066] Figure 4 It is a schematic diagram of the defect image data collected by the edge detection device in the road deduplication method based on GPS positioning, defect type and defect size comprehensive consideration of the present invention;

[0067] Figure 5 A schematic diagram of calculating the size of a defect in step S124 of the road deduplication method based on comprehensive consideration of GPS positioning, defect type and defect size of the present invention;

[0068] Figure 6 A schematic diagram of calculating the size of a defect in step S124 of the road deduplication method based on comprehensive consideration of GPS positioning, defect type and defect size of the present invention;

[0069] Figure 7 It is a schematic diagram of the database after clustering in the road deduplication method based on comprehensive consideration of GPS positioning, disease type and disease size of the present invention;

[0070] Figure 8 Schematic diagram of checking the specific distribution of the diseases on the road on the map in step S3 of the road deduplication method based on GPS positioning, disease type and disease size comprehensive consideration of the present invention Figure 1 ;

[0071] Fig. 9 Schematic diagram of checking the specific distribution of the diseases on the road on the map in step S3 of the road deduplication method based on GPS positioning, disease type and disease size comprehensive consideration of the present invention Figure 2 ;

[0072] Fig.10 Schematic diagram of checking the specific distribution of the diseases on the road on the map in step S3 of the road deduplication method based on GPS positioning, disease type and disease size comprehensive consideration of the present invention Figure 3 . DETAILED DESCRIPTION

[0073] The embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the protection scope of the present invention.

[0074] Example: Figure 1 As shown, the road deduplication method based on comprehensive consideration of GPS positioning, disease type and disease size specifically includes the following steps:

[0075] S1 Data collection and processing: Use edge detection equipment to collect road image data, identify road damage on the road image data, obtain damage data, process it and then store it;

[0076] The specific steps of step S1 are:

[0077] S11 data collection: first, the intelligent road inspection system is used to collect data through the edge detection device and store it in the database; in the step S11, the road image data collected by the binocular camera in the edge detection device includes GPS positioning information, disease type, disease original image, disease depth map and disease annotated pixel coordinate information; wherein the disease depth map and disease annotated pixel coordinate information will be processed into disease size after being saved, and then saved in the database again;

[0078] S12 Disease Identification: Identify diseases in road image data through image recognition models, determine the disease type, and calculate the disease size;

[0079] S13 data preprocessing: data formatting is performed on the corresponding disease location coordinates, disease types and disease sizes in the collected road image data to form a unified data structure, which is then stored in the database again;

[0080] like Figure 3 As shown in the figure, when the edge detection device detects a defect E, it will not upload the relevant information immediately, but will cache the relevant data. If the same type of defect is photographed multiple times in a very short period of time, and the change in longitude and latitude coordinates is less than the set threshold, it will be determined that this belongs to the same defect point. At this time, the confidence value of the recognition result will be checked in all cached data. The confidence value indicates the certainty or trust level of the algorithm model in estimating the defect point. The record with the highest confidence value will be selected, and the detected road defect type, screenshot, annotated identification box, recording time, corresponding depth and GPS positioning information will be uploaded to the cloud platform through the 5G network, and the entire set of equipment and programs will be started with the vehicle;

[0081] S2 Disease deduplication: The processed disease data is matched with the location information, disease type and size information respectively. If the location information, disease type and size information are all within the preset matching threshold, it is determined to be a duplicate item, the duplicate item is removed, and the deduplicated disease data is stored;

[0082] In step S12, the defect depth map and the defect annotated pixel coordinate information are used to calculate the defect size through the defect size calculation service module, and the defect size and center point distance are obtained and stored in the database again;

[0083] The specific steps of calculating the disease size in step S12 are:

[0084] S121 Four-corner pixel coordinates: Due to equipment conditions and objective shooting environment, the depth corresponding to the pixel coordinate points is marked Figure 2 The depth value of the dimension array may be 0. In this case, the defect size calculation service module searches for coordinate points in the set range in reverse according to the specific position of the defect-annotated pixel coordinate points in the image (upper left, lower left, lower right, upper right). If all the depth values ​​of the current round of search are still 0, the search round is expanded.

[0085] In step S121, assuming that the coordinates of the defect-annotated pixel coordinate point are (x, y), the initial search range is m, and the search round is n, the search ranges of the four points corresponding to the specific positions are:

[0086] Upper left: (x~x+m*n, ym*n~y);

[0087] Lower left: (x~x+m*n, y~y+m*n);

[0088] Lower right: (xm*n~x, y~y+m*n);

[0089] Upper right: (xm*n~x, ym*n~y);

[0090] This will shrink the points into the rectangular frame formed by the initial annotation coordinates to ensure accuracy; at the same time, each round of search will block the coordinate points searched in the previous search round;

[0091] S122 spatial coordinates: combine the four-corner pixel coordinates with the binocular camera internal reference to obtain the spatial coordinate points of the disease-marked pixels;

[0092] The specific steps of step S122 are:

[0093] S1221: The camera's intrinsic parameter matrix K describes the camera's internal characteristics, including focal length and principal point (reference point on the image plane); the intrinsic parameter matrix K is expressed as:

[0094]

[0095] Among them, f x and f y is the focal length along the x-axis and y-axis; c x and c yare the coordinates of the principal point;

[0096] S1222: When the upper left original coordinate point (x, y) is used for calculation, the coordinate set range obtained in the first round is (x~x+m, ym~y) with a total of (m+1) 2 coordinate points, the depth data of each cross position is expressed as:

[0097]

[0098] Take the non-zero values ​​and calculate the depth average d (if all values ​​are 0, d is 0);

[0099] Similarly, the same method is used to calculate the lower left, upper right, and lower right;

[0100] S1223: Calculate the three-dimensional Z coordinate value of the plane pixel coordinate in the camera coordinate system according to the focal length f of the binocular camera and the baseline parameter b, and then use the normalization formula and the principal point coordinates of the binocular camera to calculate the X, Y values ​​and the distance D (unit: mm) from the spatial coordinate to the camera;

[0101] Z = 8*f*b / d;

[0102] X=(xc x ) / f*Z;

[0103] X=(yc y ) / f*Z;

[0104]

[0105] After obtaining the spatial coordinates and distance sets corresponding to all pixel coordinates, sort and take the middle value according to D to obtain the final spatial coordinate point (X, Y, Z) of the original annotated pixel coordinates;

[0106] S123 Center point image coordinates: Obtain the pixel coordinates of the center point of the disease according to the four-corner pixel coordinates, and calculate the distance between the center point of the disease and the camera; specifically: obtain the pixel coordinates of the center point of the disease according to the four-corner pixel coordinates, and then search the surrounding range of the center point according to the similar method of the four-corner pixel coordinates, (xm*n~x+m*n, ym*n~y+m*n), and calculate the distance Dj between the center point of the disease and the camera;

[0107] S124 calculates the size of the disease: divides the spatial coordinates into two triangles, and uses vector cross product to calculate the area of ​​the two triangles to obtain the size of the disease;

[0108] In step S124, the spatial coordinates are divided into two triangles, that is, for the four points A (x1, y1, z1), B (x2, y2, z2), C (x3, y3, z3), D (x4, y4, z4) in the space, they are divided into two triangles ABC and ACD, and the areas are calculated respectively. Then, the areas of triangles ABC and ACD are calculated by vector cross multiplication to obtain the size of the defect. The formula for calculating the size of the defect is:

[0109]

[0110] Among them, A, B, and C are the coordinates of the three vertices of the triangle, AB and AC are the vectors from point A to B and C respectively, x1, y1, and z1 are the spatial coordinates of A, x2, y2, and z2 are the spatial coordinates of B, and x3, y3, and z3 are the spatial coordinates of C. i, j, and k are unit vectors in three-dimensional space, which point to the positive directions of the x-axis, y-axis, and z-axis respectively. The length of these unit vectors is 1 and they are mutually perpendicular; for the four points A (x1, y1, z1), B (x2, y2, z2), C (x3, y3, z3), and D (x4, y4, z4) in space, their areas cannot be simply calculated using AB*BC, so they are divided into two triangles ABC and ACD to calculate their areas respectively; for one of the triangles ABC, its area can be calculated by vector cross multiplication, that is, 0.5*||AB×AC||, so as to obtain the size of the defect;

[0111] like Figure 4 As shown in the figure, the edge detection device identifies and uploads the disease picture through the algorithm model, which contains information such as time, GPS, disease type, and disease identification frame. In addition, the depth information corresponding to each pixel coordinate identified by the binocular camera is also uploaded;

[0112] Mark the four points and the center point of the identification box as A, B, C, D and O respectively. Figure 5 As shown in the figure, due to the influence of objective factors such as light, weather, vehicle speed, shooting angle, road environment, etc. during the actual vehicle inspection process, the corresponding depth information of A, B, C, and D saved by the binocular camera may have errors. Therefore, each point will be used as a reference, and a small range m will be taken inside the identification frame. The depth corresponding to each pixel coordinate in the range will be calculated, and the excessively large or small values ​​will be discarded, and then the middle value will be taken. For the center point, a certain range around it will be taken for calculation;

[0113] The specific steps of step S2 are:

[0114] S21: For a road R, the starting point R0 and the end point R0 on the road will be marked when the file is created. n And the middle segmentation points R1, R2...R n-1, using the Haversine formula to calculate any segment point R m With the previous segment point R m-1 The distance between each segment point and the starting point R0 can be accumulated in sequence, with the starting point being 0 and the end point being D n And the middle segmentation points are D1, D2.....D n-1 ;

[0115] The Haversine formula is based on the spherical model of the earth and uses the relationship between spherical trigonometry and longitude and latitude to calculate the spherical distance between two points. The specific formula is:

[0116]

[0117] d = R·c;

[0118] in, are the latitudes of the two points in radians; is the latitude difference between the two points in radians;

[0119] S22: Based on the longitude Lon and latitude Lat of the disease information GPS positioning, the disease recording point, the corresponding matching road R and the inspection direction R w , also use the Haversine formula to find the disease record point R r The two points closest to each other, select the segment point R closest to the starting point R0 k , by the segmentation point R k Distance to starting point D k and disease record point R r To segment point R k The distance D rk Adding them together gives the distance D from the disease record point to the starting point of the road. r ;

[0120] S23: Considering that the different directions of vehicles lead to completely opposite shooting directions, even for the same latitude and longitude coordinates, the disease photographed is completely different; when the inspection direction is from the starting point R0 to the end point R n When , the fuzzy calculation of the disease center point D c The distance to the starting point is D r +D j , the opposite direction is D r -D j , where D j The distance between the center of the disease and the camera obtained by the disease size calculation service;

[0121] S24: When a new disease record is recorded in the database, N disease records of the same type on the road are initially screened according to the disease type T and the road R matched by GPS positioning. When N>0, the disease records are further screened according to the distance D from the disease center to the starting point. c Compare with the diseased area S to remove the duplicates. c The difference is within the preset distance threshold ±E d If the difference in the defect area S is within the preset area threshold ±Es, it is determined to be the same defect, otherwise it is saved as a new record; the defects found in each inspection are first matched with the existing information. If they do not match, they are new defects and a new defect record is created. The maintenance personnel need to be notified to carry out repairs. Otherwise, it is a repeated defect and there is no need to notify the maintenance personnel, which greatly improves the operational efficiency of road maintenance;

[0122] like Figures 6-7 As shown, if there is no disease in the database, the initial disease record information will be created. When there is a new disease record, the disease type, disease area, longitude and latitude information will be used to cluster the inspection results of different rounds on the platform;

[0123] S3 Result Display: The deduplicated road disease data is stored in the database and displayed on the map for road maintenance personnel to make maintenance decisions. Managers can view the deduplicated road disease information on the page and select the disease to assign to specific maintenance personnel. They can not only view the road disease statistics, but also view the specific distribution of the disease on the road on the map; Figures 8 to 10 shown.

[0124] The above technical solution is adopted, and the deep neural network algorithm is used to quickly and frequently detect and identify road defects, and obtain the pixel rectangular area of ​​the defect, the defect category, and GPS positioning information. The image depth information is obtained through the binocular camera, and the distance between the spatial vectors of the pixel coordinate frame is calculated based on the depth information, and finally the defect area is obtained; the inspection results of different rounds are clustered using the defect type and longitude and latitude information; finally, the similarity between the defects is determined based on the GPS positioning, defect type, and defect area.

[0125] For ordinary technicians in this field, the specific embodiments are only illustrative descriptions of the present invention. It is obvious that the specific implementation of the present invention is not limited to the above-mentioned methods. As long as various non-substantial improvements are made using the method concepts and technical solutions of the present invention, or the concepts and technical solutions of the present invention are directly applied to other occasions without improvement, they are all within the protection scope of the present invention.

Claims

1. A road deduplication method based on comprehensive consideration of GPS positioning, disease type and disease size, characterized in that: The specific steps include: S1 Data collection and processing: Use edge detection equipment to collect road image data, identify road damage on the road image data, obtain damage data, process it and then store it; S2 Disease deduplication: The processed disease data is matched with the location information, disease type and size information respectively. If the location information, disease type and size information are all within the preset matching threshold, they are determined to be duplicates, the duplicates are removed, and the deduplicated disease data is stored.

2. The road deduplication method based on comprehensive consideration of GPS positioning, disease type and disease size according to claim 1 is characterized in that: The process also includes step S3 result display: storing the deduplicated road disease data in a database and displaying it on a map for road maintenance personnel to make maintenance decisions.

3. The road deduplication method based on comprehensive consideration of GPS positioning, disease type and disease size according to claim 2 is characterized in that: The specific steps of step S1 are: S11 data collection: First, the intelligent road inspection system collects data through edge detection equipment and stores it in the database; S12 Disease Identification: Identify diseases in road image data through image recognition models, determine the disease type, and calculate the disease size; S13 Data preprocessing: Data formatting is performed on the corresponding disease location coordinates, disease types and disease sizes in the collected road image data to form a unified data structure, which is then stored in the database again.

4. The road deduplication method based on comprehensive consideration of GPS positioning, disease type and disease size according to claim 2 is characterized in that: In the step S11, the road image data collected by the binocular camera in the edge detection device includes GPS positioning information, disease type, original disease image, disease depth map and disease annotated pixel coordinate information; wherein the disease depth map and the disease annotated pixel coordinate information will be processed twice to the disease size after being saved, and saved again in the database; in the step S12, the disease depth map and the disease annotated pixel coordinate information are calculated by the disease size calculation service module, and the disease size and center point distance are obtained and stored in the database again.

5. The road deduplication method based on comprehensive consideration of GPS positioning, disease type and disease size according to claim 4 is characterized in that: The specific steps of calculating the disease size in step S12 are: S121 Four-corner pixel coordinates: The defect size calculation service module reversely searches for coordinate points in a set range according to the specific positions of the defect-annotated pixel coordinate points in the image. If all the depth values ​​of the current round of search are still 0, the search round is expanded; S122 spatial coordinates: The spatial coordinate points of the diseased annotated pixels are obtained by combining the four-corner pixel coordinates with the binocular camera internal references; S123 Center point image coordinates: Obtain the pixel coordinates of the center point of the disease according to the four-corner pixel coordinates, and calculate the distance between the center point of the disease and the camera; S124 calculates the size of the disease: divides the spatial coordinates into two triangles, and uses vector cross product to calculate the area of ​​the two triangles to obtain the size of the disease.

6. The road deduplication method based on comprehensive consideration of GPS positioning, disease type and disease size according to claim 5 is characterized in that: In step S121, assuming that the coordinates of the defect-annotated pixel coordinate point are (x, y), the initial search range is m, and the search round is n, the search ranges of the four points corresponding to the specific positions are: Upper left: (x~x+m*n, ym*n~y); Lower left: (x~x+m*n, y~y+m*n); Lower right: (xm*n~x, y~y+m*n); Upper right: (xm*n~x, ym*n~y).

7. The road deduplication method based on comprehensive consideration of GPS positioning, disease type and disease size according to claim 6 is characterized in that: The specific steps of step S122 are: S1221: The camera's intrinsic parameter matrix K describes the camera's internal characteristics, including focal length and principal point; the intrinsic parameter matrix K is expressed as: Among them, f x and f y is the focal length along the x-axis and y-axis; c x and c y are the coordinates of the principal point; S1222: When the upper left original coordinate point (x, y) is used for calculation, the coordinate set range obtained in the first round is (x~x+m, ym~y) with a total of (m+1) 2 coordinate points, the depth data of each cross position is expressed as: Take the non-zero values ​​to calculate the depth average d; Similarly, the same method is used to calculate the lower left, upper right, and lower right; S1223: Calculate the three-dimensional Z coordinate value of the plane pixel coordinate in the camera coordinate system according to the focal length f of the binocular camera and the baseline parameter b, and then use the normalization formula and the principal point coordinates of the binocular camera to calculate the X, Y values ​​and the distance D from the space coordinate to the camera; Z = 8*f*b / d; X=(x-c x ) / f*Z; X=(y-c y ) / f*Z; After obtaining the spatial coordinates and distance sets corresponding to all pixel coordinates, the middle value is taken according to D sorting to obtain the final spatial coordinate point (X, Y, Z) of the original annotated pixel coordinates.

8. The road deduplication method based on comprehensive consideration of GPS positioning, disease type and disease size according to claim 7 is characterized in that: In step S124, the spatial coordinates are divided into two triangles, that is, for the four points A (x1, y1, z1), B (x2, y2, z2), C (x3, y3, z3), D (x4, y4, z4) in the space, they are divided into two triangles ABC and ACD, and the areas are calculated respectively. Then, the areas of triangles ABC and ACD are calculated by vector cross multiplication to obtain the size of the defect. The formula for calculating the size of the defect is: Among them, A, B, and C are the coordinates of the three vertices of the triangle, AB and AC are the vectors from point A to B and C respectively, x1, y1, z1 are the spatial coordinates of A, x2, y2, z2 are the spatial coordinates of B, x3, y3, z3 are the spatial coordinates of C, i, j, k are unit vectors in three-dimensional space, and they point to the positive directions of the x-axis, y-axis, and z-axis respectively.

9. The road deduplication method based on comprehensive consideration of GPS positioning, disease type and disease size according to claim 8, characterized in that: The specific steps of step S2 are: S21: For a road R, the starting point R0 and the end point R0 on the road will be marked when the file is created. n And the middle segmentation points R1, R2...R n-1 , using the Haversine formula to calculate any segment point R m With the previous segment point R m-1 The distance between each segment point and the starting point R0 can be accumulated in sequence, with the starting point being 0 and the end point being D n And the middle segmentation points are D1, D2.....D n-1 ; S22: Based on the longitude Lon and latitude Lat of the disease information GPS positioning, the disease recording point, the corresponding matching road R and the inspection direction R w , also use the Haversine formula to find the disease record point R r The two points closest to each other, select the segment point R closest to the starting point R0 k , by the segmentation point R k Distance to starting point D k and disease record point R r To segment point R k The distance D rk Adding them together gives the distance D from the disease record point to the starting point of the road. r ; S23: When the inspection direction is from the starting point R0 to the end point R n When , the disease center point D is calculated c The distance to the starting point is D r +D j , the opposite direction is D r -D j , where D j The distance between the center of the disease and the camera obtained by the disease size calculation service; S24: When a new disease record is recorded in the database, N disease records of the same type on the road are initially screened according to the disease type T and the road R matched by GPS positioning. When N>0, the disease records are further screened according to the distance D from the disease center to the starting point. c Compare with the diseased area S to remove the duplicates. c The difference is within the preset distance threshold ±E d If the difference in the disease area S is within the preset area threshold ±Es, they are judged to be the same disease, otherwise they are saved as new records.

10. The road deduplication method based on comprehensive consideration of GPS positioning, disease type and disease size according to claim 9, characterized in that: The Haversine formula used in step S21 and step S22 is based on the spherical model of the earth, and uses the relationship between spherical trigonometry and longitude and latitude to calculate the spherical distance between two points. The specific formula is: d = R·c; in, are the latitudes of the two points in radians; is the latitude difference between the two points in radians.

Images