A method, medium and system for drawing multi-lane disease distribution map

Abstract

The present invention discloses a method, medium, and system for drawing a multi-lane defect distribution map, comprising: capturing a first multi-lane road surface image in front of an inspection vehicle at preset intervals using a camera mounted on the inspection vehicle; processing the first multi-lane road surface image using an inverse perspective transformation method to obtain a second multi-lane road surface image; marking valid image areas in the second multi-lane road surface image to obtain a third multi-lane road surface image; sequentially inputting two alternating third multi-lane road surface images into a convolutional neural network, outputting the state transition relationship between the pixel values ​​of each row of pixels in the valid image area of ​​the middle third multi-lane road surface image; processing the third multi-lane road surface image using a Kalman filter method to obtain a fourth multi-lane road surface image; identifying the locations of defects in the fourth multi-lane road surface image, and drawing the defects on a road surface drawing based on the locations of the defects. The present invention enables accurate and efficient drawing of a multi-lane defect distribution map.

Description

Technical Field

[0001] The present invention relates to the technical field of pavement maintenance engineering, and in particular to a method, medium and system for drawing a multi-lane disease distribution map. Background Art

[0002] Defect distribution maps are crucial data in maintenance projects. They provide insights into the specific location and morphology of defects, enabling better analysis of their causes and progression, and targeted treatment. Existing technologies use automated detection data to automatically create defect distribution maps, improving drawing speed. However, these technologies utilize traditional single-lane data collection, identification, and mapping techniques. This can lead to issues such as duplicated or missing defects and inconsistent pile numbers in multi-lane defect maps, resulting in inaccurate multi-lane defect distribution mapping. Summary of the Invention

[0003] Embodiments of the present invention provide a method, medium, and system for drawing a multi-lane defect distribution map to solve the problem of inaccurate multi-lane defect distribution drawing in the prior art.

[0004] In a first aspect, a method for drawing a multi-lane disease distribution map is provided, comprising:

[0005] When the inspection vehicle is traveling on the multi-lane road surface, a first multi-lane road surface image in front of the inspection vehicle is captured by a camera installed on the inspection vehicle at every preset distance;

[0006] Processing each of the first multi-lane road surface images by an inverse perspective transformation method to obtain each of the second multi-lane road surface images;

[0007] marking a valid image area in each of the second multi-lane road surface images to obtain each third multi-lane road surface image;

[0008] Inputting the two alternate third multi-lane road surface images into a convolutional neural network in sequence according to an acquisition order, and outputting a state transition relationship between pixel values ​​of each row of pixel points in a valid image area of ​​the middle third multi-lane road surface image of the two alternate images;

[0009] After obtaining the state transition matrix in the state transition relationship and the noise matrix of the prediction process, each of the third multi-lane road surface images is processed by a Kalman filter method to obtain each fourth multi-lane road surface image;

[0010] The location of each defect in the fourth multi-lane road surface image is identified, and the defect is drawn on the road surface drawing according to the location of the defect.

[0011] In a second aspect, a computer-readable storage medium is provided, on which computer program instructions are stored; when the computer program instructions are executed by a processor, a method for drawing a multi-lane defect distribution map as described in the embodiment of the first aspect is implemented.

[0012] In a third aspect, a system for drawing a multi-lane disease distribution map is provided, comprising: a computer-readable storage medium as described in the embodiment of the second aspect above.

[0013] In this way, the embodiment of the present invention can improve the clarity of the captured image, resulting in more accurate identification of the location of the defect, thereby achieving accurate drawing of the multi-lane defect distribution map and improving drawing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0015] Figure 1 is a flow chart of a method for drawing a multi-lane disease distribution map according to an embodiment of the present invention;

[0016] Figure 2 Schematic diagrams of road surface images captured by a camera according to an embodiment of the present invention, wherein (a) is a schematic diagram of a road surface image captured by the camera at the far right, and (b) is a schematic diagram of a road surface image captured by the camera at the far left;

[0017] Figure 3 is a three-dimensional schematic diagram of the camera projection relationship of an embodiment of the present invention;

[0018] Figure 4 is a side view schematic diagram of the camera projection relationship of an embodiment of the present invention;

[0019] Figure 5 is a schematic diagram of three adjacent images captured according to an embodiment of the present invention;

[0020] Figure 6 Schematic diagram of the process of determining the state transition relationship according to an embodiment of the present invention;

[0021] Figure 7 This is a multi-lane disease distribution map drawn by an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments derived by ordinary technicians in this field based on the embodiments of the present invention without creative work are within the scope of protection of the present invention.

[0023] The embodiment of the present invention discloses a method for drawing a multi-lane disease distribution map. Figure 1 As shown, the method of the embodiment of the present invention includes the following steps:

[0024] Step S101: When the inspection vehicle is traveling on a multi-lane road surface, a camera installed on the inspection vehicle collects a first multi-lane road surface image in front of the inspection vehicle at every preset distance.

[0025] The images captured by this embodiment of the present invention are grayscale images. The camera is typically an area array camera, mounted on the front roof of the inspection vehicle. In addition, encoders are installed on the non-guide wheels of the inspection vehicle to measure the vehicle's travel distance and obtain position information, ensuring that images are captured at predetermined intervals.

[0026] Step S102: Process each first multi-lane road surface image by using an inverse perspective transformation method to obtain each second multi-lane road surface image.

[0027] Specifically, the calculation formula of the inverse perspective transformation method is:

[0028]

[0029] in, represents the road surface coordinates after the image coordinates of the pixel points of the first multi-lane road surface image are inversely perspective transformed, represents the image coordinates of the pixel points of the first multi-lane road surface image, Represents the preset transformation matrix. The image coordinate system takes the center point of the first multi-lane road surface image as the origin, the X-axis is parallel to the width direction of the first multi-lane road surface image, and the Y-axis is parallel to the height direction of the first multi-lane road surface image. The road surface coordinate system takes the intersection of the camera optical axis and the multi-lane road surface as the origin, the Y-axis is parallel to the projection direction of the camera optical axis on the multi-lane road surface, and the X-axis is perpendicular to the projection direction of the camera optical axis on the multi-lane road surface.

[0030] The first multi-lane road surface image is transformed into a second multi-lane road surface image in a road surface coordinate system through an inverse perspective transformation.

[0031] Step S103: Marking a valid image area in each second multi-lane road surface image to obtain each third multi-lane road surface image.

[0032] During the detection process, the detection vehicle may be traveling on any lane. In order to make the image cover all lanes of the road, it is necessary to select a road image of an appropriate range. Figure 2 Taking (a) and (b) as examples of the image effects of the inspection vehicle at the far right and far left positions, the rectangular area d×L in the figure is the ideal effective image area. That is, the effective image area described in the embodiment of the present invention is a rectangular area that can fully display the multi-lane road surface in the width direction of the multi-lane road surface. Therefore, the width of the effective image area is the total width of the multi-lane road surface. The length of the effective image area is a preset length, which can be set based on experience. As long as the captured image contains the above-mentioned effective image area, the inspection vehicle can travel in any lane.

[0033] like Figure 3 and 4 As shown, the calculation formula for the vertical coordinates of the vertices in the length direction of the effective image area in the road coordinate system includes:

[0034]

[0035]

[0036] Where β represents half of the camera's horizontal field of view, γ represents the camera's pitch angle, h represents the camera's height, L represents the total width of the multi-lane road surface, and d represents the preset length.

[0037] Step S104: Input the two alternating third multi-lane road surface images into the convolutional neural network in sequence according to the acquisition order, and output the state transition relationship between the pixel values ​​of each row of pixel points in the effective image area of ​​the third multi-lane road surface image located in the middle of the two alternating images.

[0038] Since the image after inverse perspective transformation is larger when it is near and smaller when it is far, the image becomes less clear as it gets farther away. However, since the lane width needs to be taken into account during labeling, a certain range of image areas needs to be labeled, so the labeled image areas will be unclear.

[0039] Since the camera takes pictures at a certain interval, there is also a certain distance between the pictures. Therefore, there are repeated areas of the nth image before and after the nth image, that is, in the n-1th and n+1th images, such as Figure 5 As shown in the figure, the image in the n-1th image is complete but the clarity of the annotated effective image area is poor. The clarity of the annotated effective image area in the n+1th image is good but the annotated effective image area is incomplete. Therefore, combining these two images can obtain a complete state transition relationship for subsequent steps. Specifically, the state transition relationship includes: F(x k-1 )=Ax k-1 +w k-1 Among them, F(xk-1 ) represents the pixel value state transition relationship between the k-row pixel points and the k-1-row pixel points in the third multi-lane road image, A represents the state transition matrix, and w k-1 Represents the noise matrix of the prediction process, x k-1 Represents the pixel value of the k-1 row pixel point. The process diagram of determining the state transition relationship is as follows Figure 6 shown.

[0040] Specifically, the n-1th image and the n+1th image are input into the convolutional neural network, and a new complete image is obtained by using the convolutional neural network. According to the new image, the relationship between the pixel value of each row of pixels and the previous row is obtained, that is, the state transfer relationship F(x k-1 ), the following steps use this transfer relationship to complete the prediction of the pixel values ​​of each row of pixels in the effective image area of ​​the nth image, so as to obtain a clearer picture.

[0041] The convolutional neural network can adopt the network disclosed in CN 109360154 A.

[0042] Step S105: After obtaining the state transition matrix in the state transition relationship and the noise matrix of the prediction process, each third multi-lane road surface image is processed by a Kalman filter method to obtain each fourth multi-lane road surface image.

[0043] Based on the principle of Kalman filtering, the image is processed to improve the clarity of the image. First, the prediction quantity is clearly and the observed quantity z k , the pixel value corresponding to each row of pixels in the image processed in step S103 is the observed value, that is, z k The pixel value matrix of the kth row of pixels in the effective image area of ​​the third multi-lane road image is represented. The pixel value of the previous row is used as the state quantity, and the state prediction is completed according to the state transition relationship obtained after the processing of S104, that is, is the predicted pixel value matrix of the k-th row of pixels in the effective image area of ​​the third multi-lane road surface image, and then the optimal value is estimated according to the prediction equation and the update equation.

[0044] Specifically, the prediction equation includes:

[0045]

[0046]

[0047] The update equations include:

[0048]

[0049]

[0050]

[0051] in, represents the optimal estimation matrix of the pixel values ​​of the k-1th row of pixels in the valid image area of ​​the third multi-lane road image, The optimal estimation matrix z represents the pixel value of the kth row of pixels in the valid image area of ​​the third multi-lane road image, k represents the pixel value matrix (grayscale value in the embodiment of the present invention) of the k-th row of pixels in the effective image area of ​​the third multi-lane road surface image, H represents the observation matrix, K k represents the Kalman coefficient, Represents the prior estimated covariance matrix of the kth row, that is The covariance matrix, P k and P k-1 Represent the posterior estimated covariance matrix of the kth row and the k-1th row, that is, and The covariance matrix of , R represents the measurement noise matrix, which can be preset based on experience, and Q represents the process excitation noise matrix, which can be preset based on experience.

[0052] Through this step, the pixel value of each pixel point in the fourth multi-lane road image is finally obtained as

[0053]

[0054] Step S106: Identify the location of defects in each fourth multi-lane road surface image, and draw the defects on the road surface drawing according to the locations of the defects.

[0055] The horizontal coordinate of the road surface drawing is parallel to the length direction of the multi-lane road surface, and the horizontal coordinate of the road surface drawing can be the position of the stake. The vertical coordinate of the road surface drawing is parallel to the width direction of the multi-lane road surface. The drawing can be created using existing drawing software, such as CAD software.

[0056] If the defect is a length defect, the pixel coordinates corresponding to the turning point of the length defect are recorded.

[0057] If the damage is an area damage, the pixel coordinates corresponding to the two vertices of the diagonal corners of the area damage are recorded, for example, the two vertices at the upper left corner and the lower right corner.

[0058] The classification of length-type defects and area-type defects is carried out in accordance with the highway technical condition assessment standard JTG5210.

[0059] The process of identifying the location of the disease can be performed using existing image recognition technology or manually.

[0060] An embodiment of the present invention further discloses a computer-readable storage medium having computer program instructions stored thereon; when the computer program instructions are executed by a processor, the method for drawing a multi-lane defect distribution map as described in the above embodiment is implemented.

[0061] An embodiment of the present invention further discloses a system for drawing a multi-lane disease distribution map, comprising: a computer-readable storage medium as described in the above embodiment.

[0062] In summary, the embodiments of the present invention can improve the clarity of the captured images, resulting in more accurate identification of the locations of defects, thereby achieving accurate drawing of multi-lane defect distribution maps and improving drawing efficiency.

[0063] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for drawing a multi-lane disease distribution map, characterized in that: include: When the inspection vehicle is traveling on the multi-lane road surface, a first multi-lane road surface image in front of the inspection vehicle is captured by a camera installed on the inspection vehicle at every preset distance; Processing each of the first multi-lane road surface images by an inverse perspective transformation method to obtain each of the second multi-lane road surface images; marking a valid image area in each of the second multi-lane road surface images to obtain each third multi-lane road surface image; Inputting the two alternate third multi-lane road surface images into a convolutional neural network in sequence according to an acquisition order, and outputting a state transition relationship between pixel values ​​of each row of pixel points in a valid image area of ​​the middle third multi-lane road surface image of the two alternate images; After obtaining the state transition matrix in the state transition relationship and the noise matrix of the prediction process, each of the third multi-lane road surface images is processed by a Kalman filter method to obtain each fourth multi-lane road surface image; Identifying a location of a defect in each of the fourth multi-lane road surface images, and drawing the defect on a road surface drawing according to the location of the defect; The effective image area is a rectangular area that can fully display the multi-lane road surface in the width direction of the multi-lane road surface, wherein the width of the effective image area is the total width of the multi-lane road surface, and the length of the effective image area is a preset length; The calculation formula for the vertical coordinates of the vertices in the length direction of the effective image area in the road surface coordinate system includes: ； ； in, Y 1 and Y 2 represents the vertical coordinate of the vertex in the length direction of the effective image area in the road coordinate system, β represents half the horizontal field of view of the camera, γ represents the pitch angle of the camera, h represents the height of the camera, L represents the total width of the multi-lane road surface, d Indicates the preset length.

2. The method for drawing a multi-lane disease distribution map according to claim 1, characterized in that: The calculation formula of the inverse perspective transformation method is: ； in, road surface coordinates after inverse perspective transformation of the image coordinates of the pixel points of the first multi-lane road surface image, represents the image coordinates of the pixel points of the first multi-lane road surface image, represents a preset transformation matrix, wherein the image coordinate system takes the center point of the first multi-lane road surface image as the origin, the X-axis is parallel to the width direction of the first multi-lane road surface image, and the Y-axis is parallel to the height direction of the first multi-lane road surface image; the road surface coordinate system takes the intersection of the camera optical axis and the multi-lane road surface as the origin, the Y-axis is parallel to the projection direction of the camera optical axis on the multi-lane road surface, and the X-axis is perpendicular to the projection direction of the camera optical axis on the multi-lane road surface.

3. The method for drawing a multi-lane disease distribution map according to claim 1, characterized in that: The state transition relationship includes: ； Among them, F(x k-1 ) represents the third multi-lane road surface image k Row pixels and k -The state transition relationship of the pixel values ​​of 1 row of pixels, A represents the state transition matrix, Represents the noise matrix of the prediction process, x k-1 express k -1 pixel value of the pixel in the row.

4. The method for drawing a multi-lane disease distribution map according to claim 3, characterized in that: The calculation formula of the Kalman filtering method includes: ， ， ， ， ； in, The first image region representing the effective image area of ​​the third multi-lane road image k The predicted pixel value matrix of the row pixel point, The first image region representing the effective image area of ​​the third multi-lane road image k -1 row of pixel values ​​of the optimal estimation matrix, The first image region representing the effective image area of ​​the third multi-lane road image k The optimal estimation matrix of the pixel values ​​of the row pixels, The first image region representing the effective image area of ​​the third multi-lane road image k The pixel value matrix of the pixel points in the row, H represents the observation matrix, represents the Kalman coefficient, The third multi-lane road image is represented by k The prior estimate covariance matrix of the rows, and represent the first k Row and k -1 row of the posterior estimated covariance matrix, R represents the measurement noise matrix, Q represents the process excitation noise matrix.

5. The method for drawing a multi-lane disease distribution map according to claim 1, characterized in that: The horizontal coordinate of the road surface drawing is parallel to the length direction of the multi-lane road surface, and the vertical coordinate of the road surface drawing is parallel to the width direction of the multi-lane road surface.

6. The method for drawing a multi-lane disease distribution map according to claim 1, characterized in that: If the defect is a length defect, the pixel coordinates corresponding to the turning point of the length defect are recorded; If the disease is an area disease, the pixel coordinates corresponding to the two diagonal vertices of the area disease are recorded.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by a processor, the method for drawing a multi-lane defect distribution map according to any one of claims 1 to 6 is implemented.

8. A system for drawing a multi-lane disease distribution map, characterized in that: include: The computer-readable storage medium of claim 7.

Images