A highway construction whole process information management system and method

Abstract

The present invention relates to the field of information management technology, and in particular to an information management system and method for the entire process of highway construction. An information management system for the entire process of highway construction includes: a highway construction real-scene image acquisition module, a highway area segmentation module, and a highway construction progress calculation module. The present invention shoots a highway construction real-scene image at a set shooting location, and obtains a highway area image corresponding to each highway construction real-scene image through a highway area segmentation model, and then matches the highway area image with the corresponding standard highway area image according to the shooting location, so as to characterize the highway construction progress by the proportion of completed highways, thereby realizing the information management of the entire process of highway construction.

Description

Technical Field

[0001] The present invention relates to the technical field of information management, and in particular to an information management system and method for the entire process of highway construction. Background Art

[0002] At present, the progress management of highway construction projects mostly relies on traditional management methods, such as manual on-site measurement of engineering quantities, manual filling out of reports and reporting layer by layer. This method is time-consuming and labor-intensive, inefficient, and easily affected by subjective factors such as measurement errors, reporting delays, data tampering, etc., resulting in inaccurate progress data and untimely updates, making it difficult to truly reflect the actual progress of the project. In addition, the traditional method of information transmission is inefficient, the communication cost is high, and it is easy to create information islands, which is not conducive to macro-control and scientific decision-making by project managers, and may eventually lead to problems such as delays in construction and cost overruns. Summary of the invention

[0003] The present invention shoots a real-life picture of highway construction at a set shooting location, obtains a highway area picture corresponding to each real-life picture of highway construction through a highway area segmentation model, and then matches the highway area picture with the corresponding standard highway area picture according to the shooting location, so as to characterize the progress of highway construction by the proportion of completed highways, thereby realizing the information management of the entire highway construction process; in the process of obtaining the highway area picture corresponding to each real-life picture of highway construction through the highway area segmentation model, the highway area corresponding to the target real-life picture of highway construction is supplemented by reference provided by the surrounding real-life pictures of highway construction, so as to effectively reduce the difference between the highway area and the standard highway area caused by occlusion, and improve the accuracy of subsequent highway construction progress calculation.

[0004] The present invention provides a method for information management of the entire process of highway construction, comprising:

[0005] Obtain several real-life pictures of highway construction during the highway construction process, and record shooting data corresponding to each real-life picture of highway construction, the shooting data including shooting location and shooting direction;

[0006] Traverse all highway construction real scene pictures, and perform the following operations for the selected highway construction real scene picture: record the selected highway construction real scene picture as the target highway construction real scene picture, calculate the distance between the shooting location of all highway construction real scene pictures except the target highway construction real scene picture and the shooting location of the target highway construction real scene picture, and record it as the target distance, and arrange all highway construction real scene pictures except the target highway construction real scene picture from large to small according to the corresponding target distance, and form the first N highway construction real scene pictures after arrangement into a neighbor real scene picture set, and record the highway construction real scene pictures in the neighbor real scene picture set as reference highway construction real scene pictures, and then send the target highway construction real scene picture and its corresponding neighbor real scene picture set into the highway area segmentation model for processing, and output the corresponding highway area picture; until all highway construction real scene pictures are traversed, the highway area pictures corresponding to all highway construction real scene pictures are output;

[0007] Match all highway area pictures with corresponding standard highway area pictures according to shooting data, calculate the highway construction progress, and record the highway construction progress;

[0008] The highway area segmentation model is established based on the U-net network, including five encoding blocks, respectively recorded as the first encoding block, the second encoding block, the third encoding block, the fourth encoding block and the fifth encoding block, four decoding blocks, respectively recorded as the first decoding block, the second decoding block, the third decoding block and the fourth decoding block, four feature fusion blocks, respectively recorded as the first feature fusion block, the second feature fusion block, the third feature fusion block and the fourth feature fusion block, and a highway area segmentation layer; wherein the encoding block is used to perform feature extraction operations, and the decoder is used to perform feature reconstruction operations; the feature fusion block is used to fuse the features extracted from the target highway construction real scene picture by referring to the features extracted from the highway construction real scene picture; the highway area segmentation layer is used to segment the highway area corresponding to the target highway construction real scene picture, and output the corresponding highway area picture.

[0009] As a preferred aspect of the present invention, the target highway construction real scene image and its corresponding neighbor real scene image set are sent to the highway area segmentation model for processing, and the corresponding highway area image is output, which specifically includes the following steps:

[0010] Processing the target highway construction real scene picture through the first coding block, the second coding block, the third coding block, the fourth coding block and the fifth coding block in sequence, and outputting the corresponding first target feature map, the second target feature map, the third target feature map, the fourth target feature map and the fifth target feature map;

[0011] Processing the reference highway construction real scene pictures in the neighbor real scene picture set through the first coding block, the second coding block, the third coding block and the fourth coding block in sequence, and outputting the corresponding first reference feature picture, the second reference feature picture, the third reference feature picture and the fourth reference feature picture;

[0012] The fourth target feature map and all the fourth reference feature maps are processed by the fourth feature fusion block to obtain the sixth target feature map, and then the fifth target feature map and the sixth target feature map are spliced ​​according to the channel and processed by the fourth decoding block to obtain the seventh target feature map; the third target feature map and all the third reference feature maps are processed by the third feature fusion block to obtain the eighth target feature map, and then the seventh target feature map and the eighth target feature map are spliced ​​according to the channel and processed by the third decoding block to obtain the ninth target feature map; the second target feature map and all the second reference feature maps are processed by the second feature fusion block to obtain the tenth target feature map, and then the ninth target feature map and the tenth target feature map are spliced ​​according to the channel and processed by the second decoding block to obtain the eleventh target feature map; the first target feature map and all the first reference feature maps are processed by the first feature fusion block to obtain the twelfth feature map, and then the eleventh target feature map and the twelfth feature map are spliced ​​according to the channel and processed by the first decoding block to obtain the supplemented highway construction real scene map;

[0013] The supplemented highway construction real scene map is sent to the highway area segmentation layer for processing, and the corresponding highway area picture is output.

[0014] As a preferred aspect of the present invention, in the feature fusion block, the following operations are performed:

[0015] The feature maps input into the feature fusion block are respectively recorded as target feature maps and reference feature maps, the target feature maps include the first target feature map, the second target feature map, the third target feature map, the fourth target feature map and the fifth target feature map, and the reference feature maps include the first reference feature map, the second reference feature map, the third reference feature map and the fourth reference feature map; the feature map output by the feature fusion block is recorded as the output feature map, and the feature map output by the feature fusion block referred to here includes the sixth target feature map, the eighth target feature map, the tenth target feature map and the twelfth target feature map;

[0016] The feature fusion block includes a reference region change matrix generation layer and a feature fusion layer;

[0017] The i-th reference feature map and the target feature map are sent to the reference region change matrix generation layer for processing, and the reference region change matrix W corresponding to the i-th reference feature map is output. i , i=1,2,3,…,I, I is the total number of highway construction real scene pictures in the neighbor real scene picture set;

[0018] In the feature fusion layer, the following formula is used for processing: , where U is the output feature map, σ() is the nonlinear activation operation, F is the target feature map, and H i is the reference feature map.

[0019] As a preferred aspect of the present invention, the i-th reference feature map and the target feature map are sent to the reference region change matrix generation layer for processing, and the reference region change matrix W corresponding to the i-th reference feature map is output. i , specifically including the following steps:

[0020] The i-th reference feature map is fully connected to obtain the i-th reference feature vector, and the shooting data is added to the end of the i-th reference feature vector to construct the i-th vector to be analyzed; the target feature map is fully connected to obtain the target feature vector; the i-th vector to be analyzed and the target feature vector are concatenated and sent to the multi-layer perceptron for processing, and the reference area change matrix W corresponding to the i-th reference feature map is output. i .

[0021] As a preferred aspect of the present invention, training is performed on a highway area segmentation model, specifically comprising the following steps:

[0022] A number of highway area segmentation training samples are obtained, wherein the highway area segmentation training samples include N+1 highway construction real scene pictures, and the N+1 highway construction real scene pictures include a target highway construction real scene picture and N reference highway construction real scene pictures, and the highway area segmentation training samples are annotated through standard highway construction real scene pictures and highway area pictures, and all annotated highway area segmentation training samples are composed of a highway area segmentation training set, and the highway area segmentation model is trained through the highway area segmentation training set, and a comprehensive loss value is calculated, wherein the comprehensive loss value includes a loss value between the predicted supplemented highway construction real scene picture and the annotated standard highway construction real scene picture and a loss value between the predicted highway area picture and the annotated highway area picture, and the loss value between the predicted supplemented highway construction real scene picture and the annotated standard highway construction real scene picture and the loss value between the predicted highway area picture and the annotated highway area picture are obtained by weighted summation, and it is judged whether the comprehensive loss value is within a preset range, and if the comprehensive loss value is within the preset range, the trained highway area segmentation model is output; otherwise, the highway area segmentation model is continuously trained through the highway area segmentation training set.

[0023] As a preferred aspect of the present invention, all highway area pictures are matched with corresponding standard highway area pictures according to the shooting data, and the highway construction progress is calculated, which specifically includes the following steps: calculating the matching value between each highway area picture according to the shooting data and the corresponding standard highway area picture, calculating the ratio of the sum of the matching values ​​corresponding to all highway area pictures to M, and recording it as the highway construction progress, where M is the total number of shooting data.

[0024] As a preferred aspect of the present invention, the standard highway area picture is obtained in the following manner: obtain a highway design three-dimensional model, set a virtual camera in the three-dimensional space corresponding to the highway design three-dimensional model according to the shooting data, and obtain the standard highway area picture taken by the virtual camera.

[0025] The present invention also provides an information management system for the entire process of highway construction, comprising:

[0026] The highway construction real scene picture acquisition module is used to acquire several highway construction real scene pictures during the highway construction process and record the shooting data corresponding to each highway construction real scene picture, the shooting data including the shooting location and shooting direction;

[0027] The highway area segmentation module is used to traverse all highway construction real scene pictures, and for the selected highway construction real scene picture, perform the following operations: record the selected highway construction real scene picture as the target highway construction real scene picture, calculate the distance between the shooting location of all highway construction real scene pictures except the target highway construction real scene picture and the shooting location of the target highway construction real scene picture, and record it as the target distance, and arrange all highway construction real scene pictures except the target highway construction real scene picture from large to small according to the corresponding target distance, and form the first N highway construction real scene pictures after arrangement into a neighbor real scene picture set, and record the highway construction real scene pictures in the neighbor real scene picture set as reference highway construction real scene pictures, and then send the target highway construction real scene picture and its corresponding neighbor real scene picture set into the highway area segmentation model for processing, and output the corresponding highway area picture; until all highway construction real scene pictures are traversed, the highway area pictures corresponding to all highway construction real scene pictures are output;

[0028] The highway construction progress calculation module is used to match all highway area pictures with corresponding standard highway area pictures according to the shooting data, calculate the highway construction progress, and record the highway construction progress;

[0029] The highway area segmentation model is established based on the U-net network, including five encoding blocks, respectively recorded as the first encoding block, the second encoding block, the third encoding block, the fourth encoding block and the fifth encoding block, four decoding blocks, respectively recorded as the first decoding block, the second decoding block, the third decoding block and the fourth decoding block, four feature fusion blocks, respectively recorded as the first feature fusion block, the second feature fusion block, the third feature fusion block and the fourth feature fusion block, and a highway area segmentation layer; wherein the encoding block is used to perform feature extraction operations, and the decoder is used to perform feature reconstruction operations; the feature fusion block is used to fuse the features extracted from the target highway construction real scene picture by referring to the features extracted from the highway construction real scene picture; the highway area segmentation layer is used to segment the highway area corresponding to the target highway construction real scene picture, and output the corresponding highway area picture.

[0030] The present invention has the following advantages:

[0031] The present invention shoots a real-life picture of highway construction at a set shooting location, obtains a highway area picture corresponding to each real-life picture of highway construction through a highway area segmentation model, and then matches the highway area picture with the corresponding standard highway area picture according to the shooting location, so as to characterize the progress of highway construction by the proportion of completed highways, thereby realizing the information management of the entire highway construction process; in the process of obtaining the highway area picture corresponding to each real-life picture of highway construction through the highway area segmentation model, the highway area corresponding to the target real-life picture of highway construction is supplemented by reference provided by the surrounding real-life pictures of highway construction, so as to effectively reduce the difference between the highway area and the standard highway area caused by occlusion, and improve the accuracy of subsequent highway construction progress calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a schematic diagram of the structure of the highway area segmentation model adopted in the embodiment of the present invention.

[0033] Figure 2 This is a schematic diagram of the structure of the highway construction whole process information management system adopted in an embodiment of the present invention. DETAILED DESCRIPTION

[0034] In order to enable persons skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0035] Embodiment 1, a method for information management of the entire process of highway construction, comprising:

[0036] Obtain several real-life pictures of highway construction during the highway construction process, and record the shooting data corresponding to each real-life picture of highway construction. The shooting data includes the shooting location and shooting direction. It should be noted that during the highway construction process, drones can be used to shoot real-life pictures of highway construction at set shooting locations, and these set shooting locations are generally distributed in a grid, covering all areas where highway construction is being carried out, and there are overlapping areas between adjacent real-life pictures of highway construction, generally overlapping by 30%-40%. The shooting direction refers to the direction of the drone camera. The drone performs shooting tasks according to the set route. These real-life pictures of highway construction taken will be uploaded to the cloud server and subsequent operations will be performed;

[0037] Traverse all highway construction real scene pictures, and perform the following operations for the selected highway construction real scene picture: record the selected highway construction real scene picture as the target highway construction real scene picture, calculate the distance between the shooting location of all highway construction real scene pictures except the target highway construction real scene picture and the shooting location of the target highway construction real scene picture, and record it as the target distance. The distance calculation adopts Euclidean distance, and all highway construction real scene pictures except the target highway construction real scene picture are arranged from large to small according to the corresponding target distance, and the first N highway construction real scene pictures after arrangement are composed of a neighbor real scene picture set. Yes, since the set shooting locations are generally distributed in a grid, the N value corresponding to the real-life highway construction picture obtained at the shooting location in the middle is generally 8, and the N value corresponding to the real-life highway construction picture obtained at the shooting location at the edge is generally 5. The real-life highway construction picture in the neighbor real-life picture set is recorded as the reference real-life highway construction picture, and then the target real-life highway construction picture and its corresponding neighbor real-life picture set are sent to the highway area segmentation model for processing, and the corresponding highway area picture is output; until all the real-life highway construction pictures are traversed, the highway area pictures corresponding to all the real-life highway construction pictures are output;

[0038] Match all highway area pictures with corresponding standard highway area pictures according to shooting data, calculate and record the progress of highway construction. During the process of highway construction, the progress of highway construction is automatically recorded to realize the information management of the whole process of highway construction;

[0039] All highway area pictures are matched with corresponding standard highway area pictures according to the shooting data, and the highway construction progress is calculated, which specifically includes the following steps:

[0040] Calculate the matching value between each highway area picture according to the shooting data and the corresponding standard highway area picture. The matching value here can be the ratio between the area of ​​the highway area picture and the area of ​​the corresponding standard highway area picture. If it is a complete match, it means that the road section corresponding to the highway area picture has been completed, and the matching value is 1. Calculate the ratio of the sum of the matching values ​​corresponding to all highway area pictures to M, and record it as the highway construction progress. M is the total number of shooting data;

[0041] The standard highway area images here are obtained in the following way:

[0042] Obtain a 3D model of the highway design. The 3D model of the highway design is generally obtained by the designer through 3D modeling software (such as UG) based on the design drawings. A virtual camera is set in the 3D space corresponding to the 3D model of the highway design according to the shooting data, and a standard highway area image taken by the virtual camera is obtained. It should be noted that the 3D model of the highway design here does not contain plants, people, vehicles, etc. in the actual scene, so it does not need to be segmented;

[0043] See also Figure 1 The highway area segmentation model is established based on the U-net network, including five encoding blocks, which are respectively recorded as the first encoding block, the second encoding block, the third encoding block, the fourth encoding block and the fifth encoding block. The specific operation of the encoding block refers to the U-net network, performing convolution operations and downsampling operations, four decoding blocks, which are respectively recorded as the first decoding block, the second decoding block, the third decoding block and the fourth decoding block. The specific operation of the decoding block refers to the U-net network, performing convolution operations and upsampling operations, four feature fusion blocks, which are respectively recorded as the first feature fusion block, the second feature fusion block, the third feature fusion block and the fourth feature fusion block, and a highway area segmentation layer; wherein the encoding block is used to perform feature extraction operations, and the decoder is used to perform feature reconstruction operations. The feature reconstruction operation here refers to the image size. The feature fusion block is used to fuse the features extracted from the target highway construction real scene picture by referring to the features extracted from the highway construction real scene picture, so as to supplement the highway area corresponding to the target highway construction real scene picture and ensure the integrity of the highway area under construction. Due to the occlusion caused by plants, people and vehicles when the drone is shooting, for each target highway construction real scene picture, the highway area corresponding to the target highway construction real scene picture is supplemented by the reference provided by the surrounding highway construction real scene pictures, which can effectively reduce the difference between the highway area and the standard highway area caused by the occlusion, and improve the accuracy of the subsequent highway construction progress calculation; the highway area segmentation layer is used to segment the highway area corresponding to the target highway construction real scene picture and output the corresponding highway area picture.

[0044] The present application shoots real-life pictures of highway construction at a set shooting location, obtains highway area pictures corresponding to each real-life picture of highway construction through a highway area segmentation model, and then matches the highway area pictures with the corresponding standard highway area pictures according to the shooting location, so as to characterize the progress of highway construction by the proportion of completed highways, thereby realizing the information management of the entire highway construction process; in the process of obtaining the highway area pictures corresponding to each real-life picture of highway construction through the highway area segmentation model, the highway area corresponding to the target real-life picture of highway construction is supplemented by reference provided by the surrounding real-life pictures of highway construction, thereby effectively reducing the difference between the highway area and the standard highway area caused by occlusion, and improving the accuracy of subsequent highway construction progress calculations.

[0045] The target highway construction real scene image and its corresponding neighbor real scene image set are sent to the highway area segmentation model for processing, and the corresponding highway area image is output, which specifically includes the following steps:

[0046] Processing the target highway construction real scene picture through the first coding block, the second coding block, the third coding block, the fourth coding block and the fifth coding block in sequence, and outputting the corresponding first target feature map, the second target feature map, the third target feature map, the fourth target feature map and the fifth target feature map;

[0047] Processing the reference highway construction real scene pictures in the neighbor real scene picture set through the first coding block, the second coding block, the third coding block and the fourth coding block in sequence, and outputting the corresponding first reference feature picture, the second reference feature picture, the third reference feature picture and the fourth reference feature picture;

[0048] The fourth target feature map and all the fourth reference feature maps are processed by the fourth feature fusion block to obtain a sixth target feature map, and then the fifth target feature map and the sixth target feature map are spliced ​​according to the channel and processed by the fourth decoding block to obtain a seventh target feature map; the third target feature map and all the third reference feature maps are processed by the third feature fusion block to obtain an eighth target feature map, and then the seventh target feature map and the eighth target feature map are spliced ​​according to the channel and processed by the third decoding block to obtain a ninth target feature map; the second target feature map and all the second reference feature maps are processed by the second feature fusion block to obtain a tenth target feature map, and then the ninth target feature map is processed by the third decoding block. The target feature map and the tenth target feature map are spliced ​​according to the channel and processed by the second decoding block to obtain the eleventh target feature map; the first target feature map and all the first reference feature maps are processed by the first feature fusion block to obtain the twelfth feature map, and then the eleventh target feature map and the twelfth feature map are spliced ​​according to the channel and processed by the first decoding block to obtain the supplemented highway construction real scene map; it should be noted that, with reference to the U-net network, the processing in the decoder includes convolution operation and upsampling operation, and the size of the feature map can be continuously restored by the upsampling operation until the size of the supplemented highway construction real scene map is consistent with the size of the target highway construction real scene map;

[0049] The supplemented highway construction scene image is sent to the highway area segmentation layer for processing. The processing here can adopt a full connection operation, and the probability of each pixel of the supplemented highway construction scene image belonging to the highway area is output, and the highway area segmentation is realized by probability, and the corresponding highway area image is output;

[0050] In the feature fusion block, perform the following operations:

[0051] The feature maps input to the feature fusion block are respectively recorded as target feature maps and reference feature maps, the target feature maps include the first target feature map, the second target feature map, the third target feature map, the fourth target feature map and the fifth target feature map, and the reference feature maps include the first reference feature map, the second reference feature map, the third reference feature map and the fourth reference feature map; the feature map output by the feature fusion block is recorded as the output feature map, and the feature map output by the feature fusion block referred to here includes the sixth target feature map, the eighth target feature map, the tenth target feature map and the twelfth target feature map; since the operations in the feature fusion block are the same, only the input feature map and the output feature map are different, in order to avoid repeated description, the specific operations of the feature fusion block are summarized by the target feature map and the reference feature map;

[0052] The feature fusion block includes a reference region change matrix generation layer and a feature fusion layer;

[0053] The i-th reference feature map and the target feature map are sent to the reference region change matrix generation layer for processing, and the reference region change matrix W corresponding to the i-th reference feature map is output. i , i=1,2,3,…,I, I is the total number of highway construction scenes in the neighbor scene set; it should be noted that, since the reference highway construction scene and the target highway construction scene correspond to different shooting data, feature fusion cannot be performed directly, but it is necessary to analyze the correspondence between the features of the reference highway construction scene and the features of the target highway construction scene, and then determine the reference area that can be supplemented, so as to generate a change matrix corresponding to the reference area, and each feature fusion block corresponds to feature information at different levels, so that the highway area can be supplemented under the reference of feature information at different levels;

[0054] In the feature fusion layer, the following formula is used for processing: , where U is the output feature map, σ() is a nonlinear activation operation, generally using the ReLU function, F is the target feature map, and H i The reference feature map is used as the reference feature map; the features of the reference highway construction scene map around the target highway construction scene map are integrated at different levels, which can effectively supplement the highway area of ​​the target highway construction scene map;

[0055] The i-th reference feature map and the target feature map are sent to the reference region change matrix generation layer for processing, and the reference region change matrix W corresponding to the i-th reference feature map is output. i , specifically including the following steps:

[0056] The i-th reference feature map is fully connected to obtain the i-th reference feature vector, and the shooting data is added to the end of the i-th reference feature vector to construct the i-th vector to be analyzed; the target feature map is fully connected to obtain the target feature vector; the i-th vector to be analyzed and the target feature vector are concatenated and sent to the multi-layer perceptron for processing, and the reference area change matrix W corresponding to the i-th reference feature map is output. i ,It should be noted that the parameters of the multilayer perceptron here are ,adjusted following the training of the highway area segmentation ,model;

[0057] Training the highway area segmentation model includes the following steps:

[0058] A number of highway area segmentation training samples are obtained, and the highway area segmentation training samples include N+1 highway construction real scene pictures, and the N+1 highway construction real scene pictures include a target highway construction real scene picture and N reference highway construction real scene pictures, wherein the target highway construction real scene picture is the picture that needs to be segmented into highway areas, and the highway area segmentation training samples are annotated by standard highway construction real scene pictures and highway area pictures. It should be noted that the standard highway construction real scene picture here refers to the picture obtained after artificial correction and occlusion of the target highway construction real scene picture, and the highway area picture is the highway area segmented artificially. All the annotated highway area segmentation training samples are composed of a highway area segmentation training set, and the highway area segmentation model is trained by the highway area segmentation training set. Training, calculating the comprehensive loss value, the comprehensive loss value includes the loss value between the predicted supplemented highway construction real scene picture and the annotated standard highway construction real scene picture and the loss value between the predicted highway area picture and the annotated highway area picture, and the loss value between the predicted supplemented highway construction real scene picture and the annotated standard highway construction real scene picture and the loss value between the predicted highway area picture and the annotated highway area picture are obtained by weighted summation, the weight here can be determined by the developer, and it is judged whether the comprehensive loss value is within the preset range, the preset range is determined by the developer, if the comprehensive loss value is within the preset range, the trained highway area segmentation model is output; otherwise, the highway area segmentation model is continued to be trained through the highway area segmentation training set.

[0059] It should be added that in order to avoid repeated drone flights, whenever the highway area corresponding to the real-life highway construction picture is completely matched with the standard highway area picture, it means that the section corresponding to the real-life highway construction picture has been completed, then the shooting locations on the completed section will no longer be photographed.

[0060] Embodiment 2, a highway construction whole process information management system, such as Figure 2 As shown, including:

[0061] The highway construction real scene picture acquisition module is used to obtain several highway construction real scene pictures during the highway construction process, and record the shooting data corresponding to each highway construction real scene picture. The shooting data includes the shooting location and shooting direction. It should be noted that during the highway construction process, the highway construction real scene pictures can be shot by drones at the set shooting locations, and these set shooting locations are generally distributed in a grid, covering all areas where highway construction is carried out, and there are overlapping areas between adjacent highway construction real scenes, generally overlapping by 30%-40%. The shooting direction refers to the direction of the drone camera. The drone performs the shooting task according to the set route. These shot highway construction real scene pictures will be uploaded to the cloud server and subsequent operations will be performed;

[0062] The highway area segmentation module is used to traverse all highway construction real scene pictures, and perform the following operations for the selected highway construction real scene picture: record the selected highway construction real scene picture as the target highway construction real scene picture, calculate the distance between the shooting location of all highway construction real scene pictures except the target highway construction real scene picture and the shooting location of the target highway construction real scene picture, and record it as the target distance. The distance calculation adopts Euclidean distance, and all highway construction real scene pictures except the target highway construction real scene picture are arranged from large to small according to the corresponding target distance, and the first N highway construction real scene pictures after arrangement are composed of a neighbor real scene picture set It should be noted that, since the set shooting locations are generally distributed in a grid, the N value corresponding to the real-scene image of highway construction obtained at the shooting location in the middle is generally 8, and the N value corresponding to the real-scene image of highway construction obtained at the shooting location at the edge is generally 5. The real-scene image of highway construction in the neighbor real-scene image set is recorded as the reference real-scene image of highway construction, and then the target real-scene image of highway construction and its corresponding set of neighbor real-scene images are sent to the highway area segmentation model for processing, and the corresponding highway area image is output; until all highway construction real-scene images are traversed, the highway area images corresponding to all highway construction real-scene images are output;

[0063] The highway construction progress calculation module is used to match all highway area pictures with the corresponding standard highway area pictures according to the shooting data, calculate the highway construction progress, and record the highway construction progress. During the highway construction process, the progress of highway construction is automatically recorded to realize the information management of the entire highway construction process;

[0064] The highway area segmentation model is established based on the U-net network, including five encoding blocks, which are respectively recorded as the first encoding block, the second encoding block, the third encoding block, the fourth encoding block and the fifth encoding block. The specific operation of the encoding block refers to the U-net network, performing convolution operations and downsampling operations, four decoding blocks, which are respectively recorded as the first decoding block, the second decoding block, the third decoding block and the fourth decoding block. The specific operation of the decoding block refers to the U-net network, performing convolution operations and upsampling operations, four feature fusion blocks, which are respectively recorded as the first feature fusion block, the second feature fusion block, the third feature fusion block and the fourth feature fusion block, and a highway area segmentation layer; the encoding block is used to perform feature extraction operations, and the decoder is used to perform feature reconstruction operations. The feature reconstruction operation here refers to the image size. The feature fusion block is used to fuse the features extracted from the target highway construction real scene picture by referring to the features extracted from the highway construction real scene picture, so as to supplement the highway area corresponding to the target highway construction real scene picture and ensure the integrity of the highway area under construction. Due to the occlusion caused by plants, people and vehicles when the drone is shooting, for each target highway construction real scene picture, the highway area corresponding to the target highway construction real scene picture is supplemented by the reference provided by the surrounding highway construction real scene pictures, which can effectively reduce the difference between the highway area and the standard highway area caused by the occlusion, and improve the accuracy of the subsequent highway construction progress calculation; the highway area segmentation layer is used to segment the highway area corresponding to the target highway construction real scene picture and output the corresponding highway area picture.

[0065] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all such improvements and changes should fall within the scope of protection of the appended claims of the present invention. Parts not described in detail in this specification belong to the prior art known to those skilled in the art.

Claims

1. A method for information management of the entire process of highway construction, characterized in that: include: Obtain several real-life pictures of highway construction during the highway construction process, and record shooting data corresponding to each real-life picture of highway construction, the shooting data including shooting location and shooting direction; Traverse all highway construction real scene pictures, and perform the following operations for the selected highway construction real scene picture: record the selected highway construction real scene picture as the target highway construction real scene picture, calculate the distance between the shooting location of all highway construction real scene pictures except the target highway construction real scene picture and the shooting location of the target highway construction real scene picture, and record it as the target distance, and arrange all highway construction real scene pictures except the target highway construction real scene picture from large to small according to the corresponding target distance, and form the first N highway construction real scene pictures after arrangement into a neighbor real scene picture set, and record the highway construction real scene pictures in the neighbor real scene picture set as reference highway construction real scene pictures, and then send the target highway construction real scene picture and its corresponding neighbor real scene picture set into the highway area segmentation model for processing, and output the corresponding highway area picture; until all highway construction real scene pictures are traversed, the highway area pictures corresponding to all highway construction real scene pictures are output; Match all highway area pictures with corresponding standard highway area pictures according to shooting data, calculate the highway construction progress, and record the highway construction progress; The highway area segmentation model is established based on the U-net network, including five encoding blocks, which are respectively recorded as the first encoding block, the second encoding block, the third encoding block, the fourth encoding block and the fifth encoding block, four decoding blocks, which are respectively recorded as the first decoding block, the second decoding block, the third decoding block and the fourth decoding block, four feature fusion blocks, which are respectively recorded as the first feature fusion block, the second feature fusion block, the third feature fusion block and the fourth feature fusion block, and a highway area segmentation layer; The encoding block is used to perform feature extraction operations, and the decoder is used to perform feature reconstruction operations; the feature fusion block is used to fuse the features extracted from the target highway construction real scene image by referring to the features extracted from the highway construction real scene image; the highway area segmentation layer is used to segment the highway area corresponding to the target highway construction real scene image and output the corresponding highway area image; The target highway construction real scene image and its corresponding neighbor real scene image set are sent to the highway area segmentation model for processing, and the corresponding highway area image is output, which specifically includes the following steps: Processing the target highway construction real scene picture through the first coding block, the second coding block, the third coding block, the fourth coding block and the fifth coding block in sequence, and outputting the corresponding first target feature map, the second target feature map, the third target feature map, the fourth target feature map and the fifth target feature map; Processing the reference highway construction real scene pictures in the neighbor real scene picture set through the first coding block, the second coding block, the third coding block and the fourth coding block in sequence, and outputting the corresponding first reference feature picture, the second reference feature picture, the third reference feature picture and the fourth reference feature picture; The fourth target feature map and all the fourth reference feature maps are processed by the fourth feature fusion block to obtain the sixth target feature map, and then the fifth target feature map and the sixth target feature map are spliced ​​according to the channel and processed by the fourth decoding block to obtain the seventh target feature map; the third target feature map and all the third reference feature maps are processed by the third feature fusion block to obtain the eighth target feature map, and then the seventh target feature map and the eighth target feature map are spliced ​​according to the channel and processed by the third decoding block to obtain the ninth target feature map; the second target feature map and all the second reference feature maps are processed by the second feature fusion block to obtain the tenth target feature map, and then the ninth target feature map and the tenth target feature map are spliced ​​according to the channel and processed by the second decoding block to obtain the eleventh target feature map; the first target feature map and all the first reference feature maps are processed by the first feature fusion block to obtain the twelfth feature map, and then the eleventh target feature map and the twelfth feature map are spliced ​​according to the channel and processed by the first decoding block to obtain the supplemented highway construction real scene map; Send the supplemented highway construction real scene picture to the highway area segmentation layer for processing, and output the corresponding highway area picture; In the feature fusion block, perform the following operations: The feature maps input into the feature fusion block are respectively recorded as target feature maps and reference feature maps, the target feature maps include the first target feature map, the second target feature map, the third target feature map, the fourth target feature map and the fifth target feature map, and the reference feature maps include the first reference feature map, the second reference feature map, the third reference feature map and the fourth reference feature map; The feature map output by the feature fusion block is recorded as the output feature map, and the feature map output by the feature fusion block referred to here includes the sixth target feature map, the eighth target feature map, the tenth target feature map and the twelfth target feature map; The feature fusion block includes a reference region change matrix generation layer and a feature fusion layer; The i-th reference feature map and the target feature map are sent to the reference region change matrix generation layer for processing, and the reference region change matrix W corresponding to the i-th reference feature map is output. i , i=1,2,3,…,I, I is the total number of highway construction real scene pictures in the neighbor real scene picture set; In the feature fusion layer, the following formula is used for processing: , where U is the output feature map, σ() is the nonlinear activation operation, F is the target feature map, and H i is the reference feature map; The i-th reference feature map and the target feature map are sent to the reference region change matrix generation layer for processing, and the reference region change matrix W corresponding to the i-th reference feature map is output. i , specifically including the following steps: The i-th reference feature map is fully connected to obtain the i-th reference feature vector, and the shooting data is added to the end of the i-th reference feature vector to construct the i-th vector to be analyzed; the target feature map is fully connected to obtain the target feature vector; the i-th vector to be analyzed and the target feature vector are concatenated and sent to the multi-layer perceptron for processing, and the reference area change matrix W corresponding to the i-th reference feature map is output. i .

2. A highway construction whole process information management method according to claim 1, characterized in that: Training the highway area segmentation model includes the following steps: A number of highway area segmentation training samples are obtained, wherein the highway area segmentation training samples include N+1 highway construction real scene pictures, and the N+1 highway construction real scene pictures include a target highway construction real scene picture and N reference highway construction real scene pictures, and the highway area segmentation training samples are annotated through standard highway construction real scene pictures and highway area pictures, and all annotated highway area segmentation training samples are composed of a highway area segmentation training set, and the highway area segmentation model is trained through the highway area segmentation training set, and a comprehensive loss value is calculated, wherein the comprehensive loss value includes a loss value between the predicted supplemented highway construction real scene picture and the annotated standard highway construction real scene picture and a loss value between the predicted highway area picture and the annotated highway area picture, and the loss value between the predicted supplemented highway construction real scene picture and the annotated standard highway construction real scene picture and the loss value between the predicted highway area picture and the annotated highway area picture are obtained by weighted summation, and it is judged whether the comprehensive loss value is within a preset range, and if the comprehensive loss value is within the preset range, the trained highway area segmentation model is output; otherwise, the highway area segmentation model is continuously trained through the highway area segmentation training set.

3. A highway construction whole process information management method according to claim 2, characterized in that: All highway area pictures are matched with the corresponding standard highway area pictures according to the shooting data, and the highway construction progress is calculated, which specifically includes the following steps: calculating the matching value between each highway area picture according to the shooting data and the corresponding standard highway area picture, calculating the ratio of the sum of the matching values ​​corresponding to all highway area pictures to M, and recording it as the highway construction progress, where M is the total number of shooting data.

4. A highway construction whole process information management method according to claim 3, characterized in that: The standard highway area picture is obtained in the following manner: obtain a highway design three-dimensional model, set a virtual camera in the three-dimensional space corresponding to the highway design three-dimensional model according to the shooting data, and obtain the standard highway area picture taken by the virtual camera.

5. A highway construction whole process information management system, characterized in that: The system is a method for information management of the entire process of highway construction as described in any one of claims 1 to 4, comprising: The highway construction real scene picture acquisition module is used to acquire several highway construction real scene pictures during the highway construction process and record the shooting data corresponding to each highway construction real scene picture, the shooting data including the shooting location and shooting direction; The highway area segmentation module is used to traverse all highway construction real scene pictures, and for the selected highway construction real scene picture, perform the following operations: record the selected highway construction real scene picture as the target highway construction real scene picture, calculate the distance between the shooting location of all highway construction real scene pictures except the target highway construction real scene picture and the shooting location of the target highway construction real scene picture, and record it as the target distance, and arrange all highway construction real scene pictures except the target highway construction real scene picture from large to small according to the corresponding target distance, and form the first N highway construction real scene pictures after arrangement into a neighbor real scene picture set, and record the highway construction real scene pictures in the neighbor real scene picture set as reference highway construction real scene pictures, and then send the target highway construction real scene picture and its corresponding neighbor real scene picture set into the highway area segmentation model for processing, and output the corresponding highway area picture; until all highway construction real scene pictures are traversed, the highway area pictures corresponding to all highway construction real scene pictures are output; The highway construction progress calculation module is used to match all highway area pictures with corresponding standard highway area pictures according to the shooting data, calculate the highway construction progress, and record the highway construction progress; The highway area segmentation model is established based on the U-net network, including five encoding blocks, respectively recorded as the first encoding block, the second encoding block, the third encoding block, the fourth encoding block and the fifth encoding block, four decoding blocks, respectively recorded as the first decoding block, the second decoding block, the third decoding block and the fourth decoding block, four feature fusion blocks, respectively recorded as the first feature fusion block, the second feature fusion block, the third feature fusion block and the fourth feature fusion block, and a highway area segmentation layer; wherein the encoding block is used to perform feature extraction operations, and the decoder is used to perform feature reconstruction operations; the feature fusion block is used to fuse the features extracted from the target highway construction real scene picture by referring to the features extracted from the highway construction real scene picture; the highway area segmentation layer is used to segment the highway area corresponding to the target highway construction real scene picture, and output the corresponding highway area picture.

Images