Road surface rolling method and device based on situation awareness and electronic equipment

Abstract

The embodiment of the invention discloses a road surface rolling method and device based on situation awareness and electronic equipment. A specific embodiment of the method comprises the following steps: collecting point location information corresponding to each monitoring point location in a monitoring point location set when the road roller travels along an initial rolling path; for each piece of point location information in the point location information set, determining road surface state information according to a pre-trained road surface state recognition model and the first image, the first echo signal, the second image and the second echo signal included in the point location information; screening out the monitoring point positions of which the corresponding road surface state information meets the screening condition from the monitoring point position set, and taking the monitoring point positions as to-be-rolled point positions; according to the to-be-rolled point position set, a secondary rolling route is generated; and controlling the road roller to advance along the secondary rolling path so as to carry out secondary rolling on the to-be-rolled point. According to the embodiment, defect positioning and repeated rolling can be performed in real time in the pavement rolling stage, so that the pavement quality is ensured.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the fields of engineering construction and computer technology, and in particular to a road compaction method, device and electronic equipment based on situation awareness. Background Art

[0002] A road roller is an engineering machine that compacts the road surface by using its own weight or vibration to generate additional downforce. In practice, the compaction quality of a road roller seriously affects the subsequent life cycle of the road surface. Currently, visual inspection is usually used to check the compaction quality.

[0003] However, when the above method is adopted, the following technical problems often occur: when the construction area is large, the inspection efficiency is low by visual inspection, and it is difficult to effectively identify defects such as pavement cavities.

[0004] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Summary of the invention

[0005] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0006] Some embodiments of the present disclosure propose a road compaction method, device, and electronic device based on situational awareness to solve the technical problems mentioned in the above background technology section.

[0007] In a first aspect, some embodiments of the present disclosure provide a road rolling method based on situation awareness, the method comprising: collecting point information corresponding to each monitoring point in a monitoring point set when a roller is moving along an initial rolling route, and obtaining a point information set, wherein the point information comprises: point position, a first image, a first echo signal, a second image and a second echo signal, the first image and the first echo signal are respectively a road surface image and a road surface echo signal of the point corresponding to the point information when the roller is not rolling, and the second image and the second echo signal are respectively a road surface image and a road surface echo signal of the point corresponding to the point information after the roller is rolling; for each point information in the above point information set, according to a pre-trained road state recognition The identification model, the first image, the first echo signal, the second image and the second echo signal included in the above-mentioned point information are used to determine the road surface condition information, wherein the road surface condition information includes: road surface compactness and a defect information set, and the defect information includes: defect location and defect type; the monitoring points whose corresponding road surface condition information meets the screening conditions are screened out from the above-mentioned monitoring point set as the points to be rolled, and a set of points to be rolled is obtained, wherein the screening conditions are: the road surface compactness included in the road surface condition information is less than or equal to the preset road surface compactness or the defect information set included in the road surface condition information is not empty; according to the above-mentioned set of points to be rolled, a secondary rolling route is generated; the above-mentioned roller is controlled to move along the above-mentioned secondary rolling route to perform secondary rolling on the points to be rolled.

[0008] In a second aspect, some embodiments of the present disclosure provide a road rolling device based on situation awareness, the device comprising: a collection unit, configured to collect point information corresponding to each monitoring point in a monitoring point set when the roller is moving along an initial rolling route, to obtain a point information set, wherein the point information comprises: a point position, a first image, a first echo signal, a second image and a second echo signal, the first image and the first echo signal are respectively a road surface image and a road surface echo signal of the point corresponding to the point information when the roller has not rolled, and the second image and the second echo signal are respectively a road surface image and a road surface echo signal of the point corresponding to the point information after the roller has rolled; a determination unit, configured to, for each point information in the above point information set, identify the road surface state according to a pre-trained road surface state recognition method. The model comprises the first image, the first echo signal, the second image and the second echo signal included in the point information, and determines the road surface condition information, wherein the road surface condition information comprises: road surface compactness and a defect information set, and the defect information comprises: defect location and defect type; a screening unit is configured to screen out monitoring points whose corresponding road surface condition information meets the screening conditions from the above-mentioned monitoring point set as points to be rolled, and obtain a set of points to be rolled, wherein the screening conditions are: the road surface compactness included in the road surface condition information is less than or equal to the preset road surface compactness or the defect information set included in the road surface condition information is not empty; a secondary rolling route is generated according to the above-mentioned set of points to be rolled; a control unit is configured to control the above-mentioned roller to move along the above-mentioned secondary rolling route to perform secondary rolling on the points to be rolled.

[0009] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.

[0010] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.

[0011] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: through the road surface rolling method based on situational awareness of some embodiments of the present disclosure, the defect position can be efficiently and accurately located and repeated rolling can be performed during the road surface rolling stage to ensure the road surface quality. Specifically, first, the point information corresponding to each monitoring point in the monitoring point set when the roller moves along the initial rolling route is collected to obtain a point information set, wherein the point information includes: point position, first image, first echo signal, second image and second echo signal, the first image and the first echo signal are respectively the road surface image and the road surface echo signal of the point corresponding to the point information when the roller is not rolling, and the second image and the second echo signal are respectively the road surface image and the road surface echo signal of the point corresponding to the point information after the roller is rolling. By planning the initial rolling route, the roller can roll the rolling area without dead angles as much as possible. At the same time, the images and echo signals before and after rolling are collected respectively as the data basis for subsequent defect positioning. Secondly, for each point information in the above point information set, the road surface state information is determined according to the pre-trained road surface state recognition model, the first image, the first echo signal, the second image and the second echo signal included in the above point information, wherein the road surface state information includes: road surface compactness and defect information set, and the defect information includes: defect location and defect type. In this way, the defect location and rolling quality (road surface compactness characterization) in the road surface after rolling are determined by combining the image and the echo signal. Next, the monitoring points whose corresponding road surface state information meets the screening conditions are screened out from the above monitoring point set as the points to be rolled, and the set of points to be rolled is obtained, wherein the screening conditions are: the road surface compactness included in the road surface state information is less than or equal to the preset road surface compactness or the defect information set included in the road surface state information is not empty; according to the above set of points to be rolled, a secondary rolling route is generated. Finally, the above roller is controlled to move along the above secondary rolling route to perform secondary rolling on the points to be rolled. By screening out monitoring points that meet the screening conditions and re-planning the secondary rolling route, the points that need secondary rolling can be repeatedly rolled to eliminate road defects and improve road compaction, thereby ensuring road quality. In this way, defects can be located in real time during the road rolling stage and repeated rolling can be performed to ensure road quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0013] Figure 1 is a flow chart of some embodiments of the road surface compaction method based on situation awareness according to the present disclosure; Figure 2 It is a schematic diagram of the positional relationship between the initial rolling route and the monitoring points; Figure 3 It is a schematic diagram of the process of collecting point information; Figure 4 It is a schematic diagram of the scenarios corresponding to different defect types; Figure 5 It is a schematic diagram of the generation process of road surface state information; Figure 6 It is a position relationship diagram between the points to be rolled and the secondary rolling route; Figure 7 It is a schematic diagram of the graph structure of an undirected graph; Figure 8 is a schematic structural diagram of some embodiments of a road surface rolling device based on situation awareness according to the present disclosure; Fig. 9 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0014] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0015] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0016] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0017] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0018] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0019] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0020] refer to Figure 1 , shows a process 100 of some embodiments of the road surface rolling method based on situation awareness according to the present disclosure. The road surface rolling method based on situation awareness includes the following steps: Step 101 : collecting point information corresponding to each monitoring point in a monitoring point set when the roller travels along an initial rolling route, to obtain a point information set.

[0021] In some embodiments, the execution subject (for example, a computing device) of the situational awareness-based road rolling method can collect point information corresponding to each monitoring point in the monitoring point set when the roller moves along the initial rolling route to obtain a point information set. The point information includes: point position, a first image, a first echo signal, a second image, and a second echo signal. The first image and the first echo signal are respectively the road image and the road echo signal of the point corresponding to the point information when the roller is not rolling. The second image and the second echo signal are respectively the road image and the road echo signal of the point corresponding to the point information after the roller is rolling. The initial rolling route is the initial route of the roller for the rolling area. Specifically, the first image and the second image can be obtained by collecting a camera. The first echo signal and the second echo signal can be obtained by collecting a laser radar.

[0022] As an example, see Figure 2 The schematic diagram of the positional relationship between the initial rolling route and the monitoring points is shown in FIG. Figure 2 The initial rolling routes under two different road types are shown. Specifically, the initial rolling route is within the road boundary, and the route is set to cover the rolling area as much as possible. The monitoring points are evenly set in the rolling area. According to actual needs, the density of the monitoring points can be adaptively adjusted to meet different monitoring needs.

[0023] It should be noted that the above-mentioned computing device can be hardware or software. When the computing device is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or it can be implemented as a single server or a single terminal device. When the computing device is embodied as software, it can be installed in the hardware devices listed above. It can be implemented as multiple software or software modules for providing distributed services, for example, or it can be implemented as a single software or software module. No specific limitation is made here.

[0024] In some optional implementations of some embodiments, the execution subject collects point information corresponding to each monitoring point in the monitoring point set when the roller travels along the initial rolling route, including: The first step is to control the first image acquisition module and the first echo signal acquisition module arranged on the first side of the roller to acquire the first image and the first echo signal including the point information corresponding to the monitoring point in response to the distance value between the current position of the roller and the monitoring point being a preset distance value and the moving direction of the roller being toward the monitoring point.

[0025] In practice, the first side may be the front side of the roller. The first image acquisition module and the first echo signal acquisition module may both be arranged on the front side of the roller. Specifically, the first image acquisition module and the first echo signal acquisition module may be arranged on the front side of the rolling wheel included in the roller. The first image acquisition module may include at least one camera. The first echo signal acquisition module may include: a laser radar.

[0026] As an example, see Figure 3 The schematic diagram of the point information acquisition process shown in the figure, wherein the first image acquisition module and the first echo signal acquisition module are arranged on the front side of the rolling wheel. The first image acquisition module may include: two cameras. The first echo signal acquisition module may include: a laser radar. Specifically, when the roller is moving along the initial rolling route, when the front side of the roller is at a preset distance value from the monitoring point, the first echo signal is acquired by a laser radar included in the first echo signal acquisition module. And the first image is acquired by the two cameras included in the first image acquisition module. In particular, when the first image acquisition module includes at least two cameras, it is necessary to perform image stitching on the images acquired by the at least two cameras as the first image.

[0027] In the second step, in response to the distance value between the current position of the above-mentioned roller and the above-mentioned monitoring point being a preset distance value, and the moving direction of the above-mentioned roller being away from the above-mentioned monitoring point, the second image acquisition module and the second echo signal acquisition module arranged on the second side of the above-mentioned roller are controlled to acquire the second image and the second echo signal including the point information corresponding to the monitoring point.

[0028] In practice, the second side may be the rear side of the roller. The second image acquisition module and the second echo signal acquisition module may both be arranged at the rear side of the roller. The first image acquisition module may include at least one camera. The first echo signal acquisition module may include: a laser radar.

[0029] Specifically, by setting distance measuring sensors on the front and rear sides of the roller, it is possible to determine whether the front and rear sides of the roller are at a preset distance value from the monitoring point. In addition, in order to reduce the implementation cost and monitoring intensity, the monitoring points can be set non-repetitively and continuously. Therefore, it is only necessary to control the time interval for collecting the point information. Taking the first image and the first echo signal acquisition process as an example, when the monitoring points are continuous, the acquisition of the first image and the first echo signal included in the point information corresponding to different monitoring points is periodic acquisition, and the acquisition period of the periodic acquisition is determined by the length of the monitoring point and the travel speed of the roller. For example, acquisition period = length of the monitoring point / travel speed.

[0030] As an example, see further Figure 3 , wherein the second image acquisition module may include: two cameras. The second echo signal acquisition module may include: a laser radar. Specifically, when the roller continues to move along the initial rolling route, when the rear side of the roller is at a preset distance value from the monitoring point, a second echo signal is acquired by a laser radar included in the second echo signal acquisition module. And the second image is acquired by the two cameras included in the second image acquisition module. In particular, when the second image acquisition module includes at least two cameras, it is necessary to perform image stitching on the images acquired by the at least two cameras as the second image.

[0031] Step 102: for each point information in the point information set, determine the road surface condition information according to the pre-trained road surface condition recognition model, the first image, the first echo signal, the second image and the second echo signal included in the point information.

[0032] In some embodiments, the above-mentioned execution subject can determine the road surface state information for each point information in the point information set according to the pre-trained road surface state recognition model, the first image, the first echo signal, the second image and the second echo signal included in the point information. Among them, the road surface state recognition model is a machine learning model for identifying the road surface compactness and road surface defects of the monitoring point corresponding to the point information. The road surface state information characterizes the road surface state of the road surface corresponding to the monitoring point. The road surface state information includes: road surface compactness and defect information set. Specifically, the compactness of the road surface will change after being rolled by a roller. Therefore, the compactness of the road surface after rolling is characterized by the road surface compactness. The defect information includes: defect location and defect type. In practice, the defect types include: crack type, cavity type, and foreign body type. Among them, the crack type characterizes the presence of cracks in the road surface. The cavity type characterizes the presence of cavities inside the road surface. The foreign body type characterizes the presence of foreign bodies in the road surface. For example, the above-mentioned road surface state recognition model can be a VG-W3D model.

[0033] As an example, see Figure 4Schematic diagram of scenarios corresponding to different defect types shown, where: Figure 4 The road surface shows a crack type from a top view, a foreign object type from a top view, and a cavity type from a side view.

[0034] Optionally, the road surface state recognition model includes: a first image feature extraction network, a road surface defect position locator, a signal feature extraction network, an area of ​​interest positioning network, a second image feature extraction network and a road surface compactness classifier.

[0035] In practice, the signal feature extraction network uses PointNet as the backbone network. In particular, the signal feature extraction network removes the MaxPooling layer and the MLP layer included in the PointNet network. So that the signal feature extraction network outputs the point cloud feature map corresponding to the echo signal. Specifically, since there is a time difference in the acquisition of the first echo signal and the second echo signal, the first echo signal and the second echo signal can be sequentially input into the signal feature extraction network to obtain the point cloud feature map A corresponding to the first echo signal and the point cloud feature map B corresponding to the second echo signal. Considering that the radar wave divergence is relatively small on the rolled road surface, in principle, the signal strength of the second echo signal is greater than that of the first echo signal. Therefore, the point cloud feature map B is subtracted from the point cloud feature map A to obtain a signal heat map. In this way, the clutter (feature) interference is filtered out and the abnormal points are more prominent. Secondly, the echo signal (point cloud data) is converted into a feature map form through the signal feature extraction network, which is convenient for the subsequent positioning of the area of ​​interest through image feature processing.

[0036] In practice, considering the positioning speed, the region of interest positioning network adopts a one-stage positioning model. Specifically, the region of interest positioning network uses YOLOv3-Tiny as the backbone network. Considering that the signal feature extraction network has already performed feature extraction once, in order to avoid repeated feature extraction and further reduce the amount of data processing in the feature extraction process, the first four convolutional layers (conv) and the corresponding pooling layers (max) in the YOLOv3-Tiny network are removed. In particular, considering that the input size of the fifth convolutional layer in the YOLOv3-Tiny network is different from the size of the signal heat map, the signal heat map cannot be input into the region of interest positioning network. Therefore, the signal heat map is divided into K signal heat map blocks with the same input size as the region of interest positioning network by local segmentation, so as to perform region-by-region positioning of the region of interest.

[0037] In practice, both the first image feature extraction network and the second image feature extraction network use conventional convolutional neural network models. For example, ResNet-50 can be used as the backbone network structure. The road surface compactness classifier can be implemented using a fully connected layer to obtain the road surface compactness. Specifically, K road surface compactness can be set, so the road surface compactness classifier outputs a one-dimensional vector of 1×K.

[0038] In practice, the pavement defect location locator uses the RPN (Region Proposal Network) network as the backbone network to regress the defect location. At the same time, the pavement defect location locator also includes a classifier to output the defect type.

[0039] The above-mentioned road surface state recognition model, as an inventive point of the present disclosure, realizes real-time recognition of road surface defects and compactness by combining images and echo signals, thereby improving the quality of road surface construction.

[0040] In some optional implementations of some embodiments, the execution subject determines the road surface state information according to a pre-trained road surface state recognition model, the first image, the first echo signal, the second image, and the second echo signal included in the point information, including: In the first step, the second image is used as an image reference to flip the first image to obtain a flipped first image.

[0041] In practice, see Figure 3 The acquisition of the first image and the second image is symmetrical with the detection point as the center. From the imaging principle, the positions corresponding to the same pixel coordinates actually correspond to different image contents, so the image needs to be flipped.

[0042] The second step is to extract the image feature points included in the second image as the first image feature points.

[0043] In practice, the first image acquisition module and the second image acquisition module can be calibrated in advance. During the construction of the road roller, the construction environment is complex, so the first image acquisition module and the second image acquisition module may become loose or shifted due to mechanical vibration. Therefore, it is necessary to align the images in combination with feature point matching to improve robustness. Specifically, the SUFT (Speeded Up Robust Features) algorithm can be used to extract feature points.

[0044] The third step is to extract the image feature points included in the inverted first image as the second image feature points.

[0045] In practice, feature points can be extracted using the SUFT (Speeded Up Robust Features) algorithm.

[0046] In the fourth step, the second image and the inverted first image are aligned according to the feature points of the first image and the feature points of the second image to obtain an aligned first image and an aligned second image.

[0047] In practice, the second image and the inverted first image may be aligned by feature matching to obtain the aligned first image and the aligned second image.

[0048] The fifth step is to determine the image difference between the aligned second image and the aligned first image as a difference image.

[0049] In practice, after alignment, the (reference) feature points in the aligned first image and the (reference) feature points in the aligned second image are used as alignment positions, and the image difference between the aligned second image and the aligned first image is determined as a difference image.

[0050] In the sixth step, a signal heat map is generated through the signal feature extraction network, the first echo signal and the second echo signal.

[0051] As an example, see Figure 5 Schematic diagram of the generation process of road state information shown in FIG. 1 , wherein, since there is a time difference between the first echo signal 501 and the second echo signal 502 during the acquisition process, the first echo signal 501 can be first input into the signal feature extraction network to obtain the point cloud feature map A corresponding to the first echo signal 501. Then, the second echo signal 502 is first input into the signal feature extraction network to obtain the point cloud feature map B corresponding to the second echo signal 502. The difference between the point cloud feature map B and the point cloud feature map A is used as the signal heat map 503.

[0052] Step 7: Determine the region of interest based on the signal heat map and the region of interest positioning network.

[0053] As an example, see further Figure 5 , wherein the execution subject uses the signal heat map as the input of the region of interest positioning network to obtain the region of interest 504 where defects may occur in the monitoring point. In particular, before the signal heat map is input into the region of interest positioning network, the signal heat map is divided into K signal heat map blocks with the same input size as the region of interest positioning network by local segmentation, so as to locate the region of interest region by region.

[0054] In the eighth step, according to the above-mentioned region of interest, the difference image is preprocessed to obtain a preprocessed image.

[0055] In practice, directly setting the pixels outside the region of interest to 0 can reduce the amount of data processing, but the boundary features at the boundary of the region of interest will be lost. This may affect the positioning accuracy. Therefore, the above-mentioned execution subject can keep the pixel value unchanged at the position framed by the region of interest in the difference image, and reduce the corresponding pixel value of the pixel value outside the region of interest in a decreasing manner according to the distance of the pixel from the region of interest. In this way, the corresponding pixel value of the pixel far from the region of interest can be made to approach 0. At the same time, the image features in the region of interest are more prominent.

[0056] In the ninth step, a first image feature map is generated through the first image feature extraction network and the preprocessed image.

[0057] As an example, see further Figure 5 , wherein the first image feature extraction network takes the preprocessed image 505 as input to perform image feature extraction and obtain a first image feature map.

[0058] The tenth step is to generate a defect information set included in the road surface status information based on the first image feature map and the road surface defect position locator.

[0059] As an example, see further Figure 5 , where the defect information set 506 may include: {[defect A; P: (x1, y1); T: 01], [defect B; P: (x2, y2); T: 10]}. Wherein, "P: (x1, y1)" represents the pixel coordinates corresponding to the defect position where defect A is located. "P: (x2, y2)" represents the pixel coordinates corresponding to the defect position where defect B is located. "T: 01" represents the defect type corresponding to defect A. "T: 10" represents the defect type corresponding to defect B. In practice, "01" may correspond to a crack type. "10" may correspond to a foreign body type. "11" may correspond to a void type.

[0060] In the eleventh step, a second image feature map is generated based on the second image feature extraction network and the aligned second image.

[0061] As an example, see further Figure 5 , wherein the above-mentioned execution subject can extract image features from the aligned image 507 through a second image feature extraction network to obtain a second image feature map.

[0062] In the twelfth step, the road surface compactness included in the road surface state information is generated based on the second image feature map, the signal heat map and the road surface compactness classifier.

[0063] As an example, see further Figure 5, wherein the execution subject can extract image features from the aligned image 507 through a second image feature extraction network to obtain a second image feature map. In particular, before the second image feature map and the signal heat map are input into the road surface compactness classifier, it is necessary to unify the feature sizes of the second image feature map and the signal heat map, and perform feature superposition after unification as the input of the road surface compactness classifier. The road surface compactness classifier can be implemented using a fully connected layer to obtain the road surface compactness. Specifically, K road surface compactnesses can be set, so the road surface compactness classifier outputs a one-dimensional vector of 1×K. For example, Figure 5 The road surface firmness in 508 is shown as S.

[0064] Step 103, selecting monitoring points whose corresponding road surface status information meets the selection conditions from the monitoring point set as points to be rolled, and obtaining a set of points to be rolled.

[0065] In some embodiments, the execution subject can filter out the monitoring points whose corresponding road surface status information meets the filtering conditions from the monitoring point set as the points to be rolled, and obtain the set of points to be rolled. The filtering conditions are: the road surface compactness included in the road surface status information is less than or equal to the preset road surface compactness or the defect information set included in the road surface status information is not empty; according to the above set of points to be rolled, a secondary rolling route is generated. In practice, a single rolling cannot effectively guarantee that all points in the rolling area are rolled in place. If the rolling is repeated directly, the construction time will be increased. Therefore, it is necessary to combine the road surface status information to determine a new rolling route (secondary rolling route) to accurately roll the abnormal points.

[0066] Step 104, generating a secondary rolling route according to the set of points to be rolled.

[0067] In some embodiments, the execution subject can generate a secondary rolling route according to the set of points to be rolled. In practice, first, a directed graph including the set of points to be rolled can be constructed according to the positions of the points included in the points to be rolled. Then, path planning is performed by, for example, the Dijkstra algorithm to obtain the secondary rolling route.

[0068] In some optional implementations of some embodiments, the execution subject generates a secondary rolling route according to the set of points to be rolled, including: The first step is to generate a basic mesh based on the vehicle information of the above road roller.

[0069] The vehicle information includes: vehicle width and vehicle length. In practice, the grid length of the basic grid can be the vehicle length. The grid width of the basic grid is the vehicle width.

[0070] The second step is to project the points to be rolled in the above set of points to be rolled to the rolling area to obtain the projected rolling area.

[0071] As an example, see Figure 6 The position relationship diagram between the points to be rolled and the secondary rolling route is shown, wherein the points to be rolled correspond to the point positions, and therefore can be projected to the rolling area to obtain the projected area.

[0072] The third step is to use the basic grid as the grid unit to grid the projected rolling area to obtain the area to be rolled after grid division.

[0073] As an example, Figure 6 The area to be rolled after grid division is shown. Specifically, the projected rolling area is divided into 4×10, a total of 40 grid units, with the basic grid as the grid unit.

[0074] The fourth step is to generate a local rolling route for each layer in the area to be rolled after the above grid division along the road direction corresponding to the rolling area, and obtain a local rolling route set.

[0075] Among them, each local rolling route in the local rolling route set is parallel to the above-mentioned road direction, contains at least one to-be-rolled point in the local rolling route, and any two local rolling routes in the same layer are at least K basic grids apart, where K is 2. For example, any two local rolling routes in the same horizontal layer are at least K basic grids apart. By setting K, there is space for the roller to turn around between two local rolling routes.

[0076] As an example, see further Figure 6 , wherein, for any point to be rolled, a horizontal search is conducted for points to be rolled that are less than 2 basic grid lengths apart, and a horizontal connection is formed between at least one point to be rolled that is less than 2 basic grid lengths apart and located in the same layer as a local rolling route.

[0077] The fifth step is to use the current position of the above-mentioned roller as the starting position and generate a candidate rolling route set based on the above-mentioned local rolling route set.

[0078] Among them, the candidate rolling routes in the candidate rolling route set all cover the above-mentioned local rolling route set.

[0079] As an example, see Figure 7 The diagram of the graph structure of the undirected graph shown in Figure 6 Simplified to Figure 7The undirected graph shown in the figure. Each graph node in the undirected graph corresponds to a local rolling route. There is a graph edge between any two graph nodes corresponding to local rolling routes. Because the candidate rolling routes all cover the above local rolling route set, the candidate rolling routes are Hamiltonian paths. Then the number of Hamiltonian paths (candidate rolling paths) contained in the undirected graph is equal to the factorial of the number of nodes in the undirected graph. Figure 7 The undirected graph shown in the figure includes 6 nodes, so the candidate rolling route set includes 720 candidate rolling routes. Therefore, the above execution subject can determine the local rolling route set by Hamiltonian path traversal. It can be found that as the number of graph nodes increases, the number of Hamiltonian paths increases factorially. Considering that in the candidate rolling route planning process, the existence of leapfrogging routes should be avoided, for example Figure 7 Therefore, when there are dotted graph edges in the Hamiltonian path, the Hamiltonian path is directly abandoned and no longer used as a candidate rolling route, thereby reducing the number of traversals.

[0080] The sixth step is to determine the proportion of blank grids corresponding to each candidate rolling route in the candidate rolling route set.

[0081] Among them, the blank grid ratio represents the ratio of the basic grids that the candidate rolling route passes through and does not contain the points to be rolled to the basic grids that the candidate rolling route passes through.

[0082] As an example, Figure 6 Take the secondary rolling route (candidate rolling route) shown in the figure as an example, where the secondary rolling route passes through 27 basic grids. Among them, 9 are basic grids that do not contain the points to be rolled. Therefore, Figure 6 The blank grids corresponding to the candidate rolling routes shown account for 1 / 3.

[0083] In the sixth step, according to the proportion of blank grids corresponding to the candidate rolling routes, a route recommendation value corresponding to each candidate rolling route in the candidate rolling route set is generated.

[0084] In practice, the route recommendation value = 1 / (A1 × blank grid ratio + A2 × route length). A1 and A2 are weights. The route length is the route length of the candidate rolling route, which can be represented by the number of basic grids passed through. Specifically, the lower the blank grid ratio and the shorter the route length, the smaller the value of A1 × blank grid ratio + A2 × route length. Therefore, the inverse is taken as the route recommendation value. That is, the lower the blank grid ratio and the shorter the route length, the higher the corresponding route recommendation value.

[0085] In the seventh step, a candidate rolling route whose corresponding route recommendation value meets the route screening condition is selected from the above-mentioned candidate rolling route set as the above-mentioned secondary rolling route.

[0086] Among them, the route screening condition is: the route recommendation value is the maximum value.

[0087] Step 105, controlling the roller to move along the secondary rolling route to perform secondary rolling on the points to be rolled.

[0088] In some embodiments, the execution subject may control the roller to move along the secondary rolling route to perform secondary rolling on the points to be rolled. In practice, during the secondary rolling process, steps 101 to 105 may be repeatedly executed until the number of points to be rolled in the selected set of points to be rolled is less than the preset number of points to be rolled (the minimum number of abnormal points that meet the engineering quality).

[0089] Optionally, the above method further includes: The first step is to determine the number of rolling times corresponding to each basic grid in the projected rolling area according to the initial rolling route and the secondary rolling route.

[0090] In practice, since both the initial rolling route and the above-mentioned secondary rolling route pass through the basic grid, the number of times each basic grid is rolled can be counted by means of statistics.

[0091] The second step is to generate a heat map of the rolling area corresponding to the above-projected rolling area according to the rolling times corresponding to the basic grid.

[0092] In practice, the higher the number of rolling times, the higher the thermal value of the corresponding base grid in the rolling thermal map.

[0093] The third step is to visualize the above rolling area heat map on the remote visualization terminal.

[0094] In practice, the remote visualization terminal may be a visualization terminal for remotely monitoring the progress of construction. Specifically, the rolling area heat map may be synchronized to the remote visualization terminal in real time through a wired connection or a wireless connection.

[0095] Optionally, the above method further includes: The first step is to load a local map of the location of the above-mentioned roller on the remote visualization terminal.

[0096] In practice, you can call Amap's map interface to load a local map of the roller's location.

[0097] The second step is to determine the real-time positioning position corresponding to the above-mentioned roller.

[0098] In practice, when the position of the roller changes, the real-time positioning position of the roller can be determined by GPS (Global Positioning System).

[0099] The third step is to mark the above real-time positioning position in the above local map in real time.

[0100] In practice, the real-time positioning position is marked to display the changes in the roller's working route in a visual way.

[0101] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: through the road surface rolling method based on situational awareness of some embodiments of the present disclosure, the defect position can be efficiently and accurately located and repeated rolling can be performed during the road surface rolling stage to ensure the road surface quality. Specifically, first, the point information corresponding to each monitoring point in the monitoring point set when the roller moves along the initial rolling route is collected to obtain a point information set, wherein the point information includes: point position, first image, first echo signal, second image and second echo signal, the first image and the first echo signal are respectively the road surface image and the road surface echo signal of the point corresponding to the point information when the roller is not rolling, and the second image and the second echo signal are respectively the road surface image and the road surface echo signal of the point corresponding to the point information after the roller is rolling. By planning the initial rolling route, the roller can roll the rolling area without dead angles as much as possible. At the same time, the images and echo signals before and after rolling are collected respectively as the data basis for subsequent defect positioning. Secondly, for each point information in the above point information set, the road surface state information is determined according to the pre-trained road surface state recognition model, the first image, the first echo signal, the second image and the second echo signal included in the above point information, wherein the road surface state information includes: road surface compactness and defect information set, and the defect information includes: defect location and defect type. In this way, the defect location and rolling quality (road surface compactness characterization) in the road surface after rolling are determined by combining the image and the echo signal. Next, the monitoring points whose corresponding road surface state information meets the screening conditions are screened out from the above monitoring point set as the points to be rolled, and the set of points to be rolled is obtained, wherein the screening conditions are: the road surface compactness included in the road surface state information is less than or equal to the preset road surface compactness or the defect information set included in the road surface state information is not empty; according to the above set of points to be rolled, a secondary rolling route is generated. Finally, the above roller is controlled to move along the above secondary rolling route to perform secondary rolling on the points to be rolled. By screening out monitoring points that meet the screening conditions and re-planning the secondary rolling route, the points that need secondary rolling can be repeatedly rolled to eliminate road defects and improve road compaction, thereby ensuring road quality. In this way, defects can be located in real time during the road rolling stage and repeated rolling can be performed to ensure road quality. Further references Figure 8 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a road surface rolling device based on situation awareness. These device embodiments are similar to Figure 1Corresponding to the method embodiments shown, the road surface rolling device based on situation awareness can be specifically applied to various electronic devices.

[0102] like Figure 8 As shown, the road rolling device 800 based on situation awareness in some embodiments includes: a collection unit 801, a determination unit 802, a screening unit 803, a generation unit 804 and a control unit 805. The collection unit 801 is configured to collect point information corresponding to each monitoring point in the monitoring point set when the roller moves along the initial rolling route, and obtain a point information set, wherein the point information includes: point position, a first image, a first echo signal, a second image and a second echo signal, the first image and the first echo signal are respectively the road image and the road echo signal of the point corresponding to the point information when the roller is not rolling, and the second image and the second echo signal are respectively the road image and the road echo signal of the point corresponding to the point information after the roller is rolling; the determination unit 802 is configured to, for each point information in the above point information set, according to a pre-trained road state recognition model, the first image, the first echo signal included in the above point information, and the road state recognition model. , the second image and the second echo signal, to determine the road surface state information, wherein the road surface state information includes: road surface compactness and defect information set, and the defect information includes: defect location and defect type; the screening unit 803 is configured to screen out the monitoring points whose corresponding road surface state information meets the screening conditions from the above-mentioned monitoring point set as the points to be rolled, and obtain the set of points to be rolled, wherein the screening conditions are: the road surface compactness included in the road surface state information is less than or equal to the preset road surface compactness or the defect information set included in the road surface state information is not empty; the generating unit 804 is configured to generate a secondary rolling route according to the above-mentioned set of points to be rolled; the control unit 805 is configured to control the above-mentioned roller to move along the above-mentioned secondary rolling route to perform secondary rolling on the points to be rolled. It can be understood that the units recorded in the situation awareness-based road rolling device 800 are the same as those in the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the road surface compaction device 800 based on situation awareness and the units contained therein, and will not be described in detail here. Reference below Fig. 9 , which shows a schematic diagram of the structure of an electronic device (eg, a computing device) suitable for implementing some embodiments of the present disclosure. Fig. 9 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure. Fig. 9As shown, the computer device includes a processor, a memory and a network interface connected via a system bus, wherein the memory may include a non-volatile storage medium and an internal memory. The non-volatile storage medium may store an operating system and a computer program. The computer program includes program instructions, which, when executed, may enable the processor to execute any of the above methods. The processor is used to provide computing and control capabilities to support the operation of the entire computer device. The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium, which, when executed by the processor, may enable the processor to execute any of the above methods. The network interface is used for network communications, such as sending assigned tasks, etc. Those skilled in the art will appreciate that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of the present disclosure, and does not constitute a limitation on the computer device to which the scheme of the present disclosure is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0103] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0104] In one embodiment, the processor is used to run a computer program stored in a memory to implement the following steps: collecting point information corresponding to each monitoring point in a monitoring point set when the roller is moving along an initial rolling route to obtain a point information set, wherein the point information includes: point position, a first image, a first echo signal, a second image, and a second echo signal, wherein the first image and the first echo signal are respectively a road surface image and a road surface echo signal of the point corresponding to the point information when the roller is not rolling, and the second image and the second echo signal are respectively a road surface image and a road surface echo signal of the point corresponding to the point information after the roller is rolling; for each point information in the point information set, according to the pre-trained road state Identify the model, the first image, the first echo signal, the second image and the second echo signal included in the above-mentioned point information, and determine the road surface condition information, wherein the road surface condition information includes: road surface compactness and a defect information set, and the defect information includes: defect location and defect type; filter out the monitoring points whose corresponding road surface condition information meets the filtering conditions from the above-mentioned monitoring point set as the points to be rolled, and obtain the set of points to be rolled, wherein the filtering conditions are: the road surface compactness included in the road surface condition information is less than or equal to the preset road surface compactness or the defect information set included in the road surface condition information is not empty; generate a secondary rolling route according to the above-mentioned set of points to be rolled; control the above-mentioned roller to move along the above-mentioned secondary rolling route to perform secondary rolling on the points to be rolled.

[0105] An embodiment of the present disclosure also provides a computer-readable storage medium, on which a computer program is stored. The computer program includes program instructions. The method implemented when the program instructions are executed can refer to the various embodiments of the road compaction method based on situation awareness disclosed in the present disclosure.

[0106] The computer-readable storage medium may be an internal storage unit of the computer device described in the above embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., provided on the computer device.

[0107] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0108] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A road surface rolling method based on situation awareness, characterized in that: include: Collecting point information corresponding to each monitoring point in the monitoring point set when the roller travels along the initial rolling route, to obtain a point information set, wherein the point information includes: point position, a first image, a first echo signal, a second image, and a second echo signal, wherein the first image and the first echo signal are respectively a road surface image and a road surface echo signal of the point corresponding to the point information when the roller is not rolling, and the second image and the second echo signal are respectively a road surface image and a road surface echo signal of the point corresponding to the point information after the roller is rolling; For each point information in the point information set, determine the road surface state information according to the pre-trained road surface state recognition model, the first image, the first echo signal, the second image and the second echo signal included in the point information, wherein the road surface state information includes: road surface compactness and defect information set, and the defect information includes: defect location and defect type; Selecting monitoring points whose corresponding road surface state information meets the screening conditions from the monitoring point set as points to be rolled, and obtaining a set of points to be rolled, wherein the screening conditions are: the road surface compactness included in the road surface state information is less than or equal to the preset road surface compactness or the defect information set included in the road surface state information is not empty; Generating a secondary rolling route according to the set of points to be rolled; The roller is controlled to move along the secondary rolling route to perform secondary rolling on the points to be rolled.

2. The method according to claim 1, characterized in that: The collecting of point information corresponding to each monitoring point in the monitoring point set when the roller travels along the initial rolling route includes: In response to the distance value between the current position of the roller and the monitoring point being a preset distance value, and the moving direction of the roller being toward the monitoring point, controlling a first image acquisition module and a first echo signal acquisition module disposed on a first side of the roller to acquire a first image and a first echo signal including point information corresponding to the monitoring point; In response to the distance value between the current position of the roller and the monitoring point being a preset distance value, and the moving direction of the roller being away from the monitoring point, the second image acquisition module and the second echo signal acquisition module arranged on the second side of the roller are controlled to acquire the second image and the second echo signal including the point information corresponding to the monitoring point.

3. The method according to claim 2, characterized in that The step of generating a secondary rolling route according to the set of points to be rolled includes: Generate a basic grid according to the vehicle information of the road roller; Projecting the points to be rolled in the set of points to be rolled to the rolling area to obtain the projected rolling area; Taking the basic grid as a grid unit, gridding the projected rolling area to obtain a grid-divided area to be rolled; Following the road direction corresponding to the rolling area, a local rolling route is generated for each layer in the area to be rolled after the grid division, and a local rolling route set is obtained, wherein each local rolling route in the local rolling route set is parallel to the road direction, the local rolling route contains at least one point to be rolled, and any two local rolling routes in the same layer are at least K basic grids apart, where K is 2; Taking the current position of the roller as the starting position, generating a candidate rolling route set according to the local rolling route set, wherein the candidate rolling routes in the candidate rolling route set all cover the local rolling route set; For each candidate rolling route in the candidate rolling route set, determining the blank grid ratio corresponding to the candidate rolling route, wherein the blank grid ratio represents the ratio of the basic grids that the candidate rolling route passes through and does not contain the to-be-rolled points to the basic grids that the candidate rolling route passes through; Generate a route recommendation value corresponding to each candidate rolling route in the candidate rolling route set according to the proportion of blank grids corresponding to the candidate rolling routes; A candidate rolling route whose corresponding route recommendation value meets the route screening condition is screened out from the candidate rolling route set as the secondary rolling route.

4. The method according to claim 3, characterized in that The method further comprises: Determine the number of rolling times corresponding to each basic grid in the projected rolling area according to the initial rolling route and the secondary rolling route; According to the rolling times corresponding to the basic grid, a rolling area heat map corresponding to the projected rolling area is generated; The thermal map of the rolling area is visualized on a remote visualization terminal.

5. The method according to claim 4, characterized in that The method further comprises: Loading a local map of the location of the roller on a remote visualization terminal; Determine the real-time positioning position corresponding to the road roller; The real-time positioning position is marked in the local map in real time.

6. The method according to claim 5, characterized in that The road surface state recognition model includes: a first image feature extraction network, a road surface defect position locator, a signal feature extraction network, an area of ​​interest location network, a second image feature extraction network and a road surface compactness classifier; and The determining of the road surface state information according to the pre-trained road surface state recognition model and the first image, the first echo signal, the second image and the second echo signal included in the point information comprises: Taking the second image as an image reference, performing image flipping on the first image to obtain a flipped first image; extracting image feature points included in the second image as first image feature points; Extracting image feature points included in the inverted first image as second image feature points; Performing image alignment on the second image and the inverted first image according to the first image feature points and the second image feature points to obtain an aligned first image and an aligned second image; An image difference between the aligned second image and the aligned first image is determined as a difference image.

7. The method according to claim 6, characterized in that The method of determining the road surface state information according to the pre-trained road surface state recognition model and the first image, the first echo signal, the second image and the second echo signal included in the point information further includes: Generate a signal heat map through the signal feature extraction network, the first echo signal and the second echo signal; Determine a region of interest according to the signal heat map and the region of interest positioning network; According to the region of interest, performing image preprocessing on the difference image to obtain a preprocessed image; Generate a first image feature map through the first image feature extraction network and the preprocessed image; Generate a defect information set included in the road surface state information according to the first image feature map and the road surface defect position locator; Generate a second image feature map according to the second image feature extraction network and the aligned second image; The road surface compactness included in the road surface state information is generated according to the second image feature map, the signal heat map and the road surface compactness classifier.

8. A road surface rolling device based on situation awareness, characterized in that: include: The collecting unit is configured to collect point information corresponding to each monitoring point in the monitoring point set when the roller travels along the initial rolling route, and obtain a point information set, wherein the point information includes: point position, a first image, a first echo signal, a second image, and a second echo signal, wherein the first image and the first echo signal are respectively a road surface image and a road surface echo signal of the point corresponding to the point information when the roller is not rolling, and the second image and the second echo signal are respectively a road surface image and a road surface echo signal of the point corresponding to the point information after the roller is rolling; A determination unit is configured to determine the pavement state information for each point information in the point information set according to a pre-trained pavement state recognition model, the first image, the first echo signal, the second image and the second echo signal included in the point information, wherein the pavement state information includes: pavement compactness and a defect information set, and the defect information includes: defect location and defect type; A screening unit is configured to screen out monitoring points whose corresponding road surface state information meets a screening condition from the monitoring point set as points to be rolled, thereby obtaining a set of points to be rolled, wherein the screening condition is: the road surface compactness included in the road surface state information is less than or equal to a preset road surface compactness or the defect information set included in the road surface state information is not empty; A generating unit is configured to generate a secondary rolling route according to the set of points to be rolled; The control unit is configured to control the roller to move along the secondary rolling route to perform secondary rolling on the points to be rolled.

9. An electronic device, characterized in that: include: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer-readable medium, characterized in that A computer program is stored thereon, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images