Lane surface detection method, device, electronic device and computer readable medium

By constructing a three-dimensional model of the tunnel and detecting surface feature information, the problems of limited tunnel detection information and slow response in the existing technology are solved, and timely monitoring and safety improvement of the surface state of the tunnel are achieved.

CN114663357BActive Publication Date: 2024-11-05SHENHUA GRP WUDA MINING DISTRICT INFORMATION MANAGEMENT CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210193891.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-03-01
Publication Date
2024-11-05
Estimated Expiration
2042-03-01

AI Technical Summary

Technical Problem

The existing tunnel detection methods can adjust the angle when detecting the information on the tunnel surface, but the information is limited, and usually only can alarm be issued after the top rise occurs, which fails to detect potential safety hazards early, resulting in insufficient safety.

Method used

By obtaining the surface information of the tunnel, a target three-dimensional model is constructed, surface feature information is detected, including protrusions, depressions, cracks and color changes, and alarm information is generated to prevent capping accidents.

Benefits of technology

It improves the safety of tunnel operations, can promptly detect potential hidden dangers of overhead, reduces the risk of accidents, and enhances the timeliness and accuracy of detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114663357B_ABST
    Figure CN114663357B_ABST
Patent Text Reader

Abstract

The embodiments of the present disclosure disclose a tunnel surface detection method, device, electronic device and computer-readable medium. A specific implementation of the method includes: obtaining the surface information of the tunnel, the surface information includes the structural features of the tunnel; constructing a target three-dimensional model of the tunnel through the surface information; and detecting the surface feature information of the tunnel based on the target three-dimensional model. This implementation can detect the surface feature information of the tunnel through the target three-dimensional model. The surface feature information can reflect the surface state of the tunnel, and then warn of possible roof collapse accidents in the tunnel. In this way, the safety of tunnel operations is greatly improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of safety detection technology, and in particular to a tunnel surface detection method, device, electronic device, and computer-readable medium. Background Art

[0002] Laneways can be divided into surface laneways (for example, road lanes that pass through mountains) and underground lanes (for example, coal mine lanes, etc.). Roof fall is one of the main safety hazards of laneway safety. Once a roof fall accident occurs, it may cause significant casualties and economic losses. In order to prevent the occurrence of roof fall accidents, the existing laneway detection method can be to set up monitoring cameras at designated locations in the lane. The monitoring cameras can continuously detect whether there are objects falling in the lane for a long time. Once a fallen object is found in the lane, a roof fall alarm can be issued.

[0003] The existing tunnel detection method is usually fixed at a designated position. Even if the camera angle is adjustable, the tunnel information that can be detected is limited. In addition, the existing tunnel detection method usually can only issue an alarm after the tunnel roof collapses, which is not conducive to the early detection of tunnel roof collapse accidents. Summary of the invention

[0004] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0005] Some embodiments of the present disclosure propose a tunnel surface detection method, device, electronic device and computer-readable medium to solve the technical problems mentioned in the above background technology section.

[0006] In a first aspect, some embodiments of the present disclosure provide a tunnel surface detection method, the method comprising: acquiring surface information of the tunnel, the surface information including structural features of the tunnel; constructing a target three-dimensional model of the tunnel through the surface information; and detecting surface feature information of the tunnel based on the target three-dimensional model.

[0007] Optionally, the obtaining of the surface information of the lane includes: obtaining point cloud data along a designated route passing through the lane.

[0008] Optionally, the acquiring of point cloud data along the designated route passing through the lane includes: acquiring an image sequence along the designated route passing through the lane, wherein the images in the image sequence are acquired at at least one preset position on the designated route.

[0009] Optionally, the above-mentioned acquisition of the image sequence passing through the above-mentioned alley along the above-mentioned designated route includes: acquiring shooting range information of the camera in the above-mentioned alley, the above-mentioned shooting range information is used to characterize the range of the alley image captured by the above-mentioned camera in the above-mentioned alley; determining at least one preset position on the above-mentioned designated route through the above-mentioned shooting range information; and controlling the above-mentioned camera to capture the above-mentioned image sequence based on the above-mentioned at least one preset position.

[0010] Optionally, constructing the target three-dimensional model of the lane through the surface information includes: constructing an initial three-dimensional model of the lane through the point cloud data; matching the images in the image sequence on the initial three-dimensional model to obtain the target three-dimensional model.

[0011] Optionally, the above-mentioned matching of the images in the above-mentioned image sequence on the above-mentioned initial three-dimensional model to obtain the above-mentioned target three-dimensional model includes: determining the matching relationship between each image in the above-mentioned image sequence and the above-mentioned point cloud data through the above-mentioned at least one preset position; based on the above-mentioned matching relationship, matching the images in the above-mentioned image sequence on the above-mentioned initial three-dimensional model to obtain the above-mentioned target three-dimensional model.

[0012] Optionally, the above-mentioned detection of surface feature information of the above-mentioned alley based on the above-mentioned target three-dimensional model includes: querying at least one point cloud image of the surface of at least one alley in the above-mentioned initial three-dimensional model; for each point cloud image in the above-mentioned at least one point cloud image, in response to a first designated image feature corresponding to the first surface feature information, marking a first position of the above-mentioned first designated image feature in the point cloud image, and generating first warning information corresponding to the point cloud image, wherein the above-mentioned first surface feature information corresponds to at least one of the following: a protruding structure, a concave structure, and the above-mentioned first warning information includes the above-mentioned first position information.

[0013] Optionally, the above-mentioned detection of surface feature information of the above-mentioned tunnel based on the above-mentioned target three-dimensional model includes: in response to the absence of the above-mentioned first specified image feature, querying at least one surface image of the surface of at least one tunnel in the above-mentioned target three-dimensional model; for each of the above-mentioned at least one surface image, in response to the existence of a second specified image feature corresponding to the second surface feature information, marking the second position of the above-mentioned second specified image feature in the surface image, and generating second alarm information corresponding to the surface image, wherein the above-mentioned second surface feature information includes at least one of the following: a crack image, a color image, and the above-mentioned second alarm information includes the above-mentioned second position information.

[0014] Optionally, the above-mentioned detection of surface feature information of the above-mentioned alley based on the above-mentioned target three-dimensional model includes: matching the above-mentioned target three-dimensional model with a pre-stored reference three-dimensional model of the above-mentioned alley, and determining third surface feature information of the above-mentioned alley; in response to the above-mentioned third surface feature information exceeding a preset scale threshold, marking a third position corresponding to the above-mentioned third surface feature information in the above-mentioned target three-dimensional space, and generating third alarm information.

[0015] In a second aspect, some embodiments of the present disclosure provide a tunnel surface detection device, which includes: a surface information acquisition unit, configured to acquire surface information of the tunnel, wherein the surface information includes structural features of the tunnel; a target three-dimensional model construction unit, configured to construct a target three-dimensional model of the tunnel through the surface information; and a surface feature information detection unit, configured to detect surface feature information of the tunnel based on the target three-dimensional model.

[0016] Optionally, the surface information acquisition unit includes: a surface information acquisition subunit configured to acquire point cloud data passing through the lane along a designated route.

[0017] Optionally, the surface information acquisition subunit includes: a surface information acquisition module configured to acquire an image sequence passing through the lane along the designated route, wherein the images in the image sequence are acquired at at least one preset position on the designated route.

[0018] Optionally, the surface information acquisition module includes: a shooting range information acquisition submodule, configured to acquire the shooting range information of the camera in the alley, the shooting range information being used to characterize the range of the alley image captured by the camera in the alley; a preset position determination submodule, configured to determine at least one preset position on the designated route through the shooting range information; and an image sequence acquisition submodule, configured to control the camera to acquire the image sequence based on the at least one preset position.

[0019] Optionally, the target three-dimensional model construction unit includes: an initial three-dimensional model construction subunit, configured to construct an initial three-dimensional model of the tunnel through the point cloud data; and a target three-dimensional model construction subunit, configured to match the images in the image sequence on the initial three-dimensional model to obtain the target three-dimensional model.

[0020] Optionally, the target three-dimensional model construction subunit includes: a matching relationship determination module, configured to determine the matching relationship between each image in the image sequence and the point cloud data through the at least one preset position; a target three-dimensional model construction module, configured to match the images in the image sequence on the initial three-dimensional model based on the matching relationship to obtain the target three-dimensional model.

[0021] Optionally, the surface feature information detection unit includes: a point cloud image query subunit, configured to query at least one point cloud image of at least one tunnel surface in the initial three-dimensional model; a first alarm information generation subunit, configured to, for each of the at least one point cloud image, in response to the existence of a first specified image feature, mark the first position of the first specified image feature in the point cloud image, and generate first alarm information corresponding to the point cloud image, wherein the first specified image feature includes at least one of the following: a protruding structure, a concave structure, and the first alarm information includes the first position information.

[0022] Optionally, the surface feature information detection unit includes: a surface image query subunit, configured to query at least one surface image of at least one tunnel surface in the target three-dimensional model in response to the absence of the first specified image feature; a second alarm information generation subunit, configured to mark the second position of the second specified image feature in the surface image for each of the at least one surface image in response to the presence of the second specified image feature, and generate second alarm information corresponding to the surface image, wherein the second specified image feature includes at least one of the following: a crack image, a color image, and the second alarm information includes the second position information.

[0023] Optionally, the surface feature information detection unit includes: a third surface feature information determination subunit, configured to match the target three-dimensional model with a pre-stored reference three-dimensional model of the lane, and determine the third surface feature information of the lane; a third alarm information generation subunit, configured to respond to the third surface feature information exceeding a preset scale threshold, mark a third position corresponding to the third surface feature information in the target three-dimensional space, and generate third alarm information.

[0024] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.

[0025] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.

[0026] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: by detecting the surface feature information of the tunnel through the tunnel surface detection method of some embodiments of the present disclosure, the safety of tunnel operations is improved. Specifically, the reason for the low safety of tunnel operations is that the existing tunnel detection methods usually can only detect roof collapse accidents after the tunnel roof collapse occurs. Based on this, the tunnel surface detection method of some embodiments of the present disclosure first obtains the surface information of the tunnel. Since the surface information contains the structural features of the tunnel, the target three-dimensional model of the tunnel can be constructed through the surface information. Then, the surface feature information of the tunnel can be detected through the target three-dimensional model. The surface feature information can reflect the surface state of the tunnel, and then warn of the roof collapse accident that may occur in the tunnel. In this way, the safety of tunnel operations is greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0028] Figure 1 is a schematic diagram of an application scenario of a roadway surface detection method according to some embodiments of the present disclosure;

[0029] Figure 2 is a flow chart of some embodiments of the roadway surface detection method according to the present disclosure;

[0030] Figure 3 are flow charts of other embodiments of the roadway surface detection method according to the present disclosure;

[0031] Figure 4 are flow charts of further embodiments of the roadway surface detection method according to the present disclosure;

[0032] Figure 5 is a schematic structural diagram of some embodiments of the tunnel surface detection device according to the present disclosure;

[0033] Figure 6 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0034] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0035] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0036] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0037] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0038] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0039] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0040] Figure 1 It is a schematic diagram of an application scenario of the tunnel surface detection method according to some embodiments of the present disclosure.

[0041] like Figure 1 As shown, the present application involves a tunnel 101 and a user terminal 102. Among them, the user terminal 102 has the ability to scan the tunnel surface and perform data processing on the tunnel surface information. The user terminal 102 can be, for example, a device such as smart glasses. In practice, a technician can walk around in the tunnel 101 wearing the user terminal 102. The user terminal 102 can collect the surface information of the tunnel 101 through its own point cloud acquisition module, camera, etc. Then, the processor of the user terminal 102 can construct a target three-dimensional model of the tunnel 101 through the surface information, and detect the surface feature information of the tunnel 101 based on the target three-dimensional model to detect possible accidents such as roof collapse in the tunnel 101. The surface feature information can be, for example, bulges, depressions, cracks in the surface structure, and can also be abnormalities in the surface color, etc., depending on the actual situation.

[0042] Continue to refer Figure 2 , Figure 2 The process 200 of some embodiments of the tunnel surface detection method according to the present disclosure is shown. The tunnel surface detection method comprises the following steps:

[0043] Step 201, obtaining the surface information of the lane.

[0044] In some embodiments, the execution body of the roadway surface detection method (for example Figure 1 The user terminal 102 shown in the figure can obtain the surface information of the tunnel through a variety of data acquisition devices.

[0045] For example, a user can walk in the alley wearing an execution subject (such as smart glasses, etc.). The point cloud acquisition module, camera and other modules on the execution subject can collect surface information of each surface of the alley. Among them, the surface of the alley can be the top surface, side surface, etc. inside the alley. The surface information includes the structural features of the alley, which can be point cloud data, images, videos, etc. For example, the point cloud data includes the point cloud data of the alley before the technician enters the alley, then the point cloud data at this time can obtain the peripheral structural features of the alley. After the technician enters the alley, the point cloud data obtained by the execution subject can include the structural features inside the alley. Similarly, images and videos can also obtain the structural features of the alley.

[0046] Step 202: construct a target three-dimensional model of the tunnel using the surface information.

[0047] In some embodiments, since the surface information includes the structural features of the lane, the execution subject can construct a target three-dimensional model of the lane based on the surface information. In the process of constructing the target three-dimensional model, the position signals of various positions of the lane can also be obtained through ultra-wideband wireless communication technology. In this way, the target three-dimensional model constructed by the position signal can accurately describe the structure of the lane in space.

[0048] Step 203: Detect surface feature information of the lane based on the target three-dimensional model.

[0049] After obtaining the target 3D model, the execution subject can query each surface in the target 3D model to determine the surface feature information of the tunnel.

[0050] In practice, technicians can wear smart glasses and walk in tunnels and other passages in coal mines. Smart glasses can automatically obtain images of the surroundings and identify objects in the images. When there are tunnels and other construction projects that need to be detected for safety issues in the image, smart glasses can automatically obtain the surface information of the tunnel, build a target three-dimensional model of the tunnel, and detect the surface feature information of the tunnel. When the tunnel is detected to have safety hazards through surface feature information, an alarm signal can be issued in time to inform the technicians to evacuate in time. In this way, the safety of tunnel operations is greatly improved.

[0051] The above scheme provides a solution for technicians to wear smart glasses and walk in the tunnel to detect the surface of the tunnel. When the tunnel already has safety hazards, this method is undoubtedly highly dangerous. To this end, technicians can also set smart glasses on a small remote-controlled aircraft, or set point cloud acquisition modules, cameras, processors and other modules on the remote-controlled aircraft, and control the remote-controlled aircraft through the above technical solution to achieve safety detection of construction projects such as tunnels. The safety of the detection process is further improved.

[0052] The tunnel surface detection method disclosed in some embodiments of the present disclosure first obtains the surface information of the tunnel. Since the surface information contains the structural features of the tunnel, a target three-dimensional model of the tunnel can be constructed through the surface information. Then, the surface feature information of the tunnel can be detected through the target three-dimensional model. The surface feature information can reflect the surface state of the tunnel, and then warn of possible roof collapse accidents in the tunnel. In this way, the safety of tunnel operations is greatly improved.

[0053] Continue to refer Figure 3 , Figure 3 The process 300 of some embodiments of the tunnel surface detection method according to the present disclosure is shown. The tunnel surface detection method comprises the following steps:

[0054] Step 301, obtaining point cloud data along a designated route passing through the lane.

[0055] The technician can wear the execution body (for example, smart glasses) and walk along the designated route in the tunnel. The point cloud acquisition module of the execution body can collect the point cloud data of the tunnel while the technician is walking. For example, the designated route can be a straight line passing through the tunnel. In practice, various pipelines or other equipment can be set in the tunnel. These pipelines or other equipment are often accompanied by vibrations during operation, which may induce roof falls. Correspondingly, the designated route can also be a curve passing through a designated position in the tunnel. In this way, point cloud data at locations where roof falls may occur can be obtained in a targeted manner, thereby improving the timeliness and effectiveness of detecting roof falls.

[0056] In some optional implementations of some embodiments, the step of acquiring point cloud data along a designated route passing through the lane may include: acquiring an image sequence along the designated route passing through the lane.

[0057] In practice, point cloud data mainly reflects the structural features of the tunnel surface, and some possible roof falls (such as cracks) are not easy to accurately determine through point cloud data. For this reason, a corresponding image sequence can also be obtained in the process of obtaining point cloud data. Among them, the images in the above image sequence can be obtained at least one preset position on the above specified route. Assuming that the technician moves forward at a constant speed, the images in the image sequence can be images of the tunnel surface collected by the camera at a set distance or a set time.

[0058] In some optional implementations of some embodiments, the acquiring of the image sequence passing through the lane along the designated route may include the following steps:

[0059] The first step is to obtain the shooting range information of the camera in the above-mentioned alley.

[0060] In practice, due to reasons such as the wearing method of the technician, the shooting angle of the camera of the executing subject may be different each time it is worn. In order to obtain accurate and effective images, the executing subject may first obtain the shooting range information of the camera in the lane. The above shooting range information can be used to characterize the range of the lane image captured by the above camera in the above lane. For example, the executing subject can take several images in advance, then determine the area of ​​the lane surface covered by the image content, and determine the shooting range information based on the area. That is, the shooting range information can be expressed by area.

[0061] In the second step, at least one preset position on the designated route is determined through the shooting range information.

[0062] After obtaining the shooting range information, in order to avoid the situation where the tunnel surface image is missing, the execution subject can determine at least one preset position on the above-mentioned specified route through the shooting range information. That is, at least one image taken at at least one preset position includes the entire surface in the tunnel, avoiding the situation where the tunnel surface image is missing. In this way, the comprehensiveness and effectiveness of obtaining surface feature information are improved, thereby improving the accuracy of identifying tunnel roof falls.

[0063] The third step is to control the camera to collect the image sequence based on the at least one preset position.

[0064] After determining at least one preset position, the execution subject can control the camera to collect images at each preset position in the at least one preset position to obtain an image sequence of the tunnel. Since the images in the image sequence are acquired according to the specified route, the order between the images can correspond to the tunnel surfaces that appear in sequence along the specified route in the tunnel. In this way, the complexity of the subsequent construction of the target three-dimensional model is reduced, the construction speed of the target three-dimensional model is increased, and the speed of detecting tunnel roof falls is accelerated.

[0065] In some optional implementations of some embodiments, in practice, when the technician walks, his body will vibrate, or the technician may face different directions. This may easily cause the image captured by the camera to miss some surface images in the tunnel. In order to solve these problems, the execution subject may also record a video. Since the video contains a large number of continuous images, even if the technician has problems such as body vibration or facing different directions, a comprehensive image containing the tunnel surface can be selected from the video. In this way, the effectiveness of the image is improved, and the recognition accuracy of accidents such as tunnel roof collapse is improved.

[0066] Step 302: construct an initial three-dimensional model of the tunnel using the point cloud data.

[0067] In some embodiments, after obtaining the point cloud data, the execution entity may construct an initial three-dimensional model of the tunnel locally or on another device that communicates with the execution entity.

[0068] Step 303: Match the images in the image sequence on the initial three-dimensional model to obtain the target three-dimensional model.

[0069] In some embodiments, the initial three-dimensional model is constructed by point cloud data, which usually does not contain color information. The color of the tunnel surface may also be related to the roof fall. For example, water seepage occurs, which easily reduces the stability of the tunnel structure, but the surface is still complete and no stones or the like fall. To this end, the execution subject can match the images in the above-mentioned image sequence on the initial three-dimensional model to obtain the above-mentioned target three-dimensional model.

[0070] In some optional implementations of some embodiments, matching the images in the image sequence on the initial three-dimensional model to obtain the target three-dimensional model may include the following steps:

[0071] In the first step, a matching relationship between each image in the image sequence and the point cloud data is determined through the at least one preset position.

[0072] The image sequence and the point cloud data are both acquired based on the specified route, and the images in the image sequence are acquired at the preset positions of the specified route. Therefore, the execution subject can determine the matching relationship between the point cloud data and the image at each preset position. For example, the matching relationship can be which point cloud data each image in the image sequence corresponds to.

[0073] The second step is to match the images in the image sequence on the initial three-dimensional model based on the matching relationship to obtain the target three-dimensional model.

[0074] After determining the matching relationship, the execution subject can perform operations such as stretching on the images in the image sequence, and attach the operated images to the initial 3D model to obtain the target 3D model. In this way, the target 3D model can maximize the restoration of the real roadway, which is conducive to improving the effectiveness of the surface feature information of the roadway surface detection.

[0075] Step 304: query at least one point cloud image of at least one lane surface in the initial three-dimensional model.

[0076] After the initial 3D model is determined, the execution subject can query the point cloud image of each tunnel surface based on the initial 3D model. For example, if the tunnel surface is an arched surface at the top of the tunnel, the corresponding point cloud image can be an image obtained by combining the 3D points corresponding to the point cloud data corresponding to the arched surface.

[0077] Step 305, for each point cloud image of the at least one point cloud image, in response to the existence of a first designated image feature corresponding to the first surface feature information, mark the first position of the first designated image feature in the point cloud image, and generate first warning information for the corresponding point cloud image.

[0078] The execution subject may perform data processing on the point cloud image. When there is a first designated image feature corresponding to the first surface feature information in the point cloud image, the first position information of the first designated image feature in the point cloud image may be marked, and a first warning message may be generated. The first surface feature information may correspond to at least one of the following: a protruding structure, a concave structure. The first warning message may include the first position information.

[0079] For highway lanes, the inner surface of the lane is usually a flat surface (or smooth surface) completely covered by reinforced concrete. When there is a raised structure in the point cloud image, there is a high possibility of roof collapse. At this time, the execution subject can mark the first position of the raised structure in the point cloud image and generate a first warning information containing the second position of the raised structure. For coal mine lanes, the lanes are usually rough surfaces that are not completely covered by various supporting structures, and coal blocks may fall in coal mine lanes even when they are safe. In order to improve the accuracy of the alarm, the execution subject can adjust the size threshold of the raised structure and the sunken structure. Only when the coal mine lane has raised structures and sunken structures that exceed the size threshold, is it considered that there is a risk of roof collapse. It can be seen from the above description that in the process of constructing the target three-dimensional model, the initial three-dimensional model is first constructed through point cloud data, and then the target three-dimensional model is obtained through the image sequence. Therefore, the initial three-dimensional model can be used to detect the lane surface before the target three-dimensional model is constructed. In this way, surface detection of various tunnels (surface tunnels and underground tunnels) is achieved, which greatly shortens the time for issuing the first warning information and improves the efficiency of warning. At the same time, problems such as roof collapse are usually major safety accidents. By obtaining the first warning information through the initial 3D model, safety problems can be discovered early and major safety accidents can be avoided.

[0080] Step 306 : In response to the absence of the first specified image feature, query at least one surface image of at least one lane surface in the target three-dimensional model.

[0081] When the first specified image feature does not exist, the execution subject may continue to construct the target three-dimensional model. When the target three-dimensional model is constructed, at least one surface image of at least one lane surface in the target three-dimensional model may be queried.

[0082] Step 307: for each surface image of the at least one surface image, in response to the existence of a second designated image feature corresponding to the second surface feature information, mark a second position of the second designated image feature in the surface image, and generate second warning information corresponding to the surface image.

[0083] The execution subject may further detect a second designated image feature on the surface image. When the second surface feature exists, a second position of the second designated image feature may be marked in the surface image, and then a second warning message is generated. The second surface feature information includes at least one of the following: a crack image, a color image. The second warning message includes the second position information.

[0084] For example, if a crack image appears, it may be caused by cracks generated before the above-mentioned raised structure appears. At this time, the execution subject can mark the second position of the crack image in the surface image and generate a second alarm information of the second position containing the crack information. In addition, when a color image appears (for example, it may be a color change caused by redness or sweat on the roadway surface), it means that the surface image area corresponding to the color image is different from other areas of the surface image, and there is a risk of water seepage. At this time, the execution subject can mark the second position of the area where the color image is located, and generate a second alarm information of the second position of the color image.

[0085] Continue to refer Figure 4 , Figure 4 A process 400 of some embodiments of the tunnel surface detection method according to the present disclosure is shown. The tunnel surface detection method comprises the following steps:

[0086] Step 401, obtaining the surface information of the lane.

[0087] Step 402: construct a target three-dimensional model of the tunnel using the surface information.

[0088] Step 403: Detect surface feature information of the lane based on the target three-dimensional model.

[0089] The contents of step 401 and step 403 are the same as those of step 201 and step 203, and are not described in detail here.

[0090] Step 404: Match the target three-dimensional model with the pre-stored reference three-dimensional model of the lane to determine third surface feature information of the lane.

[0091] The execution subject may also match the acquired target 3D model with a pre-stored reference 3D model of the tunnel. The reference 3D model of the tunnel may be constructed when there are no safety hazards in the tunnel, and may characterize the structural features and surface features of the tunnel when it is safe. The execution subject may match the target 3D model with the reference 3D model, and set the different parts of the target 3D model and the reference 3D model as the third surface feature information. The third surface feature information characterizes the changes in the tunnel structure and surface.

[0092] Step 405: In response to the third surface feature information exceeding a preset scale threshold, a third position corresponding to the third surface feature information is marked in the target three-dimensional space, and third warning information is generated.

[0093] The above has detected the tunnel surface from specific locations such as more obvious protruding structures, concave structures, cracks or colors. In practice, there may also be larger safety hazards, such as large-scale landslides. To this end, when the execution subject detects the third surface feature information, it can also compare the third surface feature information with a preset scale threshold. Among them, the scale threshold can be the change in length, width, thickness or displacement of the surface corresponding to the third surface feature information. For example, coal blocks often fall in coal mine tunnels under safe conditions. Judging the safety of the tunnel only from the above-mentioned protruding structures and concave structures is prone to misjudgment. However, when the third surface feature information of the coal mine tunnel exceeds the preset scale threshold, it means that there is a possibility of a major accident in the tunnel as a whole. At this time, the execution subject can mark the third position corresponding to the third surface feature information in the target three-dimensional space and generate a third warning information. In this way, the overall judgment of various tunnel surfaces is achieved, and the safety of tunnel operations is improved.

[0094] Further references Figure 5 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a tunnel surface detection device. These device embodiments are similar to Figure 2 Corresponding to the method embodiments shown, the device can be specifically applied to various electronic devices.

[0095] like Figure 5 As shown, the tunnel surface detection device 500 of some embodiments includes: a surface information acquisition unit 501, a target three-dimensional model construction unit 502 and a surface feature information detection unit 503. The surface information acquisition unit 501 is configured to acquire the surface information of the tunnel, and the surface information includes the structural features of the tunnel; the target three-dimensional model construction unit 502 is configured to construct a target three-dimensional model of the tunnel through the surface information; and the surface feature information detection unit 503 is configured to detect the surface feature information of the tunnel based on the target three-dimensional model.

[0096] In an optional implementation of some embodiments, the surface information acquisition unit 501 may include: a surface information acquisition subunit (not shown in the figure), configured to acquire point cloud data passing through the lane along a designated route.

[0097] In an optional implementation of some embodiments, the surface information acquisition subunit may include: a surface information acquisition module (not shown in the figure), configured to acquire an image sequence passing through the lane along the designated route, wherein the images in the image sequence are acquired at at least one preset position on the designated route.

[0098] In an optional implementation of some embodiments, the surface information acquisition module may include: a shooting range information acquisition submodule (not shown in the figure), a preset position determination submodule (not shown in the figure), and an image sequence acquisition submodule (not shown in the figure). The shooting range information acquisition submodule is configured to acquire the shooting range information of the camera in the lane, and the shooting range information is used to characterize the lane image range captured by the camera in the lane; the preset position determination submodule is configured to determine at least one preset position on the designated route through the shooting range information; and the image sequence acquisition submodule is configured to control the camera to acquire the image sequence based on the at least one preset position.

[0099] In an optional implementation of some embodiments, the target 3D model construction unit 502 may include: an initial 3D model construction subunit (not shown in the figure) and a target 3D model construction subunit (not shown in the figure). The initial 3D model construction subunit is configured to construct the initial 3D model of the lane through the point cloud data; the target 3D model construction subunit is configured to match the image in the image sequence on the initial 3D model to obtain the target 3D model.

[0100] In an optional implementation of some embodiments, the target 3D model construction subunit may include: a matching relationship determination module (not shown in the figure) and a target 3D model construction module (not shown in the figure). The matching relationship determination module is configured to determine the matching relationship between each image in the image sequence and the point cloud data through the at least one preset position; the target 3D model construction module is configured to match the images in the image sequence on the initial 3D model based on the matching relationship to obtain the target 3D model.

[0101] In an optional implementation of some embodiments, the surface feature information detection unit 503 may include: a point cloud image query subunit (not shown in the figure) and a first warning information generation subunit (not shown in the figure). The point cloud image query subunit is configured to query at least one point cloud image of at least one lane surface in the initial three-dimensional model; the first warning information generation subunit is configured to, for each point cloud image in the at least one point cloud image, mark the first position of the first specified image feature in the point cloud image in response to the existence of the first specified image feature corresponding to the first surface feature information, and generate the first warning information corresponding to the point cloud image, wherein the first surface feature information corresponds to at least one of the following: a convex structure, a concave structure, and the first warning information includes the first position information.

[0102] In an optional implementation of some embodiments, the surface feature information detection unit 503 may include: a surface image query subunit (not shown in the figure) and a second alarm information generation subunit (not shown in the figure). The surface image query subunit is configured to query at least one surface image of at least one tunnel surface in the target three-dimensional model in response to the absence of the first specified image feature; the second alarm information generation subunit is configured to mark the second position of the second specified image feature in the surface image for each of the at least one surface image in response to the presence of the second specified image feature corresponding to the second surface feature information, and generate the second alarm information of the corresponding surface image, wherein the second surface feature information includes at least one of the following: a crack image, a color image, and the second alarm information includes the second position information.

[0103] In an optional implementation of some embodiments, the surface feature information detection unit 503 may further include: a third surface feature information determination subunit (not shown in the figure) and a third warning information generation subunit (not shown in the figure). The third surface feature information determination subunit is configured to match the target three-dimensional model with the pre-stored reference three-dimensional model of the lane to determine the third surface feature information of the lane; the third warning information generation subunit is configured to respond to the third surface feature information exceeding a preset scale threshold, mark a third position corresponding to the third surface feature information in the target three-dimensional space, and generate third warning information.

[0104] It is understood that the units described in the device 500 are similar to those described in the reference Figure 2 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the device 500 and the units included therein, and will not be described in detail here.

[0105] like Figure 6 As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0106] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 6 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0107] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0108] It should be noted that the computer-readable medium in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0109] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0110] The computer-readable medium may be included in the electronic device, or may exist independently without being installed in the electronic device. The computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device: obtains surface information of the lane, the surface information includes structural features of the lane; constructs a target three-dimensional model of the lane based on the surface information; and detects surface feature information of the lane based on the target three-dimensional model.

[0111] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0112] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0113] The units described in some embodiments of the present disclosure may be implemented by software or hardware. The units described may also be set in a processor, for example, it may be described as: a processor includes a surface information acquisition unit, a target three-dimensional model construction unit and a surface feature information detection unit. The names of these units do not constitute a limitation on the units themselves in some cases, for example, the surface feature information detection unit may also be described as a "unit for detecting feature information of the roadway surface".

[0114] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0115] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A roadway surface detection method, comprising: Acquiring surface information of the lane, wherein the surface information includes structural features of the lane; Constructing a target three-dimensional model of the lane through the surface information; Detecting surface feature information of the lane based on the target three-dimensional model; Wherein, the obtaining of the surface information of the lane includes: obtaining point cloud data passing through the lane along a designated route; The acquiring of point cloud data along a designated route passing through the lane comprises: acquiring an image sequence along the designated route passing through the lane, wherein images in the image sequence are acquired at at least one preset position on the designated route; The constructing the target three-dimensional model of the lane through the surface information includes: constructing an initial three-dimensional model of the lane through the point cloud data, matching the images in the image sequence on the initial three-dimensional model to obtain the target three-dimensional model; After the initial three-dimensional model is constructed, the method further includes: Query at least one point cloud image of at least one lane surface in the initial three-dimensional model, and for each point cloud image in the at least one point cloud image, in response to the existence of a first designated image feature corresponding to the first surface feature information, mark a first position of the first designated image feature in the point cloud image, and generate first warning information corresponding to the point cloud image, wherein the first warning information includes the first position information, and the first surface feature information corresponds to at least one of the following: a convex structure, a concave structure; And in response to the absence of the first specified image feature, query at least one surface image of at least one tunnel surface in the target three-dimensional model, and for each of the at least one surface image, in response to the existence of a second specified image feature corresponding to second surface feature information, mark the second position of the second specified image feature in the surface image, and generate second alarm information corresponding to the surface image, wherein the second alarm information includes the second position information, and the second surface feature information includes at least one of the following: a crack image and a color image.

2. The method according to claim 1, wherein: The acquiring of an image sequence passing through the lane along the designated route comprises: Acquire shooting range information of the camera in the lane, wherein the shooting range information is used to represent the range of the lane image captured by the camera in the lane; Determine at least one preset location on the designated route through the shooting range information; The camera is controlled to capture the image sequence based on the at least one preset position.

3. The method according to claim 1, wherein: The step of matching the images in the image sequence on the initial three-dimensional model to obtain the target three-dimensional model comprises: Determining a matching relationship between each image in the image sequence and the point cloud data through the at least one preset position; Based on the matching relationship, the images in the image sequence are matched on the initial three-dimensional model to obtain the target three-dimensional model.

4. The method according to claim 1, wherein: The detecting of the surface feature information of the lane based on the target three-dimensional model also includes: Matching the target three-dimensional model with a pre-stored reference three-dimensional model of the lane to determine third surface feature information of the lane; In response to the third surface feature information exceeding a preset scale threshold, a third position corresponding to the third surface feature information is marked in the target three-dimensional space, and third warning information is generated.

5. A tunnel surface detection device, comprising: A surface information acquisition unit is configured to acquire surface information of the lane, wherein the surface information includes structural features of the lane; a target three-dimensional model construction unit, configured to construct a target three-dimensional model of the lane through the surface information; a surface feature information detection unit, configured to detect surface feature information of the lane based on the target three-dimensional model; in, The surface information acquisition unit includes a surface information acquisition subunit configured to acquire point cloud data passing through the lane along a designated route; The surface information acquisition subunit comprises a surface information acquisition module configured to acquire an image sequence passing through the lane along the designated route, wherein images in the image sequence are acquired at at least one preset position on the designated route; The target three-dimensional model construction unit includes an initial three-dimensional model construction subunit and a target three-dimensional model construction subunit; The initial three-dimensional model construction subunit is configured to construct an initial three-dimensional model of the lane through the point cloud data; The target three-dimensional model construction subunit is configured to match the images in the image sequence on the initial three-dimensional model to obtain the target three-dimensional model; The surface feature information detection unit includes a point cloud image query subunit and a first warning information generation subunit; The point cloud image query subunit is configured to query at least one point cloud image of at least one lane surface in the initial three-dimensional model; The first warning information generating subunit is configured to, for each point cloud image of the at least one point cloud image, mark a first position of the first designated image feature in the point cloud image in response to the presence of a first designated image feature corresponding to the first surface feature information, and generate first warning information corresponding to the point cloud image, wherein the first surface feature information corresponds to at least one of the following: a convex structure, a concave structure, and the first warning information includes the first position information; The surface feature information detection unit further includes a surface image query subunit and a second warning information generation subunit; The surface image query subunit is configured to query at least one surface image of at least one lane surface in the target three-dimensional model in response to the absence of the first specified image feature; The second alarm information generating subunit is configured to, for each surface image of the at least one surface image, mark the second position of the second designated image feature in the surface image in response to the existence of the second designated image feature corresponding to the second surface feature information, and generate second alarm information for the corresponding surface image, wherein the second surface feature information includes at least one of the following: a crack image, a color image, and the second alarm information includes the second position information.

6. An electronic device comprising: one or more processors; A camera for collecting surface images of the lane; Point cloud acquisition equipment, used to collect point cloud data; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 4.

7. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.