Bridge construction potential safety hazard digital management system based on visual identification

Abstract

The invention relates to a bridge construction potential safety hazard digital management system based on visual identification, the system comprises an inspection robot and a control terminal, and the inspection robot is in communication connection with the control terminal; the control terminal is set to respond to an inspection task sent by the user terminal and extract a target bridge section construction drawing indicated by the inspection task in a preset bridge section construction drawing library; according to a target inspection object indicated by the inspection task, determining an image acquisition area marked for inspection of the inspection robot in the target bridge section construction drawing; sending the target bridge section construction drawing to the inspection robot; the control terminal is configured to: receive a sampled image; determining object features of the target inspection object according to the sampling image; and when the object features accord with the set potential safety hazard condition, sending a prompt message that the target inspection object has the potential safety hazard to the user terminal.

Description

Technical Field

[0001] The present disclosure relates to the technical field of municipal and traffic engineering construction, and more specifically, to a digital management system for bridge construction safety hazards based on visual recognition. Background Art

[0002] With the rapid development of visual recognition technology, it is applied to different industries, such as the construction industry, the medical industry, and the public transportation industry, etc. At present, during the bridge construction process, safety officers need to regularly inspect the construction site, or record videos of the bridge construction process through cameras so that safety officers can confirm the safety hazards existing at the bridge construction site through the videos, which results in low efficiency and high safety risks. Summary of the Invention

[0003] An object of an embodiment of the present disclosure is to provide a new technical solution for digital management of bridge construction safety hazards based on visual recognition.

[0004] According to a first aspect of the present disclosure, there is provided a digital management system for bridge construction safety hazards based on visual recognition, the system includes an inspection robot and a control terminal, and the inspection robot is communicatively connected to the control terminal;

[0005] The control terminal is configured to: in response to an inspection task sent by a user terminal, extract a target bridge construction drawing indicated by the inspection task from a preset bridge construction drawing library; determine an image acquisition area marked in the target bridge construction drawing for the inspection robot to inspect according to a target inspection object indicated by the inspection task; send the target bridge construction drawing to the inspection robot;

[0006] The inspection robot is configured to: receive the target bridge construction drawing; set a moving path according to the target bridge construction drawing; control the operation of a moving device and an image acquisition device of the inspection robot according to path features of the moving path; send a sampled image output by the image acquisition device to the control terminal;

[0007] The control terminal is configured to: receive the sampled image; determine an object feature of the target inspection object according to the sampled image; when the object feature meets a set safety hazard condition, send a prompt message indicating that the target inspection object has a safety hazard to the user terminal.

[0008] Optionally, the control terminal is configured to: the determining the image acquisition area marked in the target bridge construction drawing for the inspection robot to inspect according to a target inspection object indicated by the inspection task includes:

[0009] Determine the target construction stage indicated by the inspection task;

[0010] Determine the object feature information of the target inspection object in the target bridge construction drawing at the target construction stage;

[0011] Based on the object feature information, determine the radiation area of the target inspection object as the image acquisition area marked in the target bridge construction drawing for the inspection robot to inspect.

[0012] Optionally, the control terminal is set to: The determining the object feature information of the target inspection object in the target bridge construction drawing at the target construction stage includes:

[0013] When the target construction stage is the preparation stage, determine the horizontal position information of the target inspection object in the target bridge construction drawing as the object feature information of the target inspection object in the target bridge construction drawing at the target construction stage;

[0014] The control terminal is set to: The determining the radiation area of the target inspection object based on the object feature information includes:

[0015] Based on the horizontal contour of the target inspection object reflected by the object feature information, determine the radiation area of the target inspection object.

[0016] Optionally, the control terminal is set to: The determining the object feature information of the target inspection object in the target bridge construction drawing at the target construction stage includes:

[0017] When the target construction stage is the main implementation stage, determine the assembly object associated with the target inspection object and the relative position of the assembly object with respect to the target inspection object; according to the relative position, determine the spatial position information of the assembly object and the target inspection object in the target bridge construction drawing as the object feature information of the target inspection object in the target bridge construction drawing at the target construction stage;

[0018] The control terminal is set to: The determining the radiation area of the target inspection object based on the object feature information includes:

[0019] Based on the minimum bounding box where the target inspection object and the assembly object are located reflected by the object feature information, determine the radiation area of the target inspection object.

[0020] Optionally, the control terminal is set to: The determining the object feature information of the target inspection object in the target bridge construction drawing at the target construction stage includes:

[0021] When the target construction stage is the acceptance stage, determine the spatial position information of the target inspection object in the construction drawing of the target bridge section, and use it as the object feature information of the target inspection object in the target bridge section in the target construction stage;

[0022] The control terminal is configured to: based on the object feature information, determine the radiation area of the target inspection object, including:

[0023] Based on the minimum bounding box where the target inspection object is located as reflected by the object feature information, determine the radiation area of the target inspection object.

[0024] Optionally, the control terminal is configured to: according to the sampled image, determine the object features of the target inspection object, including:

[0025] Through a pre-set image recognition model, recognize the target inspection object, reference object, and abnormal object in the sampled image;

[0026] Determine the first relative position distance between the target inspection object and the reference object and the second relative position distance between the target inspection object and the abnormal object, and use them as the object features of the target inspection object.

[0027] Optionally, the potential safety hazard conditions include that the first relative position distance is greater than or equal to a set first distance threshold and the second relative position distance is less than or equal to a set second distance threshold.

[0028] According to the second aspect of the present disclosure, there is also provided a method for digital management of potential safety hazards in bridge construction based on visual recognition. The method is implemented using the system for digital management of potential safety hazards in bridge construction based on visual recognition. The system includes an inspection robot and a control terminal, and the inspection robot is communicatively connected to the control terminal; the execution subject of the method is the control terminal, and the method includes:

[0029] In response to an inspection task sent by a user terminal, extract the construction drawing of the target bridge section indicated by the inspection task from a preset bridge section construction drawing library;

[0030] According to the target inspection object indicated by the inspection task, determine the image acquisition area marked in the construction drawing of the target bridge section for the inspection robot to perform inspections;

[0031] Send the construction drawing of the target bridge section to the inspection robot so that the inspection robot receives the construction drawing of the target bridge section; set a moving path according to the construction drawing of the target bridge section; control the operation of the moving device and the image acquisition device of the inspection robot according to the path characteristics of the moving path; send the sampled image output by the image acquisition device to the control terminal;

[0032] Receive the sampled image;

[0033] Determine the object characteristics of the target inspection object according to the sampled image;

[0034] When the object characteristics meet the set safety hazard conditions, send a prompt message indicating that the target inspection object has a safety hazard to the user terminal.

[0035] According to the third aspect of the present disclosure, there is also provided a control terminal, and the control terminal includes:

[0036] A response module, configured to respond to an inspection task sent by a user terminal, and extract the construction drawing of the target bridge section indicated by the inspection task from a preset bridge section construction drawing library;

[0037] A first determination module, configured to determine an image acquisition area marked for the inspection robot to inspect in the construction drawing of the target bridge section according to the target inspection object indicated by the inspection task;

[0038] A first sending module, configured to send the construction drawing of the target bridge section to the inspection robot so that the inspection robot receives the construction drawing of the target bridge section; set a moving path according to the construction drawing of the target bridge section; control the operation of the moving device and the image acquisition device of the inspection robot according to the path characteristics of the moving path; send the sampled image output by the image acquisition device to the control terminal;

[0039] A receiving module, configured to receive the sampled image;

[0040] A second determination module, configured to determine the object characteristics of the target inspection object according to the sampled image;

[0041] A second sending module, configured to send a prompt message indicating that the target inspection object has a safety hazard to the user terminal when the object characteristics meet the set safety hazard conditions.

[0042] According to the fourth aspect of the present disclosure, there is also provided a computer-readable storage medium, and a computer program is stored in the computer-readable storage medium, wherein the computer program is set to execute the above-mentioned digital management method for bridge construction safety hazards based on visual recognition when running.

[0043] According to a fifth aspect of the present disclosure, there is also provided a computer program product including a computer program which, when executed, causes a computer to perform the steps of the above-mentioned digital management method for potential safety hazards in bridge construction based on visual recognition.

[0044] According to a sixth aspect of the present disclosure, there is also provided a control terminal including a memory and a processor. A computer program is stored in the memory, and the processor is configured to execute the above-mentioned digital management method for potential safety hazards in bridge construction based on visual recognition through the computer program.

[0045] One beneficial effect of the embodiments of the present disclosure is that the digital management system for potential safety hazards in bridge construction based on visual recognition provided by the embodiments of the present disclosure can instruct an inspection robot to inspect an image acquisition area in a construction drawing of a target bridge section through a control terminal and feed back sampled images collected during the inspection process to the control terminal. The control terminal then determines whether there are potential safety hazards in the bridge. And when there are potential safety hazards in the bridge, a prompt message with potential safety hazards is sent to a user terminal, effectively reducing labor costs and improving the efficiency of bridge inspection.

[0046] Through the following detailed description of the exemplary embodiments of the present specification with reference to the accompanying drawings, the features and advantages of the embodiments of the present specification will become clear. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The drawings incorporated in the specification and constituting a part of the specification illustrate embodiments of the present specification and, together with the description, are used to explain the principles of the embodiments of the present specification.

[0048] Figure 1 Shows a schematic hardware structure diagram that can be used to implement a digital management system for potential safety hazards in bridge construction based on visual recognition according to an embodiment of the present disclosure;

[0049] Figure 2 Shows a schematic diagram of an image sampling area according to some embodiments;

[0050] Figure 3 Shows a schematic flowchart of a digital management method for potential safety hazards in bridge construction based on visual recognition according to some embodiments;

[0051] Figure 4 Shows a schematic diagram of the structure of a control terminal according to some embodiments;

[0052] Figure 5 Shows a schematic hardware structure diagram of a control terminal according to some embodiments. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0053] Now, various exemplary embodiments of the present specification will be described in detail with reference to the accompanying drawings.

[0054] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way a limitation on the embodiments of this specification or their application or use.

[0055] It should be noted that like reference numerals and letters denote like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.

[0056] It should be noted that all actions of obtaining signals, information, or data in the embodiments of the present disclosure are carried out on the premise of complying with the corresponding data protection regulations and policies of the country where the location is located and obtaining the authorization given by the corresponding device owner.

[0057] <System Embodiment>

[0058] The embodiments of the present disclosure provide a new digital management system for visual recognition of bridge safety hazards. As Figure 1 shown, the system includes an inspection robot 20 and a control terminal 10, and the inspection robot 20 is communicatively connected to the control terminal 10.

[0059] In this embodiment, the mobile device 21 configured by the inspection robot 20 may be a wheeled drive device, a tracked drive device, a legged drive device, or a flying drive device. The image acquisition device 22 configured by the inspection robot 20 may be an RGB camera, a depth camera, or a combination of both. The inspection robot 20 may also be configured with a controller 23 and a wireless communication 24 module, etc., so that the inspection robot 20 can control the above-mentioned mobile device 21 and image acquisition device 22, etc., to realize functions such as navigation, obstacle avoidance, and data transmission of the inspection robot 20.

[0060] In this embodiment, the control terminal 10 may be various types of servers, and the control terminal 10 has functions such as data processing, data transmission, and data storage.

[0061] The control terminal 10 is configured to: in response to an inspection task sent by the user terminal 30, extract the target bridge construction drawing indicated by the inspection task from a preset bridge construction drawing library; determine the image acquisition area marked for inspection by the inspection robot 20 in the target bridge construction drawing according to the target inspection object indicated by the inspection task; and send the target bridge construction drawing to the inspection robot 20.

[0062] In this embodiment, the user terminal 30 is, for example, the user's mobile phone, a touch device configured by the control terminal 10, etc.

[0063] In this embodiment, the inspection task can be information output by the user on the user interface of the user terminal 30. For example, the inspection is for potential safety hazards of the street lights on the bridge Br1 during the acceptance stage.

[0064] In this embodiment, each bridge section construction drawing in the bridge section construction drawing library can be imported into the control terminal 10 by the above-mentioned user or other users. The control terminal 10 can extract the information about a certain bridge section in the inspection task and extract the corresponding bridge section construction drawing of that bridge section from the bridge section construction drawing library. For example, if the inspection task is to inspect the potential safety hazards of the street lights on the bridge Br1 during the acceptance stage, then the control terminal 10 can extract the bridge section construction drawing of bridge Br1 from the bridge section construction drawing library.

[0065] In some examples, if the inspection task is to inspect the potential safety hazards of the street lights on the bridge Br1 during the acceptance stage, then the target inspection object is the street lights.

[0066] The inspection robot 20 is configured to: receive the target bridge section construction drawing; set a moving path according to the target bridge section construction drawing; control the moving device 21 and the image acquisition device 22 of the inspection robot 20 to run according to the path characteristics of the moving path; and send the sampled image output by the image acquisition device 22 to the control terminal 10.

[0067] In some examples, as Figure 2 shown, when the inspection robot 20 receives the target bridge section construction drawing marked with the image sampling area D1, the inspection robot 20 can set the moving path L1 according to the image sampling area being area D1. Among the path nodes J1 - J8 set in the moving path L1, the inspection robot 20 controls its moving device 21 to sequentially pass through the path nodes J1 - J8, and controls the image acquisition device 22 to run during the process of the path nodes J1 - J8, so that the inspection robot 20 can send the sampled image output by the image acquisition device 22 to the control terminal 10.

[0068] The control terminal 10 is configured to: receive the sampled image; determine the object characteristics of the target inspection object according to the sampled image; and send a prompt message indicating that the target inspection object has potential safety hazards to the user terminal 30 when the object characteristics meet the set potential safety hazard conditions.

[0069] In some examples, the target inspection object is, for example, a guardrail. The object characteristics of the guardrail are, for example, the height of the guardrail, the damaged area, etc. The potential safety hazard conditions are, for example, that the height of the guardrail is less than or equal to the set height or the damaged area is greater than or equal to the set area.

[0070] In some examples, during the inspection process, the inspection robot can identify smoke or personnel intrusion into the operating area of construction machinery, and feedback the abnormal information of the identified smoke or personnel intrusion into the operating area of construction machinery to the user terminal.

[0071] In this embodiment, the digital management system for bridge construction safety hazards based on visual recognition can instruct the inspection robot 20 to inspect the image acquisition area in the construction drawing of the target bridge section through the control terminal 10 and feedback the sampled images collected during the inspection process to the control terminal 10. The control terminal 10 then determines whether there are safety hazards on the bridge. And when there are safety hazards on the bridge, it sends a prompt message with safety hazards to the user terminal 30, effectively reducing the labor cost and improving the efficiency of bridge inspection.

[0072] In some embodiments, the control terminal 10 is configured to: determine the target construction stage corresponding to the inspection task; determine the object feature information of the target inspection object in the construction drawing of the target bridge section in the target construction stage; based on the object feature information, determine the radiation area of the target inspection object as the image acquisition area marked in the construction drawing of the target bridge section for the inspection robot 20 to inspect.

[0073] In this embodiment, the construction stage may include a preparation stage, a main body implementation stage, and an acceptance stage. For example, if the inspection task is to inspect the safety hazards of the street lights of Bridge Br1 in the acceptance stage, then the target construction stage indicated by this inspection task is the acceptance stage.

[0074] In some examples, as Figure 2 shown, the target inspection object is, for example, the street lights of Bridge Br1, the target construction stage is the acceptance stage, and the object feature information of the target inspection object in the target construction stage may include the height of the street light, the length of the street light, the width of the street light, or the status of the street light source, etc. Then, the radiation area of the target inspection object is, for example, area D1.

[0075] In this embodiment, by determining the target construction stage of the inspection task, the area that the inspection robot 20 needs to inspect can be determined more accurately, effectively improving the inspection efficiency of the inspection robot 20.

[0076] In some examples, in order to improve the inspection efficiency of the inspection robot 20 for the bridge in the preparation stage, the control terminal 10 is configured to: when the target construction stage is the preparation stage, determine the horizontal position information of the target inspection object in the construction drawing of the target bridge section as the object feature information of the target inspection object in the target construction stage.

[0077] The control terminal 10 is configured to: based on the horizontal contour of the target inspection object reflected by the object feature information, determine the radiation area of the target inspection object.

[0078] In some examples, such as Figure 2 shown, the target inspection object is the street lamp of bridge Br1. Then, the horizontal position information can be the length and width of the base of the street lamp of bridge Br1. According to the length and width of the base of the street lamp of bridge Br1, the control terminal 10 can determine that the radiation area of the target inspection object is a two-dimensional area K1 - area K6, so that the control terminal 10 can determine a two-dimensional image acquisition area surrounding the two-dimensional area K1 - area K6, that is, area D1, and send this area D1 to the inspection robot 20. After obtaining the two-dimensional area D1, the inspection robot 20 can determine that the shooting angle of the image acquisition device 22 of the inspection robot 20 is a fixed value, so that the inspection robot 20 does not need to adjust the shooting angle of the image acquisition device 22 during the inspection process, effectively improving the inspection efficiency of the inspection robot 20.

[0079] In some embodiments, in order to improve the inspection efficiency of the inspection robot 20 for the bridge during the main construction stage, the control terminal 10 is set to: when the target construction stage is the main construction stage, determine the assembly object associated with the target inspection object and the relative position of the assembly object with respect to the target inspection object; according to the relative position, determine the spatial position information of the assembly object and the target inspection object in the target bridge construction drawing as the object feature information of the target inspection object in the target construction stage in the target bridge construction drawing.

[0080] The control terminal 10 is set to: based on the minimum bounding box reflecting the target inspection object and the assembly object where the object feature information is located, determine the radiation area of the target inspection object.

[0081] In some examples, such as Figure 2 shown, the target inspection object is the street lamp of bridge Br1. Then, the assembly object of this street lamp is the street lamp mounting frame. According to the length, width, height of the street lamp of bridge Br1, the length, width and height of the street lamp mounting frame, the control terminal 10 can determine that the area where the street lamp of the target inspection object is located is a three-dimensional area K1 - area K6, and the area where the street lamp mounting frame is located is a three-dimensional area C1 - area C6, so that the control terminal 10 can determine a three-dimensional image acquisition area surrounding the three-dimensional area K1 - area K6 and the three-dimensional area C1 - area C6, that is, area D1, and send this area D1 as the minimum bounding box where the target inspection object and the assembly object are located to the inspection robot 20. After obtaining the minimum bounding box, the inspection robot 20 can determine that the shooting angle of the image acquisition device 22 of the inspection robot 20 is within the angle range corresponding to the height of the street lamp or the street lamp mounting frame, for example, -10 degrees to 70 degrees. The inspection robot 20 adjusts the shooting angle of the image acquisition device 22 during the main construction stage, which can effectively ensure the accuracy of the sampled images output by the inspection robot 20.

[0082] In some embodiments, in order to improve the inspection efficiency of the inspection robot 20 for the bridge during the acceptance stage, the control terminal 10 is configured to: when the target construction stage is the acceptance stage, determine the spatial position information of the target inspection object in the construction drawing of the target bridge section as the object feature information of the target inspection object in the construction drawing of the target bridge section during the target construction stage.

[0083] The control terminal 10 is configured to: determine the radiation area of the target inspection object based on the minimum bounding box where the target inspection object is located as reflected by the object feature information.

[0084] In some examples, if the target inspection object is the street lamp of Bridge Br1, then the control terminal 10 can determine that the area where the street lamp of the target inspection object is located is the three-dimensional area K1 - area K6 according to the length, width, and height of the street lamp of this Bridge Br1, so that the control terminal 10 can determine the three-dimensional image acquisition area that encloses the three-dimensional area K1 - area K6, that is, area D1, and send this area D1 as the spatial position information to the inspection robot 20. After obtaining the spatial position information, the inspection robot 20 can determine that the shooting angle of the image acquisition device 22 of the inspection robot 20 is the angle range corresponding to the height of the street lamp. For example, -10 degrees to 60 degrees. By adjusting the shooting angle of the image acquisition device 22 during the acceptance stage, the inspection robot 20 can effectively ensure the accuracy of the sampled images output by the inspection robot 20.

[0085] In some embodiments, the control terminal 10 is configured to: identify the target inspection object, the reference object, and the abnormal object in the sampled image through a pre-set image recognition model. Determine the first relative position distance between the target inspection object and the reference object and the second relative position distance between the target inspection object and the abnormal object as the object features of the target inspection object.

[0086] In this embodiment, the reference object and the abnormal object can be pre-set, and the target inspection object corresponds to different reference objects and abnormal objects in different construction stages. Taking the street lamp as the target inspection object as an example, in the preparation stage, the reference objects of this street lamp can be the road surface paving marking line, the guardrail installation marking line, etc., and the abnormal objects are, for example, the guardrail or the street lamp assembly rack, etc. In the main body implementation stage, the reference object of this street lamp can be the street lamp assembly rack. In the acceptance stage, the reference objects of this street lamp can be the yellow line, white line on the road surface, or the guardrail, etc., and the abnormal objects of this street lamp are, for example, the street lamp assembly rack, etc. By determining the first relative distance between the reference object and the target inspection object or determining the second relative distance between the abnormal object and the target inspection object, it is possible to effectively determine whether there are potential safety hazards for the target inspection object.

[0087] In some embodiments, in order to determine whether there are potential safety hazards for the target inspection object, the potential safety hazard conditions include that the first relative position distance is greater than or equal to a set first distance threshold and the second relative position distance is less than or equal to a set second distance threshold.

[0088] <Method Embodiment>

[0089] Figure 2 is a schematic flowchart of a method for digital management of potential safety hazards in bridge construction based on visual recognition according to an embodiment. The implementation subject is, for example, Figure 1 the control terminal 10 in

[0090] As Figure 2 shown, the method for digital management of potential safety hazards in bridge construction based on visual recognition in this embodiment may include the following steps S210 to step S260:

[0091] Step S210, in response to an inspection task sent by a user terminal, extract the target bridge section construction drawing indicated by the inspection task from a preset bridge section construction drawing library.

[0092] Step S220, according to the target inspection object indicated by the inspection task, determine the image acquisition area marked in the target bridge section construction drawing for the inspection robot to perform inspections.

[0093] In some embodiments, this step S220 may include the following steps S2201 to step S2203:

[0094] Step S2201, determine the target construction stage corresponding to the inspection task indicated.

[0095] Step S2202, determine the object feature information of the target inspection object in the target bridge section construction drawing at the target construction stage.

[0096] Step S2203, based on the object feature information, determine the radiation area of the target inspection object as the image acquisition area marked in the target bridge section construction drawing for the inspection robot to perform inspections.

[0097] Step S230, send the target bridge section construction drawing to the inspection robot so that the inspection robot receives the target bridge section construction drawing; set a moving path according to the target bridge section construction drawing; control the operation of the moving device and the image acquisition device of the inspection robot according to the path features of the moving path; send the sampled image output by the image acquisition device to the control terminal.

[0098] Step S240, receive the sampled image.

[0099] Step S250, determine the object features of the target inspection object according to the sampled image.

[0100] Step S260: When the object features meet the set potential safety hazard conditions, send a prompt message indicating that the target inspection object has potential safety hazards to the user terminal.

[0101] In some embodiments, step S2202 may include the following steps S310:

[0102] Step S310: When the target construction stage is the preparation stage, determine the horizontal position information of the target inspection object in the construction drawing of the target bridge section as the object feature information of the target inspection object in the target construction stage in the construction drawing of the target bridge section.

[0103] Step S2203 may include the following steps S320:

[0104] Step S320: Based on the horizontal contour of the target inspection object reflected by the object feature information, determine the radiation area of the target inspection object.

[0105] In some embodiments, step S2202 may include the following steps S410:

[0106] When the target construction stage is the main implementation stage, determine the assembly object associated with the target inspection object and the relative position of the assembly object with respect to the target inspection object; according to the relative position, determine the spatial position information of the assembly object and the target inspection object in the construction drawing of the target bridge section as the object feature information of the target inspection object in the target construction stage in the construction drawing of the target bridge section.

[0107] Step S2203 may include the following steps S420:

[0108] Step S420: Based on the minimum bounding box where the target inspection object and the assembly object are located as reflected by the object feature information, determine the radiation area of the target inspection object.

[0109] In some embodiments, step S2202 may include the following steps S510:

[0110] Step S510: When the target construction stage is the acceptance stage, determine the spatial position information of the target inspection object in the construction drawing of the target bridge section as the object feature information of the target inspection object in the target construction stage in the construction drawing of the target bridge section.

[0111] Step S2203 may include the following steps S520:

[0112] Step S520: Based on the minimum bounding box where the target inspection object is located as reflected by the object feature information, determine the radiation area of the target inspection object.

[0113] In some embodiments, step S250 may include the following steps S610 and step S620:

[0114] Step S610: Identify the target inspection object, reference object, and abnormal object in the sampled image through a pre-set image recognition model.

[0115] Step S620: Determine the first relative position distance between the target inspection object and the reference object and the second relative position distance between the target inspection object and the abnormal object as the object features of the target inspection object.

[0116] <Device Embodiment>

[0117] Figure 4 The composition structure diagram of the digital management device for bridge construction safety hazards based on visual recognition according to an embodiment of the present disclosure is shown. As Figure 4 shown, the digital management device 400 for bridge construction safety hazards based on visual recognition includes:

[0118] The response module 410 is configured to respond to the inspection task sent by the user terminal, and extract the target bridge construction drawing indicated by the inspection task from the pre-set bridge construction drawing library;

[0119] The first determination module 420 is configured to determine the image acquisition area marked for the inspection robot to inspect in the target bridge construction drawing according to the target inspection object indicated by the inspection task;

[0120] The first sending module 430 is configured to send the target bridge construction drawing to the inspection robot, so that the inspection robot receives the target bridge construction drawing; set the movement path according to the target bridge construction drawing; control the movement device and the image acquisition device of the inspection robot to operate according to the path features of the movement path; send the sampled image output by the image acquisition device to the control terminal;

[0121] The receiving module 440 is configured to receive the sampled image;

[0122] The second determination module 450 is configured to determine the object features of the target inspection object according to the sampled image;

[0123] The second sending module 460 is configured to send a prompt message indicating that the target inspection object has a safety hazard to the user terminal when the object features meet the set safety hazard conditions.

[0124] In some embodiments, the first determination module 420 is further configured to determine the target construction stage corresponding to the inspection task indicated; determine the object feature information of the target inspection object in the target bridge construction drawing at the target construction stage; determine the radiation area of the target inspection object based on the object feature information as the image acquisition area marked for the inspection robot to inspect in the target bridge construction drawing

[0125] In some embodiments, the first determination module 420 is further configured to, when the target construction stage is the preparation stage, determine the horizontal position information of the target inspection object in the construction drawing of the target bridge section as the object feature information of the target inspection object in the target construction stage in the construction drawing of the target bridge section; and determine the radiation area of the target inspection object based on the minimum bounding box where the target inspection object and the assembly object are located as reflected by the object feature information.

[0126] In some embodiments, the first determination module 420 is further configured to, when the target construction stage is the main body implementation stage, determine the assembly object associated with the target inspection object and the relative position of the assembly object with respect to the target inspection object; determine the spatial position information of the assembly object and the target inspection object in the construction drawing of the target bridge section as the object feature information of the target inspection object in the target construction stage in the construction drawing of the target bridge section; and determine the radiation area of the target inspection object based on the minimum bounding box where the target inspection object and the assembly object are located as reflected by the object feature information.

[0127] In some embodiments, the first determination module 420 is further configured to, when the target construction stage is the acceptance stage, determine the spatial position information of the target inspection object in the construction drawing of the target bridge section as the object feature information of the target inspection object in the target construction stage in the construction drawing of the target bridge section; and determine the radiation area of the target inspection object based on the minimum bounding box where the target inspection object is located as reflected by the object feature information.

[0128] In some embodiments, the second determination module 450 is further configured to identify the target inspection object, the reference object, and the abnormal object in the sampled image through a pre-set image recognition model; and determine the first relative position distance between the target inspection object and the reference object and the second relative position distance between the target inspection object and the abnormal object as the object features of the target inspection object.

[0129] <Device Embodiment>

[0130] Figure 5 The hardware structure diagram of an electronic device according to some other embodiments is shown. As Figure 5 shown, the electronic device 500 includes a processor 510 and a memory 520. The memory 520 is used to store a computer program, and the computer program is used to control the processor 510 to operate so as to control the electronic device 500 to execute the digital management method for bridge construction safety hazards based on visual recognition according to any embodiment of the present disclosure.

[0131] An embodiment of the present disclosure further provides a computer-readable storage medium, which stores a computer program that, when executed by a processor, implements the digital management method for bridge construction safety hazards based on visual recognition according to any embodiment of the present disclosure.

[0132] Each embodiment in this specification is described in a progressive manner. For the identical or similar parts among the embodiments, reference can be made to each other, and the differences between each embodiment and other embodiments are emphasized. In particular, for the apparatus and device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and for the relevant parts, reference can be made to the partial description of the method embodiments.

[0133] The above describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve the desired results. Additionally, the processes depicted in the figures do not necessarily require the particular order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0134] The embodiments of this specification may be devices, methods, and / or computer program products. A computer program product may include a computer-readable storage medium having thereon computer-readable program instructions for causing a processor to implement various aspects of the embodiments of this specification.

[0135] A computer-readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanically encoded device such as a punch card or raised structures in grooves having instructions stored thereon, and any suitable combination of the foregoing. The computer-readable storage medium used herein is not construed as being a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated through a waveguide or other transmission medium (e.g., an optical pulse through an optical fiber cable), or an electrical signal transmitted through a wire.

[0136] The computer-readable program instructions described herein can be downloaded to various computing / processing devices from a computer-readable storage medium or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing / processing device.

[0137] The computer program instructions for performing the operations of the embodiments of the present specification may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via an Internet service provider through the Internet). In some embodiments, by using the state information of the computer-readable program instructions to customize an electronic circuit, such as a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit can execute the computer-readable program instructions to implement various aspects of the embodiments of the present specification.

[0138] Aspects of the embodiments of the present specification are described herein with reference to the flowcharts and / or block diagrams of methods, apparatuses (devices), and computer program products according to the embodiments of the present specification. It should be understood that each block of the flowcharts and / or block diagrams, and the combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0139] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that the instructions, when executed by the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in one or more boxes of the flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium that causes a computer, a programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer-readable medium storing the instructions comprises a manufacture, the instructions comprising aspects for implementing the functions / acts specified in one or more boxes of the flowchart and / or block diagram.

[0140] The computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process such that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in one or more boxes of the flowchart and / or block diagram.

[0141] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present specification. In this regard, each box in the flowchart or block diagram may represent a module, a segment of a program, or a portion of an instruction, the module, segment of a program, or portion of an instruction comprising one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions noted in the boxes may occur out of the order noted in the figures. For example, two consecutive boxes may in fact be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each box of the block diagrams and / or flowcharts, and combinations of boxes in the block diagrams and / or flowcharts, can be implemented by special-purpose hardware-based systems that perform the specified functions or acts, or by combinations of special-purpose hardware and computer instructions. It is well known to those skilled in the art that implementation via hardware, implementation via software, and implementation via a combination of software and hardware are equivalent.

[0142] The embodiments of the present specification have been described above. The above description is exemplary and not exhaustive, and is also not limited to the disclosed embodiments. Many modifications and variations are obvious to those of ordinary skill in the art without departing from the scope of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, the practical application, or the improvement of the technology in the market, or to enable other ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A digital management system for bridge construction safety hazards based on visual recognition, wherein, The system includes an inspection robot and a control terminal, and the inspection robot is communicatively connected to the control terminal; The control terminal is configured to: in response to an inspection task sent by a user terminal, extract a target bridge construction drawing indicated by the inspection task from a preset bridge construction drawing library; determine an image acquisition area marked in the target bridge construction drawing for the inspection robot to inspect according to a target inspection object indicated by the inspection task; and send the target bridge construction drawing to the inspection robot; The inspection robot is configured to: receive the target bridge construction drawing; set a moving path according to the target bridge construction drawing; control the operation of a moving device and an image acquisition device of the inspection robot according to path features of the moving path; and send a sampled image output by the image acquisition device to the control terminal; The control terminal is configured to: receive the sampled image; determine an object feature of the target inspection object according to the sampled image; and send a prompt message indicating that the target inspection object has a safety hazard to the user terminal when the object feature meets set safety hazard conditions.

2. The system according to claim 1, wherein, The control terminal is configured to: the determining an image acquisition area marked in the target bridge construction drawing for the inspection robot to inspect according to a target inspection object indicated by the inspection task includes: Determining a target construction stage corresponding to the target inspection object indicated by the inspection task; Determining object feature information of the target inspection object in the target bridge construction drawing at the target construction stage; Based on the object feature information, determining a radiation area of the target inspection object as the image acquisition area marked in the target bridge construction drawing for the inspection robot to inspect.

3. The system according to claim 2, wherein, The control terminal is configured to: the determining object feature information of the target inspection object in the target bridge construction drawing at the target construction stage includes: When the target construction stage is a preparation stage, determining horizontal position information of the target inspection object in the target bridge construction drawing as the object feature information of the target inspection object in the target bridge construction drawing at the target construction stage; The control terminal is configured to: the determining a radiation area of the target inspection object based on the object feature information includes: Determining a radiation area of the target inspection object based on a horizontal contour of the target inspection object reflected by the object feature information.

4. The system according to claim 2, wherein, The control terminal is configured to: the determining object feature information of the target inspection object in the target bridge construction drawing at the target construction stage includes: When the target construction stage is a main body implementation stage, determining an assembly object associated with the target inspection object and an associated position of the assembly object relative to the target inspection object; and determining spatial position information of the assembly object and the target inspection object in the target bridge construction drawing as the object feature information of the target inspection object in the target bridge construction drawing at the target construction stage according to the associated position. The control terminal is configured to: determine the radiation area of the target inspection object based on the object feature information, including: Determine the radiation area of the target inspection object based on the minimum bounding box where the target inspection object and the assembly object are located as reflected by the object feature information.

5. The system according to claim 2, wherein The control terminal is configured to: determine the object feature information of the target inspection object in the target construction drawing of the target bridge section in the target construction stage, including: When the target construction stage is the acceptance stage, determine the spatial position information of the target inspection object in the target construction drawing of the target bridge section as the object feature information of the target inspection object in the target construction drawing of the target bridge section in the target construction stage; The control terminal is configured to: determine the radiation area of the target inspection object based on the object feature information, including: Determine the radiation area of the target inspection object based on the minimum bounding box where the target inspection object is located as reflected by the object feature information.

6. The system according to claim 1, wherein The control terminal is configured to: determine the object features of the target inspection object according to the sampled image, including: Identify the target inspection object, reference object, and abnormal object in the sampled image through a pre-set image recognition model; Determine the first relative position distance between the target inspection object and the reference object and the second relative position distance between the target inspection object and the abnormal object as the object features of the target inspection object.

7. The system according to claim 6, wherein The potential safety hazard conditions include that the first relative position distance is greater than or equal to a set first distance threshold and the second relative position distance is less than or equal to a set second distance threshold.

8. A digital management method for potential safety hazards in bridge construction based on visual recognition, wherein, The method uses the digital management system for bridge construction safety hazards based on visual recognition. The system includes an inspection robot and a control terminal, and the inspection robot is communicatively connected to the control terminal; The execution subject of the method is the control terminal, and the method includes: In response to the inspection task sent by the user terminal, extract the target construction drawing of the target bridge section indicated by the inspection task from a pre-set bridge construction drawing library; According to the target inspection object indicated by the inspection task, determine the image acquisition area marked in the target construction drawing of the target bridge section for the inspection robot to inspect; Send the target construction drawing of the target bridge section to the inspection robot so that the inspection robot receives the target construction drawing; set a moving path according to the target construction drawing; control the running of the moving device and the image acquisition device of the inspection robot according to the path features of the moving path; send the sampled image output by the image acquisition device to the control terminal; Receive the sampled image; Determine the object features of the target inspection object according to the sampled image; When the object features meet the set potential safety hazard conditions, send a prompt message indicating that the target inspection object has a potential safety hazard to the user terminal.

9. A control terminal, wherein, The control terminal includes: A response module, configured to extract the target construction drawing of the target bridge section indicated by the inspection task from a pre-set bridge construction drawing library in response to the inspection task sent by the user terminal; The first determination module is configured to determine an image acquisition area marked in the target bridge construction drawing for the inspection robot to inspect according to the target inspection object indicated by the inspection task; The first sending module is configured to send the target bridge construction drawing to the inspection robot, so that the inspection robot receives the target bridge construction drawing; set a moving path according to the target bridge construction drawing; control the operation of the moving device and the image acquisition device of the inspection robot according to the path characteristics of the moving path; and send the sampled image output by the image acquisition device to the control terminal; The receiving module is configured to receive the sampled image; The second determination module is configured to determine the object characteristics of the target inspection object according to the sampled image; The second sending module is configured to send a prompt message indicating that the target inspection object has a safety hazard to the user terminal when the object characteristics meet the set safety hazard conditions.

10. A control terminal, wherein, It includes a memory and a processor. The memory is used to store a computer program, and the processor is used to execute the method steps according to claim 8 under the control of the computer program.

Images