Inspection method, device, equipment, medium and program product

Abstract

The invention provides an inspection method, device and equipment, a medium and a program product, and relates to the field of risk control, and the method comprises the steps: collecting a construction fence image of a construction site; on the basis of the construction fence image, feature elements of the construction fence are identified, and the feature elements comprise a conical barrel and a warning line of a conical barrel type fence and a fence edge of a fence type fence; sorting the feature elements to obtain an ordered feature element list; based on the feature element list, checking whether an association relationship between adjacent feature elements accords with a preset association relationship or not to obtain a checking result; wherein the step of checking the incidence relation between the adjacent characteristic elements comprises whether the warning line exists between the adjacent conical barrels in the conical barrel type fence or not, and / or whether the distance between the end points of the edges of the adjacent fences in the fence type fence meets a preset condition or not; and judging whether the construction site is compliant or not according to the inspection result. And the accuracy and comprehensiveness of construction risk detection are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of risk control, in particular to a method and device for inspection, equipment, medium and program product. BACKGROUND

[0002] Inspection is an important management work, and the inspection range includes building safety, road hygiene, equipment operation and green environment, etc. However, the existing inspection method is insufficient in comprehensiveness of safety compliance inspection of the construction site, and there are safety loopholes. SUMMARY

[0003] Embodiments of the present application provide a method and device for inspection, equipment and storage medium to solve the problem of insufficient comprehensiveness of safety compliance inspection of the construction site in the existing inspection method and the existence of safety loopholes.

[0004] To solve the above technical problems, the present application is implemented as follows:

[0005] In a first aspect, the embodiments of the present application provide a method for inspection, which comprises:

[0006] acquiring a construction fence image of a construction site;

[0007] identifying feature elements of the construction fence based on the construction fence image, the feature elements including cones and warning lines of a cone fence, and fence edges of a fence;

[0008] sorting the feature elements to obtain an ordered feature element list;

[0009] checking whether an association relationship between adjacent feature elements conforms to a preset association relationship based on the feature element list to obtain a checking result, wherein the checking of the association relationship between adjacent feature elements includes checking whether the warning lines exist between adjacent cones in the cone fence, and / or checking whether an end point distance between adjacent fence edges in the fence conforms to a preset condition;

[0010] determining whether the construction site is compliant according to the checking result.

[0011] In a second aspect, the embodiments of the present application further provide a device for inspection. The device for inspection comprises:

[0012] an acquisition module configured to acquire a construction fence image of a construction site;

[0013] an identification module configured to identify feature elements of the construction fence based on the construction fence image, the feature elements including cones and warning lines of a cone fence, and fence edges of a fence;

[0014] The sorting module is configured to sort the feature elements to obtain an ordered feature element list.

[0015] The checking module is configured to check, based on the feature element list, whether the correlation between adjacent feature elements conforms to a preset correlation to obtain a checking result. The checking of the correlation between adjacent feature elements includes checking whether the warning line exists between adjacent barrels in the barrel type enclosure and / or whether the end point spacing of adjacent fence edges in the fence type enclosure conforms to a preset condition.

[0016] The determining module is configured to determine, according to the checking result, whether the construction site is compliant.

[0017] In a third aspect, an embodiment of the present application provides an electronic device, including a transceiver and a processor, and the processor is configured to:

[0018] Collect a construction enclosure image of a construction site.

[0019] Identify feature elements of the construction enclosure based on the construction enclosure image, the feature elements including barrels of a barrel type enclosure and warning lines and fence edges of a fence type enclosure.

[0020] Sort the feature elements to obtain an ordered feature element list.

[0021] Check, based on the feature element list, whether the correlation between adjacent feature elements conforms to a preset correlation to obtain a checking result. The checking of the correlation between adjacent feature elements includes checking whether the warning line exists between adjacent barrels in the barrel type enclosure and / or whether the end point spacing of adjacent fence edges in the fence type enclosure conforms to a preset condition.

[0022] Determine, according to the checking result, whether the construction site is compliant.

[0023] In a fourth aspect, an embodiment of the present application provides an electronic device, including a processor, a memory, and a program stored in the memory and executable on the processor, and the program, when executed by the processor, implements the steps of the inspection method according to the first aspect.

[0024] In a fifth aspect, an embodiment of the present application provides a computer readable storage medium, and the computer readable storage medium stores a computer program, and the computer program, when executed by a processor, implements the steps of the inspection method according to the first aspect.

[0025] In a sixth aspect, an embodiment of the present application provides a computer program product, including computer instructions, and the computer instructions, when executed by a processor, implement the steps of the inspection method according to the first aspect.

[0026] The inspection method provided by the embodiments of the present application comprises: collecting a construction fence image of a construction site; identifying feature elements of the construction fence based on the construction fence image, wherein the feature elements comprise a cone barrel of a cone barrel type fence and a warning line, and a fence edge of a fence type fence; sorting the feature elements to obtain an ordered feature element list; checking whether a correlation between adjacent feature elements conforms to a preset correlation based on the feature element list to obtain a checking result; wherein the checking of the correlation between adjacent feature elements comprises checking whether the warning line exists between adjacent cone barrels in the cone barrel type fence, and / or checking whether an end point distance between adjacent fence edges in the fence type fence conforms to a preset condition; and determining whether the construction site is compliant according to the checking result. The method realizes targeted compliance detection of the two types of construction fences by collecting the construction fence image, identifying the feature elements such as the cone barrel, the warning line and the fence edge, checking the correlation between adjacent elements and determining the compliance of the construction site, and the accuracy and comprehensiveness of the construction risk detection in the inspection are improved to some extent, and potential safety risks can be effectively avoided. BRIEF DESCRIPTION OF DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the description of the embodiments of the present application will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0028] Figure 1 is a flowchart of the inspection method provided by the embodiments of the present application;

[0029] Figure 2 is a schematic diagram of the fence type fence provided by an embodiment of the present application;

[0030] Figure 3 is a robot-based inspection architecture diagram provided by an embodiment of the present application;

[0031] Figure 4 is a structural diagram of the inspection device provided by an embodiment of the present application;

[0032] Figure 5 is a structural diagram of the electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0033] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are only some of the embodiments of the present application, but not all of the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0034] The embodiment of the present application provides a method for inspection. Referring to Figure 1 , Figure 1 The embodiment of the present application provides a method for inspection. Referring to Figure 1 The flowchart of the method for inspection is shown in the figure, and the method comprises the following steps:

[0035] Step 101, collecting a construction site construction fence image;

[0036] The above construction site can be understood as an area where construction work is being carried out, and can include a work point and a necessary protection range around it. The above construction fence image can be understood as an image or video frame obtained by an image acquisition device, such as a camera of an inspection robot, a park monitoring camera, etc., containing construction area protection facilities, such as cone barrels, fences, etc.

[0037] The above construction fence can be understood as a protection facility for isolating the construction area from the non-construction area, and can specifically include the following two types:

[0038] Cone barrel type fence: temporary protection facilities composed of cone barrels and warning lines, which need to be "linear straight, complete surrounding of construction area";

[0039] Fence type fence: protection facilities composed of continuous fences, which need to be "smoothly connected, without gaps or inclination".

[0040] For the specific way of collecting images, the embodiment of the present application does not make specific limitation, which can be real-time shooting by the video acquisition module carried by the inspection robot during the execution of the inspection task; the unmanned aerial vehicle can be used to shoot the construction area according to the preset flight route to obtain multi-angle fence images; or the fixed monitoring camera already deployed can be used to collect image data in the fixed field of view of the construction risk area.

[0041] In this step, the image data of the construction area protection facilities can be obtained by the image acquisition device to provide a basis for subsequent analysis.

[0042] Step 102, based on the construction fence image, identifying the characteristic elements of the construction fence, the characteristic elements including the cone barrels and warning lines of the cone barrel type fence, and the fence edges of the fence type fence;

[0043] In this step, the above characteristic elements can refer to the key components for determining the compliance of the fence.

[0044] In this step, key components can be extracted from the image of the enclosure through image recognition technology, including cones, warning lines, and fence edges. Alternatively, the laser radar carried by the inspection robot can be used to perform three-dimensional scanning on the construction enclosure area to obtain point cloud data of the enclosure. Through point cloud segmentation and feature extraction algorithms, the spatial position of the cones in the cone-type enclosure (based on the height and shape features of the point cloud), the continuous line features of the warning lines (based on the linear distribution of the point cloud), and the edge profile of the fence-type enclosure (based on the planar or continuous columnar features of the point cloud) are identified, thereby extracting the corresponding feature elements. The ultrasonic radar and millimeter wave radar data of the robot can also be combined to assist in identifying the feature elements. Different methods of identifying feature elements do not affect the realization of the basic functions of the embodiments of the present application.

[0045] Step 103, sorting the feature elements to obtain an ordered feature element list;

[0046] In this step, the ordered feature element list described above can be understood as a list formed by sorting the identified feature elements according to certain rules such as spatial position, arrangement order, etc. The purpose is to clearly define the logical relationship of "adjacent feature elements".

[0047] The sorting method of the feature elements in the present application is not specifically limited, and the feature elements can be sorted by coordinate information, for example, the feature elements can be divided into two groups according to the comparison result of the y coordinate value with the preset value, the upper group is sorted from small to large according to the x coordinate value, the lower group is sorted from large to small according to the x coordinate value, and then the two groups are combined to obtain a clockwise arranged ordered list; or based on the counterclockwise sorting of the spatial position, for example, first divide the upper group and the lower group according to the y coordinate value, then sort the upper group from large to small according to the x coordinate value, and sort the lower group from small to large according to the x coordinate value; or take the geometric center of the construction area or the initial observation point of the robot as the reference point, calculate the straight line distance of each feature element to the reference point, and sort the feature elements from near to far or from far to near.

[0048] Step 104, checking whether the correlation between adjacent feature elements meets a preset correlation based on the feature element list to obtain a checking result; wherein the checking of the correlation between adjacent feature elements includes whether the warning line exists between adjacent cones in the cone-type enclosure, and / or whether the distance between the end points of adjacent fence edges in the fence-type enclosure meets a preset condition;

[0049] In this step, the above-mentioned preset correlation can be understood as a rule that is set in advance and is used to determine whether the feature elements meet the safety specifications. Specifically, it can include:

[0050] In the cone-type enclosure, "warning lines must exist between adjacent cones";

[0051] The end-to-end distance of the adjacent fence edges in the fence type enclosure needs to be within a specified range.

[0052] The above preset condition can be understood as a specific numerical standard for determining whether the distance between adjacent edges of the fence type enclosure is compliant. For example, the preset condition can include:

[0053] When the fence type enclosure is compliant, the distance is less than 1 / 4 of the length of the shorter edge of the adjacent two edges.

[0054] When the distance between the fence type enclosures is too wide, 1 / 4 ≤ the distance < 1 / 2.

[0055] When the fence type enclosure is missing, the distance ≥ 1 / 2.

[0056] The inspection result can be understood as a determination of the association relationship of the feature elements, including "compliance with the preset association relationship" (compliant) or "non-compliance with the preset association relationship" (non-compliant, such as missing warning lines, excessive distance between fences, etc.).

[0057] In this step, the feature elements are sorted according to spatial logic, such as clockwise logic, counterclockwise logic, etc., to determine the positional relationship of adjacent elements. Then, based on the sorted list, it can be verified whether the adjacent elements meet the safety specifications, such as whether there is a warning line between the cone and the bucket, and whether the distance between the fences is compliant.

[0058] Step 105, determining whether the construction site is compliant according to the inspection result.

[0059] In this step, the inspection results of the above association relationships can be combined to output the final compliance conclusion.

[0060] For example, if it is a cone and bucket type enclosure, it needs to check if the missing warning line list is empty: if it is empty, it is determined to be compliant; if there is a missing record, it is determined to be non-compliant and the number of missing records is specified.

[0061] If it is a fence type enclosure, the number of excessively wide distances and the number of missing fences need to be counted: if both are 0, it is determined to be compliant; if there is any count, it is determined to be non-compliant and the type of problem can be specified, such as excessively wide distance or missing fence, etc.

[0062] In this embodiment, by collecting construction enclosure images, identifying features such as cones, warning lines and fence edges, checking the association relationship of adjacent elements and determining the compliance of the construction site, the two types of scene of the construction enclosure are detected for compliance, which solves the problem of missing compliance determination of the construction site in the existing inspection technology. Through clear feature extraction, sorting and association checking logic, the accuracy and comprehensiveness of the construction risk detection in the inspection are improved to some extent, which can more effectively avoid potential safety risks.

[0063] Optionally, sorting the feature elements to obtain an ordered list of feature elements includes:

[0064] Obtain multiple target coordinate values ​​for multiple feature elements, wherein the target coordinate values ​​include the coordinates of the center point of the cone or the coordinates of the endpoint of the fence edge;

[0065] Compare the eigenvalue of the y-coordinate in the target coordinate values ​​with a preset value;

[0066] In this process, multiple first feature elements are sorted according to the size of the feature values ​​of their corresponding x-coordinates to obtain a list of first feature elements, wherein the first feature elements are feature elements whose feature values ​​are less than the preset value.

[0067] Multiple second feature elements are sorted in reverse order according to the size of the feature values ​​of their corresponding x-coordinates to obtain a list of second feature elements, wherein the second feature elements are feature elements whose feature values ​​are greater than or equal to the preset value;

[0068] The first feature element list and the second feature element list are merged to obtain the ordered feature element list.

[0069] In the inspection method of this application embodiment, the above-mentioned target coordinate value can be understood as the position coordinate of the feature element in the image or space, and specifically may include:

[0070] The coordinates of the center point of the cone can be calculated from the coordinates of the diagonal vertices of the cone detection frame;

[0071] The endpoint coordinates of the fence edge can be understood as the coordinates of the two endpoints of the upper edge of the fence detection frame.

[0072] The above preset value can be understood as a pre-set threshold for the y-coordinate used to divide the upper and lower regions of the feature element.

[0073] In some alternative implementations, a target detection model can be used to detect cone targets and obtain a list of cone targets. The center point T of the cone is obtained from equation (1-1). io For the cone target, T is obtained by sorting the points clockwise according to equations (1-2) to (1-4). sort The first feature element mentioned above is sorted in ascending order of its x-coordinate, i.e., T up according to Sorting; the above second feature elements are sorted in descending order of their x-coordinates, i.e., T down according to Reverse sorting, equation (1-4) represents the result of merging the first and second feature element lists to obtain T. sort .

[0074]

[0075] T sort =T up +T down (1-4)

[0076] In other optional embodiments, a target detection model can be used to detect the fence-like target to obtain a quadrilateral detection frame list wherein the coordinates of F i are the corners of the quadrilateral in a clockwise arrangement, is the top-left corner.

[0077] The upper edge of the fence-like target in the list F is extracted to obtain a list F e is sorted according to formula (2-3) to obtain F e_sort in a clockwise arrangement of the fence edge, formula (2-1), (2-2) sort (a, b), inv_sort (a, b) indicates that a is sorted according to b in order.

[0078]

[0079] F e_sort =F e_up +F e_down (2-3)

[0080] In this embodiment, by obtaining the coordinates of the cone barrel center point or the fence edge end point, comparing the y coordinate characteristic value with the preset value to divide the first and second characteristic elements, and then respectively sorting the x coordinate characteristic value in order and in reverse order and merging, the ordered arrangement of the characteristic elements is realized, which provides a clear logical order for the subsequent accurate checking of the correlation between adjacent characteristic elements, and ensures the orderliness and accuracy of the construction fence compliance determination.

[0081] Optionally, in the case where the characteristic elements are the cone barrels and warning lines of the cone barrel type construction fence, the checking of whether the correlation between adjacent characteristic elements conforms to the preset correlation to obtain a checking result comprises:

[0082] Based on the construction fence image, the start and end points of each warning line are extracted, and the start and end points are fitted with a straight line to obtain the slope corresponding to each warning line;

[0083] In the case where the difference between the slopes corresponding to any two warning lines is less than a preset threshold, the start and end points of the two warning lines are merged to obtain a merged warning line set;

[0084] If there is a warning line between adjacent feature elements, the inspection result is in line with the preset association relationship;

[0085] If there is no warning line between adjacent feature elements, the inspection result is not in line with the preset association relationship, and the adjacent feature elements are recorded in a warning line missing list. The total number of records in the warning line missing list is counted to obtain the number of missing warning lines.

[0086] In the inspection method of the embodiments of the present application, the start and end points of the above-mentioned warning line can be understood as the start and end coordinates of each segment of the warning line in the image. The start and end points of the warning line can be fitted by a linear regression algorithm to obtain a straight line equation, and then the slope of the corresponding warning line is obtained through the straight line equation. The above-mentioned merged warning line set is a set formed after merging multiple segments of warning lines with similar slopes into continuous line segments, which can be used to eliminate the interference of local occlusion on the association relationship check. The above-mentioned preset threshold value can be used to judge whether the two segments of warning lines are continuous as a whole. The slope difference threshold value; the above-mentioned warning line missing list can be understood as a list recording the bucket numbers of adjacent buckets without corresponding warning lines, which can be used to count the number of missing warning lines. In some optional embodiments, the warning line missing list can also record the center point coordinates of adjacent buckets to facilitate accurate positioning of the missing position and provide coordinate guidance for on-site rectification of workers; or based on the straight line distance between adjacent buckets, such as the Euclidean distance between two points, record the "estimated length of missing warning line" to help workers prepare warning line materials of corresponding length in advance.

[0087] In some optional embodiments, an image segmentation model can be used to obtain the warning line region, and a warning line segmentation region mask M is returned. Based on the warning line segmentation mask M, the warning line region m i is iterated one by one i The start and end points of each segment of warning line m are obtained by using a contour detection algorithm The corresponding straight line fitting formula is obtained The warning lines m i , m j are screened i The end points of the sub-regions are merged to restore the line breaks caused by local occlusion, i.e. Merging into The sorted warning line set is obtained An ordering process can be performed based on the sorted bucket target list T sort and the post-processed warning line L merge , to determine whether the adjacent buckets T i , Tj whether there is a warning line between the cones, and if there is a missing warning line, recording the missing cone number (T i , T j ) to a missing list E.

[0088] In some optional embodiments, the specific determination method of the missing warning line is as follows: iteratively determining the cone target clockwise, recording the current cone target as T i , and the adjacent target clockwise as T j ; iteratively determining the warning line target clockwise, and recording the current warning line target as m i ; obtaining the fitting formula of the adjacent cone connecting line In L merge , find the warning line close to clockwise from m i ; if there is no such warning line, it is considered that the warning line is missing; continue to iterate the next cone target until the elements in T sort are completely traversed.

[0089] In some optional embodiments, it can be determined whether the list E is empty by the following steps: if the list E is empty, output the compliance determination result “cone type enclosure setting is compliant”; if the list E is not empty, calculate the number n of records in the list E, and issue an alarm “n warning lines are missing, and the enclosure is not compliant!”.

[0090] In this embodiment, by fitting the warning lines, merging the broken warning lines, and then checking the correlation relationship, and finally counting the missing warning lines, the problem of broken warning lines caused by shielding in the cone type enclosure is solved, and it is difficult to determine whether the adjacent cones are effectively connected, which can provide more accurate inspection results for the cone type scene for subsequent “construction site compliance determination”.

[0091] Optionally, in the case where the feature element is a fence edge of the fence type enclosure, the method comprises:

[0092] obtaining the distance between the end points of adjacent fence edges based on the construction enclosure image;

[0093] if the distance between the end points is less than a first ratio of the length of the shorter side of the adjacent two sides, the inspection result is that the correlation relationship meets the preset correlation relationship;

[0094] if the distance between the end points is greater than or equal to the first ratio of the length of the shorter side of the adjacent two sides and less than a second ratio, the inspection result is that the correlation relationship does not meet the preset correlation relationship, and the inspection result further comprises the number of excessively wide distances, and the second ratio is greater than the first ratio;

[0095] If the distance between the end points is greater than or equal to the second ratio of the length of the shorter of the two adjacent sides, it is determined that there is a missing fence between the adjacent fence edges, and the checking result is that the preset correlation is not met, and the checking result further includes the number of missing fences.

[0096] In the inspection method of the embodiments of the present application, the fence edge of the fence type enclosure can be understood as the boundary line of the fence type enclosure, and can be, for example, the upper edge of the fence detection frame, such as the two end points of the upper edge of the quadrilateral detection frame, which can be used to determine whether the fence connection is smooth.

[0097] The distance between the end points of the adjacent fence edges can be understood as the distance between the adjacent end points of the two adjacent fence edges after sorting. The length of the shorter of the two adjacent sides can refer to the smaller one of the lengths of the two adjacent fence edges. The first ratio and the second ratio can be understood as a preset critical ratio for determining compliance of the distance, and the first ratio can be, for example, 1 / 4, and the second ratio can be, for example, 1 / 2, which can be used to distinguish between the three states of “compliance”, “excessively wide distance” and “missing fence”.

[0098] The number of excessively wide distances can be used to record the number of times when the distance between the end points of the adjacent fence edges is in the range of “first ratio ≤ distance < second ratio”. The number of missing fences can be used to record the number of times when the distance between the end points of the adjacent fence edges is greater than or equal to the second ratio.

[0099] For example, reference can be made to Figure 2 For F e_sort The middle edge is traversed one by one. For edge A, the corner points are denoted as A1 and A2, and the length of edge A is denoted as d A The adjacent edge is denoted as B, and the length is d B All edges are traversed and judged according to Table 1, and the excessively wide distance count c d and the missing count c loss are recorded, where dis(a, b) represents the distance between a and b. Table 1 is a compliance judgment table for fence type enclosures:

[0100] Table 1 Compliance Judgment Table for Fence Type Enclosures

[0101]

[0102] In some optional embodiments, it can be determined whether the excessively wide distance count c d and the missing count c loss are 0. If both are 0, the compliance determination result “fence type enclosure setting is compliant” is output. If c d > 0, an alarm “c d excessively wide distance between fences, and the enclosure is not compliant!” is issued. If c loss > 0, an alarm “c lossThe fence is missing, and the fence is not in compliance!

[0103] In this embodiment, by calculating the interval, the contrast value is determined and classified and counted, which solves the quantitative determination problem of the problems such as the connection gap and the interval exceeding the standard caused by improper installation in the fence type fence, and improves the effectiveness of the fence in isolating the construction area to a certain extent, and can better avoid the safety risks caused by protection loopholes.

[0104] Optionally, after determining whether the construction site is compliant according to the inspection result, the method further comprises:

[0105] In the case where it is determined that the construction site is not compliant, an alarm information is generated and sent to a patrol management platform;

[0106] The area of the construction site that is not compliant is marked as a forbidden area in a map, and a detour route applicable to a robot is planned based on the forbidden area.

[0107] In the patrol method of the embodiments of the present application, the above-mentioned alarm information can be understood as prompt information generated when the construction site is determined to be non-compliant, and can include non-compliance types such as missing warning line, fence interval too wide, location, and severity.

[0108] The above-mentioned patrol management platform can be understood as the core control platform of the patrol system, which can be responsible for receiving alarms, managing patrol tasks, and scheduling robots. When it is determined that the construction site is not compliant, such as missing cone warning line, fence interval exceeding the standard, etc., the system can automatically generate an alarm information and send it to the patrol management platform in real time. In some optional embodiments, the corresponding staff or responsible person can be notified by SMS at the same time, to ensure that the relevant parties are aware of the risks in a timely manner.

[0109] The above-mentioned forbidden area can be understood as an area of the construction site that is determined to be non-compliant, which is marked on the map as a range that the robot is prohibited to enter. The robot can mark the range of the area on the map, which will be used as a risk forbidden area for detouring in the later stage. The above-mentioned detour route can be understood as a re-planned patrol path of the robot after avoiding the forbidden area.

[0110] In some optional embodiments, after the alarm information is sent to the inspection management platform, if there is an alarm, the park staff can be notified by short message at the same time. If the robot does not receive the work instruction, it can return to the standby point if it is a robot in the construction risk confirmation task; if it is a robot in the inspection task, it can continue the task according to the subsequent inspection route after the latest no-entry area planning. It should be noted that the above construction risk confirmation task is a directional verification task for "discovered potential construction risk", and the inspection task is a preset routine inspection task. The inspection management platform sets the inspection route and point in advance, and the robot executes according to the plan, covering the safety, environment and other routine inspection of buildings, roads, equipment and other areas in the park.

[0111] In this embodiment, the alarm notification and the no-entry mark on the map facilitate the re-planning of the robot route, solving the problem of "task interruption due to construction risk area and the need to set a new route" in the existing fixed path inspection. On the one hand, the alarm information can ensure that the risk is timely communicated to the staff, facilitating quick rectification; on the other hand, the no-entry area mark and the detour route planning ensure the continuity and safety of the robot inspection, which can improve the ability to respond to construction risks dynamically and to a certain extent, improve the efficiency of the inspection.

[0112] Optionally, before collecting the construction site construction fence image, it further includes:

[0113] controlling the robot to collect the surrounding environment video in real time, uploading the surrounding environment video to the cloud environment for safety analysis, in the case that the analysis result is that there is a construction situation in the travel route or field of view, triggering a construction risk confirmation task, and / or controlling the monitoring system to collect fixed field of view video and uploading to the cloud environment for safety analysis, in the case that the analysis result is that there is a construction fence or construction behavior, sending the monitoring point to the inspection management platform, triggering the construction risk confirmation task;

[0114] sending the construction risk confirmation task to the robot, the robot reaching the vicinity of the risk site according to the construction risk confirmation task, detouring the risk site;

[0115] in the case that the robot confirms that the risk site is the construction site, marking the area where the construction site is located as a no-entry area on the map, and planning a detour route suitable for the robot based on the no-entry area.

[0116] In the inspection method of the embodiments of the present application, the above-mentioned surrounding environment video can be understood as the park environment picture captured by the body video acquisition module in the conventional inspection process of the inspection robot, which is used for preliminary identification of the construction situation. For example, the dynamic environment monitoring or the robot sensor collects the field video V, which can include continuous video frames F_i. The video frames can enter the behavior detection network and the target detection network of the environment safety analysis module respectively. If the personnel construction behavior or the construction related target object such as the fence or the safety helmet is found, it is considered that there is a construction risk, and the construction risk alarm and the risk location P position are sent to the inspection management platform. After the robot arrives near the risk location P, it can bypass the risk area P, confirm the construction range through the laser radar and the ultrasonic radar, mark the area range on the map, and bypass as a risk forbidden area in the later period. If it is confirmed that the construction has ended when the new task passes through, the area is restored to the drivable area.

[0117] The above-mentioned cloud environment safety analysis can be real-time processing of the uploaded video by the environment safety analysis module of the cloud computing center, identifying the construction behavior, fence and other risk characteristics. The above-mentioned construction risk confirmation task can be understood as a directional verification task for suspected construction area, triggered by the cloud analysis result, the purpose is to confirm whether the risk location is a construction site and to judge the compliance.

[0118] The above-mentioned monitoring system can be understood as a pre-set fixed camera group and other hardware facilities in a certain area, which is used for collecting the fixed field video of a specific area as a supplementary monitoring means of the robot inspection. The above-mentioned sending of the monitoring point to the inspection management platform can be understood as that after the monitoring system detects the position information of the construction risk, the position information, i.e. the monitoring point, is sent to the inspection management platform for guiding the robot to go to the verification.

[0119] In this embodiment, the construction risk confirmation task is triggered by the robot and the monitoring system, the robot bypasses and verifies and marks the forbidden area, and plans a new route, which can improve the accuracy of dynamic identification of construction risk and the safety of continuous execution of inspection task, and thus can improve the timeliness of construction risk control and the adaptability of robot inspection in the inspection.

[0120] In some optional embodiments, the robot collects the personnel image or video frame I t (x,y) of the construction site, obtains the person target detection frame and the wearing equipment target detection frame of the construction site based on the target recognition network, wherein c represents different types of wearing equipment such as safety helmet, reflective vest, etc. If the number of is less than the number of or the center position of deviates from by more thanIf the number of the detected equipment is more than half of the width of the equipment, it is considered to be non-compliant and an alarm "construction work wearing non-compliant" is issued. In some other optional embodiments, the construction wearing compliance can also be determined based on the construction site wearing compliance determination, and it is considered to be non-compliant and an alarm is issued. If there is an alarm, the park staff is notified by SMS. If the robot does not receive the work instruction, it returns to the standby point if it is a robot in the construction risk confirmation task; if it is a robot in the inspection task, it continues the task according to the subsequent inspection route based on the latest forbidden area planning.

[0121] In this embodiment, the compliance of personnel wearing is determined by inputting the personnel image collected by the robot into the target recognition network to detect the person and the wearing equipment frame, and comparing the number and position to determine the compliance of personnel wearing. The automation and precision of the construction personnel wearing standard are realized, and the subsequent action is adapted according to the task type to return to the standby point or continue the inspection, which not only ensures the timely and effective construction safety supervision to a certain extent, but also maintains the continuity of the robot inspection task, and can improve the efficiency of construction safety control and inspection.

[0122] In some optional embodiments, in the park environment, the robot inspection architecture diagram can be as shown in Figure 3 The park monitoring system can collect fixed field video by the video collection module to supplement the robot inspection blind area; the robot can complete environment collection, field adjustment, and mobile detour by the environment information collection, mechanical arm field control, and chassis motion control module to support inspection and risk confirmation; the cloud computing center can make video analysis and fence compliance determination by the environment safety and construction risk analysis module, which is the core of risk identification; the inspection management platform can receive alarms and issue tasks based on the alarm management module and the task management module, and can realize risk closed loop and inspection coordination.

[0123] Referring to Figure 4 , Figure 4 is a structural diagram of an inspection device provided by another embodiment of the present application.

[0124] As shown in Figure 4 , the inspection device 400 comprises:

[0125] The collection module 401 is configured to collect a construction fence image of a construction site.

[0126] The identification module 402 is configured to identify feature elements of the construction fence based on the construction fence image, wherein the feature elements comprise a cone barrel of a cone barrel type fence and a warning line, and a fence edge of a fence type fence.

[0127] The sorting module 403 is configured to sort the feature elements to obtain an ordered feature element list.

[0128] The checking module 404 is configured to check whether the correlation between adjacent feature elements conforms to a preset correlation based on the feature element list, to obtain a checking result. The checking of the correlation between adjacent feature elements includes whether the warning line exists between adjacent cone barrels in the cone barrel type enclosure and / or whether the end point distance between adjacent fence edges in the fence type enclosure conforms to a preset condition.

[0129] The determining module 405 is configured to determine whether the construction site is compliant according to the checking result.

[0130] Optionally, the sorting module 403 is further configured to:

[0131] obtain a plurality of target coordinate values of the plurality of feature elements, wherein the target coordinate values include a center point coordinate of a cone barrel or an end point coordinate of a fence edge;

[0132] compare the feature values of y coordinates in the target coordinate values with a preset value;

[0133] wherein the plurality of first feature elements are sorted according to the size of the feature values of the corresponding x coordinates to obtain a first feature element list, and the first feature elements are feature elements with feature values less than the preset value;

[0134] the plurality of second feature elements are sorted according to the inverse order of the size of the feature values of the corresponding x coordinates to obtain a second feature element list, and the second feature elements are feature elements with feature values greater than or equal to the preset value;

[0135] the first feature element list and the second feature element list are merged to obtain the ordered feature element list.

[0136] Optionally, the checking module 404 is further configured to:

[0137] extract the start and end points of each warning line based on the construction enclosure image, and perform linear fitting on the start and end points to obtain the slope corresponding to each warning line;

[0138] in a case where the difference between the slopes corresponding to any two warning lines is less than a preset threshold, the start and end points of the two warning lines are merged to obtain a merged warning line set;

[0139] based on the feature element list and the warning line set, if there is a warning line between adjacent feature elements, the checking result is that the preset correlation is met;

[0140] If there is no warning line between adjacent feature elements, the checking result is that the preset correlation relationship is not met, and the adjacent feature elements are recorded in a warning line missing list. The total number of records in the warning line missing list is counted to obtain the number of missing warning lines.

[0141] Optionally, the inspection module 404 can also be configured to:

[0142] acquire the distance between the endpoints of adjacent fence edges based on the construction fence image;

[0143] If the distance between the endpoints is less than a first ratio of the length of the shorter side of the adjacent two sides, the checking result is that the preset correlation relationship is met.

[0144] If the distance between the endpoints is greater than or equal to the first ratio of the length of the shorter side of the adjacent two sides and less than a second ratio, the checking result is that the preset correlation relationship is not met, and the checking result also includes the number of excessively wide distances, the second ratio being greater than the first ratio.

[0145] If the distance between the endpoints is greater than or equal to the second ratio of the length of the shorter side of the adjacent two sides, it is determined that the fence of the adjacent fence edge is missing, the checking result is that the preset correlation relationship is not met, and the checking result also includes the number of missing fences.

[0146] Optionally, the inspection device 400 can also be configured to:

[0147] In the case where the construction site is determined to be non-compliant, generate an alarm information and send it to the inspection management platform;

[0148] Mark the area of the construction site that is non-compliant as a forbidden area on a map, and plan a detour route for the robot based on the forbidden area.

[0149] Optionally, the inspection device 400 can also be configured to:

[0150] Control the robot to collect real-time surrounding environment video, upload the surrounding environment video to the cloud environment for safety analysis, in the case where the analysis result is that there is a construction situation in the travel route or field of view, trigger a construction risk confirmation task, and / or control the monitoring system to collect fixed field of view video and upload it to the cloud environment for safety analysis, in the case where the analysis result is that there is a construction fence or construction behavior, send the monitoring point to the inspection management platform, and trigger the construction risk confirmation task;

[0151] Send the construction risk confirmation task to the robot, and the robot reaches the vicinity of the risk location according to the construction risk confirmation task and detours around the risk location.

[0152] In a case where the robot confirms that the risk location is the construction site, marking an area where the construction site is located as a forbidden area on a map, and planning a detour route applicable to the robot based on the forbidden area.

[0153] Specifically, referring to Figure 5 As shown in the figure, the embodiment of the application further provides an electronic device, including a bus 501, a transceiver 502, an antenna 503, a bus interface 504, a processor 505 and a memory 506.

[0154] The processor 505 is configured to:

[0155] Collect a construction fence image of the construction site;

[0156] Based on the construction fence image, identify feature elements of the construction fence, the feature elements including cones and warning lines of a cone fence, and fence edges of a fence;

[0157] Sort the feature elements to obtain an ordered feature element list;

[0158] Based on the feature element list, check whether an association relationship between adjacent feature elements conforms to a preset association relationship to obtain a checking result; wherein, the checking of the association relationship between adjacent feature elements includes whether the warning lines exist between adjacent cones in the cone fence, and / or whether an end point distance between adjacent fence edges in the fence conforms to a preset condition;

[0159] According to the checking result, determine whether the construction site is compliant.

[0160] In the Figure 5 bus architecture (represented by the bus 501), the bus 501 can include any number of interconnected buses and bridges, which link various circuits including one or more processors represented by the processor 505 and the memory represented by the memory 506. The bus 501 can also link various other circuits such as peripheral devices, voltage stabilizers and power management circuits, which are well known in the art, and therefore, will not be further described herein. The bus interface 504 provides an interface between the bus 501 and the transceiver 502. The transceiver 502 can be one element or multiple elements such as multiple receivers and transmitters, which provide a unit for communicating with various other devices on a transmission medium. Data processed by the processor 505 is transmitted on a wireless medium through the antenna 503, and further, the antenna 503 also receives data and transmits the data to the processor 505.

[0161] The processor 505 is responsible for managing the bus 501 and general processing, and can also provide various functions, including timing, peripheral interface, voltage regulation, power management, and other control functions. The memory 506 can be used to store data used by the processor 505 in performing operations.

[0162] Optionally, the processor 505 can be a central processing unit (CPU), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or a complex programmable logic device (CPLD).

[0163] Optionally, the processor 505 is specifically configured to:

[0164] Obtain a plurality of target coordinate values of the plurality of feature elements, wherein the target coordinate values include a center point coordinate of a cone barrel or an end point coordinate of a fence edge;

[0165] Compare the feature values of the y coordinates in the target coordinate values with the preset value;

[0166] Wherein, the plurality of first feature elements are sorted according to the size of the feature values of the corresponding x coordinates to obtain a first feature element list, and the first feature elements are feature elements with feature values less than the preset value;

[0167] The plurality of second feature elements are sorted according to the inverse order of the size of the feature values of the corresponding x coordinates to obtain a second feature element list, and the second feature elements are feature elements with feature values greater than or equal to the preset value;

[0168] The first feature element list and the second feature element list are merged to obtain the ordered feature element list.

[0169] Optionally, the processor 505 is specifically configured to:

[0170] Based on the construction fence image, extract the start and end points of each warning line, and perform linear fitting on the start and end points to obtain the slope corresponding to each warning line;

[0171] In the case where the difference between the slopes corresponding to any two segments of warning lines is less than a preset threshold, the start and end points of the two segments of warning lines are merged to obtain a merged warning line set;

[0172] determining, based on the feature element list and the set of warning lines, that the inspection result is in conformity with the preset association relationship if there is a warning line between adjacent feature elements;

[0173] if there is no warning line between adjacent feature elements, the inspection result is not in conformity with the preset association relationship, and the adjacent feature elements are recorded in a warning line missing list, and a total number of records in the warning line missing list is counted to obtain a number of missing warning lines.

[0174] Optionally, the processor 505 is specifically configured to:

[0175] obtaining an end-to-end distance between adjacent fence edges based on the construction fence image;

[0176] if the end-to-end distance is less than a first ratio of the length of the shorter side of the adjacent two sides, the inspection result is in conformity with the preset association relationship;

[0177] if the end-to-end distance is greater than or equal to the first ratio of the length of the shorter side of the adjacent two sides and less than a second ratio, the inspection result is not in conformity with the preset association relationship, and the inspection result further includes a number of excessively wide distances, and the second ratio is greater than the first ratio;

[0178] if the end-to-end distance is greater than or equal to the second ratio of the length of the shorter side of the adjacent two sides, it is determined that there is a missing fence between the adjacent fence edges, the inspection result is not in conformity with the preset association relationship, and the inspection result further includes a number of missing fences.

[0179] Optionally, the processor 505 is specifically configured to:

[0180] generating an alarm information and sending it to the inspection management platform if it is determined that the construction site is not in conformity with the regulation;

[0181] marking the area of the construction site that is not in conformity with the regulation as a forbidden area in a map, and planning a detour route applicable to the robot based on the forbidden area.

[0182] Optionally, the processor 505 is specifically configured to:

[0183] controlling the robot to collect a surrounding environment video in real time, uploading the surrounding environment video to a cloud environment for safety analysis, triggering a construction risk confirmation task if the analysis result is that there is a construction situation in the travel route or field of view, and / or controlling the monitoring system to collect a fixed field of view video and uploading it to the cloud environment for safety analysis, sending the monitoring point to the inspection management platform if the analysis result is that there is a construction fence or construction behavior, and triggering the construction risk confirmation task;

[0184] send the construction risk confirmation task to the robot, and the robot reaches the vicinity of the risk location according to the construction risk confirmation task, and detours the risk location;

[0185] in a case where the robot confirms that the risk location is the construction site, mark the area where the construction site is located as a forbidden area on a map, and plan a detour route applicable to the robot based on the forbidden area.

[0186] It should be noted that the electronic device provided by the embodiments of the present application is a device capable of executing the above-mentioned inspection method, and all implementation manners in the above-mentioned inspection method embodiments are applicable to the electronic device, and all can achieve the same or similar beneficial effects. To avoid repetition, the embodiments will not be described again.

[0187] The embodiments of the present application also provide an electronic device, comprising a processor, a memory, and a program stored in the memory and executable on the processor, wherein the program is executed by the processor to implement each process of the above-mentioned inspection method embodiments, and achieve the same technical effects. To avoid repetition, this will not be described again.

[0188] The embodiments of the present application also provide a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement each process of the above-mentioned inspection method embodiments, and achieve the same technical effects. To avoid repetition, this will not be described again. The computer readable storage medium includes a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0189] The embodiments of the present application also provide a computer program product, comprising computer instructions, which are executed by a processor to implement each process of the above-mentioned inspection method embodiments, and achieve the same technical effects. To avoid repetition, this will not be described again.

[0190] It should be noted that in this document, the term "comprising" or "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or device. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or device including the element.

[0191] Through the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned example methods can be realized by means of software and a necessary general hardware platform, and of course, can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product in essence or in the form of a part that contributes to the prior art. The computer software product is stored in a storage medium (such as a ROM / RAM, a magnetic disk, or an optical disk) and includes a plurality of instructions for causing a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in the various embodiments of the present application.

[0192] The embodiments of the present application are described above in combination with the drawings, but the present application is not limited to the above-mentioned specific embodiments, and the above-mentioned specific embodiments are only illustrative and not restrictive. Those skilled in the art can make many forms without departing from the scope of the present application under the inspiration of the present application, and all of them belong to the protection of the present application.

Claims

1. An inspection method, characterized in that, The method includes: Collect images of the construction site fencing; Based on the construction site enclosure image, the feature elements of the construction site enclosure are identified, including cones and warning lines for cone-type enclosures, and the fence edges for fence-type enclosures. The feature elements are sorted to obtain an ordered list of feature elements; Based on the list of feature elements, check whether the relationship between adjacent feature elements conforms to the preset relationship to obtain the check result; wherein, checking the relationship between adjacent feature elements includes whether the warning line exists between adjacent cones in the cone-type enclosure, and / or whether the distance between the endpoints of adjacent fence edges in the fence-type enclosure meets the preset conditions; Based on the inspection results, it is determined whether the construction site is compliant.

2. The method according to claim 1, characterized in that, The step of sorting the feature elements to obtain an ordered list of feature elements includes: Obtain multiple target coordinate values ​​for multiple feature elements, wherein the target coordinate values ​​include the coordinates of the center point of the cone or the coordinates of the endpoint of the fence edge; Compare the eigenvalue of the y-coordinate in the target coordinate values ​​with a preset value; In this process, multiple first feature elements are sorted according to the size of the feature values ​​of their corresponding x-coordinates to obtain a list of first feature elements, wherein the first feature elements are feature elements whose feature values ​​are less than the preset value. Multiple second feature elements are sorted in reverse order according to the size of the feature values ​​of their corresponding x-coordinates to obtain a list of second feature elements, wherein the second feature elements are feature elements whose feature values ​​are greater than or equal to the preset value; The first feature element list and the second feature element list are merged to obtain the ordered feature element list.

3. The method according to claim 2, characterized in that, When the feature elements are the cones and warning lines of the cone-type enclosure, checking whether the correlation between adjacent feature elements conforms to a preset correlation relationship to obtain the check result includes: Based on the construction site fence image, the start and end points of each warning line segment are extracted, and a straight line is fitted to the start and end points to obtain the slope corresponding to each warning line segment. If the difference between the slopes of any two warning lines is less than a preset threshold, the start and end points of the two warning lines are merged to obtain a merged set of warning lines. Based on the list of feature elements and the set of warning lines, if there is a warning line between adjacent feature elements, the inspection result is that it conforms to the preset association relationship. If there is no warning line between adjacent feature elements, the inspection result is that the preset association relationship is not met, and the adjacent feature elements are recorded in the warning line missing list. The total number of records in the warning line missing list is counted to obtain the number of warning lines missing.

4. The method according to claim 2, characterized in that, When the feature element is the edge of the fence of the fence type, the step of checking whether the association relationship between adjacent feature elements conforms to a preset association relationship to obtain the check result includes: The distance between the endpoints of adjacent fence edges is obtained based on the construction site fence image; If the distance between the endpoints is less than the first ratio of the length of the shorter side among the two adjacent sides, then the inspection result is in accordance with the preset correlation relationship; If the endpoint spacing is greater than or equal to the first ratio of the shorter side length of the two adjacent sides and less than the second ratio, then the inspection result is that it does not conform to the preset association relationship. The inspection result also includes the number of times the spacing is too wide, and the second ratio is greater than the first ratio. If the distance between the endpoints is greater than or equal to the second ratio of the shorter side length of the two adjacent sides, it is determined that the fence of the adjacent fence edge is missing. The inspection result is that it does not conform to the preset association relationship. The inspection result also includes the number of missing fences.

5. The method according to any one of claims 1 to 4, characterized in that, After determining whether the construction site is compliant based on the inspection results, the process further includes: If the construction site is determined to be non-compliant, an alarm message is generated and sent to the inspection management platform; The non-compliant areas of the construction site are marked as no-entry zones on the map, and detour routes suitable for robots are planned based on the no-entry zones.

6. The method according to any one of claims 1 to 4, characterized in that, Before acquiring images of the construction site enclosure, the following steps are also included: The robot is controlled to collect real-time video of the surrounding environment and upload the video to the cloud environment for safety analysis. If the analysis result indicates that there is construction in the route or field of view, a construction risk confirmation task is triggered. Or, the monitoring system is controlled to collect fixed field of view video and upload it to the cloud environment for safety analysis. If the analysis result indicates that there is construction enclosure or construction activity, the monitoring point is sent to the inspection management platform, triggering a construction risk confirmation task. The construction risk confirmation task is sent to the robot, and the robot arrives near the risk location according to the construction risk confirmation task and detours around the risk location; If the robot confirms that the risk location is the construction site, the area where the construction site is located is marked as a restricted area on the map, and a detour route suitable for the robot is planned based on the restricted area.

7. An inspection device, characterized in that, include: The data acquisition module is used to acquire images of construction site fencing. The recognition module is used to identify the feature elements of the construction site enclosure based on the construction site enclosure image. The feature elements include cones and warning lines for cone-type enclosures, and the fence edges for fence-type enclosures. The sorting module is used to sort the feature elements to obtain an ordered list of feature elements; The inspection module is used to check whether the association relationship between adjacent feature elements conforms to the preset association relationship based on the feature element list, so as to obtain the inspection result; wherein, the inspection of the association relationship between adjacent feature elements includes whether the warning line exists between adjacent cones in the cone-type fence, and / or whether the endpoint spacing of adjacent fence edges in the fence-type fence meets the preset conditions; Based on the inspection results, it is determined whether the construction site is compliant.

8. An electronic device, characterized in that, include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the inspection method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the inspection method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, It includes computer instructions, which, when executed by a processor, implement the steps of the inspection method as described in any one of claims 1 to 6.

Images