A method, device, equipment, medium and program product for inspection

Abstract

The application provides a method, device, equipment, medium and program product for inspection, and relates to the field of risk control. The method comprises the following steps: collecting a construction fence image of a construction site; identifying feature elements of the construction fence based on the construction fence image, wherein the feature elements comprise cones and warning lines of a cone fence, and fence edges of a fence; sorting the feature elements to obtain an ordered feature element list; checking whether a correlation between adjacent feature elements conforms to a preset correlation based on the feature element list to obtain a checking result; wherein the checking of the correlation between adjacent feature elements comprises checking whether the warning lines exist between adjacent cones in the cone fence, and / or checking whether the distance between the end points of adjacent fence edges in the fence conforms to a preset condition; and determining whether the construction site is compliant according to the checking result. The accuracy and comprehensiveness of construction risk detection are improved.

Description

Technical Field

[0001] This application relates to the field of risk control, and in particular to an inspection method, apparatus, equipment, medium, and procedure product. Background Technology

[0002] Inspection is an important management task, and its scope includes building safety, road cleanliness, equipment operation, and green environment. However, existing inspection methods are not comprehensive enough to check the safety compliance of construction sites, and there are safety loopholes. Summary of the Invention

[0003] This application provides an inspection method, device, equipment, and storage medium to address the problem that existing inspection methods lack comprehensiveness in checking the safety compliance of construction sites and have security vulnerabilities.

[0004] To solve the above-mentioned technical problems, this application is implemented as follows:

[0005] In a first aspect, embodiments of this application provide an inspection method, the method comprising:

[0006] Collect images of the construction site fencing;

[0007] Based on the construction site enclosure image, the feature elements of the construction site enclosure are identified, including cones and warning lines for cone-type enclosures, and the fence edges for fence-type enclosures.

[0008] The feature elements are sorted to obtain an ordered list of feature elements;

[0009] Based on the list of feature elements, check whether the relationship between adjacent feature elements conforms to the preset relationship to obtain the check result; wherein, checking the relationship between adjacent feature elements includes whether the warning line exists between adjacent cones in the cone-type enclosure, and / or whether the distance between the endpoints of adjacent fence edges in the fence-type enclosure meets the preset conditions;

[0010] Based on the inspection results, it is determined whether the construction site is compliant.

[0011] Secondly, embodiments of this application also provide an inspection device. The inspection device includes:

[0012] The data acquisition module is used to acquire images of construction site fencing.

[0013] The recognition module is used to identify the feature elements of the construction site enclosure based on the construction site enclosure image. The feature elements include cones and warning lines for cone-type enclosures, and the fence edges for fence-type enclosures.

[0014] The sorting module is used to sort the feature elements to obtain an ordered list of feature elements;

[0015] The inspection module is used to check whether the association relationship between adjacent feature elements conforms to the preset association relationship based on the feature element list, so as to obtain the inspection result; wherein, the inspection of the association relationship between adjacent feature elements includes whether the warning line exists between adjacent cones in the cone-type fence, and / or whether the endpoint spacing of adjacent fence edges in the fence-type fence meets the preset conditions;

[0016] The judgment module is used to determine whether the construction site is compliant based on the inspection results.

[0017] Thirdly, embodiments of this application provide an electronic device, including a transceiver and a processor, wherein the processor is used for:

[0018] Collect images of the construction site fencing;

[0019] Based on the construction site enclosure image, the feature elements of the construction site enclosure are identified, including cones and warning lines for cone-type enclosures, and the fence edges for fence-type enclosures.

[0020] The feature elements are sorted to obtain an ordered list of feature elements;

[0021] Based on the list of feature elements, check whether the relationship between adjacent feature elements conforms to the preset relationship to obtain the check result; wherein, checking the relationship between adjacent feature elements includes whether the warning line exists between adjacent cones in the cone-type enclosure, and / or whether the distance between the endpoints of adjacent fence edges in the fence-type enclosure meets the preset conditions;

[0022] Based on the inspection results, it is determined whether the construction site is compliant.

[0023] Fourthly, embodiments of this application provide an electronic device, including: a processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, it implements the steps of the inspection method described in the first aspect above.

[0024] Fifthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the inspection method described in the first aspect above.

[0025] Sixthly, embodiments of this application provide a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the inspection method as described in the first aspect above.

[0026] The inspection method of this application includes acquiring images of construction site fencing; identifying feature elements of the construction site fencing based on the images, including cones and warning lines in cone-type fencing and fence edges in fence-type fencing; sorting the feature elements to obtain an ordered list of feature elements; checking whether the correlation between adjacent feature elements conforms to a preset correlation relationship based on the feature element list to obtain an inspection result; wherein, checking the correlation between adjacent feature elements includes whether the warning line exists between adjacent cones in cone-type fencing, and / or whether the distance between the endpoints of adjacent fence edges in fence-type fencing meets a preset condition; and determining whether the construction site is compliant based on the inspection result. This method, by acquiring images of construction site fencing, identifying feature elements such as cones, warning lines, and fence edges, checking the correlation between adjacent elements, and determining the compliance of the construction site, achieves targeted compliance detection for two types of construction site fencing scenarios; through clear feature extraction, sorting, and correlation checking logic, this method improves the accuracy and comprehensiveness of construction risk detection during inspections to a certain extent, and can more effectively avoid potential safety risks. Attached Figure Description

[0027] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart of the inspection method provided in the embodiments of this application;

[0029] Figure 2 This is a schematic diagram of a fence-like enclosure provided in one embodiment of this application;

[0030] Figure 3 This is a diagram of a robot-based inspection architecture provided in one embodiment of this application;

[0031] Figure 4 This is a structural diagram of an inspection device provided in one embodiment of this application;

[0032] Figure 5 This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0033] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0034] This application provides an inspection method. See also... Figure 1 , Figure 1 This is a flowchart of the inspection method provided in the embodiments of this application, such as... Figure 1 As shown, it includes the following steps:

[0035] Step 101: Collect images of the construction site enclosure;

[0036] The aforementioned construction site can be understood as an area where construction work is underway, which may include the work site and the necessary surrounding protection area; the aforementioned construction fence image can be understood as an image or video frame obtained through image acquisition equipment, such as the camera of an inspection robot, a park surveillance camera, etc., which includes protective facilities in the construction area, such as cones, fences, etc.

[0037] The aforementioned construction site fencing can be understood as protective facilities used to separate construction areas from non-construction areas, and can specifically include the following two categories:

[0038] Cone-type barriers: Temporary protective facilities consisting of cones and warning lines must be "straight in line and completely surround the construction area";

[0039] Fence-type barriers: Protective facilities consisting of continuous fences must be "smoothly connected, without gaps or tilts".

[0040] The specific method of image acquisition is not limited in this application embodiment. It can be that the inspection robot takes real-time pictures through the video acquisition module on its body during the inspection task; it can be that a drone takes pictures of the construction area along a preset route to obtain multi-angle images of the fence; or it can be that a fixed monitoring camera that has been deployed can be used to collect images of the fixed field of view of the construction risk area.

[0041] In this step, image data of the protective facilities in the construction area can be acquired using image acquisition equipment, providing a basis for subsequent analysis.

[0042] Step 102: Based on the construction site enclosure image, identify the feature elements of the construction site enclosure, including cones and warning lines for cone-type enclosures, and the fence edges for fence-type enclosures.

[0043] In this step, the aforementioned feature elements may refer to key components used to determine the compliance of the construction site.

[0044] In this step, key components, including cones, warning lines, and fence edges, can be extracted from the construction site enclosure image using image recognition technology. Alternatively, a LiDAR-equipped inspection robot can be used to perform a 3D scan of the construction site enclosure area to obtain point cloud data. Point cloud segmentation and feature extraction algorithms are then used to identify the spatial location of cones in cone-type enclosures (based on the height and shape features of the point cloud), the continuous line features of warning lines (based on the linear distribution of the point cloud), and the edge contours of fence-type enclosures (based on the planar or continuous columnar features of the point cloud), thereby extracting the corresponding feature elements. Furthermore, data from the robot's ultrasonic and millimeter-wave radars can be combined to assist in identifying feature elements. Different methods for identifying feature elements do not affect the implementation of the basic functions of the embodiments of this application.

[0045] Step 103: Sort the feature elements to obtain an ordered list of feature elements;

[0046] In this step, the aforementioned ordered list of feature elements can be understood as a list formed by sorting the identified feature elements according to specific rules such as spatial location and arrangement order. The purpose is to clarify the logical relationship between "adjacent feature elements".

[0047] This application does not specify a particular method for sorting feature elements. Sorting can be done using the coordinate information of the feature elements. For example, feature elements can be divided into two groups based on the comparison results of their y-coordinate values ​​with preset values. The upper group can be sorted by x-coordinate values ​​from smallest to largest, and the lower group can be sorted by x-coordinate values ​​from largest to smallest. Then, the two groups can be merged to obtain an ordered list arranged clockwise. Alternatively, sorting can be done counterclockwise based on spatial location. For example, the upper and lower groups can be divided by y-coordinate values. The upper group can be sorted by x-coordinate values ​​from largest to smallest, and the lower group can be sorted by x-coordinate values ​​from smallest to largest. The geometric center of the construction area or the initial observation point of the robot can also be used as a reference point. The straight-line distance from each feature element to the reference point can be calculated, and the feature elements can be sorted by distance from nearest to farthest or farthest to nearest.

[0048] Step 104: Based on the feature element list, check whether the association relationship between adjacent feature elements conforms to the preset association relationship to obtain the check result; wherein, checking the association relationship between adjacent feature elements includes whether the warning line exists between adjacent cones in the cone-type enclosure, and / or whether the endpoint spacing of adjacent fence edges in the fence-type enclosure meets the preset conditions;

[0049] In this step, the aforementioned preset association relationship can be understood as a pre-defined rule used to determine whether the feature elements conform to security specifications, specifically including:

[0050] For cone-type fencing, "a warning line must be present between adjacent cones";

[0051] For fence-type enclosures, "the distance between the endpoints of adjacent fence edges must be within the specified range."

[0052] The aforementioned preset conditions can be understood as specific numerical standards for determining whether the spacing between adjacent edges of a fence or similar structure is compliant. For example, the preset conditions may include:

[0053] When the aforementioned fences or barriers are compliant, the spacing must be less than 1 / 4 of the shorter side length of any two adjacent sides.

[0054] When the spacing of the aforementioned fences is too wide, 1 / 4 ≤ the aforementioned spacing < 1 / 2;

[0055] When the above-mentioned fences or barriers are missing: the above-mentioned spacing is ≥1 / 2.

[0056] The above inspection results can be understood as a judgment on the relationship between feature elements, including "meets the preset relationship" (compliant) or "does not meet the preset relationship" (non-compliant, such as missing warning lines, excessively wide fence spacing, etc.).

[0057] In this step, the feature elements are first sorted according to spatial logic, such as clockwise or counterclockwise, to clarify the positional relationships between adjacent elements. Then, based on the sorted list, it can be verified whether adjacent elements meet safety regulations, such as whether there are warning lines between cones and whether the spacing between fences is compliant.

[0058] Step 105: Based on the inspection results, determine whether the construction site is compliant.

[0059] In this step, the results of the above-mentioned checks on related relationships can be combined to output a final compliance conclusion.

[0060] For example, if it is a cone-shaped fence, it is necessary to check whether the list of missing warning lines is empty: if it is empty, it is considered compliant; if there are missing records, it is considered non-compliant and the number of missing records is specified.

[0061] For fences, it is necessary to count the number of excessively wide spacing and the number of missing fences: if both are 0, it is considered compliant; if either count exists, it is considered non-compliant and the type of problem can be clearly identified, such as excessively wide spacing or missing fences.

[0062] In this implementation, by collecting images of construction site hoardings, identifying feature elements such as cones, warning lines, and fence edges, checking the correlation between adjacent elements, and determining the compliance of the construction site, targeted compliance detection for two types of construction site hoarding scenarios is achieved. This method solves the problem of the lack of compliance determination for construction site hoardings in existing inspection technologies. Through clear feature extraction, sorting, and correlation inspection logic, it improves the accuracy and comprehensiveness of construction risk detection during inspections to a certain extent, and can more effectively avoid potential safety risks.

[0063] Optionally, sorting the feature elements to obtain an ordered list of feature elements includes:

[0064] Obtain multiple target coordinate values ​​for multiple feature elements, wherein the target coordinate values ​​include the coordinates of the center point of the cone or the coordinates of the endpoint of the fence edge;

[0065] Compare the eigenvalue of the y-coordinate in the target coordinate values ​​with a preset value;

[0066] In this process, multiple first feature elements are sorted according to the size of the feature values ​​of their corresponding x-coordinates to obtain a list of first feature elements, wherein the first feature elements are feature elements whose feature values ​​are less than the preset value.

[0067] Multiple second feature elements are sorted in reverse order according to the size of the feature values ​​of their corresponding x-coordinates to obtain a list of second feature elements, wherein the second feature elements are feature elements whose feature values ​​are greater than or equal to the preset value;

[0068] The first feature element list and the second feature element list are merged to obtain the ordered feature element list.

[0069] In the inspection method of this application embodiment, the above-mentioned target coordinate value can be understood as the position coordinate of the feature element in the image or space, and specifically may include:

[0070] The coordinates of the center point of the cone can be calculated from the coordinates of the diagonal vertices of the cone detection frame;

[0071] The endpoint coordinates of the fence edge can be understood as the coordinates of the two endpoints of the upper edge of the fence detection frame.

[0072] The above preset value can be understood as a pre-set threshold for the y-coordinate used to divide the upper and lower regions of the feature element.

[0073] In some alternative implementations, a target detection model can be used to detect cone targets and obtain a list of cone targets. The center point T of the cone is obtained from equation (1-1). io For the cone target, T is obtained by sorting the points clockwise according to equations (1-2) to (1-4). sort The first feature element mentioned above is sorted in ascending order of its x-coordinate, i.e., T up according to Sorting; the above second feature elements are sorted in descending order of their x-coordinates, i.e., T down according to Reverse sorting, equation (1-4) represents the result of merging the first and second feature element lists to obtain T. sort .

[0074]

[0075] T sort =T up +T down (1-4)

[0076] In some other alternative implementations, a target detection model can be used to detect fence-like targets, resulting in a list of quadrilateral detection boxes. Where F i The coordinates are the corner points of the quadrilateral arranged clockwise. The top left corner point.

[0077] Extract the top edge of the fence-type targets in list F to obtain the list. For F e According to equation (2-3), the F values ​​arranged clockwise along the edge of the fence are obtained. e_sort In equations (2-1) and (2-2), sort(a,b) and inv_sort(a,b) represent sorting a in order and in reverse order according to b.

[0078]

[0079] F e_sort =F e_up +F e_down (2-3)

[0080] In this implementation, by obtaining the coordinates of the center point of the cone or the edge endpoint of the fence, comparing the y-coordinate feature value with the preset value to divide the first and second feature elements, and then merging them according to the ascending and descending order of the x-coordinate feature values ​​respectively, the orderly arrangement of feature elements is realized. This provides a clear logical order for subsequent accurate inspection of the relationship between adjacent feature elements, ensuring the orderliness and accuracy of the compliance determination of the construction fence.

[0081] Optionally, when the feature elements are the cones and warning lines of the cone-type enclosure, checking whether the association relationship between adjacent feature elements conforms to a preset association relationship to obtain the check result includes:

[0082] Based on the construction site fence image, the start and end points of each warning line segment are extracted, and a straight line is fitted to the start and end points to obtain the slope corresponding to each warning line segment.

[0083] If the difference between the slopes of any two warning lines is less than a preset threshold, the start and end points of the two warning lines are merged to obtain a merged set of warning lines.

[0084] Based on the list of feature elements and the set of warning lines, if there is a warning line between adjacent feature elements, the inspection result is that it conforms to the preset association relationship.

[0085] If there is no warning line between adjacent feature elements, the inspection result is that the preset association relationship is not met, and the adjacent feature elements are recorded in the warning line missing list. The total number of records in the warning line missing list is counted to obtain the number of warning lines missing.

[0086] In the inspection method of this application embodiment, the start and end points of the warning line can be understood as the start and end coordinates of each warning line segment in the image. The start and end points of the warning line can be fitted by a linear regression algorithm to obtain a straight line equation, and then the slope of the corresponding warning line can be obtained through the straight line equation. The above-mentioned merged warning line set can be formed by merging multiple warning lines with similar slopes into a set of continuous line segments, which can be used to eliminate the interference of local occlusion on the correlation inspection. The above-mentioned preset threshold can be used to determine the critical value of the slope difference between two warning lines as a continuous whole; the above-mentioned warning line missing list can be understood as a list recording the cone numbers of adjacent cones without corresponding warning lines, which can be used to count the number of missing warning lines. In some optional embodiments, the warning line missing list can also record the center point coordinates of adjacent cones to facilitate accurate location of the missing position and provide coordinate guidance for on-site rectification by staff; it can also be calculated based on the straight-line distance between adjacent cones, such as the Euclidean distance between two points, and recorded as "estimated length of missing warning line" to help staff prepare warning line materials of the corresponding length in advance.

[0087] In some alternative implementations, an image segmentation model can be used to obtain the warning line regions, returning a warning line segmentation region mask M. The warning line regions m are then iterated sequentially based on the warning line segmentation mask M. i The contour detection algorithm is used to obtain each warning line segment m. i start and end points The corresponding linear fitting formula is obtained. filter Warning line m i m j Merge the endpoints of its sub-regions to restore line breaks caused by local occlusion, i.e., m i → merged into Obtain the merged and sorted set of warning lines. It can be further sorted, based on the sorted list of bucket targets T. sort and post-processing warning line L merge Determine the adjacent cones T within the enclosed area. i ,Tj Are there any warning lines between cones? If any warning lines are missing, record the number of the missing cone (T). i ,T j ) to the missing list E.

[0088] In some optional implementations, the specific method for determining the missing warning line is as follows: Iterate through the cone targets clockwise, and denote the current cone target as T. i The adjacent target in the clockwise direction is T. j Similarly, iterate through the warning line targets clockwise, and denote the current warning line target as m. i ; Obtain the fitting equation for the line connecting adjacent cones In L merge From m i Start searching clockwise and If a similar warning line does not exist, it is considered that the warning line is missing; continue iterating to the next cone target, until T. sort Complete traversal of all elements.

[0089] In some optional implementations, the following steps can be used to determine whether list E is empty. If list E is empty, the compliance determination result "cone-type fence setting is compliant" is output; if list E is not empty, the number of records n in list E is calculated, and an alarm "n warning lines are missing, fence is not compliant!" is issued.

[0090] In this implementation, by fitting the warning lines, merging the broken warning lines, checking the correlation, and finally counting the missing warning lines, the problem of warning lines breaking due to obstruction and difficulty in determining whether adjacent cones are effectively connected in cone-type enclosures is solved. This can provide more accurate inspection results for cone-type scenarios in subsequent "construction site compliance determination".

[0091] Optionally, when the feature element is the edge of the fence of the fence type, the step of checking whether the association relationship between adjacent feature elements conforms to a preset association relationship to obtain the check result includes:

[0092] The distance between the endpoints of adjacent fence edges is obtained based on the construction site fence image;

[0093] If the distance between the endpoints is less than the first ratio of the length of the shorter side among the two adjacent sides, then the inspection result is in accordance with the preset correlation relationship;

[0094] If the endpoint spacing is greater than or equal to the first ratio of the shorter side length of the two adjacent sides and less than the second ratio, then the inspection result is that it does not conform to the preset association relationship. The inspection result also includes the number of times the spacing is too wide, and the second ratio is greater than the first ratio.

[0095] If the distance between the endpoints is greater than or equal to the second ratio of the shorter side length of the two adjacent sides, it is determined that the fence of the adjacent fence edge is missing. The inspection result is that it does not conform to the preset association relationship. The inspection result also includes the number of missing fences.

[0096] In the inspection method of this application embodiment, the fence edge of the above-mentioned fence-like enclosure can be understood as the boundary line of the fence-like enclosure. For example, it can be the upper edge of the fence detection frame, such as the two upper ends of the quadrilateral detection frame, which can be used to determine whether the fence connection is smooth.

[0097] The aforementioned spacing between the endpoints of adjacent fence edges can be understood as the distance between the adjacent endpoints of two adjacent fence edges after sorting. The shorter side length among the two adjacent sides can refer to the smaller of the lengths of the two adjacent fence edges. The aforementioned first ratio and second ratio can be understood as preset critical ratios for determining the compliance of spacing. For example, the first ratio can be 1 / 4 and the second ratio can be 1 / 2, which can be used to distinguish between the three states of "compliant", "spacing too wide", and "fence missing".

[0098] The number of times the spacing is too wide can be used to record the number of times the distance between the endpoints of adjacent fence edges falls within the range of "first ratio ≤ spacing < second ratio". The number of times the fence is missing can be used to record the number of times the distance between the endpoints of adjacent fence edges is greater than or equal to the second ratio.

[0099] For example, you can refer to Figure 2 For F e_sort Traverse the edges one by one. For edge A, denote its corner points as A1 and A2, and the length of edge A as d. A Let the adjacent edge be B, and its length be d. B According to Table 1, traverse and judge all edges, and record the number of edges with excessively wide spacing c. d and missing count c loss Where dis(a,b) represents the calculation of the distance between a and b. Table 1 is the compliance judgment table for fences and barriers:

[0100] Table 1 Compliance Judgment Table for Fences and Barriers

[0101]

[0102] In some alternative implementations, the spacing can be determined by counting c if it is too wide. d and missing count c loss If both are 0, output the compliance judgment result "Fence-type enclosure setting is compliant"; if c d If the value is greater than 0, an alarm "c" will be issued. d The spacing between the fence sections is too wide; the fencing is not up to standard! If c loss If the value is greater than 0, an alarm "c" will be issued. lossThe fence is missing and the enclosure is not up to standard!

[0103] In this implementation, by calculating the spacing, comparing the values, and making judgments and statistical counts, the problem of quantitatively judging problems such as connection gaps and excessive spacing caused by improper installation in fence-type barriers is solved. To a certain extent, this improves the effectiveness of the fence in isolating the construction area and can better avoid safety risks caused by protective loopholes.

[0104] Optionally, after determining whether the construction site is compliant based on the inspection results, the method further includes:

[0105] If the construction site is determined to be non-compliant, an alarm message is generated and sent to the inspection management platform;

[0106] The non-compliant areas of the construction site are marked as no-entry zones on the map, and detour routes suitable for robots are planned based on the no-entry zones.

[0107] In the inspection method of this application embodiment, the above alarm information can be understood as prompt information generated when the construction site is determined to be non-compliant. It may include the type of non-compliance, such as missing warning lines, excessively wide fence spacing, etc., as well as the location and severity.

[0108] The aforementioned inspection management platform can be understood as the core control platform of the inspection system, responsible for receiving alarms, managing inspection tasks, and scheduling robots. When a non-compliant construction site is determined, such as missing cone warning lines or excessive fence spacing, the system can automatically generate alarm information and send it to the inspection management platform in real time. Simultaneously, in some optional implementations, SMS notifications can be sent to relevant staff or responsible personnel to ensure that all parties are promptly aware of the risks.

[0109] The aforementioned restricted areas can be understood as areas within the construction site deemed non-compliant, marked on the map as areas where robots are prohibited from entering. Robots can mark these areas on the map and subsequently detour around them as high-risk restricted zones. The aforementioned detour routes can be understood as the inspection paths replanned by the robot after avoiding the restricted areas.

[0110] In some optional implementations, after sending alarm information to the inspection management platform, park staff can be notified simultaneously via SMS if an alarm exists. If the robot does not receive the on-site instruction, it can return to the standby point if it is in a construction risk confirmation task; if it is in an inspection task, it can continue the task according to the latest restricted area plan. It should be noted that the above-mentioned construction risk confirmation task is a targeted verification task for "potential construction risks that have been discovered," while the inspection task is a pre-set routine inspection task. The inspection management platform pre-sets the inspection route and points, and the robot executes according to the plan, covering routine safety and environmental inspections of buildings, roads, equipment, and other areas within the park.

[0111] In this implementation, alarm notifications and no-entry markings on the map facilitate robot route replanning, solving the problem of "task interruption due to construction risk areas requiring route resetting" in existing fixed-path inspections. On one hand, alarm information ensures that risks are promptly communicated to staff for rapid rectification; on the other hand, no-entry zone markings and detour route planning guarantee the continuity and safety of robot inspections, improve the ability to dynamically respond to construction risks, and to some extent, enhance inspection efficiency.

[0112] Optionally, before acquiring images of the construction site fence, the method further includes:

[0113] The robot is controlled to collect real-time video of the surrounding environment and upload the video to the cloud environment for safety analysis. If the analysis result indicates that there is construction in the route or field of view, a construction risk confirmation task is triggered. Or, the monitoring system is controlled to collect fixed field of view video and upload it to the cloud environment for safety analysis. If the analysis result indicates that there is construction enclosure or construction activity, the monitoring point is sent to the inspection management platform, triggering a construction risk confirmation task.

[0114] The construction risk confirmation task is sent to the robot, and the robot arrives near the risk location according to the construction risk confirmation task and detours around the risk location;

[0115] If the robot confirms that the risk location is the construction site, the area where the construction site is located is marked as a restricted area on the map, and a detour route suitable for the robot is planned based on the restricted area.

[0116] In the inspection method of this application embodiment, the aforementioned surrounding environment video can be understood as the park environment image captured in real time by the inspection robot through its main body video acquisition module during routine inspections, used for preliminary identification of construction conditions. For example, environmental monitoring or robot sensors acquire on-site video V, which may contain continuous video frames F_i. These video frames can be fed into the behavior detection network and target detection network of the environmental safety analysis module, respectively. If construction activities by personnel or construction-related targets such as fences or safety helmets are detected, a construction risk is considered to exist, and a construction risk alarm and the location of the risk location P are sent to the inspection management platform. After the robot arrives near the risk location P, it can detour around the risk area P, confirm the construction range using lidar and ultrasonic radar, mark the area on the map, and later detour as a risk-restricted zone. If a new task passes through and confirms that construction has ended, the area is restored to a drivable zone.

[0117] The aforementioned cloud-based environmental security analysis can be performed in real-time by the environmental security analysis module of the cloud computing center, processing uploaded videos to identify risk characteristics such as construction activities and fencing. The aforementioned construction risk confirmation task can be understood as a targeted verification task for suspected construction areas, triggered by the cloud analysis results, with the aim of confirming whether the risk location is a construction site and determining compliance.

[0118] The aforementioned monitoring system can be understood as a set of fixed cameras and other hardware facilities pre-installed within a certain area, used to collect fixed-view video of a specific area as a supplementary monitoring method for robot inspections. Sending monitoring points to the inspection management platform can be understood as the monitoring system, after detecting location information of construction risks, sending the location information (i.e., the monitoring point) to the inspection management platform to guide the robot to conduct verification.

[0119] In this implementation, the construction risk confirmation task is triggered by both the robot and the monitoring system. The robot detours to check and mark restricted areas and plans new routes, which can improve the accuracy of dynamic identification of construction risks and the safety of continuous execution of inspection tasks. This can further improve the timeliness of construction risk control during inspections and the adaptability of robot inspections.

[0120] In some alternative implementations, the robot captures images or video frames of personnel at the construction site. t (x,y) represents the bounding boxes for human targets at the construction site, obtained using a target recognition network. Wearable equipment target detection box Where 'c' represents different types of wearable equipment, such as helmets, reflective vests, etc. If The number is less than The number or and The center position deviation is greater than If the width exceeds half of the target width, it is considered non-compliant and an alarm "Construction work attire non-compliant" is issued. In some other optional implementations, the compliance of construction work attire can also be determined based on the compliance of attire at the construction site. If non-compliance is deemed non-compliant, an alarm is issued, and park staff are simultaneously notified via SMS. If the robot does not receive an on-site instruction, it returns to the standby point if it is a robot performing a construction risk confirmation task; if it is a robot performing an inspection task, it continues the task according to the latest restricted area plan for subsequent inspection routes.

[0121] In this implementation, by collecting images of personnel with a robot, inputting them into a target recognition network to detect the personnel and their equipment frames, and comparing the quantity and location to determine the compliance of personnel's attire, the compliance of construction personnel's attire is determined. This achieves automated and accurate verification of the attire of construction personnel. Combined with the task type, subsequent actions are adapted, and the robot can return to the standby point or continue the inspection. This not only ensures timely and effective supervision of construction safety to a certain extent, but also maintains the continuity of the robot's inspection tasks, thereby improving the efficiency of construction safety management and inspection.

[0122] In some alternative implementations, in a park environment, the robot-based inspection architecture diagram can be as follows: Figure 3 As shown, the park monitoring system can use video acquisition modules to collect fixed-view video to fill blind spots in robot inspections; the robot can use environmental information acquisition, robotic arm vision control, and chassis motion control modules to complete environmental acquisition, vision adjustment, and movement around obstacles, supporting inspections and risk confirmation; the cloud computing center, with its environmental safety and construction risk analysis modules, performs video analysis and fence compliance judgments, serving as the core of risk identification; the inspection management platform, based on alarm management and task management modules, can receive alarms and send tasks, enabling risk closure and inspection coordination.

[0123] See Figure 4 , Figure 4 This is a structural diagram of an inspection device provided in another embodiment of this application.

[0124] like Figure 4 As shown, the inspection device 400 includes:

[0125] The acquisition module 401 is used to acquire images of the construction site enclosure.

[0126] The identification module 402 is used to identify the feature elements of the construction site enclosure based on the construction site enclosure image. The feature elements include cones and warning lines of cone-type enclosures, and the fence edges of fence-type enclosures.

[0127] The sorting module 403 is used to sort the feature elements to obtain an ordered list of feature elements;

[0128] The inspection module 404 is used to check whether the association relationship between adjacent feature elements conforms to the preset association relationship based on the feature element list, so as to obtain the inspection result; wherein, the inspection of the association relationship between adjacent feature elements includes whether the warning line exists between adjacent cones in the cone-type enclosure, and / or whether the endpoint spacing of adjacent fence edges in the fence-type enclosure meets the preset conditions;

[0129] The judgment module 405 is used to determine whether the construction site is compliant based on the inspection results.

[0130] Optionally, the sorting module 403 can also be used for:

[0131] Obtain multiple target coordinate values ​​for multiple feature elements, wherein the target coordinate values ​​include the coordinates of the center point of the cone or the coordinates of the endpoint of the fence edge;

[0132] Compare the eigenvalue of the y-coordinate in the target coordinate values ​​with a preset value;

[0133] In this process, multiple first feature elements are sorted according to the size of the feature values ​​of their corresponding x-coordinates to obtain a list of first feature elements, wherein the first feature elements are feature elements whose feature values ​​are less than the preset value.

[0134] Multiple second feature elements are sorted in reverse order according to the size of the feature values ​​of their corresponding x-coordinates to obtain a list of second feature elements, wherein the second feature elements are feature elements whose feature values ​​are greater than or equal to the preset value;

[0135] The first feature element list and the second feature element list are merged to obtain the ordered feature element list.

[0136] Optionally, the inspection module 404 can also be used for:

[0137] Based on the construction site fence image, the start and end points of each warning line segment are extracted, and a straight line is fitted to the start and end points to obtain the slope corresponding to each warning line segment.

[0138] If the difference between the slopes of any two warning lines is less than a preset threshold, the start and end points of the two warning lines are merged to obtain a merged set of warning lines.

[0139] Based on the list of feature elements and the set of warning lines, if there is a warning line between adjacent feature elements, the inspection result is that it conforms to the preset association relationship.

[0140] If there is no warning line between adjacent feature elements, the inspection result is that the preset association relationship is not met, and the adjacent feature elements are recorded in the warning line missing list. The total number of records in the warning line missing list is counted to obtain the number of warning lines missing.

[0141] Optionally, the inspection module 404 can also be used for:

[0142] The distance between the endpoints of adjacent fence edges is obtained based on the construction site fence image;

[0143] If the distance between the endpoints is less than the first ratio of the length of the shorter side among the two adjacent sides, then the inspection result is in accordance with the preset correlation relationship;

[0144] If the endpoint spacing is greater than or equal to the first ratio of the shorter side length of the two adjacent sides and less than the second ratio, then the inspection result is that it does not conform to the preset association relationship. The inspection result also includes the number of times the spacing is too wide, and the second ratio is greater than the first ratio.

[0145] If the distance between the endpoints is greater than or equal to the second ratio of the shorter side length of the two adjacent sides, it is determined that the fence of the adjacent fence edge is missing. The inspection result is that it does not conform to the preset association relationship. The inspection result also includes the number of missing fences.

[0146] Optionally, the inspection device 400 can also be used for:

[0147] If the construction site is determined to be non-compliant, an alarm message is generated and sent to the inspection management platform;

[0148] The non-compliant areas of the construction site are marked as no-entry zones on the map, and detour routes suitable for robots are planned based on the no-entry zones.

[0149] Optionally, the inspection device 400 can also be used for:

[0150] The robot is controlled to collect real-time video of the surrounding environment and upload the video to the cloud environment for safety analysis. If the analysis result indicates that there is construction in the route or field of view, a construction risk confirmation task is triggered. Or, the monitoring system is controlled to collect fixed field of view video and upload it to the cloud environment for safety analysis. If the analysis result indicates that there is construction enclosure or construction activity, the monitoring point is sent to the inspection management platform, triggering a construction risk confirmation task.

[0151] The construction risk confirmation task is sent to the robot, and the robot arrives near the risk location according to the construction risk confirmation task and detours around the risk location;

[0152] If the robot confirms that the risk location is the construction site, the area where the construction site is located is marked as a restricted area on the map, and a detour route suitable for the robot is planned based on the restricted area.

[0153] For details, see Figure 5 As shown in the figure, this application embodiment also provides an electronic device, including a bus 501, a transceiver 502, an antenna 503, a bus interface 504, a processor 505, and a memory 506.

[0154] Processor 505, used for:

[0155] Collect images of the construction site fencing;

[0156] Based on the construction site enclosure image, the feature elements of the construction site enclosure are identified, including cones and warning lines for cone-type enclosures, and the fence edges for fence-type enclosures.

[0157] The feature elements are sorted to obtain an ordered list of feature elements;

[0158] Based on the list of feature elements, check whether the relationship between adjacent feature elements conforms to the preset relationship to obtain the check result; wherein, checking the relationship between adjacent feature elements includes whether the warning line exists between adjacent cones in the cone-type enclosure, and / or whether the distance between the endpoints of adjacent fence edges in the fence-type enclosure meets the preset conditions;

[0159] Based on the inspection results, it is determined whether the construction site is compliant.

[0160] exist Figure 5 In this document, a bus architecture (represented by bus 501) is used. Bus 501 can include any number of interconnected buses and bridges, linking various circuits including one or more processors represented by processor 505 and memory represented by memory 506. Bus 501 can also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. Bus interface 504 provides an interface between bus 501 and transceiver 502. Transceiver 502 can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by processor 505 is transmitted over a wireless medium via antenna 503, which further receives data and transmits it to processor 505.

[0161] Processor 505 manages bus 501 and general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory 506 can be used to store data used by processor 505 during operation.

[0162] Optionally, the processor 505 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a complex programmable logic device (CPLD).

[0163] Optionally, the processor 505 is specifically used for:

[0164] Obtain multiple target coordinate values ​​for multiple feature elements, wherein the target coordinate values ​​include the coordinates of the center point of the cone or the coordinates of the endpoint of the fence edge;

[0165] Compare the eigenvalue of the y-coordinate in the target coordinate values ​​with a preset value;

[0166] In this process, multiple first feature elements are sorted according to the size of the feature values ​​of their corresponding x-coordinates to obtain a list of first feature elements, wherein the first feature elements are feature elements whose feature values ​​are less than the preset value.

[0167] Multiple second feature elements are sorted in reverse order according to the size of the feature values ​​of their corresponding x-coordinates to obtain a list of second feature elements, wherein the second feature elements are feature elements whose feature values ​​are greater than or equal to the preset value;

[0168] The first feature element list and the second feature element list are merged to obtain the ordered feature element list.

[0169] Optionally, the processor 505 is specifically used for:

[0170] Based on the construction site fence image, the start and end points of each warning line segment are extracted, and a straight line is fitted to the start and end points to obtain the slope corresponding to each warning line segment.

[0171] If the difference between the slopes of any two warning lines is less than a preset threshold, the start and end points of the two warning lines are merged to obtain a merged set of warning lines.

[0172] Based on the list of feature elements and the set of warning lines, if there is a warning line between adjacent feature elements, the inspection result is that it conforms to the preset association relationship.

[0173] If there is no warning line between adjacent feature elements, the inspection result is that the preset association relationship is not met, and the adjacent feature elements are recorded in the warning line missing list. The total number of records in the warning line missing list is counted to obtain the number of warning lines missing.

[0174] Optionally, the processor 505 is specifically used for:

[0175] The distance between the endpoints of adjacent fence edges is obtained based on the construction site fence image;

[0176] If the distance between the endpoints is less than the first ratio of the length of the shorter side among the two adjacent sides, then the inspection result is in accordance with the preset correlation relationship;

[0177] If the endpoint spacing is greater than or equal to the first ratio of the shorter side length of the two adjacent sides and less than the second ratio, then the inspection result is that it does not conform to the preset association relationship. The inspection result also includes the number of times the spacing is too wide, and the second ratio is greater than the first ratio.

[0178] If the distance between the endpoints is greater than or equal to the second ratio of the shorter side length of the two adjacent sides, it is determined that the fence of the adjacent fence edge is missing. The inspection result is that it does not conform to the preset association relationship. The inspection result also includes the number of missing fences.

[0179] Optionally, the processor 505 is specifically used for:

[0180] If the construction site is determined to be non-compliant, an alarm message is generated and sent to the inspection management platform;

[0181] The non-compliant areas of the construction site are marked as no-entry zones on the map, and detour routes suitable for robots are planned based on the no-entry zones.

[0182] Optionally, the processor 505 is specifically used for:

[0183] The robot is controlled to collect real-time video of the surrounding environment and upload the video to the cloud environment for safety analysis. If the analysis result indicates that there is construction in the route or field of view, a construction risk confirmation task is triggered. Or, the monitoring system is controlled to collect fixed field of view video and upload it to the cloud environment for safety analysis. If the analysis result indicates that there is construction enclosure or construction activity, the monitoring point is sent to the inspection management platform, triggering a construction risk confirmation task.

[0184] The construction risk confirmation task is sent to the robot, and the robot arrives near the risk location according to the construction risk confirmation task and detours around the risk location;

[0185] If the robot confirms that the risk location is the construction site, the area where the construction site is located is marked as a restricted area on the map, and a detour route suitable for the robot is planned based on the restricted area.

[0186] It should be noted that the electronic device provided in this application embodiment is a device capable of executing the above-described inspection method. Therefore, all implementation methods in the above-described inspection method embodiments are applicable to this electronic device and can achieve the same or similar beneficial effects. To avoid repetition, this embodiment will not elaborate further.

[0187] This application also provides an electronic device, including: a processor, a memory, and a program stored in the memory and executable on the processor. When the program is executed by the processor, it implements the various processes of the above-described inspection method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0188] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the various processes of the above-described inspection method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0189] This application also provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, they implement the various processes of the above-described inspection method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0190] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0191] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0192] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An inspection method, characterized in that, The method includes: Collect images of the construction site fencing; Based on the construction site enclosure image, the feature elements of the construction site enclosure are identified, including cones and warning lines for cone-type enclosures, and the fence edges for fence-type enclosures. The feature elements are sorted to obtain an ordered list of feature elements; Based on the list of feature elements, check whether the relationship between adjacent feature elements conforms to the preset relationship to obtain the check result; wherein, checking the relationship between adjacent feature elements includes whether the warning line exists between adjacent cones in the cone-type enclosure, and / or whether the distance between the endpoints of adjacent fence edges in the fence-type enclosure meets the preset conditions; Based on the inspection results, it is determined whether the construction site is compliant. The step of sorting the feature elements to obtain an ordered list of feature elements includes: Obtain multiple target coordinate values ​​for multiple feature elements, wherein the target coordinate values ​​include the coordinates of the center point of the cone or the coordinates of the endpoint of the fence edge; Compare the eigenvalue of the y-coordinate in the target coordinate values ​​with the preset value; In this process, multiple first feature elements are sorted according to the size of the feature values ​​of their corresponding x-coordinates to obtain a list of first feature elements, wherein the first feature elements are feature elements whose feature values ​​are less than the preset value. Multiple second feature elements are sorted in reverse order according to the size of the feature values ​​of their corresponding x-coordinates to obtain a list of second feature elements, wherein the second feature elements are feature elements whose feature values ​​are greater than or equal to the preset value; The first feature element list and the second feature element list are merged to obtain the ordered feature element list; When the feature elements are the cones and warning lines of the cone-type enclosure, checking whether the correlation between adjacent feature elements conforms to a preset correlation relationship to obtain the check result includes: Based on the construction site fence image, the start and end points of each warning line segment are extracted, and a straight line is fitted to the start and end points to obtain the slope corresponding to each warning line segment. If the difference between the slopes of any two warning lines is less than a preset threshold, the start and end points of the two warning lines are merged to obtain a merged set of warning lines. Based on the list of feature elements and the set of warning lines, if there is a warning line between adjacent feature elements, the inspection result is that it conforms to the preset association relationship. If there is no warning line between adjacent feature elements, the inspection result is that the preset association relationship is not met, and the adjacent feature elements are recorded in the warning line missing list. The total number of records in the warning line missing list is counted to obtain the number of warning lines missing.

2. The method according to claim 1, characterized in that, When the feature element is the edge of the fence of the fence type, the step of checking whether the association relationship between adjacent feature elements conforms to a preset association relationship to obtain the check result includes: The distance between the endpoints of adjacent fence edges is obtained based on the construction site fence image; If the distance between the endpoints is less than the first ratio of the length of the shorter side among the two adjacent sides, then the inspection result is in accordance with the preset correlation relationship; If the endpoint spacing is greater than or equal to the first ratio of the shorter side length of the two adjacent sides and less than the second ratio, then the inspection result is that it does not conform to the preset association relationship. The inspection result also includes the number of times the spacing is too wide, and the second ratio is greater than the first ratio. If the distance between the endpoints is greater than or equal to the second ratio of the shorter side length of the two adjacent sides, it is determined that the fence of the adjacent fence edge is missing. The inspection result is that it does not conform to the preset association relationship. The inspection result also includes the number of missing fences.

3. The method according to claim 1 or 2, characterized in that, After determining whether the construction site is compliant based on the inspection results, the process further includes: If the construction site is determined to be non-compliant, an alarm message is generated and sent to the inspection management platform; The non-compliant areas of the construction site are marked as no-entry zones on the map, and detour routes suitable for robots are planned based on the no-entry zones.

4. The method according to claim 1 or 2, characterized in that, Before acquiring images of the construction site enclosure, the following steps are also included: The robot is controlled to collect real-time video of the surrounding environment and upload the video to the cloud environment for safety analysis. If the analysis result indicates that there is construction in the route or field of view, a construction risk confirmation task is triggered. Or, the monitoring system is controlled to collect fixed field of view video and upload it to the cloud environment for safety analysis. If the analysis result indicates that there is construction enclosure or construction activity, the monitoring point is sent to the inspection management platform, triggering a construction risk confirmation task. The construction risk confirmation task is sent to the robot, and the robot arrives near the risk location according to the construction risk confirmation task and detours around the risk location; If the robot confirms that the risk location is the construction site, the area where the construction site is located is marked as a restricted area on the map, and a detour route suitable for the robot is planned based on the restricted area.

5. An inspection device, characterized in that, include: The data acquisition module is used to acquire images of construction site fencing. The identification module is used to identify the feature elements of the construction site enclosure based on the construction site enclosure image. The feature elements include cones and warning lines for cone-type enclosures, and the fence edges for fence-type enclosures. The sorting module is used to sort the feature elements to obtain an ordered list of feature elements; The inspection module is used to check whether the association relationship between adjacent feature elements conforms to the preset association relationship based on the feature element list, so as to obtain the inspection result; wherein, the inspection of the association relationship between adjacent feature elements includes whether the warning line exists between adjacent cones in the cone-type fence, and / or whether the endpoint spacing of adjacent fence edges in the fence-type fence meets the preset conditions; Based on the inspection results, it is determined whether the construction site is compliant. The sorting module is used for: Obtain multiple target coordinate values ​​for multiple feature elements, wherein the target coordinate values ​​include the coordinates of the center point of the cone or the coordinates of the endpoint of the fence edge; Compare the eigenvalue of the y-coordinate in the target coordinate values ​​with the preset value; In this process, multiple first feature elements are sorted according to the size of the feature values ​​of their corresponding x-coordinates to obtain a list of first feature elements, wherein the first feature elements are feature elements whose feature values ​​are less than the preset value. Multiple second feature elements are sorted in reverse order according to the size of the feature values ​​of their corresponding x-coordinates to obtain a list of second feature elements, wherein the second feature elements are feature elements whose feature values ​​are greater than or equal to the preset value; The first feature element list and the second feature element list are merged to obtain the ordered feature element list; The inspection module is used for: When the feature elements are the cones and warning lines of the cone-type enclosure, the start and end points of each warning line are extracted based on the construction enclosure image, and the start and end points are fitted with straight lines to obtain the slope corresponding to each warning line. If the difference between the slopes of any two warning lines is less than a preset threshold, the start and end points of the two warning lines are merged to obtain a merged set of warning lines. Based on the list of feature elements and the set of warning lines, if there is a warning line between adjacent feature elements, the inspection result is that it conforms to the preset association relationship. If there is no warning line between adjacent feature elements, the inspection result is that the preset association relationship is not met, and the adjacent feature elements are recorded in the warning line missing list. The total number of records in the warning line missing list is counted to obtain the number of warning lines missing.

6. An electronic device, characterized in that, include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the inspection method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the inspection method as described in any one of claims 1 to 4.

8. A computer program product, characterized in that, It includes computer instructions, which, when executed by a processor, implement the steps of the inspection method as described in any one of claims 1 to 4.

Images