Intelligent supervision system and method

Abstract

The invention discloses an intelligent supervision system and method. The system comprises an unmanned aerial vehicle, shooting equipment, a cloud supervision center and a handheld terminal, a first detection module arranged in the unmanned aerial vehicle is used for determining first information of a target object based on a target image shot by the shooting equipment, and a second detection module arranged in the unmanned aerial vehicle is used for determining second information of the target object based on the target image shot by the shooting equipment; the cloud supervision center determines whether the target object meets the high-altitude operation qualification or not according to the first information and outputs a first detection result, and determines whether the safety equipment worn by the target object meets a preset wearing requirement or not according to the second information and outputs a second detection result; and the handheld terminal receives the first detection result and the second detection result sent by the cloud supervision center, so that a supervisor can take corresponding measures in time according to the first detection result and the second detection result. According to the invention, the intelligent system is adopted to accurately and efficiently carry out safety supervision and inspection in the overhead working scene of the power transmission line.

Description

Technical Field

[0001] The present invention relates to the technical field of power transmission lines, and in particular to an intelligent inspection system and method. Background Art

[0002] In the high-altitude operation scenario of power transmission lines, there will be many safety hazards for workers during the high-altitude operation. Therefore, safety supervision is required when workers are working at high altitudes.

[0003] At present, there are two methods for safety inspection of high-altitude operation scenes of power transmission lines: visual online inspection and traditional offline inspection. For traditional offline inspection, high-altitude operations are observed through telescopes. This method of evidence collection is inefficient and easily interferes with high-altitude operations, which poses risks. For visual online inspection, cameras are usually installed on the ground to film high-altitude workers. However, since high-altitude workers are far away from the ground and the targets are small, the images may be blurred and indistinguishable, which greatly affects the accuracy of the inspection. Summary of the invention

[0004] The present invention provides an intelligent inspection system and method to realize accurate and efficient safety inspection in the high-altitude operation scene of power transmission lines.

[0005] According to one aspect of the present invention, there is provided an intelligent inspection system, which includes: a drone, a photographing device, a cloud-based supervision center, and a handheld terminal; the drone is equipped with a gimbal, and the photographing device is connected to the gimbal; the drone is provided with a first detection module and a second detection module;

[0006] The photographing device is used to photograph at least one target image of the target object and send the target image to the first detection module and the second detection module;

[0007] The first detection module is used to determine first information of the target object based on the target image, and the second detection module is used to determine second information of the target object based on the target image; the first information is feature information representing the identity of the target object; the second information is feature information representing the safety device worn by the target object;

[0008] The cloud-based monitoring center is configured to receive the first information and the second information sent by the drone, determine whether the target object meets the qualifications for high-altitude operations based on the first information, and output a first detection result, determine whether the safety equipment worn by the target object meets the preset wearing requirements based on the second information, and output a second detection result;

[0009] The handheld terminal is used to receive the first test result and the second test result sent by the cloud-based supervision center, so that the inspector can take corresponding measures in time according to the first test result and the second test result.

[0010] According to another aspect of the present invention, there is provided an intelligent inspection method, which is applied to the intelligent inspection system according to any one of claims 1 to 8, wherein the intelligent inspection system comprises: an unmanned aerial vehicle, a photographing device, a cloud-based supervision center and a handheld terminal; the unmanned aerial vehicle is provided with a pan-tilt platform, and the photographing device is connected to the pan-tilt platform; the unmanned aerial vehicle is provided with a first detection module and a second detection module, and the method comprises:

[0011] The first detection module determines first information of the target object based on at least one target image of the target object, and the second detection module determines second information of the target object based on at least one target image of the target object; the first information is feature information representing the identity of the target object; the second information is feature information representing the safety device worn by the target object; the at least one target image of the target object is taken by the shooting device;

[0012] The cloud-based monitoring center receives the first information and the second information sent by the drone, determines whether the target object meets the qualifications for high-altitude operations according to the first information, and outputs a first detection result, determines whether the safety equipment worn by the target object meets the preset wearing requirements according to the second information, and outputs a second detection result;

[0013] The handheld terminal receives the first detection result and the second detection result sent by the cloud-based supervision center, so that the inspector can take corresponding measures in a timely manner according to the first detection result and the second detection result.

[0014] The intelligent inspection system of the technical solution of the embodiment of the present invention includes: an unmanned aerial vehicle, a shooting device, a cloud-based supervision center and a handheld terminal; the unmanned aerial vehicle is equipped with a gimbal, and the shooting device is connected to the gimbal through the gimbal; a first detection module and a second detection module are arranged in the unmanned aerial vehicle; the shooting device is used to shoot at least one target image of the target object and send the target image to the first detection module and the second detection module; the first detection module is used to determine the first information of the target object based on the target image, and the second detection module is used to determine the second information of the target object based on the target image; the cloud-based supervision center is used to receive the first information and the second information sent by the unmanned aerial vehicle, determine whether the target object meets the qualifications for high-altitude operations according to the first information, and output the first detection result, determine whether the safety equipment worn by the target object meets the preset wearing requirements according to the second information, and output the second detection result; the handheld terminal is used to receive the first detection result and the second detection result sent by the cloud-based supervision center, so that the inspectors can take corresponding measures in time according to the first detection result and the second detection result. The present invention adopts the mutual cooperation between drones, shooting equipment, cloud-based monitoring centers and handheld terminals in an intelligent system to achieve real-time supervision, so that potential safety hazards can be discovered and dealt with in a timely manner. Drones can achieve all-round and multi-angle detection, improve the accuracy of supervision, and thus achieve accurate and efficient safety supervision in high-altitude operation scenarios of power transmission lines.

[0015] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 is a structural schematic diagram of an intelligent inspection system provided according to an embodiment of the present invention;

[0018] Figure 2 It is a flow chart of an intelligent inspection method provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0020] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0021] Embodiment 1

[0022] Figure 1 This is a schematic diagram of the structure of an intelligent inspection system provided by an embodiment of the present invention. This embodiment is applicable to the situation of real-time inspection of operators in the scene of high-altitude operations. Figure 1 As shown, the intelligent inspection system of the present invention includes: a drone 110, a shooting device 120, a cloud-based supervision center 130 and a handheld terminal 140; the drone is equipped with a gimbal, and the shooting device is connected via the gimbal; a first detection module 1101 and a second detection module 1102 are provided in the drone.

[0023] The shooting device 120 is used to shoot at least one target image of the target object and send the target image to the first detection module and the second detection module. The target object may be a staff member performing high-altitude left and right. The shooting device may be a device with a high-power zoom function.

[0024] The first detection module 1101 is used to determine first information of the target object based on the target image, and the second detection module 1102 is used to determine second information of the target object based on the target image; the first information is feature information representing the identity of the target object; the second information is feature information representing the safety device worn by the target object;

[0025] The cloud monitoring center 130 is used to receive the first information and the second information sent by the drone, determine whether the target object meets the qualifications for high-altitude operations according to the first information, and output a first detection result, determine whether the safety equipment worn by the target object meets the preset wearing requirements according to the second information, and output a second detection result;

[0026] The handheld terminal 140 is used to receive the first detection result and the second detection result sent by the cloud supervision center, so that the inspectors can take corresponding measures in time according to the first detection result and the second detection result.

[0027] Optionally, after the drone rises to a preset height, the camera is controlled to capture at least one reference image, the pan tilt angle of the pan tilt is determined according to the reference image, and the shooting angle of the camera is determined according to the pan tilt angle, so that the camera captures at least one target image of the target object according to the shooting angle. The target object is displayed in the target image at a preset size and is located in the middle of the target image.

[0028] For example, the preset height can be determined according to the actual working height of the staff. After the UAV rises to the preset height, the shooting device connected to the gimbal on the UAV takes at least one reference image and transmits it back to the UAV. The UAV determines the gimbal angle of the gimbal by analyzing the at least one reference image, and the shooting angle of the shooting device can be adjusted according to the gimbal angle so that the shooting device reaches the optimal shooting angle, that is, the clarity of the captured image reaches the preset clarity, and the target object in the image is displayed according to the preset size and is located in the middle of the target image.

[0029] In the embodiment of the present invention, optionally, the first detection module determines the first information of the target object based on the target image, which may include the following steps A1-A3:

[0030] Step A1, performing posture estimation on the face to be identified in each target image, and determining the posture type of the face to be identified, wherein the posture type is used to characterize the orientation of the face to be identified;

[0031] Step A2, determining a reference difference between the orientation angle corresponding to the posture type of the face to be identified and the front orientation angle, and taking a target image corresponding to the face to be identified whose reference difference is less than or equal to a preset deviation angle as a reference image;

[0032] Step A3: perform image enhancement processing on the reference image to obtain a processed image, use a deep learning model to extract features of the processed image to obtain target features, and use the target features as the first information of the target object.

[0033] This embodiment estimates the posture of the face to be identified in each target image, determines the posture type of the face to be identified, and further determines the reference difference between the orientation angle corresponding to the posture type of the face to be identified and the front orientation angle, so as to accurately obtain the deviation of the face to be identified in the target image relative to the front. Because the front image is more likely to identify the identity features of the target object, the target image corresponding to the face to be identified with a reference difference less than or equal to the preset deviation angle is used as a reference image, so as to perform image enhancement processing on the reference image to obtain a processed image, and use a deep learning model to extract features from the processed image to obtain accurate target features, that is, to obtain accurate first information of the target object, so as to accurately judge the identity qualifications of the target object later.

[0034] In an embodiment of the present invention, optionally, the second detection module determines the second information of the target object based on the target image, including: using a target detection model to mark and classify the safety equipment worn by the target object in the target image, obtaining the marked image and the device type of the safety equipment, and using the marked image and the device type of the safety equipment as the second information of the target object; the target detection model is a model based on the target detection algorithm of YOLOv5s, which is used to mark and classify objects in the image. The marking can be to enclose the safety equipment with a bounding box to indicate the specific location of the safety equipment.

[0035] In an embodiment of the present invention, optionally, the cloud-based supervision center determines whether the target object meets the qualifications for high-altitude operations according to the first information, and outputs a first detection result, including:

[0036] The first information is compared with the preset identity information in the database, and a first comparison result is determined; the first comparison result is used to describe the degree of match between the first information and the preset identity information in the database; if the first comparison result reaches a first matching degree, it is determined that the target object meets the qualifications for high-altitude work, and the target object's qualifications for high-altitude work are output as the first detection result, and continue to determine whether the safety equipment worn by the target object meets the preset wearing requirements based on the second information; if the first comparison result does not reach the first matching degree, it is determined that the target object does not meet the qualifications for high-altitude work, and the target object's qualifications for high-altitude work are output as the first detection result.

[0037] In an embodiment of the present invention, optionally, after determining that the target object does not meet the qualifications for high-altitude operations, and outputting the fact that the target object does not meet the qualifications for high-altitude operations as the first detection result, it indicates that the target object cannot perform high-altitude operations. At this time, the cloud-based supervision center of the intelligent inspection system will stop determining whether the safety equipment worn by the target object meets the preset wearing requirements based on the second information, and send the first detection result to the handheld terminal to remind the inspectors to take corresponding measures, such as notifying the target object to stop high-altitude operations and return to land.

[0038] In the embodiment of the present invention, optionally, determining whether the safety device worn by the target object meets the preset wearing requirement according to the second information includes:

[0039] Extract target feature information of the safety device of the target device type in the annotated image, compare the target feature information with the safety wearing standard of the safety device of the target device type, and determine a second comparison result; the second comparison result is used to describe the degree of matching between the target feature information and the safety wearing standard; the target feature information is used to describe the way the target object wears the safety device; if the second comparison result reaches a second matching degree, it is determined that the safety device worn by the target object meets the preset wearing requirements, and the compliance of the safety device worn by the target object with the preset wearing requirements is output as a second detection result; if the second comparison result does not reach the second matching degree, it is determined that the safety device worn by the target object does not meet the preset wearing requirements, and the compliance of the safety device worn by the target object with the preset wearing requirements is output as a second detection result.

[0040] This embodiment extracts target feature information of the safety device of the target device type in the labeled image, compares the target feature information with the safety wearing standard of the safety device of the target device type to determine the degree of match between the target feature information and the safety wearing standard, thereby achieving accurate determination of whether the target object is safely wearing the safety device.

[0041] In an embodiment of the present invention, the intelligent inspection system may further include: a digital inspection telescope, which is used to assist the inspector so that the inspector can use the digital inspection telescope to monitor the ground operation area. In addition, the object information and ground information of the target object observed by the digital inspection telescope can be uploaded to the cloud supervision center, so that the cloud supervision center can analyze whether the target object has an operation risk based on the object information and ground information, thereby greatly improving the efficiency of safety supervision and inspection.

[0042] The intelligent inspection system of the technical solution of the embodiment of the present invention includes: an unmanned aerial vehicle, a shooting device, a cloud-based supervision center and a handheld terminal; the unmanned aerial vehicle is equipped with a gimbal, and the shooting device is connected to the gimbal through the gimbal; a first detection module and a second detection module are arranged in the unmanned aerial vehicle; the shooting device is used to shoot at least one target image of the target object and send the target image to the first detection module and the second detection module; the first detection module is used to determine the first information of the target object based on the target image, and the second detection module is used to determine the second information of the target object based on the target image; the cloud-based supervision center is used to receive the first information and the second information sent by the unmanned aerial vehicle, determine whether the target object meets the qualifications for high-altitude operations according to the first information, and output the first detection result, determine whether the safety equipment worn by the target object meets the preset wearing requirements according to the second information, and output the second detection result; the handheld terminal is used to receive the first detection result and the second detection result sent by the cloud-based supervision center, so that the inspectors can take corresponding measures in time according to the first detection result and the second detection result. The present invention adopts the mutual cooperation between drones, shooting equipment, cloud-based monitoring centers and handheld terminals in an intelligent system to achieve real-time supervision, so that potential safety hazards can be discovered and dealt with in a timely manner. Drones can achieve all-round and multi-angle detection, improve the accuracy of supervision, and thus achieve accurate and efficient safety supervision in high-altitude operation scenarios of power transmission lines.

[0043] Embodiment 2

[0044] Figure 2 A flow chart of an intelligent inspection method provided in an embodiment of the present invention. This embodiment can be applied to the situation of real-time inspection of workers in high-altitude operation scenarios. The method can be executed by an intelligent inspection system.

[0045] The intelligent inspection method of the present invention is applied to an intelligent inspection system, which includes: an unmanned aerial vehicle, a shooting device, a cloud-based supervision center and a handheld terminal; the unmanned aerial vehicle is equipped with a gimbal, and the shooting device is connected to the gimbal; the unmanned aerial vehicle is provided with a first detection module and a second detection module.

[0046] like Figure 2 As shown, the intelligent inspection method of the present invention may include the following steps:

[0047] S210. A first detection module determines first information of a target object based on at least one target image of the target object, where the first information is feature information representing an identity of the target object.

[0048] Wherein, at least one target image of the target object is photographed by a photographing device.

[0049] Specifically, a dynamic face recognition algorithm may be used to analyze at least one target image to obtain the first information of the target object.

[0050] On the basis of the above embodiment, optionally, after the drone rises to a preset height, the shooting device is controlled to shoot at least one reference image, the gimbal angle of the gimbal is determined according to the reference image, and the shooting angle of the shooting device is determined according to the gimbal angle, so that the shooting device shoots at least one target image of the target object according to the shooting angle; the target object is displayed in the target image according to a preset size and is located in the middle position of the target image.

[0051] Based on the above embodiment, optionally, determining the first information of the target object based on the target image includes:

[0052] Performing posture estimation on the face to be identified in each of the target images to determine the posture type of the face to be identified, wherein the posture type is used to characterize the orientation of the face to be identified;

[0053] Determine a reference difference between the orientation angle corresponding to the posture type of the face to be identified and the front orientation angle, and use the target image corresponding to the face to be identified whose reference difference is less than or equal to a preset deviation angle as a reference image;

[0054] Perform image enhancement processing on the reference image to obtain a processed image, use a deep learning model to perform feature extraction on the processed image to obtain target features, and use the target features as the first information of the target object.

[0055] S220. The second detection module determines second information of the target object based on at least one target image of the target object, where the second information is feature information representing a safety device worn by the target object.

[0056] Specifically, feature information of the security device in the target image can be extracted and used as the second information of the target object, wherein the feature information includes the location information of the security device in the image. The extraction method can be at least one of color feature extraction, texture feature extraction, and shape feature extraction.

[0057] Based on the above embodiment, optionally, determining the second information of the target object based on the target image includes:

[0058] A target detection model is used to label and classify the safety equipment worn by the target object in the target image, to obtain the labeled image and the device type of the safety equipment, and the labeled image and the device type of the safety equipment are used as the second information of the target object; the target detection model is a model based on the YOLOv5s target detection algorithm, which is used to label and classify objects in the image.

[0059] S230, the cloud-based monitoring center receives the first information sent by the drone, determines whether the target object meets the qualifications for high-altitude operations based on the first information, and outputs a first detection result.

[0060] Based on the above embodiment, optionally, determining whether the target object meets the qualification for high-altitude work according to the first information and outputting a first detection result includes:

[0061] The first information is compared with the preset identity information in the database, and a first comparison result is determined; the first comparison result is used to describe the degree of match between the first information and the preset identity information in the database; if the first comparison result reaches a first matching degree, it is determined that the target object meets the qualifications for high-altitude work, and the target object's qualifications for high-altitude work are output as a first detection result, and further determined based on the second information whether the safety equipment worn by the target object meets the preset wearing requirements; if the first comparison result does not reach the first matching degree, it is determined that the target object does not meet the qualifications for high-altitude work, and the target object's qualifications for high-altitude work are output as a first detection result.

[0062] Based on the above embodiment, optionally, after determining that the target object does not meet the qualifications for high-altitude work, and outputting the target object's failure to meet the qualifications for high-altitude work as a first detection result, the system also includes: the cloud-based supervision center stops determining whether the safety equipment worn by the target object meets the preset wearing requirements based on the second information, and sends the first detection result to the handheld terminal to remind the inspectors to take corresponding measures.

[0063] S240: The cloud monitoring center receives the second information sent by the drone, determines whether the safety equipment worn by the target object meets the preset wearing requirements according to the second information, and outputs a second detection result.

[0064] Based on the above embodiment, optionally, determining whether the safety device worn by the target object meets the preset wearing requirement according to the second information includes:

[0065] Extract target feature information of the safety equipment of the target equipment type in the annotated image, compare the target feature information with the safety wearing standard of the safety equipment of the target equipment type, and determine a second comparison result; the second comparison result is used to describe the degree of matching between the target feature information and the safety wearing standard; the target feature information is used to describe the way the target object wears the safety equipment; if the second comparison result reaches a second matching degree, it is determined that the safety equipment worn by the target object meets the preset wearing requirements, and the compliance of the safety equipment worn by the target object with the preset wearing requirements is output as a second detection result; if the second comparison result does not reach the second matching degree, it is determined that the safety equipment worn by the target object does not meet the preset wearing requirements, and the compliance of the safety equipment worn by the target object with the preset wearing requirements is output as a second detection result.

[0066] S250. The handheld terminal receives the first test result and the second test result sent by the cloud supervision center, so that the inspectors can take corresponding measures in time according to the first test result and the second test result.

[0067] On the basis of the above embodiment, optionally, the intelligent inspection system further includes: a digital inspection telescope, which is used to assist the inspectors so that the inspectors can use the digital inspection telescope to monitor the ground operation area.

[0068] The intelligent inspection method provided in the embodiment of the present invention can be executed by the intelligent inspection system provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0069] Embodiment 3

[0070] The embodiment of the present invention further describes in detail the model of the target detection algorithm based on YOLOv5s in the target detection model. This embodiment can be combined with various optional schemes in one or more of the above embodiments.

[0071] The model based on the target detection algorithm of YOLOv5s is used for training and detection. The structure of YOLOv5s includes input end, Backbone structure, Neck structure, and Prediction structure. The input end includes Mosaic data enhancement module, adaptive anchor box calculation module, and adaptive image scaling module. The Backbone structure is a convolutional neural network that extracts features from images at different granularities. The Neck structure is a network layer that mixes and combines image features and passes image features to the prediction layer, namely the Prediction structure. The Prediction structure is used to predict image features and generate bounding boxes and predicted device categories. The bounding box is the framework for annotating security devices in the target image.

[0072] The Mosaic data enhancement module combines multiple target images into one image in a certain proportion, so that the model can identify the target in a smaller range. The steps of Mosaic data enhancement include:

[0073] (1) Randomly select the coordinates (xc, yc) of the image stitching reference point and randomly select four target images;

[0074] (2) The four target images are resized and scaled according to the reference points and placed at the upper left, upper right, lower left, and lower right positions of the large image of the specified size;

[0075] (3) According to the size transformation method of each target image, the mapping relationship is matched to the target image label;

[0076] (4) According to the specified horizontal and vertical coordinates, the large image is spliced ​​and the coordinates of the detection box that exceed the boundary are processed.

[0077] (5) The processed target images are organized into different data sets.

[0078] The Mosaic data enhancement method has the following advantages:

[0079] (1) Enriching the dataset: We randomly use four images, randomly scale them, and randomly splice them together, adding many small objects, which greatly increases the data diversity.

[0080] (2) Enhance model robustness: Mixing four images with different semantic information allows the model to detect objects beyond the conventional context.

[0081] (3) Enhance the effect of Batch Normalization: When the model is set with the BN operation, the total number of batch samples will be increased as much as possible during training, because the BI principle is to calculate the mean and variance of each feature layer. The larger the total number of batch samples, the closer the mean and variance calculated by BN are to the mean and variance of the entire data set, and the better the effect.

[0082] (4) The Mosaic data enhancement algorithm is beneficial to improving the performance of small target detection: The Mosaic data enhanced image is composed of four original images, so that each image has a greater probability of containing small targets, thereby improving the detection ability of the model.

[0083] Among them, the adaptive anchor box calculation module adaptively calculates the prior box anchor according to different data sets before each training starts. The specific steps are as follows:

[0084] ① Get the width and height of all target images in the dataset;

[0085] ② Scale each target image to the specified size in proportion to ensure that the maximum value of width, height and height meets the specified size;

[0086] ③ Change the target image from relative coordinates to absolute coordinates, where the width and height after scaling are multiplied;

[0087] ④ Filter the target images and retain the target images whose width and height are greater than or equal to two pixels;

[0088] ⑤Use k-means clustering to obtain n prior box anchors;

[0089] ⑥ Use genetic algorithm to randomly mutate the width and height of anchors; if the effect after mutation is good, assign the result to anchors; if the effect after mutation is worse, skip it; the default mutation is 1000 times.

[0090] Furthermore, the adaptive image scaling module uniformly scales the original image to a standard size and then feeds it into the detection network.

[0091] In the commonly used target detection algorithms, different pictures have different lengths and widths, so the common method is to uniformly scale the original picture to a standard size and then send it to the detection network. For example, the yolo algorithm often uses sizes such as 416416, 608608, etc. This has been improved in the yolov5 code, making the yolov5 reasoning speed very fast. In actual project use, many pictures have different aspect ratios, so after scaling and filling, the sizes of the black edges at both ends are different. If there are more fillings, there is information redundancy, which affects the reasoning speed. Therefore, in the yolov5 code, the letterbox function of datasets.py has been modified to adaptively add the least black edges to the original image. The black edges at both ends of the image height have become less, and the amount of calculation will also be reduced during reasoning, that is, the target detection speed will be improved.

[0092] Furthermore, the Backbone structure includes the Focus structure and the CSP structure;

[0093] The Focus structure slices the target image before it enters the Backbone, and obtains a value for every other pixel in an image, thereby improving computing power without losing information. The CSP structure divides the input feature map into two parts in the channel dimension. The first part is retained, and the second part is passed backward through multiple residual blocks. Finally, the two are merged at the end of the CSP structure to achieve cross-stage splitting and merging.

[0094] Furthermore, the Neck structure adopts the FPN+PAN structure and the CSP2 structure designed by CSPNet to enhance the network feature fusion capability.

[0095] Among them, the model of the target detection algorithm based on YOLOv5s uses the following SiLU activation function:

[0096]

[0097] This embodiment describes in detail the structure and calculation process of the model of the target detection algorithm based on YOLOv5s, and can more intuitively conclude that the safety equipment worn by the target object in the target image is labeled and classified using the model based on the target detection algorithm based on YOLOv5s. The labeled image and the device type of the safety equipment, that is, the second information of the target object, can be accurately obtained, so that the safety equipment worn by the target object meets the preset wearing requirements can be accurately determined based on the second information, thereby greatly improving the accuracy of supervision.

Claims

1. An intelligent inspection system, characterized in that: The system includes: a drone, a shooting device, a cloud monitoring center and a handheld terminal; the drone is equipped with a gimbal, and the shooting device is connected to the gimbal; the drone is provided with a first detection module and a second detection module; The photographing device is used to photograph at least one target image of the target object and send the target image to the first detection module and the second detection module; The first detection module is used to determine first information of the target object based on the target image, and the second detection module is used to determine second information of the target object based on the target image; the first information is feature information representing the identity of the target object; the second information is feature information representing the safety device worn by the target object; The cloud-based monitoring center is configured to receive the first information and the second information sent by the drone, determine whether the target object meets the qualifications for high-altitude operations based on the first information, and output a first detection result, determine whether the safety equipment worn by the target object meets the preset wearing requirements based on the second information, and output a second detection result; The handheld terminal is used to receive the first test result and the second test result sent by the cloud-based supervision center, so that the inspector can take corresponding measures in time according to the first test result and the second test result.

2. The system according to claim 1, characterized in that After the drone rises to a preset height, the shooting device is controlled to shoot at least one reference image, the gimbal angle of the gimbal is determined according to the reference image, and the shooting angle of the shooting device is determined according to the gimbal angle, so that the shooting device shoots at least one target image of the target object according to the shooting angle; the target object is displayed in the target image according to a preset size and is located in the middle position of the target image.

3. The system according to claim 1, characterized in that Determining first information of the target object based on the target image includes: Performing posture estimation on the face to be identified in each of the target images to determine the posture type of the face to be identified, wherein the posture type is used to characterize the orientation of the face to be identified; Determine a reference difference between the orientation angle corresponding to the posture type of the face to be identified and the front orientation angle, and use the target image corresponding to the face to be identified whose reference difference is less than or equal to a preset deviation angle as a reference image; Perform image enhancement processing on the reference image to obtain a processed image, use a deep learning model to perform feature extraction on the processed image to obtain target features, and use the target features as the first information of the target object.

4. The system according to claim 1, characterized in that Determining second information of the target object based on the target image includes: A target detection model is used to label and classify the safety equipment worn by the target object in the target image, to obtain the labeled image and the device type of the safety equipment, and the labeled image and the device type of the safety equipment are used as the second information of the target object; the target detection model is a model based on the YOLOv5s target detection algorithm, which is used to label and classify objects in the image.

5. The system according to claim 1 or 3, characterized in that: Determining whether the target object meets the qualifications for high-altitude work according to the first information, and outputting a first detection result, including: Comparing the first information with the preset identity information in the database and determining a first comparison result; the first comparison result is used to describe the degree of matching between the first information and the preset identity information in the database; If the first comparison result reaches a first matching degree, it is determined that the target object meets the qualifications for high-altitude work, and the qualification of the target object for high-altitude work is output as the first detection result, and whether the safety equipment worn by the target object meets the preset wearing requirements is further determined according to the second information; If the first comparison result does not reach the first matching degree, it is determined that the target object does not meet the qualifications for high-altitude work, and the target object does not meet the qualifications for high-altitude work is output as the first detection result.

6. The system according to claim 5, characterized in that After determining that the target object does not meet the qualifications for high-altitude work, and outputting the target object not meeting the qualifications for high-altitude work as a first detection result, the system further includes: The cloud-based supervision center stops determining whether the safety equipment worn by the target object meets the preset wearing requirements based on the second information, and sends the first detection result to the handheld terminal to remind the inspector to take corresponding measures.

7. The system according to claim 4, characterized in that Determining whether the safety equipment worn by the target object meets the preset wearing requirement according to the second information includes: Extracting target feature information of a safety device of the target device type in the annotated image, comparing the target feature information with a safety wearing standard for the safety device of the target device type, and determining a second comparison result; the second comparison result is used to describe the degree of matching between the target feature information and the safety wearing standard; the target feature information is used to describe the way in which the target object wears the safety device; If the second comparison result reaches a second matching degree, it is determined that the safety device worn by the target object meets the preset wearing requirement, and the safety device worn by the target object meets the preset wearing requirement is output as a second detection result; If the second comparison result does not reach the second matching degree, it is determined that the safety device worn by the target object does not meet the preset wearing requirement, and the safety device worn by the target object does not meet the preset wearing requirement is output as the second detection result.

8. The system according to claim 1, characterized in that The intelligent inspection system also includes: a digital inspection telescope, which is used to assist the inspectors so that the inspectors can use the digital inspection telescope to monitor the ground operation area.

9. An intelligent inspection method, characterized in that: The intelligent inspection system applied to any one of claims 1 to 8 comprises: a drone, a photographing device, a cloud-based supervision center and a handheld terminal; the drone is provided with a gimbal, and the photographing device is connected to the gimbal; the drone is provided with a first detection module and a second detection module, and the method comprises: The first detection module determines first information of the target object based on at least one target image of the target object, and the second detection module determines second information of the target object based on at least one target image of the target object; the first information is feature information representing the identity of the target object; the second information is feature information representing the safety device worn by the target object; the at least one target image of the target object is taken by the shooting device; The cloud-based monitoring center receives the first information and the second information sent by the drone, determines whether the target object meets the qualifications for high-altitude operations according to the first information, and outputs a first detection result, determines whether the safety equipment worn by the target object meets the preset wearing requirements according to the second information, and outputs a second detection result; The handheld terminal receives the first detection result and the second detection result sent by the cloud-based supervision center, so that the inspector can take corresponding measures in a timely manner according to the first detection result and the second detection result.

10. The method according to claim 9, characterized in that The first detection module determines first information of the target object based on at least one target image of the target object, including: Performing posture estimation on the face to be identified in each of the target images to determine the posture type of the face to be identified, wherein the posture type is used to characterize the orientation of the face to be identified; Determine a reference difference between the orientation angle corresponding to the posture type of the face to be identified and the front orientation angle, and use the target image corresponding to the face to be identified whose reference difference is less than or equal to a preset deviation angle as a reference image; Performing image enhancement processing on the reference image to obtain a processed image, performing feature extraction on the processed image using a deep learning model to obtain target features, and using the target features as the first information of the target object; Accordingly, the second detection module determines the second information of the target object based on at least one target image of the target object, including: A target detection model is used to label and classify the safety equipment worn by the target object in the target image, to obtain the labeled image and the device type of the safety equipment, and the labeled image and the device type of the safety equipment are used as the second information of the target object; the target detection model is a model based on the YOLOv5s target detection algorithm, which is used to label and classify objects in the image.

Images