Identification system and method for wiring equipment based on images

Abstract

The invention discloses an image-based wiring equipment identification system and method. The method comprises the following steps: acquiring a panoramic image of a working scene by using an unmanned aerial vehicle; judging whether wiring equipment exists in the panoramic image or not; if the wiring device exists in the panoramic image, determining the position of each wiring device in the panoramic image; setting a flight route of the unmanned aerial vehicle according to the position of each wiring device in the panoramic image; the unmanned aerial vehicle flies according to the flight route and obtains data information of each wiring device one by one; judging whether the wiring equipment meets wiring requirements or not according to the data information; and if the wiring equipment does not meet the wiring requirement, alarming. The unmanned aerial vehicle is used for collectively recognizing the wiring equipment and intelligently recognizing the wiring of the wiring equipment, and human eye recognition of workers is not needed any more, so that on one hand, the accuracy of wiring recognition is improved, and various potential safety hazards caused by wiring errors are avoided; on the other hand, the labor cost burden can be greatly reduced, the production efficiency is improved, and the labor gap is supplemented.

Description

Technical Field

[0001] The present invention belongs to the technical field of wiring equipment operation and maintenance, and in particular relates to an image-based wiring equipment recognition system and method. Background Art

[0002] With the rapid development of my country's economy, urban electricity consumption has increased dramatically, which has also led to a large investment in wiring equipment. Wiring equipment distributes and controls power to electrical equipment and provides power-off protection when the circuit is overloaded, short-circuited or leaking. Therefore, the correct wiring of wiring equipment is a necessary measure to ensure safe operation.

[0003] The existing electrical wiring has different wiring rules and requirements due to the different types, properties and levels of signals. However, there are still some problems, such as the design and layout of the cabinets are too crowded, making it difficult to quickly check after wiring; the unreasonable layout of components is not conducive to wiring; the strong and weak current wiring is mixed and intertwined; there is no good grounding measure; the drive and power device heat dissipation space is insufficient, etc. These are all safety hazards. It is not conducive to the rapid and effective maintenance of equipment by engineers.

[0004] Existing construction sites and large-scale projects are equipped with multiple wiring devices, and the inspection of wiring equipment wiring mainly relies on the staff's visual recognition, color calibration, position inspection, etc. When faced with complex electrical cabinets, it seems that they are unable to do so, the accuracy is not high enough, and it is easy to cause safety hazards; and relying on the workers for visual recognition will cause a waste of manpower. Summary of the invention

[0005] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide an image-based wiring device identification system and method thereof. During the inspection of electrical wiring, the wiring of the wiring device can be quickly and accurately inspected without touching the wires under test to meet the needs of safe work of personnel and equipment. The outstanding advantage of the present invention is that the inspection device is less affected by environmental factors, and the required data can be obtained through image processing methods in the face of complex lines to realize the inspection function.

[0006] In the first aspect, the present invention provides an image-based wiring device identification system, the system comprising: an unmanned aerial vehicle, used to obtain a panoramic image of a working scene; a first judgment module, used to determine whether there is a wiring device in the panoramic image; a position determination module, when there is a wiring device in the panoramic image, used to determine the position of one or more wiring devices; a flight route setting module, used to set the flight route of the unmanned aerial vehicle according to the position of each one or more wiring devices; a data acquisition module, used to obtain a wiring image, ID information and a standard wiring scheme paired with the ID information of the wiring device; a second judgment module, used to compare the wiring image with the standard wiring scheme to determine whether the wiring device meets the wiring requirements; an alarm module, which sends an alarm signal when the wiring device does not meet the wiring requirements.

[0007] In the second aspect, the present invention provides an image-based wiring device identification method, comprising the following steps:

[0008] S1, using drones to obtain panoramic images of the working scene;

[0009] S2, determining whether there is a wiring device in the panoramic image;

[0010] S3, if the wiring device exists in the panoramic image, determining the position of each wiring device in the panoramic image;

[0011] S4, setting a flight route of the UAV according to the positions of each wiring device in the panoramic image;

[0012] S5, the UAV flies according to the flight route and obtains data information of each wiring device one by one;

[0013] S6, judging whether the wiring device meets the wiring requirements according to the data information;

[0014] S7: If the wiring device does not meet the wiring requirements, an alarm is issued.

[0015] Preferably, the step S1 further comprises: S101, using an image redundancy removal algorithm to remove redundant images in the image to be identified;

[0016] S102, performing image grayscale transformation processing on the image;

[0017] S103, through histogram equalization, the grayscale of fewer pixels will be allocated to other grayscales, and the pixels are relatively concentrated;

[0018] S104, using Gabor filtering algorithm to remove noise interference in the image.

[0019] Preferably, step S2 further includes: collecting wiring device image samples and non-wiring device image samples, classifying, labeling and training, and obtaining a wiring device feature model; using k different rectangular frames to nominate on the convolution feature layer of the panoramic image, and screening out candidate areas where wiring devices may exist, where k is a positive integer; extracting features of wiring devices from the candidate areas respectively; using the wiring device feature model to classify and regress the features to identify whether the panoramic image has wiring devices; if it is identified that the detection image has wiring devices, performing border position regression processing on the wiring devices to obtain a wiring device frame.

[0020] Preferably, the step S3 further includes:

[0021] S301, constructing a Faster R-CNN network model, and inputting the extracted features of the wiring device into the Faster R-CNN network model for training;

[0022] S302, optimizing the parameters of the Faster R-CNN network model through real-time feedback of the neuron node recognition results to obtain a trained Faster R-CNN network model;

[0023] S303, inputting the extracted features of the wiring device into the trained Faster R-CNN network model for verification, and when the accuracy of the output result reaches a set threshold, a target object positioning model is obtained;

[0024] S304: Use the target object positioning model to locate each wiring device, and output the location information of each wiring device.

[0025] Preferably, step S4 also includes: determining the distance between each wiring device and the drone and the distance between each wiring device according to the position of each wiring device, taking the wiring device closest to the drone as the first station of the drone, and then taking the wiring device closest to the first station as the second station of the drone, and in this way until the drone reaches the last wiring device in the panoramic image, thereby completing the setting of the entire flight route.

[0026] Preferably, the step S5 further comprises: acquiring a wiring image, ID information and a standard wiring scheme matched with the ID information of the wiring device.

[0027] Preferably, step S5 also includes: the drone flies according to the set flight route, and when the drone approaches the wiring device, it captures the wiring image of the wiring device through a camera, and scans the QR code set on the wiring device through the camera to obtain the ID information of the wiring device and the standard wiring scheme matched with the ID information.

[0028] Preferably, the step S6 further comprises: comparing the wiring image of the wiring device with the standard wiring scheme, and if the wiring image is consistent with the standard wiring scheme, determining that the wiring device meets the wiring requirements.

[0029] Preferably, the step S7 further comprises: if the wiring device does not meet the wiring requirements, sending the ID information and the standard wiring scheme to the mobile terminal for alarm.

[0030] The beneficial effects of the present invention are:

[0031] Drones are used to identify wiring devices and intelligently identify the wiring of wiring devices, rather than relying on human visual identification by staff. On the one hand, this improves the accuracy of wiring identification and avoids various safety hazards caused by wiring errors; on the other hand, it can greatly reduce the burden of labor costs, improve production efficiency, and make up for labor shortages.

[0032] Description of the drawings

[0033] Figure 1 A flow chart of a wiring device identification method based on an image provided by an embodiment of the present invention;

[0034] Figure 2 A schematic diagram of an image-based wiring device identification system provided by another embodiment of the present invention;

[0035] Figure 3 A schematic diagram of the detection model training process provided by an embodiment of the present invention;

[0036] Figure 4 Schematic diagram of the specific implementation process of step S2 of the embodiment of the present invention. DETAILED DESCRIPTION

[0037] See also Figure 1 , which is a flow chart of a wiring device identification method based on an image provided by an embodiment of the present invention. As can be seen from the figure, it includes the following steps:

[0038] S1, using drones to obtain panoramic images of the working scene.

[0039] In a specific embodiment, a drone can be used to take multi-angle photos in a work scene, and then the multiple photos can be synthesized into a panoramic image of the work scene through panoramic image synthesis technology, wherein the panoramic image synthesis technology adopts existing conventional technology and will not be described in detail here.

[0040] In a specific embodiment, the following steps are also included:

[0041] S101, using an image redundancy removal algorithm to remove redundant images from the image to be identified;

[0042] S102, performing image grayscale transformation processing on the image;

[0043] S103, through histogram equalization, the grayscale of fewer pixels will be allocated to other grayscales, and the pixels are relatively concentrated;

[0044] S104, using Gabor filtering algorithm to remove noise interference in the image.

[0045] Generally, images are interfered and affected by various noises during the process of acquisition, generation and transmission, which causes the images to contain various noises and distortions. Direct use will affect the correctness and accuracy of the subsequent recognition system. Therefore, before target recognition, some preprocessing techniques are required to make useful information easier to extract.

[0046] The present invention adopts MATLAB software to pre-process the image to be identified. First, an image redundancy removal algorithm is adopted to remove redundant images in the image; then, the image is processed by image grayscale transformation to avoid stripe distortion; then, the grayscale of fewer pixels is allocated to other grayscales through histogram equalization, and the pixels are relatively concentrated. After the processing, the grayscale range, contrast and clarity become larger, and the image can be effectively enhanced; finally, a Gabor filtering algorithm is adopted to remove noise interference in the image, and the spatial information of the target object in the image can be highlighted, and other irrelevant information can be suppressed.

[0047] S2, determining whether there is a wiring device in the panoramic image.

[0048] See also Figure 3 In the specific implementation, it is necessary to train the wiring device feature model. Collect a variety of wiring device image samples and non-wiring device image samples, classify, label and train them, and obtain the wiring device feature model.

[0049] A large number of wiring equipment image samples and non-wiring equipment image samples are collected, annotated using special tools, and divided into training samples and test samples. The training samples are used to train the model parameters, and the test samples are used to test the model effect. This example uses 8,000 wiring equipment image samples and non-wiring equipment image samples, and uses the ImgLabel tool to annotate the area and category. The training set and the test set are divided into 7:3 ratios.

[0050] See also Figure 4 , using k different rectangular boxes to nominate on the convolutional feature layer of the panoramic image, and screening out candidate areas where wiring devices may exist, where k is a positive integer.

[0051] The feature extraction of the wiring device is performed on the candidate area respectively; in the specific implementation, the extracted image features mainly include: texture features, edge features and motion features of the image. (1) Texture features mainly include grayscale histogram, edge direction histogram, grayscale co-occurrence matrix, etc. of the image. (2) Edge features directly show the outline of the image, mainly including the perimeter, area, aspect ratio (principal axis ratio), dispersion, compactness, etc. of the image. (3) Motion features are related to motion behavior, generally including motion center of mass, speed, displacement, gradient, etc.

[0052] In the specific implementation, the entire image is input and the Darknet51 network is used to automatically obtain the features of the image.

[0053] The features are classified and regressed using a wiring device feature model to identify whether the detection image has a wiring device; if it is identified that the detection image has a wiring device to be worn, a frame position regression process is performed on the wiring device to be worn to obtain a wiring device frame.

[0054] In the specific implementation, the Feature Map after feature extraction is classified into two categories to determine whether it contains wiring devices, and then k regression models (each corresponding to a different Anchor Box) are used to adjust the position and size of the candidate box, and finally the wiring device classification is performed.

[0055] In this example, the model further processes the predicted values ​​of the bounding boxes of each Feature in the Feature Map (the relative coordinates of the center point and the relative width and height of the box) to obtain the actual values. The predicted center coordinates are normalized using the sigmoid function, and the actual coordinates are obtained by adding the offset value of the current Feature relative to the upper left corner of the image. The predicted object width and height are normalized using the exp function, and then multiplied by the width and height of the prior box corresponding to the current Anchor Box to obtain the actual width and height. The width and height of the prior box corresponding to each Anchor Box can be obtained by clustering the data set.

[0056] This embodiment uses the detection and classification model and GPU to determine whether each area contains a specific type of target. The detection model is Darknet, and the classifier is the YOLOV3 model based on the focal loss function. Then, the detected wiring device to be worn is processed by the frame position regression post-processing operation to obtain the final wiring device frame.

[0057] S3: If the wiring device exists in the panoramic image, determine the position of each wiring device in the panoramic image.

[0058] In a specific embodiment, a Faster R-CNN network model is constructed, and the features of the extracted wiring devices are input into the Faster R-CNN network model for training; the parameters of the Faster R-CNN network model are optimized through real-time feedback of the neuron node recognition results to obtain a trained Faster R-CNN network model; the features of the extracted wiring devices are input into the trained Faster R-CNN network model for verification, and when the accuracy of the output result reaches a set threshold, a target object positioning model is obtained; each wiring device is positioned using the target object positioning model, and the location information of each wiring device is output.

[0059] The present invention adopts Faster R-CNN, a special deep neural network algorithm for target detection and positioning, to identify the appearance characteristics of targets with and without stacking occlusion and determine the robot's grasping position.

[0060] First, a Faster R-CNN network model is constructed, and the outer contour features of the objects extracted from the training set are input into the Faster R-CNN network model for training. By feeding back the recognition results of the neuron nodes, the network parameters are continuously optimized, so as to solve the problem that stacked and partially occluded targets in complex environments cannot be accurately recognized, improve the target recognition accuracy and speed, and obtain the trained Faster R-CNN network model; then, the test set is output to the trained Faster R-CNN network model for verification, so that the accuracy of the output result of the model meets the set threshold requirement of the present invention, and a high-precision and high-speed target object positioning model can be obtained, and finally the position information of each object is output.

[0061] S4, setting a flight route of the UAV according to the positions of each wiring device in the panoramic image.

[0062] In a specific embodiment, according to the position of each wiring device, the distance between each wiring device and the drone and the distance between each wiring device are determined, and the wiring device closest to the drone is used as the first station of the drone, and then the wiring device closest to the first station is used as the second station of the drone, and the wiring device closest to the nth station is used as the n+1th station of the drone, and this method is used until the drone reaches the last wiring device in the panoramic image, thereby completing the setting of the entire flight route.

[0063] S5, the UAV flies according to the flight route and obtains data information of each wiring device one by one.

[0064] In a specific embodiment, the drone flies according to a set flight route and stops near each wiring device. When the drone approaches the wiring device, the camera scans the QR code set on the wiring device to obtain the ID information of the wiring device and the standard wiring scheme matched with the ID information; wherein the ID information and the standard wiring scheme matched with the ID information are stored in a corresponding storage module in a pre-stored manner. At the same time, the camera is used to capture the wiring image of the wiring device, and the wiring image and the standard wiring scheme corresponding to the ID information of the wiring device are sent to the judgment module to judge whether the wiring device meets the wiring requirements.

[0065] S6: Determine whether the wiring device meets wiring requirements based on the data information.

[0066] In a specific embodiment, a wiring image of the wiring device is compared with the standard wiring scheme. If the wiring image is consistent with the standard wiring scheme, it is determined that the wiring device meets the wiring requirements.

[0067] S7: If the wiring device does not meet the wiring requirements, an alarm is issued.

[0068] In a specific embodiment, when it is found that the wiring device does not meet the wiring requirements by comparing the wiring image of the wiring device with the standard wiring scheme, the ID information and the standard wiring scheme are sent to the mobile terminal for alarm, so that the staff can quickly locate the wiring device and rewire according to the marked wiring scheme.

[0069] See also Figure 2 , which is a schematic diagram of an image-based wiring device identification system provided by an embodiment of the present invention; as shown in the figure, the system 100 of the present invention includes: a drone 101, used to obtain a panoramic image of a working scene; a first judgment module 102, used to judge whether there is a wiring device in the panoramic image; a position determination module 103, when there is a wiring device in the panoramic image, used to determine the position of one or more wiring devices; a flight route setting module 104, used to set the flight route of the drone according to the position of each one or more wiring devices; a data acquisition module 105, used to obtain a wiring image, ID information and a standard wiring scheme paired with the ID information of the wiring device; a second judgment module 106, used to compare the wiring image with the standard wiring scheme to determine whether the wiring device meets the wiring requirements; an alarm module 107, which sends an alarm signal when the wiring device does not meet the wiring requirements.

[0070] In one embodiment, the drone 101 takes photos at multiple angles in the working scene, and then uses a panoramic image synthesis technology to synthesize the multiple photos into a panoramic image of the working scene.

[0071] In one embodiment, the first judgment module 102 collects wiring device image samples and non-wiring device image samples, classifies, labels and trains them, and obtains a wiring device feature model; uses k different rectangular frames to nominate on the convolution feature layer of the panoramic image to screen out candidate areas where wiring devices may exist, where k is a positive integer; extracts features of wiring devices from the candidate areas respectively; uses the wiring device feature model to classify and regress the features to identify whether the panoramic image has wiring devices; if it is identified that the detection image has wiring devices, the wiring device is regressed to obtain a wiring device frame.

[0072] In one embodiment, the location determination module 103 constructs a Faster R-CNN network model, inputs the extracted features of the wiring device into the Faster R-CNN network model for training; optimizes the parameters of the Faster R-CNN network model through real-time feedback of the neuron node recognition results to obtain a trained Faster R-CNN network model; inputs the extracted features of the wiring device into the trained Faster R-CNN network model for verification, and when the accuracy of the output result reaches a set threshold, a target object positioning model is obtained; and each wiring device is positioned using the target object positioning model, and the location information of each wiring device is output.

[0073] In one embodiment, the flight route setting module 104 determines the distance between each wiring device and the drone and the distance between each wiring device according to the position of each wiring device, and uses the wiring device closest to the drone as the first station of the drone, and then uses the wiring device closest to the first station as the second station of the drone, and uses the wiring device closest to the nth station as the n+1th station of the drone, and so on until the drone reaches the last wiring device in the panoramic image, thereby completing the setting of the entire flight route.

[0074] In one embodiment, when the drone approaches the wiring device, the data acquisition module 105 scans the QR code set on the wiring device through the camera to obtain the ID information of the wiring device and the standard wiring scheme matched with the ID information; wherein the ID information and the standard wiring scheme matched with the ID information are stored in the corresponding storage module in a pre-stored manner. At the same time, the camera is used to capture the wiring image of the wiring device, and the wiring image and the standard wiring scheme corresponding to the ID information of the wiring device are sent to the judgment module to judge whether the wiring device meets the wiring requirements.

[0075] In one embodiment, the second judgment module 106 compares the wiring image of the wiring device with the standard wiring scheme, and if the wiring image is consistent with the standard wiring scheme, it is determined that the wiring device meets the wiring requirements.

[0076] In one embodiment, when the alarm module 107 compares the wiring image of the wiring device with the standard wiring plan and finds that the wiring device does not meet the wiring requirements, the ID information and the standard wiring plan are sent to the mobile terminal for alarm, so that the staff can quickly locate the wiring device and rewire it according to the marked wiring plan.

[0077] The embodiments described above are only descriptions of the preferred modes of the present invention, and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. An image-based wiring equipment identification system, characterized in that: The system includes: a drone for obtaining panoramic images of the working scene; A first determination module is used to determine whether there is a wiring device in the panoramic image; A position determination module is used to determine the position of one or more wiring devices when there are wiring devices in the panoramic image; a flight route setting module is used to set the flight route of the drone according to the position of one or more wiring devices; a data acquisition module is used to obtain the wiring image, ID information and the standard wiring scheme matched with the ID information of the wiring device; a second judgment module is used to compare the wiring image with the standard wiring scheme to determine whether the wiring device meets the wiring requirements; an alarm module sends an alarm signal when the wiring device does not meet the wiring requirements.

2. A wiring device identification method based on an image, characterized in that: The following steps are involved: S1, using drones to obtain panoramic images of the working scene; S2, determining whether there is a wiring device in the panoramic image; S3, if the wiring device exists in the panoramic image, determining the position of each wiring device in the panoramic image; S4, setting a flight route of the drone according to the positions of each wiring device in the panoramic image; S5, the UAV flies according to the flight route and obtains data information of each wiring device one by one; S6, judging whether the wiring device meets the wiring requirements according to the data information; S7: If the wiring device does not meet the wiring requirements, an alarm is issued.

3. The image-based wiring device identification method according to claim 2, characterized in that: The step S1 further includes: S101, using an image redundancy removal algorithm to remove redundant images from the image to be identified; S102, performing image grayscale transformation processing on the image; S103, through histogram equalization, the grayscale of fewer pixels will be allocated to other grayscales, and the pixels are relatively concentrated; S104, using Gabor filtering algorithm to remove noise interference in the image.

4. The image-based wiring device identification method according to claim 3, characterized in that: The step S2 also includes: collecting wiring device image samples and non-wiring device image samples, classifying, labeling and training them, and obtaining a wiring device feature model; using k different rectangular frames to nominate on the convolution feature layer of the panoramic image to screen out candidate areas where wiring devices may exist, where k is a positive integer; extracting features of wiring devices from the candidate areas respectively; using the wiring device feature model to classify and regress the features to identify whether the panoramic image has wiring devices; if it is identified that the detection image has wiring devices, performing border position regression processing on the wiring devices to obtain a wiring device frame.

5. The method for identifying wiring devices based on images according to claim 4, wherein step S3 further comprises: S301, constructing a Faster R-CNN network model, and inputting the extracted features of the wiring device into the Faster R-CNN network model for training; S302, optimizing the parameters of the Faster R-CNN network model through real-time feedback of the neuron node recognition results to obtain a trained Faster R-CNN network model; S303, inputting the extracted features of the wiring device into the trained Faster R-CNN network model for verification, and when the accuracy of the output result reaches a set threshold, a target object positioning model is obtained; S304: Use the target object positioning model to locate each wiring device, and output the location information of each wiring device.

6. The method for identifying wiring devices based on images according to claim 5, wherein step S4 further comprises: According to the position of each wiring device, the distance between each wiring device and the drone and the distance between each wiring device are determined, and the wiring device closest to the drone is used as the first station of the drone, and then the wiring device closest to the first station is used as the second station of the drone, and this method is used until the drone reaches the last wiring device in the panoramic image, thereby completing the setting of the entire flight route.

7. The method for identifying wiring devices based on images according to claim 6, wherein step S5 further comprises: The wiring image, ID information and the standard wiring scheme matched with the ID information of the wiring device are obtained.

8. The method for identifying wiring devices based on images according to claim 7, wherein step S5 further comprises: The drone flies according to the set flight route. When the drone approaches the wiring device, it uses the camera to capture the wiring image of the wiring device and scans the QR code set on the wiring device through the camera to obtain the ID information of the wiring device and the standard wiring solution matched with the ID information.

9. The method for identifying wiring devices based on images according to claim 8, wherein step S6 further comprises: The wiring image of the wiring device is compared with the standard wiring scheme. If the wiring image is consistent with the standard wiring scheme, it is determined that the wiring device meets the wiring requirements.

10. The method for identifying wiring devices based on images according to claim 9, wherein the step S7 further comprises: If the wiring device does not meet the wiring requirements, the ID information and the standard wiring solution are sent to the mobile terminal for alarm.

Images