Method, electronic device and medium for measuring knife opening and closing distance

Abstract

The present invention provides a method, electronic device and medium for measuring the opening and closing distance of a gate blade. The method of the present invention comprises: collecting a gate blade image in real time through a camera; inputting the gate blade image into a pre-trained deep learning network model to obtain a gate blade feature image; cutting out a gate blade second end image from the gate blade image; matching the gate blade feature image with the gate blade second end image to obtain the two-dimensional coordinates of the center of the gate blade second end; constructing a motion equation and a ray equation for the center of the gate blade second end; calculating the intersection of the motion equation and the ray equation for the center of the gate blade second end to obtain the three-dimensional coordinates of the center of the gate blade second end; calculating the distance value of the gate blade movement according to the three-dimensional coordinates of the center of the gate blade second end on the two previous and next gate blade images. The present invention matches the gate blade feature image with the gate blade second end, and converts the two-dimensional coordinates of the gate blade second end into three-dimensional coordinates, processes the two-dimensional problem in three dimensions, and improves the accuracy of the gate blade opening and closing distance measurement.

Description

Technical Field

[0001] The invention belongs to the technical field of power equipment detection, and in particular relates to a method for measuring the opening and closing distance of a knife switch, electronic equipment and a medium. Background Art

[0002] The high-voltage box is an important electrical equipment of the whole vehicle, and the knife switch is an important connecting device in the high-voltage box. Due to the long-term use of the knife switch, the knife switch in the high-voltage box may become loose, not closed in place, or have closing deviations. Therefore, it is very important to monitor the opening and closing of the knife switch in the high-voltage box.

[0003] At present, the measurement of the opening and closing of the knife in the high-voltage box is generally measured by an infrared distance measuring sensor. The opening and closing distance of the knife is measured by the infrared distance measuring sensor, and the measured value is returned to the control center. At the same time, in some places, the opening and closing of the knife in the high-voltage box is judged by manual observation, and only the staff judge whether it is available. Since the high-voltage box is arranged on a train or a motor vehicle, it is a confined space with a huge high-voltage current, and the train or the motor vehicle is always in motion. The entire train or the motor vehicle has a certain degree of shaking. Therefore, there are certain problems in using an infrared distance measuring sensor to measure the opening and closing distance of the knife. The infrared distance measuring sensor causes the measuring point to shift due to the shaking of the train or the motor vehicle during operation, thereby affecting the accuracy of the result. At the same time, the infrared distance measuring sensor can only transmit data and cannot provide the real-time situation of the knife in the high-voltage box.

[0004] With the continuous development of deep learning technology, especially in the work of target detection and target tracking, deep learning has good stability, fast recognition speed and strong adaptability to scene lighting changes. Therefore, deep learning technology is widely used in the rail transit industry.

[0005] Patent application CN110728171A discloses a method for refined detection and identification of knife switch status, which includes inputting the knife switch image to be detected into a pre-trained deep learning module (YOLOV3 model) to judge the opening state of the knife switch. It detects the opening and closing state of the knife switch through a deep learning method, and can only roughly judge whether the knife switch is in an open or closed state, and the detection accuracy and precision are low. Summary of the invention

[0006] The purpose of the present invention is to provide a method, electronic equipment and medium for measuring the opening and closing distance of a knife switch in view of the deficiencies in the prior art, so as to realize the detection of the opening and closing distance of the knife switch and improve the measurement accuracy of the opening and closing state of the knife switch.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is:

[0008] A method for measuring the opening and closing distance of a knife switch comprises the following steps:

[0009] S1: Real-time acquisition of blade images through the camera;

[0010] S2: inputting the gate blade image into a pre-trained deep learning network model to obtain a gate blade feature image; and cropping a gate blade second end image from the gate blade image;

[0011] S3: Matching the gate blade feature image with the gate blade second end image to obtain the two-dimensional coordinates of the center of the gate blade second end;

[0012] S4: Taking the center of the first end of the knife as the origin and the distance from the center of the second end of the knife to the center of the first end of the knife as the radius, construct the motion equation of the center of the second end of the knife, and transform the motion equation of the center of the second end of the knife into the camera coordinate system;

[0013] S5: Taking the origin of the camera coordinate system as the starting point, construct the ray equation of the center of the second end of the switch;

[0014] S6: Calculate the intersection of the motion equation of the center of the second end of the knife and the ray equation to obtain the three-dimensional coordinates of the center of the second end of the knife;

[0015] S7: Calculate the distance value of the gate blade movement according to the three-dimensional coordinates of the center of the second end of the gate blade on the two front and rear gate blade images. The expression of the distance value of the gate blade movement is as follows:

[0016]

[0017] Among them, (x1, y1, z1) is the three-dimensional coordinate of the center of the second end of the knife in the previous knife image, and (x2, y2, z2) is the three-dimensional coordinate of the center of the second end of the knife in the current knife image.

[0018] The present invention extracts features from the gate blade image data through a deep learning method, and performs secondary positioning of the gate blade feature image extracted by the deep learning network model through the image of the second end of the gate blade, thereby improving the positioning accuracy of the second end of the gate blade and the accuracy of the gate blade opening and closing distance measurement; the present invention constructs the motion equation and ray equation of the second end of the gate blade to solve the three-dimensional coordinates of the second end of the gate blade, transforms the two-dimensional problem into three-dimensional, and improves the accuracy and precision of the gate blade opening and closing distance measurement.

[0019] Furthermore, the deep convolutional network model is a YOLOV5 model, and the YOLOV5 model includes a backbone network, a neck network and a head network.

[0020] Furthermore, the two-dimensional coordinate of the center of the second end of the blade is calculated by the following formula:

[0021] R(x,y)=sumx′.y′ (T(x′,y′)-I(x+x′,y+y′)) 2

[0022] Among them, R(x,y) is the image of the second end of the knife after matching, T(x′,y′) is the image of the second end of the knife, I(x+x′,y+y′) is the characteristic image of the knife, sum x′.y′ () is the similarity calculation function, (x, y) is the two-dimensional coordinate of the center of the second end of the blade.

[0023] Furthermore, the expression of the motion equation of the center of the second end of the blade is as follows:

[0024] (xh) 2 +(yk) 2 =r 2

[0025] Among them, (h, k) is the coordinate of the center of the first end of the knife, and r is the distance from the center of the second end of the knife to the center of the first end of the knife.

[0026] Furthermore, the expression of the ray equation at the center of the second end of the blade is as follows:

[0027] L=P0+tp′

[0028] Where L is the ray from the center of the second end of the switch, p0 is the origin of the camera coordinate system, t is the scale factor, p′ is the normalized camera coordinate, and p′=K -1 P h , K -1 is the inverse of the camera intrinsic parameter matrix, P h is the homogeneous coordinate transformed from the point P(u,v) on the knife image, p h =(u,v,1).

[0029] Based on the same inventive concept, the present invention also provides an electronic device, including:

[0030] one or more processors;

[0031] A memory having one or more programs stored thereon, which, when executed by the one or more processors, enables the one or more processors to implement the steps of the method for measuring the opening and closing distance of a knife switch.

[0032] Based on the same inventive concept, the present invention also provides a computer-readable storage medium storing a computer program, which implements the steps of the method for measuring the opening and closing distance of a knife switch when the computer program is executed by a processor.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] The present invention extracts features from the gate blade image data through a deep learning method, and performs secondary positioning of the gate blade feature image extracted by the deep learning network model through the image of the second end of the gate blade, thereby improving the positioning accuracy of the second end of the gate blade and the accuracy of the gate blade opening and closing distance measurement; the present invention constructs a three-dimensional motion equation and a ray equation of the second end of the gate blade, solves the three-dimensional coordinates of the second end of the gate blade, transforms the two-dimensional problem into three-dimensional, and improves the accuracy and precision of the gate blade opening and closing distance measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a schematic diagram of the flow of the method for measuring the opening and closing distance of the knife switch of the present invention;

[0036] Figure 2 It is a schematic diagram of the knife structure;

[0037] Figure 3 Schematic diagram of obtaining the gate features for the YOLOV5 model;

[0038] Figure 4 Schematic diagram of the conversion between the knife coordinate system and the camera coordinate system. DETAILED DESCRIPTION

[0039] The present invention will be described in detail below in conjunction with the embodiments. It should be noted that the embodiments and features of the embodiments of the present invention can be combined with each other without conflict. For the convenience of description, if the words "upper", "lower", "left" and "right" appear in the following, they only indicate that the upper, lower, left and right directions are consistent with the drawings themselves, and do not limit the structure.

[0040] Example

[0041] like Figure 1 As shown, the method for measuring the opening and closing distance of the high-voltage box switch of this embodiment comprises the following steps:

[0042] Step 1: Collect the image data of the knife opening and closing in the high-voltage box to form a knife opening and closing measurement image dataset, and divide it into a training set, a validation set, and a test set;

[0043] Specifically, the image of the switch blade is collected through the high-voltage box switch blade opening and closing image data collection system, which includes a camera, a light source, and a host computer. The camera is used to collect the switch blade image in real time, the light source is used to fill the light for the switch blade in a dark environment to facilitate the imaging of the camera, and the host computer is used to visualize and save the collected data. Then, the collected switch blade image data is pre-processed by screening, segmentation, labeling, etc. to form a switch blade image data set.

[0044] like Figure 2As shown, the gate knife includes a first end and a second end, the first end is a stationary end, and the second end is a movable end; the gate knife moves through the second end, changing the position of the second end on the gate knife column to achieve the opening and closing of the gate knife. The center of the second end is the geometric center of the overlapped part of the second end and the gate knife column when the gate knife and the gate knife column are vertical, and the center of the first end is the geometric center of the first end.

[0045] The method for generating a knife image data set for knife opening and closing distance measurement by collecting on-site image data of knife opening and closing in a high-voltage box specifically comprises the following steps:

[0046] 1) The Zhang Zhengyou calibration method is used to calibrate the internal and external parameters of the camera, which is a prior art. Specifically, the camera internal parameter calibration is achieved by collecting the calibration plate image by the camera, extracting the calibration plate corner points, establishing the camera projection model, and solving the camera internal and external parameters. Then the camera and the center of the first end of the gate are jointly calibrated, which is specifically achieved by placing the calibration plate at the center of the first end of the gate, the camera collecting the calibration plate image, extracting the calibration plate corner points, establishing the camera reprojection model, and finally realizing the rigid transformation between the camera and the gate. The frame rate of the camera in this embodiment is 50fps.

[0047] 2) Label the collected blade opening and closing image data, label the blade part data as "1" and the rest as "0", and create a data set for YOLOV5 model training. In this embodiment, the labelImg tool is used to label the high-voltage box blade image data, and generate label files and original image data respectively.

[0048] 3) Divide the generated gate blade image data set into a training set, a validation set and a test set. In this embodiment, the gate blade image data set contains 3000 image data, which are divided into a training set, a validation set and a test set in a ratio of 7:2:1.

[0049] Step 2: Pass the training data set through the YOLOV5 model to learn the distribution characteristics of the gate in different image data backgrounds.

[0050] The YOLOV5 model includes a backbone network, a neck network, and a head network. The YOLOV5 model includes four operations: input image preprocessing, which is to scale and normalize the input image to meet the input requirements of the model; feature extraction, which is to send the preprocessed image to the backbone network, and the backbone network will extract features from the image; feature processing and detection, which is to send the extracted features to the head network after feature extraction; post-processing, which is the output of the target box and category information after post-processing to form the final target detection result.

[0051] In this embodiment, model training is performed in the virtual environment imageDetected_Part created by conda, and the gate knife image data are respectively input into the YOLOV5 model.

[0052] Set the loss function of the YOLOV5 model, and optimize and evaluate the YOLOV5 model based on the validation set and test set.

[0053] Specifically, the loss function of the YOLOV5 model can be expressed as:

[0054] L_seg=-∑(y_seg*log(y_pred_seg)) / N_seg

[0055] Among them, y _ seg is the true target detection label, y_pred_seg is the predicted target detection category probability, and N_seg is the number of points in the gate image.

[0056] The image of the switch in the high-voltage box captured by the camera in real time is input into the trained YOLOV5 model to obtain the switch feature image, such as Figure 3 shown.

[0057] Step 3: By performing regional cropping on the gate blade image captured by the camera, the second end area of ​​the gate blade on the gate blade image is cropped out to produce the image data of the second end of the gate blade. Then, the cropped second end image of the gate blade is used to match the second end of the gate blade in the gate blade feature image obtained by the YOLOV5 model, so as to obtain the accurate coordinate position of the second end of the gate blade.

[0058] Specifically, the image of the gate blade captured by the camera is used to crop the second end of the gate blade on the captured large image using an image cropping tool to obtain the image of the second end of the gate blade. The image of the second end of the gate blade is then slid on the gate blade feature image acquired by the YOLOV5 model, and the similarity between the image of the second end of the gate blade and the corresponding area of ​​the gate blade feature image is calculated respectively, and finally the accurate two-dimensional coordinates of the second end of the gate blade are calculated.

[0059] The similarity between the image of the second end of the gate blade and the corresponding area of ​​the gate blade feature image, and the calculation formula for solving the accurate two-dimensional coordinates of the second end of the gate blade at the second end of the gate blade are as follows:

[0060] R(x,y)=sum x′.y′ (T(x′,y′)-I(x+x′,y+y′)) 2

[0061] Among them, R(x,y) is the image of the second end of the knife after matching, T(x′,y′) is the image of the second end of the knife, I(x+x′,y+y′) is the characteristic image of the knife, sum x′.y′() is the similarity calculation function, (x, y) is the two-dimensional coordinate of the second end of the blade.

[0062] Step 4: Establish a three-dimensional space coordinate system with the center of the first end of the switch in the high-voltage box as the origin, construct the motion equation of the second end of the switch, and then convert the motion equation of the second end of the switch to the camera coordinate system. Figure 4 shown.

[0063] Specifically, the motion equation of the second end of the switch blade is converted into the camera coordinate system, which specifically includes the following steps:

[0064] 1) Take the center of the first end of the switch blade in the high-voltage box as the origin of the three-dimensional coordinate system, and take the distance from the center of the second end of the switch blade to the center of the first end as the radius, and construct the motion equation (cylindrical surface equation) of the second end of the switch blade. In this embodiment, it is carried out in the OPENCV library, and the center of the first end of the switch blade is selected as the origin coordinate, and the distance from the center of the second end to the center of the first end of the switch blade is the radius.

[0065] The equation of motion for the second end of the blade is expressed as follows:

[0066] (xh) 2 +(yk) 2 =r 2

[0067] Where (h, k) is the coordinate of the center of the first end of the knife, and r is the distance from the center of the second end of the knife to the center of the first end of the knife;

[0068] 2) The motion equation of the second end of the knife is transformed into the camera coordinate system through the calibrated rigid transformation matrix of the camera and the knife. In this embodiment, the calibrated rigid transformation matrix of the camera and the knife is RT, and the motion equation of the second end of the knife is transformed into the camera coordinate system and can be expressed as:

[0069] Plane camera =RT*Plane cylinder

[0070] Among them, Plane camera is the motion equation of the second end of the blade in the camera coordinate system, RT is the rigid transformation matrix of the camera and the blade, Plane cylinder It is the motion equation of the second end of the knife in the knife coordinate system.

[0071] The rigid transformation matrix of the camera and the blade is obtained by calibration, which is an existing technology. The specific steps are: placing a checkerboard calibration plate at the center of the first end of the blade, the calibration plate is perpendicular to the initial position of the blade, calibrating the RT between the checkerboard world coordinate system and the camera coordinate system, and the rigid transformation matrix RT can be expressed as:

[0072]

[0073] Among them, R 00 , R 10 , R 20 They represent the projection direction of the x-axis of the camera coordinate system in the world coordinate system, R 01 , R 11 , R 21 Represents the projection direction of the y-axis of the camera coordinate system in the world coordinate system, R 02 , R 12 , R 22 Indicates the projection direction of the z-axis of the camera coordinate system in the world coordinate system, T 03 is the camera translation in the x direction, T 13 is the camera's translation in the y direction, T 23 is the camera's translation in the z direction.

[0074] Step 5: Construct the ray equation of the center of the second end of the knife according to the principle of camera pinhole imaging, and then use the motion equation Plane of the second end of the knife cylinder The three-dimensional coordinates of the second end of the switch are solved by the ray equation, and then the opening and closing distance of the switch in the high-voltage box is calculated.

[0075] Specifically, the motion equation of the second end of the knife and the ray equation in the camera coordinate system are solved together to obtain the three-dimensional coordinates of the second end of the knife, which specifically includes the following steps:

[0076] 1) Based on the pinhole imaging principle of the camera, taking the origin of the camera coordinate system as the starting point, solve the ray equation of the center of the second end of the switch.

[0077] Specifically:

[0078] The camera intrinsic parameter matrix K is obtained by using Zhang Zhengyou calibration method. The camera intrinsic parameter matrix K can be expressed as:

[0079]

[0080] Among them, f x is the focal length of the camera along the x-axis, f y is the focal length of the camera in the y-axis direction, c x 、c y They represent the horizontal and vertical coordinates of the projection center of the camera optical axis on the image plane respectively.

[0081] Convert a point P(u,v) on the knife image to homogeneous coordinates p h , and then use the inverse of the camera intrinsic parameter matrix to transform the image coordinates into coordinates in the normalized camera coordinate system:

[0082]

[0083] Among them, p′ is the normalized camera coordinate, K -1 is the inverse of the camera intrinsic parameter matrix, P h is the homogeneous coordinate transformed from the point P(u,v) on the knife image, p h =(u,v,1).

[0084] The calculated normalized camera coordinate p′ is used to represent the ray, which can be expressed as starting from the origin of the camera coordinate system. The ray equation can be expressed as:

[0085] L=P0+tp′

[0086] Where t is the scale factor of the ray and L is the ray at the center of the second end of the blade.

[0087] 2) Then calculate the coordinates of the intersection of the ray and the motion equation. In this embodiment, the algorithm for implementing the ray equation and the motion equation is implemented in the OPENCV library.

[0088] 3) Measure the two images of the knife captured by the camera after the knife moves a certain distance, solve the three-dimensional coordinates of the center point of the second end of the knife in the two images in the camera coordinate system, and then solve the distance the knife moves. In this embodiment, the measurement of the knife opening and closing distance is mainly solved by the distance formula between the center point of the second end of the knife in the current image in the camera coordinate system and the center point of the second end of the knife in the previous image in the camera coordinate system. The distance formula can be expressed as

[0089]

[0090] Among them, (x1, y1, z1) is the three-dimensional coordinate of the center of the second end of the knife in the previous knife image, and (x2, y2, z2) is the three-dimensional coordinate of the center of the second end of the knife in the current knife image. The above formula can calculate the straight-line distance between two points.

[0091] This implementation method has the following advantages:

[0092] 1) The YOLOV5 model is used to detect the gate from the image captured by the camera. Compared with the traditional algorithm for extracting gate feature data, the YOLOV5 model improves the stability and accuracy of extracting the gate from the complex background.

[0093] 2) The extraction of gate blade image data through deep learning methods not only improves the accuracy of gate blade feature extraction, but also improves the adaptability of the gate blade opening and closing distance measurement algorithm to different working conditions.

[0094] 3) By making the image of the second end of the gate, the gate feature image extracted by the YOLOV5 model is accurately positioned again, which improves the positioning accuracy of the center point of the second end of the gate, and also improves the measurement accuracy of the gate opening and closing distance measurement.

[0095] 4) By constructing the motion equation of the second end and the ray equation of the second end based on the camera imaging principle in the camera coordinate system, the three-dimensional coordinates of the center of the second end in the camera coordinate system are solved jointly, and the two-dimensional problem is processed in three dimensions, thereby improving the measurement accuracy of the gate opening and closing distance measurement.

[0096] The method of this embodiment is adaptable to different operating conditions and can achieve high-precision measurement of the opening and closing distance of the switch in the high-voltage box under complex conditions (vibration, confined space, etc.).

[0097] Another embodiment of the present invention provides an electronic device, including:

[0098] one or more processors;

[0099] A memory stores one or more programs, and when the one or more programs are executed by one or more processors, the one or more processors implement the steps of the method for measuring the opening and closing distance of the knife switch.

[0100] In some implementations, the memory may be a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory.

[0101] In some other implementations, the processor may be a central processing unit (CPU), a digital signal processor (DSP), or other general-purpose processors of various types, which are not limited herein.

[0102] Another embodiment of the present invention provides a computer-readable storage medium storing a computer program, which implements the steps of the method for measuring the opening and closing distance of a knife switch when the computer program is executed by a processor.

[0103] The contents explained in the above embodiments should be understood as these embodiments are only used to more clearly illustrate the present invention, and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

Claims

1. A method for measuring the opening and closing distance of a knife switch, characterized in that: The following steps are involved: S1: Real-time acquisition of blade images through the camera; S2: inputting the gate blade image into a pre-trained deep learning network model to obtain a gate blade feature image; and cropping the gate blade second end image from the gate blade image; S3: Matching the gate blade feature image with the gate blade second end image to obtain the two-dimensional coordinates of the center of the gate blade second end; S4: Taking the center of the first end of the knife as the origin and the distance from the center of the second end of the knife to the center of the first end of the knife as the radius, construct the motion equation of the center of the second end of the knife, and transform the motion equation of the center of the second end of the knife into the camera coordinate system; S5: Taking the origin of the camera coordinate system as the starting point, construct the ray equation of the center of the second end of the switch; S6: Calculate the intersection of the motion equation of the center of the second end of the knife and the ray equation to obtain the three-dimensional coordinates of the center of the second end of the knife; S7: Calculate the distance value of the gate blade movement according to the three-dimensional coordinates of the center of the second end of the gate blade on the two front and rear gate blade images. The expression of the distance value of the gate blade movement is as follows: Among them, (x1, y1, z1) is the three-dimensional coordinate of the center of the second end of the knife in the previous knife image, and (x2, y2, z2) is the three-dimensional coordinate of the center of the second end of the knife in the current knife image.

2. The method for measuring the opening and closing distance of a knife according to claim 1, characterized in that: The deep convolutional network model is a YOLOV5 model, which includes a backbone network, a neck network and a head network.

3. The method for measuring the opening and closing distance of a knife according to claim 1, characterized in that: The two-dimensional coordinates of the center of the second end of the blade are calculated by the following formula: R(x,y)=sum x′.y′ (T(x′,y′)-I(x+x′,y+y′)) 2 Among them, R(x,y) is the image of the second end of the knife after matching, T(x′,y′) is the image of the second end of the knife, I(x+x′,y+y′) is the characteristic image of the knife, sum x′.y′ () is the similarity calculation function, (x, y) is the two-dimensional coordinate of the center of the second end of the blade.

4. The method for measuring the opening and closing distance of a knife according to claim 1, characterized in that: The expression of the motion equation of the center of the second end of the blade is as follows: (x-h) 2 +(y-k) 2 =r 2 Among them, (h, k) is the coordinate of the center of the first end of the knife, and r is the distance from the center of the second end of the knife to the center of the first end of the knife.

5. The method for measuring the opening and closing distance of a knife according to claim 1, characterized in that: The expression of the ray equation at the center of the second end of the blade is as follows: L=P0+tp′ Where L is the ray from the center of the second end of the switch, p0 is the origin of the camera coordinate system, t is the scale factor, p′ is the normalized camera coordinate, and p′=K -1 P h , K -1 is the inverse of the camera intrinsic parameter matrix, P h is the homogeneous coordinate transformed from the point P(u,v) on the knife image, p h =(u,v,1).

6. An electronic device, characterized in that: include: one or more processors; A memory having one or more programs stored thereon, which, when the one or more programs are executed by the one or more processors, enables the one or more processors to implement the steps of the method according to any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that: The computer program is stored therein, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images