A gate opening recognition method based on key point detection

Abstract

The invention discloses a gate opening recognition method based on key point detection, comprising the following steps: step one, model establishment; step two, model training; step three, video access; step four, pointer position detection; step five, scale point calibration; step six, intersection determination; step seven, pixel length calculation; step eight, actual length calculation; step nine, gate opening calculation; step ten, result output; the invention adopts a key point detection algorithm to automatically identify the position of the gate opening scale pointer, calculates the gate opening at the intersection of the gate pointer and the calibration line segment according to the pre-calibrated scale line, and can calculate the actual gate opening value, which is beneficial to improving the efficiency of gate opening measurement and has intuitive and visible measurement capabilities. In addition, the invention does not adopt sensor measurement, which is beneficial to reducing measurement errors. At the same time, the measuring equipment is all located on the water surface, avoiding the phenomenon of water damage.

Description

Technical Field

[0001] The invention relates to the technical field of water discharge gate opening measurement, and in particular to a gate opening identification method based on key point detection. Background Art

[0002] The existing arc gate opening measurement technology mostly uses sensors to measure the gate's rotation stroke and the gate's rotation angle, and then calculates the gate's opening through a specified conversion relationship. However, the actual gate opening is indirectly measured by installing specific sensors, and the gate opening cannot be directly calculated. Due to the inherent errors of the sensors and the deviations caused by long-term use, the calculated gate opening data is not completely accurate. At the same time, the sensor device is immersed in water for a long time and is prone to damage, causing certain losses. In addition, since the gate opening is measured by sensors, certain changes need to be made to the gate itself. On the one hand, the installation and deployment are more complicated, and on the other hand, it will also cause potential risks to the gate. Summary of the invention

[0003] The purpose of the present invention is to provide a gate opening recognition method based on key point detection to solve the problems raised in the above background technology.

[0004] In order to solve the above technical problems, the present invention provides the following technical solutions: a gate opening recognition method based on key point detection, comprising the following steps: step one, model establishment; step two, model training; step three, video access; step four, pointer position detection; step five, scale point calibration; step six, intersection determination; step seven, pixel length calculation; step eight, actual length calculation; step nine, gate opening calculation; step ten, result output;

[0005] In the above step 1, a gate opening key point detection model based on deep learning technology is used, and the key point detection model adopts a key point detection model CPN based on deep learning to annotate the gate opening scale key point sample data required for model training;

[0006] In the above step 2, the gate opening scale key point detection model in step 1 is used, and the sample data marked in step 1 is used to train the key point detection model by performing multiple iterations using the sample data;

[0007] In the above step 3, the real-time gate monitoring video is connected, and three preset positions are set to cover the range of the gate scale. The camera shoots the three preset positions respectively, and the image data of three different preset positions are intercepted for the next step of model prediction;

[0008] In the above step 4, the key point detection model trained in step 2 is used to predict the three gate opening scale images obtained in step 3. The key point detection model will detect the gate opening scale pointer position coordinates in one of the image data;

[0009] In the above step 5, according to the three preset position images of the camera obtained in step 3, the gate scale is not a straight line but a curved scale. Several groups of scale points are marked according to the curvature of the gate. Each scale point corresponds to an actual scale value. The scale segment connected by the scale points can approximate the curved scale. The line segment composed of the scale points is used to calculate the scale of the gate pointer in the next step.

[0010] In the above step 6, the gate opening scale pointer line segment in step 4 and the scale segment calibrated in step 5 are used to solve the coordinates of the intersection of the two segments, and the intersection coordinates are used for the next step of calculation;

[0011] In the above step 7, according to the coordinate point solved in step 6, the pixel length of the intersection point on the scale segment is calculated using the approximate triangle formula, and the solved pixel length is used for the subsequent calculation of the actual gate opening value of the intersection point;

[0012] In the above step eight, according to the actual scale values ​​corresponding to the several scale points calibrated in step five, the pixel length and the actual length of the scale segment can be obtained, and according to the pixel length and the actual length, the actual length l of the unit pixel corresponding to the scale segment can be solved;

[0013] In the above step nine, the pixel length of the intersection point on the scale segment obtained in step seven is multiplied by the actual length l of the unit pixel, and the actual length L corresponding to the pixel length of the intersection point on the scale segment can be obtained. The actual scale value corresponding to the intersection point obtained according to the actual length L is the actual gate opening value;

[0014] In the above step ten, the preset position of the key point predicted in step four, the key point calculated by the model in step four, the calibration point in step five and the solved intersection point are drawn on the image, the vertical coordinate of the pointer tip calculated by the model is taken, and a horizontal line is drawn in the image to indicate the position of the end of the gate pointer at this time, and the actual gate opening is drawn in the image, and the predicted gate opening analysis results are output.

[0015] According to the above technical solution, in step three, the camera takes a picture of each of the three preset positions every five minutes.

[0016] According to the above technical solution, in step three, the camera is a spherical camera.

[0017] According to the above technical solution, in step four, the gate opening scale pointer position coordinates are composed of several points, distributed at the end and corners of the gate opening scale pointer, and the straight line formed by the tail of the pointer is used for subsequent calculation of the scale pointed to by the gate pointer.

[0018] According to the above technical solution, in step five, in an actual scenario, it is necessary to further determine the number of calibration points according to the degree of curvature of the ruler. The more points there are, the more approximate the curved ruler can be.

[0019] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: the present invention adopts a key point detection algorithm to automatically identify the position of the gate opening scale pointer, and calculates the gate opening at the intersection of the gate pointer and the calibration line segment according to the pre-calibrated scale line, so as to calculate the actual gate opening value, and then outputs the measured gate opening and image. The method based on video image analysis is adopted, which is beneficial to greatly improve the efficiency of gate opening measurement and has intuitive and visible measurement capabilities. In addition, the present invention does not adopt sensor measurement, which is beneficial to reduce measurement errors. At the same time, the measuring equipment of the present invention is located on the water surface, avoiding the phenomenon of water damage. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0021] Figure 1 is a flow chart of the method of the present invention;

[0022] Figure 2 It is a simplified schematic diagram of the gate opening scale of the present invention;

[0023] Figure 3 It is the actual prediction diagram of the gate opening of the present invention. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] See also Figure 1-3The present invention provides a technical solution: a gate opening recognition method based on key point detection, comprising the following steps: step one, model establishment; step two, model training; step three, video access; step four, pointer position detection; step five, scale point calibration; step six, intersection determination; step seven, pixel length calculation; step eight, actual length calculation; step nine, gate opening calculation; step ten, result output;

[0026] In the above step 1, a gate opening key point detection model based on deep learning technology is used, and the key point detection model adopts a key point detection model CPN based on deep learning to annotate the gate opening scale key point sample data required for model training;

[0027] In the above step 2, the gate opening scale key point detection model in step 1 is used, and the sample data labeled in step 1 is used to train the key point detection model. The Pytorch framework is used to implement model training. The key indicator parameters of the model training are set as follows: learning rate 0.001, momentum parameter 0.9, weight decay parameter 0.0001, and the model is iterated for 12 data cycles;

[0028] In the above step 3, the real-time gate monitoring video is connected, and three preset positions are set to cover the range of the gate scale. The camera polls the three preset positions every five minutes. The camera is a spherical camera, and the image data of three different preset positions is intercepted for the next model prediction;

[0029] In the above step 4, the key point detection model trained in step 2 is used to predict the three gate opening scale images obtained in step 3. The key point detection model will detect the gate opening scale pointer position coordinates in one of the image data. The gate opening scale pointer position coordinates are composed of 4 points, distributed at the two ends and two corners of the gate opening scale pointer. Figure 2 A1-A4 correspond to the above four coordinate points respectively. The coordinates of A1-A4 constitute the key point information of the gate scale pointer. The line segment formed by the two coordinate points A3 and A4 is used for the subsequent calculation of the scale pointed by the gate pointer.

[0030] In the above step 5, according to the three preset position images of the camera obtained in step 3, such as Figure 3 is one of the preset position image data. Figure 3 The scale of the middle gate is not a straight line but a curved scale. Therefore, in order to simplify the calculation complexity, Figure 3Several groups of scale points B1-B3 are marked in the figure. Each coordinate point corresponds to an actual scale value. The line segment connecting the points B1-B3 can approximate the curved ruler. The line segment composed of the marked points is used to calculate the scale of the gate pointer in the next step. In actual scenarios, the number of calibration points needs to be further determined according to the curvature of the ruler. The more points there are, the more approximate the curved ruler can be.

[0031] In the above step 6, the gate opening scale pointer line segment A3-A4 line segment in step 4 and the B2-B3 line segment calibrated in step 5 are used to solve the coordinates of the intersection point C1 of the two line segments, and the C1 coordinate point is used to calculate the pixel length of the C1-B3 line segment in the next step;

[0032] In the above step 7, according to the coordinate point C1 solved in step 6, the triangle formed by B2B3O1 and the triangle formed by C1B3O2 are similar triangles. The actual pixel length of the line segment C1B3 can be solved by the similar triangle formula. The solved pixel length of the C1B3 line segment is used to subsequently calculate the actual gate opening value of point C1. The calculation formula is as follows:

[0033]

[0034] In the above step eight, according to the actual scale value corresponding to the B1-B3 point calibrated in step five, the pixel length and actual length of the B2B3 line segment can be obtained. According to the pixel length and actual length, the actual length l of the unit pixel corresponding to the B2B3 line segment is solved. The formula is as follows:

[0035]

[0036] In the above step nine, the pixel length of the C1B3 line segment obtained in step seven is multiplied by the actual unit pixel length l to obtain the actual length L corresponding to the C1B3 line segment. The actual gate opening value is obtained by adding the actual scale value marked at point B3 to the length L.

[0037] In the above step ten, the preset position of the key point predicted in step four, the key point calculated by the model in step four, the calibration point in step five and the solved intersection point are drawn on the image, the vertical coordinate of the key point A4 calculated by the model is taken, and a horizontal line is drawn in the image to indicate the position of the end of the gate pointer at this time, and the actual gate opening is drawn in the image, and the predicted gate opening analysis results are output.

[0038] Based on the above, the advantage of the present invention is that the present invention adopts a key point detection algorithm to automatically identify the position of the gate opening scale pointer, and calculates the gate opening at the intersection of the gate pointer and the calibration line segment according to the pre-calibrated scale line, so as to calculate the actual opening value of the gate, and then output the measured gate opening and image. The method based on video image analysis of the present invention replaces the traditional sensor measurement method, which is conducive to improving the accuracy of measurement and greatly improves the efficiency of measurement. In addition, the device of the present invention is located above the water surface, which avoids the occurrence of equipment damage caused by equipment immersion in water, and is conducive to improving the safety of use of the present invention.

[0039] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0040] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A gate opening recognition method based on key point detection, comprising the following steps: Step 1, model establishment; Step 2, model training; Step 3, video access; Step 4, pointer position detection; Step 5, scale point calibration; Step 6, intersection determination; Step 7, pixel length calculation; Step 8, actual length calculation; Step 9, gate opening calculation; Step 10, result output; It is characterized by: In the above step 1, a gate opening key point detection model based on deep learning technology is used, and the key point detection model adopts a key point detection model based on deep learning to annotate the gate opening scale key point sample data required for model training; In the above step 2, the gate opening scale key point detection model in step 1 is used, and the sample data marked in step 1 is used to train the key point detection model by performing multiple iterations using the sample data; In the above step 3, the real-time gate monitoring video is connected, and three preset positions are set to cover the range of the gate scale. The camera shoots the three preset positions respectively, and the image data of three different preset positions are intercepted for the next step of model prediction; In the above step 4, the key point detection model trained in step 2 is used to predict the three gate opening scale images obtained in step 3. The key point detection model will detect the gate opening scale pointer position coordinates in one of the image data; In the above step 5, according to the three preset position images of the camera obtained in step 3, the gate scale is not a straight line but a curved scale. Several groups of scale points are marked according to the curvature of the gate. Each scale point corresponds to an actual scale value. The scale segment connected by the scale points approximates the curved scale. The line segment composed of the scale points is used to calculate the scale of the gate pointer in the next step. In the above step 6, the gate opening scale pointer line segment in step 4 and the scale segment calibrated in step 5 are used to solve the coordinates of the intersection of the two segments, and the intersection coordinates are used for the next step of calculation; In the above step 7, according to the coordinate point solved in step 6, the pixel length of the intersection point on the scale segment is calculated using the approximate triangle formula, and the solved pixel length is used for the subsequent calculation of the actual gate opening value of the intersection point; In the above step eight, according to the actual scale values ​​corresponding to the several scale points calibrated in step five, the pixel length and the actual length of the scale segment can be obtained, and according to the pixel length and the actual length, the actual length l of the unit pixel corresponding to the scale segment can be solved; In the above step nine, the pixel length of the intersection point on the scale segment obtained in step seven is multiplied by the actual length l of the unit pixel, and the actual length L corresponding to the pixel length of the intersection point on the scale segment can be obtained. The actual scale value corresponding to the intersection point obtained according to the actual length L is the actual gate opening value; In the above step ten, the preset position of the key point predicted in step four, the key point calculated by the model in step four, the calibration point in step five and the solved intersection point are drawn on the image, the vertical coordinate of the pointer tip calculated by the model is taken, and a horizontal line is drawn in the image to indicate the position of the end of the gate pointer at this time, and the actual gate opening is drawn in the image, and the predicted gate opening analysis results are output.

2. The gate opening recognition method based on key point detection according to claim 1 is characterized in that: In the step 3, the camera takes a picture of each of the three preset positions every five minutes.

3. The gate opening recognition method based on key point detection according to claim 1 is characterized in that: In step three, the camera is a spherical camera.

4. The gate opening recognition method based on key point detection according to claim 1 is characterized in that: In step 4, the gate opening scale pointer position coordinates are composed of a number of points distributed at the end and corners of the gate opening scale pointer, wherein the straight line formed by the tail of the pointer is used for subsequent calculation of the scale pointed to by the gate pointer.

5. The gate opening recognition method based on key point detection according to claim 1 is characterized in that: In the step 5, in an actual scenario, it is necessary to further determine the number of calibration points according to the degree of curvature of the ruler. The more points there are, the more approximate the curved ruler can be.

Images