Water level identification and display system and method based on virtual imaging

Abstract

The invention relates to a water level identification and display system and method based on virtual imaging, and the method comprises the steps: obtaining the elevation of a calibration point in a visual field through an inclinometer and a range finder, and calculating the elevation corresponding to each pixel in the visual field; while monitoring is carried out, the water level is identified from the pixel level and converted into the actual water level. The calibrated elevation and water level information is directly fused into a video file, and the elevation and water level information carried in a video stream is identified while a client decodes a video. And through the D3D technology, frame-by-frame drawing and superposition are performed on the video, a virtual water gauge is displayed, the water level value is displayed in real time, the current water level is indicated by an arrow, and real-time monitoring and afterwards reviewing are facilitated. In addition, the system has rich expansibility, the warning water level can be set, so that the colored water gauge is displayed, warning water level alarming can be achieved through algorithm recognition, and meanwhile the water level changing process and the high water level warning degree can be obviously seen from videos.

Description

Technical Field

[0001] The invention belongs to the technical field of video monitoring, and in particular to a water level recognition and display system and method based on virtual imaging. Background Art

[0002] Traditional water level monitoring uses radar water level gauges or electronic water gauges. As AI technology becomes more and more mature, video algorithm monitoring of water levels has begun to be applied in reality. Deep learning trains water gauges and water surface models to obtain water level data. The water level monitoring environment is complex. It is difficult to install radar water level gauges and electronic water gauges or water gauges on some sites, and there is a high risk of working near the water surface. At the same time, there will be a large number of temporary monitoring points during the flood season. If the flood season has arrived, there will not be enough time and conditions to deploy and install water gauges. Summary of the invention

[0003] The purpose of the present invention is to overcome the shortcomings of the prior art, propose a water level recognition and display system and method based on virtual imaging, solve the problem of inconvenient installation of water gauge, and identify the water level with the help of rangefinder, inclinometer and image recognition technology, and superimpose a virtual water gauge with elevation on the video, so that the water level can be quickly identified and recorded. Compared with the traditional water gauge monitoring method, because the video directly carries the elevation information, the conversion step is omitted, which is more convenient for subsequent review and correction.

[0004] The present invention solves the technical problem by adopting the following technical solutions:

[0005] The water level recognition and display system based on image recognition and virtual water gauge includes an inclinometer, a rangefinder, a CPU, an image processor, a network module, a pan-tilt head and client software, wherein the inclinometer, the rangefinder / image processor and the pan-tilt head are respectively connected to the CPU, the CPU is connected to the network module, the network module is connected to the client software through the network, the image processor, the inclinometer and the rangefinder are installed on the spherical shell on the pan-tilt head, the image processor and the rangefinder are installed at the opening on the spherical shell for acquiring optical data, and the inclinometer is installed inside the spherical shell.

[0006] A method for identifying and displaying a water level based on image recognition and a virtual water gauge and a water level recognition and display system, comprising the following steps:

[0007] Step 1: Set up monitoring equipment and determine the equipment elevation; determine the monitoring scene, determine the size of the monitoring scene, and calibrate the monitoring scene elevation;

[0008] Step 2: Generate a virtual water gauge according to the elevation in step 1, and encode the virtual water gauge data into the video stream;

[0009] Step 3: Identify the water level line according to the generated virtual water gauge, calculate the water level value, and encode the water level line data into the video stream;

[0010] Step 4: Analyze user data and draw virtual water gauge and water level line.

[0011] Moreover, the specific implementation method of step 1 is: query geographic data or use a third-party altimeter to obtain the altitude of the ground at the monitoring location, add the distance of the device from the ground as the device elevation, and the monitoring scene is based on the river bank or wall plane near the water surface. At least two calibration points are set. The calibration points and the monitored target water surface position are on the same vertical axis in the video, and the monitoring scene elevation is calibrated by the device elevation.

[0012] Moreover, the specific implementation method of the monitoring scene elevation is:

[0013] H1=H-d1*sina1

[0014] H2=H-d2*sina2

[0015] Among them, the equipment elevation is H, the distance read from the rangefinder is d1 when calibration point 1, and the inclination angle read from the inclinometer is a1. When setting calibration point 2, the distance read from the rangefinder is d2, and the inclination angle read from the inclinometer is a2.

[0016] Set the pixel coordinates of the upper left corner of the video window to (0,0) and the lower right corner to (10000, 10000). According to the relative positions of the calibration points in the video, record the pixel heights pixH1 and pixH2 of calibration points 1 and 2 in the video.

[0017] Determine the elevation of any pixel point in the virtual water ruler based on calibration point 1 and calibration point 2:

[0018] Hx=H1+(pixHx–pixH1)*(H2–H1) / (pixH2–pixH1).

[0019] Moreover, the specific implementation method of step 2 is: according to the elevation of any position in the virtual water ruler obtained in step 1, combined with the determined scene size range, the elevation is rounded to generate the scale of the virtual water ruler, and the corresponding relationship between the pixel height and elevation of each scale is generated according to the elevation and the speed limit height, the generation of the virtual water ruler is completed, and the water ruler scale data is written into the user data of the video.

[0020] Moreover, the specific implementation method of step 3 is: during the monitoring process, the camera image processor collects the video, the intelligent analysis algorithm performs image recognition and analysis on the video data, uses the LBP feature method to identify the position of the water level line in the video, and compares it with the calibrated virtual water scale to obtain the current water level; at the same time, the elevation of the water level line and the relative coordinates of the video are also written into the user data of the video stream.

[0021] Moreover, the specific implementation method of step 4 is: when the client receives the monitored video stream, it decodes and displays the video while parsing the user data in the video stream; using D3D rendering technology, draw scale lines according to the relative position of the virtual water scale video, and display the elevation value of the scale on the right.

[0022] The advantages and positive effects of the present invention are:

[0023] The present invention obtains the elevation of the calibrated point in the field of view through an inclinometer and a rangefinder, and calculates the elevation corresponding to each pixel in the field of view. While monitoring, the water level is identified at the pixel level and converted into the actual water level. The calibrated elevation and water level information are directly integrated into the video file, and while the client decodes the video, the elevation and water level information carried in the video stream is identified. Through D3D technology, it is drawn frame by frame and superimposed on the video to display a virtual water gauge, display the water level value in real time, and indicate the current water level with an arrow, which is convenient for real-time monitoring and subsequent review. The present invention also has rich extensibility, and a warning water level can be set to display a colored water gauge. The warning water level alarm can be realized through algorithm recognition, and the change process of the water level and the warning degree of the high water level can be clearly seen from the video. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a system structure diagram of the present invention;

[0025] Figure 2 is a flow chart of the method of the present invention;

[0026] Figure 3 This is a schematic diagram of the monitoring scenario of the present invention;

[0027] Figure 4 Generate a virtual water gauge flow chart for the present invention;

[0028] Figure 5 A schematic diagram of a virtual water gauge generated by the present invention;

[0029] Figure 6 It is a schematic diagram of the installation of the system of the present invention;

[0030] Figure 7 It is a connection diagram of the system of the present invention;

[0031] Figure 8 Schematic diagram of a video frame in a video stream of the present invention. DETAILED DESCRIPTION

[0032] The present invention is further described in detail below with reference to the accompanying drawings.

[0033] Water level recognition and display system based on image recognition and virtual water gauge, such as Figure 1As shown, it includes an inclinometer, a rangefinder, a CPU, an image processor, a network module, a pan / tilt and client software, wherein the inclinometer, the rangefinder / image processor and the pan / tilt are connected to the CPU respectively, the CPU is connected to the network module, the network module is connected to the client software via the network, and the installation positions of the inclinometer, the rangefinder and the image processor are as shown in FIG. Figure 6 Figure 7 As shown, the image processor and the rangefinder need to open holes on the spherical shell to obtain optical data, but the inclinometer does not need to be installed inside the spherical shell. The three move synchronously when the gimbal moves.

[0034] A water level recognition and display method based on image recognition and virtual water gauge. Figure 2 As shown, the following steps are included:

[0035] Step 1: Set up monitoring equipment and determine the equipment elevation; determine the monitoring scene, determine the size of the monitoring scene, and calibrate the monitoring scene elevation.

[0036] The specific implementation method of step 1 is: the monitoring process is completed only at one side of the river bank, and there is no need to install a water gauge on the other side or on the water surface.

[0037] Query geographic data or use a third-party altimeter to obtain the ground elevation of the monitoring location, and add the distance of the device from the ground to obtain the device elevation. For example, Figure 3 As shown, the monitoring scene is based on the river bank or wall plane near the water surface, and at least two calibration points are set. The calibration points and the monitored target water surface position are on the same vertical axis in the video (i.e., the virtual water scale position, and the field of view can only be moved up and down during this process), and the monitoring scene elevation is calibrated by the equipment elevation.

[0038] The specific implementation method of monitoring scene elevation is:

[0039] H1=H-d1*sina1

[0040] H2=H-d2*sina2

[0041] Among them, the equipment elevation is H, the distance read from the rangefinder is d1 when calibration point 1, and the inclination angle read from the inclinometer is a1. When setting calibration point 2, the distance read from the rangefinder is d2, and the inclination angle read from the inclinometer is a2.

[0042] Set the pixel coordinates of the upper left corner of the video window to (0,0) and the lower right corner to (10000, 10000). According to the relative positions of the calibration points in the video, record the pixel heights pixH1 and pixH2 of calibration points 1 and 2 in the video.

[0043] Determine the elevation of any pixel point in the virtual water ruler based on calibration point 1 and calibration point 2:

[0044] Hx=H1+(pixHx–pixH1)*(H2–H1) / (pixH2–pixH1).

[0045] Step 2: Generate a virtual water gauge based on the elevation in step 1, and encode the virtual water gauge data into the video stream.

[0046] The specific implementation method of step 2 is: according to the elevation of any position in the virtual water ruler obtained in step 1, combined with the determined scene size range, the elevation is rounded to generate the scale of the virtual water ruler, and the corresponding relationship between the pixel height and elevation of each scale is generated according to the elevation and speed limit height to complete the generation of the virtual water ruler.

[0047] Assume that the elevation of calibration point 1 is 1.2m, the pixel height is 7000, and the elevation of calibration point 2 is 1.8m, the pixel height is 4000. Then select 0.1m as a scale, and get the corresponding relationship between pixel height and elevation of each scale:

[0048]

[0049]

[0050] Each scale consists of two values, elevation and video relative height. The virtual water gauge consists of a set of scales stored in memory.

[0051]

[0052] like Figure 4 As shown in the figure, during the video acquisition and encoding process, the water scale data is written into the user data of the video. Taking H264 encoding as an example, the video frame data in the video stream is as follows Figure 8 As shown. In the binary data, 00000001 is the delimiter between frames, and the following byte identifies the type. 0000000106 is specifically used to write custom data. The binary data of the scale information above is written into this section. When decoding the video, the water scale data can be obtained by reading in the same format.

[0053] Step 3: Identify the water level line according to the generated virtual water gauge, calculate the water level value, and encode the water level line data into the video stream.

[0054] The specific implementation method of step 3 is as follows: during the monitoring process, the camera image processor collects video, the intelligent analysis algorithm performs image recognition and analysis on the video data, uses the LBP feature method to identify the position of the water level line in the video, and compares it with the calibrated virtual water scale to obtain the current water level; at the same time, the elevation of the water level line and the relative coordinates of the video are also written into the user data of the video stream.

[0055] Step 4: Analyze user data and draw virtual water gauge and water level line.

[0056] The specific implementation method of step 4 is: when the client receives the monitored video stream, it decodes and displays the video while parsing the user data in the video stream; using D3D rendering technology (Direct3D (abbreviated as: D3D) is a set of 3D graphics programming interfaces developed on the Windows operating system. It is part of DirectX and is currently widely supported by various display cards. DirectX (Direct eXtension, abbreviated as DX) is a multimedia programming interface on the Windows operating system. It is an application programming interface (API)), draws scale lines according to the relative position of the virtual water scale video, and displays the elevation value of the scale on the right.

[0057] At the same time, the present invention innovatively adds background shadows to avoid interference with scale reading by human eyes caused by changes in the video image itself.

[0058] After the water level line is parsed, the coordinates of the two endpoints of the water level line in the video are (x1, y1) -> (x2, y2). According to the relative position of the water level line in the video, the horizontal coordinate of the virtual water ruler (i.e. the horizontal coordinate x of calibration point 1 and calibration point 2) is obtained in step 2. The intersection with the virtual water ruler is calculated. Then the vertical coordinate of the intersection of the water level line and the virtual water ruler can be calculated as:

[0059] y=y1+(x-x1)*(y2-y1) / (x2-x1).

[0060] Then, according to the formula in step 1, the elevation of the intersection is calculated as:

[0061] H=H1+(y–pixH1)*(H2–H1) / (pixH2–pixH1).

[0062] For better reading, the line is no longer displayed, but an arrow is used instead. The intersection of the water level line and the virtual water gauge is used as the vertex of the arrow, and an arrow with a black frame and a white background is drawn. The arrow's indication accuracy can reach the single pixel level.

[0063] like Figure 5 As shown, for better reading, the line is no longer displayed, but an indicating arrow is used instead. The intersection of the water level line and the virtual water gauge is used as the vertex of the arrow, and an arrow with a black frame and a white background is drawn. The indication accuracy of the arrow can reach the single pixel level.

[0064] It should be emphasized that the embodiments described in the present invention are illustrative rather than restrictive. Therefore, the present invention includes but is not limited to the embodiments described in the specific implementation manner. Any other implementation manners derived by those skilled in the art based on the technical solution of the present invention also fall within the scope of protection of the present invention.

Claims

1. A water level recognition and display system based on image recognition and virtual water gauge, characterized by: It includes an inclinometer, a rangefinder, a CPU, an image processor, a network module, a pan-tilt and client software, wherein the inclinometer, the rangefinder / image processor and the pan-tilt are respectively connected to the CPU, the CPU is connected to the network module, the network module is connected to the client software through the network, the image processor, the inclinometer and the rangefinder are installed on the spherical shell on the pan-tilt, the image processor and the rangefinder are installed at the opening on the spherical shell for acquiring optical data, and the inclinometer is installed inside the spherical shell.

2. A method for identifying and displaying a water level based on image recognition and a virtual water gauge according to claim 1, characterized in that: The following steps are involved: Step 1: Set up monitoring equipment and determine the equipment elevation; determine the monitoring scene, determine the size of the monitoring scene, and calibrate the monitoring scene elevation; Step 2: Generate a virtual water gauge according to the elevation in step 1, and encode the virtual water gauge data into the video stream; Step 3: Identify the water level line according to the generated virtual water gauge, calculate the water level value, and encode the water level line data into the video stream; Step 4: Analyze user data and draw virtual water gauge and water level line.

3. The identification and display method of the water level identification and display system based on image recognition and virtual water gauge according to claim 2, characterized in that: The specific implementation method of step 1 is: query geographic data or use a third-party altimeter to obtain the ground elevation of the monitoring location, add the distance of the device from the ground as the device elevation, and set at least two calibration points based on the river bank or wall plane near the water surface. The calibration points and the monitored target water surface position are on the same vertical axis in the video, and the monitoring scene elevation is calibrated by the device elevation.

4. The identification and display method of the water level identification and display system based on image recognition and virtual water gauge according to claim 3, characterized in that: The specific implementation method of the monitoring scene elevation is: H1=H-d1*sina1 H2=H-d2*sina2 Among them, the equipment elevation is H, the distance read from the rangefinder is d1 when calibration point 1, and the inclination angle read from the inclinometer is a1. When setting calibration point 2, the distance read from the rangefinder is d2, and the inclination angle read from the inclinometer is a2. Set the pixel coordinates of the upper left corner of the video window to (0,0) and the lower right corner to (10000, 10000). According to the relative positions of the calibration points in the video, record the pixel heights pixH1 and pixH2 of calibration points 1 and 2 in the video. Determine the elevation of any pixel point in the virtual water ruler based on calibration point 1 and calibration point 2: Hx=H1+(pixHx–pixH1)*(H2–H1) / (pixH2–pixH1).

5. The identification and display method of the water level identification and display system based on image recognition and virtual water gauge according to claim 2, characterized in that: The specific implementation method of step 2 is: according to the elevation of any position in the virtual water ruler obtained in step 1, combined with the determined scene size range, the elevation is rounded to generate the scale of the virtual water ruler, and the corresponding relationship between the pixel height and elevation of each scale is generated according to the elevation and the speed limit height, the generation of the virtual water ruler is completed, and the water ruler scale data is written into the user data of the video.

6. The identification and display method of the water level identification and display system based on image recognition and virtual water gauge according to claim 2, characterized in that: The specific implementation method of step 3 is as follows: during the monitoring process, the camera image processor collects video, the intelligent analysis algorithm performs image recognition and analysis on the video data, uses the LBP feature method to identify the position of the water level line in the video, and compares it with the calibrated virtual water scale to obtain the current water level; at the same time, the elevation of the water level line and the relative coordinates of the video are also written into the user data of the video stream.

7. The identification and display method of the water level identification and display system based on image recognition and virtual water gauge according to claim 2, characterized in that: The specific implementation method of step 4 is: when the client receives the monitored video stream, it decodes and displays the video while parsing the user data in the video stream; using D3D rendering technology, drawing scale lines according to the relative position of the virtual water scale video, and displaying the elevation value of the scale on the right.

Images