Camera-based channel surface flow state digital twinning system and camera-based channel surface flow state digital twinning method

Abstract

The invention provides a camera-based channel surface flow state digital twinning system and method, and the system comprises the steps: integrating a high-precision spherical camera, a real-time data processing module, a local memory module, a power module, a protection structure, a network server module, a remote control terminal and other modules, and building the channel surface flow state digital twinning system; the method comprises the steps of processing and analyzing image data by using an image processing algorithm and an LSPIV algorithm model, calculating flow velocity data and a signal-to-noise ratio of the data, drawing a two-dimensional flow field vector diagram and the like, so that real-time monitoring and dynamic visual display of a channel surface flow field are realized. The method aims to overcome the defects that a traditional measurement method is poor in real-time performance, complex in data processing, not suitable for large-range water area monitoring and the like, and technical support is provided for guaranteeing safe and efficient operation of a channel.

Description

Technical Field

[0001] The present invention belongs to the field of smart waterways, and specifically relates to a camera-based digital twin system and method for waterway surface flow. Background Art

[0002] The digital twin of the channel surface flow pattern refers to the use of digital technology to create a virtual model of the channel surface flow field. This model can reflect and simulate the actual situation of the channel surface flow pattern in real time.

[0003] With the development of modern shipping industry, the number and scale of ships are increasing, which puts higher requirements on shipping safety and waterway management. A key factor in ensuring shipping safety and effective waterway management is to accurately identify the surface flow state of the waterway. The surface flow state of the waterway directly affects the navigation track, speed and safety of the ship. Therefore, real-time monitoring and analysis of the surface flow state is crucial to preventing shipping accidents and optimizing waterway management.

[0004] Traditional methods for measuring water velocity mainly rely on point measurement equipment, such as current meters, buoys, etc. These methods usually require manual operation, have a limited measurement range, and are difficult to measure accurately in complex and harsh environments. In addition, traditional measurement methods have poor real-time performance and complex data processing, and are not suitable for real-time monitoring of large areas of water. In recent years, PIV technology has gradually become an important means of flow field measurement due to its non-contact and high-precision advantages. However, the existing PIV system requires the release of tracer particles, and the measured flow field is only a few square centimeters, which is difficult to meet larger-scale flow field measurements. Summary of the invention

[0005] In order to address the shortcomings of existing observation methods, the present invention proposes a camera-based digital twin system and method for channel surface flow states to meet the needs of large-scale surface flow field measurement in target waters, and can obtain flow field data in real time, accurately and conveniently.

[0006] In order to realize the above technical features, the object of the present invention is achieved as follows: a digital twin system of waterway surface flow state based on a camera, comprising: Image acquisition module, used to obtain image data of the channel surface in real time and in a wide range; The real-time data processing module includes a local server, which is used to identify and analyze the acquired image data, process the image data using an image processing algorithm, analyze the image data using an LSPIV algorithm model, calculate the flow velocity data and the signal-to-noise ratio of the data, and draw a two-dimensional flow field vector diagram; The power module and protective structure are used to supply power to all modules and have waterproof, dustproof and shockproof properties to ensure stable operation in various harsh environments; The network server module is a data transfer and processing center responsible for managing the data transmission between the image acquisition module, the local server and the remote control terminal module; The remote control terminal module is used for remote access and control of the system. It can view and analyze the flow information on the channel surface in real time through the network server module, provide control and operation instructions, and retrieve historical data to achieve remote monitoring and data playback of the water area.

[0007] Preferably, the image acquisition module adopts a high-precision spherical camera.

[0008] Preferably, when the high-precision spherical camera collects image data of the target water area, at least 6 ground control points should be set on site, and the position information of the ground control points should be recorded in the world coordinate system ( X , Y , Z ), in meters; then, a high-precision spherical camera is used to obtain the position of the ground control point in the image, and the displacement information of the ground control point in the image coordinate system is obtained ( x , y , z ), in pixels.

[0009] A digital twin method for flow state of a waterway surface based on a camera, the method being implemented by the digital twin system for flow state of a waterway surface, comprising the following steps: Step 1, digital image processing: The image data acquired by the image acquisition module is subjected to frame-by-frame extraction, frame image orthophoto correction, grayscale processing, distortion correction, gamma correction and image binarization processing using an image processing algorithm; Step 2, flow rate data calculation: Obtain two consecutive frames of images processed by step 1, set the size of the grid unit in the frame image, match the image feature points, calculate the relative displacement of the feature points using the cross-correlation function, and calculate the water flow velocity using the displacement velocity formula; Step 3, flow field diagram drawing: The signal-to-noise ratio of the cross-correlation function of the flow velocity of each grid unit is calculated, the signal-to-noise ratio of each grid unit is filtered, and a two-dimensional flow field vector diagram is drawn.

[0010] Preferably, in step 1, processing the image data using an image processing algorithm comprises the following steps: 1.1, extract the image data obtained by the image acquisition module frame by frame, and record the time interval Δ between two frames of images t ; 1.2, according to the formula , , the position information of the ground control points in the two coordinate systems (X , Y , Z ) is brought into the conversion formula to determine 11 projection coordinate coefficients, so as to perform orthographic correction on each frame of the extracted image, that is, to correct the obliquely shot image into an overhead shot image and unify the physical resolution of the image; in, A 1. A 2. A 3. A 4. B 1. B 2. B 3. B 4. C 1. C 2. C 3 is the projection coefficient; ( x , y , z ) is the displacement information of the ground control point in the image coordinate system; 1.3, grayscale processing, distortion correction and gamma correction are performed on the corrected image, and finally the frame image is processed into a binary image containing only black and white.

[0011] Preferably, the step 2 of calculating the flow rate data using the LSPIV algorithm model comprises the following steps: 2.1, obtain two consecutive frames of images from the processed image data in real time, and set the size of each grid unit in the image; 2.2, use the cross-correlation algorithm to calculate the cross-correlation function between two image blocks, match the feature points of the images, and use the formula , calculate the cross-correlation function R ( x , y ), through the cross-correlation function R ( x , y ) and calculate the displacement Δ between the feature points in the image block x , Δ y , and then through the formula , Calculate flow rate u , v ; in, I 1( i , j ) is the image block I 1 in position ( i , j ), I 2( i + x , j +y ) is the image block I 2 in position ( i + x , j + y ), Δ x and Δ y Respectively in x and y Displacement in the direction, Δ t is the time interval between two image frames.

[0012] Preferably, the step 3 of drawing the flow field diagram using the LSPIV algorithm model comprises the following steps: 3.1, through the formula Calculate the signal-to-noise ratio of the cross-correlation function of the flow velocity for each grid cell SNR ; in, P peak is the highest peak value of the cross-correlation function, P secondary peak This is the second highest peak value; 3.2, using formula The signal-to-noise ratio of each grid cell is filtered, and the missing values ​​are filled using the local mean interpolation method for results below the threshold; in, threshold It refers to the threshold for filtering the signal-to-noise ratio results; 3.3. Based on the flow velocity data, call the Matplotlib library function to draw a two-dimensional flow field vector diagram.

[0013] The present invention has the following beneficial effects: 1. The present invention adopts high-precision spherical camera and large-scale particle image velocimetry (LSPIV) technology to measure the flow field on the water surface in a non-invasive manner, avoiding interference and influence on the water flow environment. In addition, the all-round coverage of the high-precision spherical camera can monitor the water area covering hundreds of meters in length and width in real time, meet the needs of large-scale flow field measurement, and realize wide-area monitoring; 2. High-precision spherical cameras and real-time data processing modules are installed near the waterway to collect image data on site and process flow data in real time, which greatly reduces the time of data transmission, has high timeliness, and realizes real-time monitoring and analysis; 3. Traditional equipment is difficult to measure accurately in harsh environments, but the system of the present invention has the characteristics of waterproof, dustproof, shockproof, etc., and can operate stably in various complex and harsh environmental conditions to ensure the reliability of data; 4. Through the network server module and remote control terminal module, users can remotely access and control the system, view and analyze flow information in real time, make timely adjustments and decisions, and significantly improve management and response capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0015] Figure 1 It is a flow chart of each module of the system of the present invention.

[0016] Figure 2 Set up high-definition cameras for the scene.

[0017] Figure 3 The calculated flow rate data.

[0018] Figure 4 is the processed frame image and the corresponding flow field map. DETAILED DESCRIPTION

[0019] The embodiments of the present invention are further described below in conjunction with the accompanying drawings.

[0020] Embodiment 1: See also Figure 1 This embodiment provides a camera-based digital twin system of waterway surface flow, including: Image acquisition module, used to obtain image data of the channel surface in real time and in a wide range; The real-time data processing module includes a local server, which is used to identify and analyze the acquired image data, process the image data using an image processing algorithm, analyze the image data using an LSPIV algorithm model, calculate the flow velocity data and the signal-to-noise ratio of the data, and draw a two-dimensional flow field vector diagram; The power module and protective structure are used to supply power to all modules and have waterproof, dustproof and shockproof properties to ensure stable operation in various harsh environments; The network server module is a data transfer and processing center responsible for managing the data transmission between the image acquisition module, the local server and the remote control terminal module; The remote control terminal module is used for remote access and control of the system. It can view and analyze the flow information on the channel surface in real time through the network server module, provide control and operation instructions, and retrieve historical data to achieve remote monitoring and data playback of the water area.

[0021] Furthermore, the image acquisition module adopts a high-precision spherical camera.

[0022] Furthermore, when the high-precision spherical camera collects image data of the target water area, at least 6 ground control points should be set on site, and the position information of the ground control points should be recorded in the world coordinate system ( X , Y , Z ), in meters; then, a high-precision spherical camera is used to obtain the position of the ground control point in the image, and the displacement information of the ground control point in the image coordinate system is obtained ( x , y , z ), in pixels.

[0023] Embodiment 2: This embodiment provides a camera-based digital twin method for channel surface flow state, comprising the following steps: Step 1, digital image processing: The image data acquired by the image acquisition module is subjected to frame-by-frame extraction, frame image orthophoto correction, grayscale processing, distortion correction, gamma correction and image binarization processing using an image processing algorithm; Step 2, flow rate data calculation: Obtain two consecutive frames of images processed by step 1, set the size of the grid unit in the frame image, match the image feature points, calculate the relative displacement of the feature points using the cross-correlation function, and calculate the water flow velocity using the displacement velocity formula; Step 3, flow field diagram drawing: The signal-to-noise ratio of the cross-correlation function of the flow velocity of each grid unit is calculated, the signal-to-noise ratio of each grid unit is filtered, and a two-dimensional flow field vector diagram is drawn.

[0024] Furthermore, in step 1, processing the image data using an image processing algorithm comprises the following steps: 1.1, extract the image data obtained by the image acquisition module frame by frame, and record the time interval Δ between two frames of images t ; 1.2, according to the formula , , the position information of the ground control points in the two coordinate systems ( X , Y , Z ) is brought into the conversion formula to determine 11 projection coordinate coefficients, so as to perform orthographic correction on each frame of the extracted image, that is, to correct the obliquely shot image into an overhead shot image and unify the physical resolution of the image; in, A 1. A 2. A 3. A 4. B 1. B 2.B 3. B 4. C 1. C 2. C 3 is the projection coefficient; ( x , y , z ) is the displacement information of the ground control point in the image coordinate system; 1.3, grayscale processing, distortion correction and gamma correction are performed on the corrected image, and finally the frame image is processed into a binary image containing only black and white.

[0025] Furthermore, the step 2 of calculating the flow rate data using the LSPIV algorithm model comprises the following steps: 2.1, obtain two consecutive frames of images from the processed image data in real time, and set the size of each grid unit in the image; 2.2, use the cross-correlation algorithm to calculate the cross-correlation function between two image blocks, match the feature points of the images, and use the formula , calculate the cross-correlation function R ( x , y ), through the cross-correlation function R ( x , y ) and calculate the displacement Δ between the feature points in the image block x , Δ y , and then through the formula , Calculate flow rate u , v ; in, I 1( i , j ) is the image block I 1 in position ( i , j ), I 2( i + x , j + y ) is the image block I 2 in position ( i + x , j + y ), Δ x and Δ y Respectively in x and y Displacement in the direction, Δ t is the time interval between two image frames.

[0026] Furthermore, the step 3 of drawing the flow field diagram using the LSPIV algorithm model includes the following steps: 3.1, through the formula Calculate the signal-to-noise ratio of the cross-correlation function of the flow velocity for each grid cell SNR ; in, P peak is the highest peak value of the cross-correlation function, P secondary peak This is the second highest peak value; 3.2, using formula The signal-to-noise ratio of each grid cell is filtered, and the missing values ​​are filled using the local mean interpolation method for results below the threshold; in, threshold It refers to the threshold for filtering the signal-to-noise ratio results; 3.3. Based on the flow velocity data, call the Matplotlib library function to draw a two-dimensional flow field vector diagram.

[0027] Embodiment 3: Reference Figures 1 to 4 The present invention provides a real-time twin system and method for channel surface flow based on a camera. The system working steps are as follows: Step 1: Figure 2 As shown in the figure, a high-precision spherical camera is set up on the shore of the target waters of the channel and the angle and height of the camera are adjusted on site so that the camera can clearly observe the situation of the target waters. At the same time, the control points of the target waters are calibrated and the position information of the control points in the world coordinate system is recorded ( X , Y , Z ), and transmit the captured image data to the local server in the real-time data processing module; Step 2: (1) In the real-time data processing module, the captured image data is extracted frame by frame, the time interval Δt between every two frames is recorded, the position of the ground control point is found in the frame image, and the position information in the image coordinate system is recorded ( x , y , z ), and then according to the formula , , the position information of the ground control points in the two coordinate systems is brought into the conversion formula to determine 11 projection coordinate coefficients, so as to perform orthoprojection correction on each frame of the extracted image and unify the physical resolution of the image; (2) After the orthophoto correction is completed, grayscale processing, distortion correction and gamma correction are performed on each frame image, and finally the frame image is processed into a binary image containing only black and white; (3) Obtain two consecutive frames of images from the processed image data in real time, set the size of each grid unit, use the cross-correlation algorithm to calculate the cross-correlation function between the two image blocks, match the feature points of the two consecutive frames of images, and use the formula Calculate the cross-correlation function R ( x , y ), through the cross-correlation function R ( x , y ) and calculate the displacement Δ between the feature points in the image block x , Δ y , and then through the formula , Calculate flow rate u , v ; (4) In the obtained velocity data, some feature points have a low matching degree. The lower the signal-to-noise ratio, the lower the credibility. Calculate the signal-to-noise ratio of the cross-correlation function of the flow velocity for each grid cell SNR , using the formula The signal-to-noise ratio of each grid cell is filtered, and the missing values ​​are filled using the local mean interpolation method for results below the threshold; (5) Based on the flow velocity data, call the Matplotlib library function to draw a two-dimensional flow field vector diagram.

[0028] Step 3: The image data captured by the high-precision spherical camera, the flow velocity data and the two-dimensional flow field vector data obtained in the real-time data processing module are stored in the local storage module, and transmitted to the remote control terminal module through the network server module. The user can view and analyze the flow information on the channel surface in real time in the remote control terminal module, and can perform operations such as data playback and remote monitoring.

Claims

1. A digital twin system of waterway surface flow based on a camera, characterized in that: include: Image acquisition module, used to obtain image data of the channel surface in real time and in a wide range; The real-time data processing module includes a local server, which is used to identify and analyze the acquired image data, process the image data using an image processing algorithm, analyze the image data using an LSPIV algorithm model, calculate the flow velocity data and the signal-to-noise ratio of the data, and draw a two-dimensional flow field vector diagram; The power module and protective structure are used to supply power to all modules and have waterproof, dustproof and shockproof properties to ensure stable operation in various harsh environments; The network server module is a data transfer and processing center responsible for managing the data transmission between the image acquisition module, the local server and the remote control terminal module; The remote control terminal module is used for remote access and control of the system. It can view and analyze the flow information on the channel surface in real time through the network server module, provide control and operation instructions, and retrieve historical data to achieve remote monitoring and data playback of the water area.

2. According to the camera-based digital twin system of waterway surface flow state according to claim 1, it is characterized in that: The image acquisition module adopts a high-precision spherical camera.

3. According to the camera-based digital twin system of waterway surface flow state according to claim 2, it is characterized in that: When the high-precision spherical camera collects image data of the target water area, at least 6 ground control points should be set on site, and the position information of the ground control points should be recorded in the world coordinate system ( X , Y , Z ), in meters; then, a high-precision spherical camera is used to obtain the position of the ground control point in the image, and the displacement information of the ground control point in the image coordinate system is obtained ( x , y , z ), in pixels.

4. A camera-based digital twin method for channel surface flow states, characterized in that: The method is implemented by using the digital twin system of the waterway surface flow state according to any one of claims 1 to 3, and comprises the following steps: Step 1, digital image processing: The image data acquired by the image acquisition module is subjected to frame-by-frame extraction, frame image orthophoto correction, grayscale processing, distortion correction, gamma correction and image binarization processing using an image processing algorithm; Step 2, flow rate data calculation: Obtain two consecutive frames of images processed by step 1, set the size of the grid unit in the frame image, match the image feature points, calculate the relative displacement of the feature points using the cross-correlation function, and calculate the water flow velocity using the displacement velocity formula; Step 3, flow field diagram drawing: The signal-to-noise ratio of the cross-correlation function of the flow velocity of each grid unit is calculated, the signal-to-noise ratio of each grid unit is filtered, and a two-dimensional flow field vector diagram is drawn.

5. The camera-based digital twin method for channel surface flow state according to claim 4 is characterized in that: The step 1 of using an image processing algorithm to process image data comprises the following steps: 1.1, extract the image data obtained by the image acquisition module frame by frame, and record the time interval Δ between two frames of images t ; 1.2, according to the formula , , the position information of the ground control points in the two coordinate systems ( X , Y , Z ) is brought into the conversion formula to determine 11 projection coordinate coefficients, so as to perform orthographic correction on each frame of the extracted image, that is, to correct the obliquely shot image into an overhead shot image and unify the physical resolution of the image; in, A 1. A 2. A 3. A 4. B 1. B 2. B 3. B 4. C 1. C 2. C 3 is the projection coefficient; ( x , y , z ) is the displacement information of the ground control point in the image coordinate system; 1.3, grayscale processing, distortion correction and gamma correction are performed on the corrected image, and finally the frame image is processed into a binary image containing only black and white.

6. The camera-based digital twin method for channel surface flow states according to claim 5 is characterized in that: The step 2 uses the LSPIV algorithm model to calculate the flow rate data, and comprises the following steps: 2.1, obtain two consecutive frames of images from the processed image data in real time, and set the size of each grid unit in the image; 2.2, use the cross-correlation algorithm to calculate the cross-correlation function between two image blocks, match the feature points of the images, and use the formula , calculate the cross-correlation function R ( x , y ), through the cross-correlation function R ( x , y ) and calculate the displacement Δ between the feature points in the image block x , Δ y , and then through the formula , Calculate flow rate u , v ; in, I 1( i , j ) is the image block I 1 in position ( i , j ), I 2( i + x , j + y ) is the image block I 2 in position ( i + x , j + y ), Δ x and Δ y Respectively in x and y Displacement in the direction, Δ t is the time interval between two image frames.

7. The camera-based digital twin method for channel surface flow states according to claim 6 is characterized in that: The step 3 uses the LSPIV algorithm model to draw a flow field diagram, comprising the following steps: 3.1, through the formula Calculate the signal-to-noise ratio of the cross-correlation function of the flow velocity for each grid cell SNR ; in, P peak is the highest peak value of the cross-correlation function, P secondary peak This is the second highest peak value; 3.2, using formula The signal-to-noise ratio of each grid cell is filtered, and the missing values ​​are filled using the local mean interpolation method for results below the threshold; in, threshold It refers to the threshold for filtering the signal-to-noise ratio results; 3.

3. Based on the flow velocity data, call the Matplotlib library function to draw a two-dimensional flow field vector diagram.

Images