Large-view-field underwater high-speed motion measurement system and method

Abstract

The invention discloses a large-view-field underwater high-speed motion measurement system and method. The system comprises an underwater unmanned aerial vehicle, an underwater sealing box, a communication power supply unit and an upper computer. The system is based on an ultra-wideband wireless communication technology, a sound wave conversion technology and a dual-target calibration and measurement technology, and is used for high-speed acquisition and shooting of moving objects and measurement of motion parameters of underwater large-field-of-view multi-camera, so that the convenience and practicability of operation and layout are ensured on the premise of ensuring the measurement precision; another purpose of the invention is to provide a large-field-of-view underwater high-speed motion measurement method so as to improve the actual measurement accuracy and the measurement effect of the whole system.

Description

Technical Field

[0001] The present invention relates to the technical field of underwater binocular measurement, and in particular to a system and method for underwater high-speed motion measurement with a large field of view. Background Art

[0002] The scope of human activities has gradually expanded in the process of modernization, not only on land, but also in underwater space. In my country's underwater environment, in addition to aquaculture, mining, engineering experiments, and engineering applications, there are also some military tasks such as underwater launch and measurement in the field of national defense. The motion parameter measurement of underwater targets has always been an important task in the development of underwater military tasks. The motion parameters of the launch target, such as speed, acceleration, attitude angle, etc., are the parameters that the staff pays attention to. The accuracy and convenience of motion parameter measurement are also factors that the staff pays attention to.

[0003] In the currently commonly used underwater binocular measurement process, it is necessary to first perform binocular calibration and capture underwater calibration images. Usually, a fixed calibration device is used to dive into the common field of view of the underwater binocular camera for shooting and calibration. On the one hand, this method is more troublesome to set up and recover, and is limited to shallow underwater environments. On the other hand, if the shooting field of view is large, it is difficult to move the underwater calibration equipment for collaborative calibration of multiple cameras. Even if it is moved, it will take a long time.

[0004] In view of the above problems, there is an urgent need for a fast and convenient underwater large field of view calibration and measurement system that can ensure the convenience and practicality of operation and deployment while ensuring measurement accuracy.

[0005] To this end, we provide a system and method for large-field-of-view underwater high-speed motion measurement to solve the above problems. Summary of the invention

[0006] In view of the above problems, one of the objects of the present invention is to provide a system for underwater high-speed motion measurement with a large field of view in order to overcome the shortcomings of the prior art. The system is based on ultra-wideband wireless communication technology, acoustic wave conversion technology, dual-target positioning and measurement technology, and is used for high-speed acquisition and shooting of moving objects and measurement of motion parameters of underwater large-field-of-view multi-cameras, so as to ensure the convenience and practicality of operation and deployment while ensuring the measurement accuracy; another object of the present invention is to provide a method for underwater high-speed motion measurement with a large field of view, so as to improve the actual measurement accuracy and measurement effect of the entire system.

[0007] In order to achieve the above-mentioned object, the present invention adopts a system for large-field underwater high-speed motion measurement, including an underwater drone, an underwater sealing box, a communication power supply unit and a host computer;

[0008] An underwater drone, the underwater drone is located within the field of view of the underwater multi-camera and a luminous sonar transmitting module is mounted below the underwater drone;

[0009] Underwater sealed box, the following equipment is placed inside the underwater sealed box:

[0010] A sonar receiving end, which receives the signal from the light-emitting sonar transmitting module and is electrically connected to the processing module to transmit the external sound wave signal converted into an electrical signal to the processing module;

[0011] A processing module, wherein the processing module is provided with a receiving unit for receiving a signal from a sonar receiving end and an analyzing unit for analyzing a signal transmitting position;

[0012] High-speed cameras, the number of which is 4, the above-mentioned high-speed cameras form a binocular camera in pairs and obtain binocular images, the pixel position of the luminous sonar transmitting module in the large field of view, and the three-dimensional position of the luminous sonar module relative to the sonar receiving end;

[0013] A communication power supply unit, which includes a power supply subunit for supplying power to various devices in the underwater sealed box, a communication subunit for communicating with various devices in the underwater sealed box, and an NTP network timer for timing the high-speed camera. The communication subunit arranges the binocular images in a timestamp order and obtains an image sequence. The NTP network timer is externally connected to an antenna for receiving satellite signals and the NTP network timer times the high-speed camera.

[0014] The host computer includes a communication part for communicating with various devices, a control part for controlling various devices, a calibration part for underwater calibration under a large field of view, and a parsing part for obtaining parameter files based on image sequences and positioning tag positions.

[0015] As a further optimization of the above solution, it also includes:

[0016] A light source part, which corresponds one-to-one to the above-mentioned high-speed camera and is electrically connected to the communication power supply unit.

[0017] As a further optimization of the above solution, the sonar transmitting module includes a positioning tag submodule, a light emitting submodule and a sound wave conversion submodule.

[0018] As a further optimization of the above solution, the power supply subunit includes a power supply module and a light source controller, one end of the light source controller is connected to a switch and the other end is connected to each light source device;

[0019] The communication subunit includes a switch, which controls and communicates with each high-speed motor, light source and sonar receiving end.

[0020] As a further optimization of the above solution, the underwater drone is connected to the ground remote controller via wire.

[0021] As a further optimization of the above solution, the number of sonar receiving ends is three.

[0022] A method for large-field underwater high-speed motion measurement comprises the following steps:

[0023] S1. Install and fix the underwater sealed box, and reserve a power supply interface and a communication interface inside the underwater sealed box;

[0024] S2, multi-camera setup;

[0025] S3, installation and fixation of fill light source, sonar receiver and radar;

[0026] S4, the sonar receiving end communicates and coordinates with the luminous sonar transmitting module and calculates the position of the luminous tag online;

[0027] S5. Multi-camera high-speed shooting, NTP network timing module time-keeping for each high-speed camera, the system is ready;

[0028] S6, the underwater drone is equipped with a luminous sonar transmitting module and moves in multiple directions in the field of view of the underwater multi-camera, the multi-camera takes pictures synchronously, and the motion measurement of the target is performed using a multi-target stator method;

[0029] S7, the power supply communication unit arranges the multi-camera synchronized images in the order of timestamps, and records the underwater position of the luminous sonar transmitting module relative to the base station according to the timestamp, and synchronizes the binocular image with the positioning information by using the synchronization sub-method between the binocular image timestamp and the positioning information timestamp;

[0030] S8, the host computer receives the image sequence and the location of the positioning tag in real time, combines them with the sequence of timestamps through the image stitching sub-method, performs pairing, analysis and calculates the parameter file for calibration;

[0031] S9. Use the calibrated parameter file to conduct real-time underwater monitoring, real-time tracking of targets and complete motion measurement of targets.

[0032] As a further optimization of the above solution, in step S2, the distance between two adjacent cameras corresponds to the distance between each camera and the observation field of view.

[0033] As a further optimization of the above solution, the sonar receiving end position solution method in step S4 adopts a three-subarray positioning method.

[0034] As a further optimization of the above solution, step S6 also includes using a high-speed camera to extract a grayscale histogram of the drone using a brightness information addition sub-method to extract and track pixel coordinates, and using a pulse coupled neural network sub-method to filter the collected image.

[0035] The system and method for large-field-of-view underwater high-speed motion measurement of the present invention have the following beneficial effects:

[0036] 1) Through the above scheme, on the one hand, the present invention implements a system for calibration and measurement of a large field of view of multiple underwater high-speed cameras, including: an underwater sealed box; high-speed cameras A, B, C, D; fill light sources A, B, C, D; sonar processing module; luminous sonar transmitting module, sonar receiving end, underwater drone, power supply module, industrial computer, synchronous trigger and other components; on the other hand, based on the above system, a method for calibration and measurement of a large field of view of multiple underwater high-speed cameras is provided to realize high-speed acquisition and shooting of moving objects and measurement of motion parameters of multiple underwater cameras with a large field of view, while ensuring the convenience and practicality of operation and deployment under the premise of ensuring measurement accuracy.

[0037] 2) Furthermore, in the above method, the sonar receiving end position solution method adopts a three-subarray positioning method, which performs positioning in the same plane. It does not consider the vertical distribution of the channel sound velocity, nor the multipath effect of the channel, and can achieve high accuracy in positioning the close-range sound source, so as to ensure that the sonar receiving end accurately receives the sound wave signal of the luminous sonar transmitting module.

[0038] 3) Further, the power supply communication unit uses the synchronization sub-method between the binocular image timestamp and the positioning information timestamp to achieve the synchronization of the binocular image and the positioning information. Specifically, after the binocular camera receives the NTP network timing device, it adopts the Beidou satellite reference time and receives the PPS signal of pulses per second. After the high-speed cameras A, B, C, and D receive the trigger signal, they start to collect images at high speed from the next full second of that moment; at the same time, the active sonar in the underwater drone transmits an active signal, starting from the full second, and outputs the relative position data of the luminous sonar transmitting module every 100ms, such as (30.5m, 26.3m, -89.6m). In this way, for example, if the frame rate of the high-speed camera is set to 100fps, there will always be 100ms, 200ms, 300ms... full 100 milliseconds timestamp images in the collected images of the high-speed camera. The above method is used to achieve the synchronization of the time of both parties.

[0039] 4) Furthermore, the high-speed camera uses the method of adding brightness information to extract the grayscale histogram of the drone to realize the extraction and tracking of pixel coordinates. The grayscale histogram is calculated by counting the grayscale value of the drone area and the number of its occurrences to form a grayscale histogram. The coordinates matching the histogram are found in the graphics captured by the multi-camera. In real-time tracking, in order to improve the tracking speed, when traversing each local pixel area (such as a 20×20 pixel grid), the ratio between the highest brightness value and the average grayscale value is first determined. If it exceeds 3 times, the histogram is matched. If the ratio has not reached 3 times, it is directly ignored and the next local pixel grid is traversed.

[0040] 5) Furthermore, the high-speed camera uses a pulse coupled neural network (PCNN) sub-method to filter the captured image. PCNN first accurately locates the noise pixels in the image, and then determines whether it is a noise pixel. If it is useful information, it is not processed at all. If it is noise, it is filtered in a median-like method. In this way, the useful information pixels can be retained to the maximum extent, so that the image can only gradually move towards the direction of the original image after filtering. In this way, noise pollution is filtered out as much as possible while keeping the useful information from being considered destroyed, thereby greatly improving the filtering performance.

[0041] 6) Furthermore, the host computer combines the sequence of timestamps through an image stitching sub-method. The image stitching sub-method ensures accurate pairing of the image sequence and the positioning tag position through feature point extraction and matching, image registration and image fusion, which is beneficial for the host computer to subsequently parse and calculate the parameter file for calibration.

[0042] With reference to the following description and drawings, specific embodiments of the present invention are disclosed in detail, indicating the manner in which the principles of the present invention can be adopted. It should be understood that the scope of the embodiments of the present invention is not limited thereby. Within the spirit and scope of the appended claims, the embodiments of the present invention include many changes, modifications and equivalents. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a schematic diagram of the structure of a system for underwater high-speed motion measurement with a large field of view;

[0044] Figure 2 A schematic diagram of the working process of a method for underwater high-speed motion measurement with a large field of view;

[0045] Figure 3 A schematic diagram of a synchronization sub-method between binocular image timestamp and positioning information timestamp in the present invention to achieve time synchronization between the two parties;

[0046] Figure 4 Schematic diagram of the working process of the pulse coupled neural network sub-method in the present invention;

[0047] Figure 5 Schematic diagram of the weighted smoothing algorithm in the image stitching sub-method of the present invention.

[0048] In the figure: 1. Underwater drone; 2. Underwater sealing box; 3. Communication power supply unit; 4. Host computer; 6. Light source part; 11. Ground remote control; 12. Luminous sonar transmitting module; 21. Sonar receiving end; 22. Processing module; 23. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present invention more clear, the present invention is further described in detail below through the accompanying drawings and embodiments. However, it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the scope of the present invention.

[0050] It should be noted that, when an element is referred to as being "disposed on, provided with" another element, it may be directly on the other element or there may be a central element; when an element is considered to be "connected, connected to" another element, it may be directly connected to the other element or there may be a central element at the same time; "fixed connection" means a fixed connection, and there are many ways of fixed connection, which are not within the scope of protection of this document; the terms "vertical", "horizontal", "left", "right" and similar expressions used in this document are for illustrative purposes only and do not represent the only implementation method.

[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs. The terms used in the specification are only for the purpose of describing specific implementations and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0052] Please refer to the instruction manual Figure 1-5 The present invention provides a technical solution: a system for large-field underwater high-speed motion measurement, first embodiment, with particular reference to Figure 1 ,Should Figure 1 The specific structure of the system for large-field underwater high-speed motion measurement is shown. Specifically, the system includes an underwater drone 1, an underwater sealing box 2, a communication power supply unit 3 and a host computer 4;

[0053] An underwater drone 1, which is limitedly connected to a ground remote controller 11, so that ground personnel can control the movement of the underwater drone 1 through the ground remote controller 11. The underwater drone 1 is located in the field of view of an underwater multi-camera and a light-emitting sonar transmitting module 12 is mounted below the underwater drone 1. It should be noted that the main function of the underwater drone 1 is to be mounted on the light-emitting sonar transmitting module 12 and move from far to near and from bottom to top within the field of view of four high-speed cameras 23. The high-speed camera 23 in the system records the pixel position of the light-emitting sonar transmitting module 12 in the large field of view on the one hand, and records the three-dimensional position of the light-emitting sonar transmitting module 12 relative to the sonar receiving end 21 on the other hand, and performs the subsequent calibration file solution. The above-mentioned light-emitting sonar transmitting module 12 integrates a positioning tag submodule, a light-emitting submodule and a sound wave conversion submodule, which emits visible light to enable the high-speed camera 23 to detect the target on the one hand, and sends its own information to the sonar receiving end 21 through electromagnetic wave-sound wave conversion on the other hand;

[0054] The underwater sealed box 2 has the following equipment placed inside:

[0055] A sonar receiving end 21, which is used in conjunction with a processing module 22. Specifically, the sonar receiving end 21 receives a signal from the light-emitting sonar transmitting module 12. The sonar receiving end 21 is electrically connected to the processing module 22 to transmit the external sound wave signal converted into an electrical signal to the processing module 22;

[0056] In this embodiment, the sonar receiving terminals 21 are fixed in position and are three in number. For example, in this embodiment, two of the sonar receiving terminals 21 are arranged in the underwater sealing box 2 and one is placed above the underwater drone 1 .

[0057] The processing module 22 is provided with a receiving unit for receiving the signal of the sonar receiving terminal 21 and a parsing unit for parsing the signal emission position. The processing module 22 works in coordination with the sonar receiving terminal 21, that is, on the one hand, the signal of the sonar receiving terminal 21 is received; on the other hand, the signal emission position is parsed. It should be noted that the positions of the three sonar receiving terminals 21 of the system are measurable or known when they are installed. The three sonar receiving terminals 21 receive the sound waves emitted by the active sonar, and the position of the active sonar can be calculated in combination with their own positions. In this embodiment, the above-mentioned active sonar is mounted on the above-mentioned underwater drone 1.

[0058] High-speed cameras 23, the number of which is 4. In this embodiment, the four high-speed cameras 23 are marked as A, B, C, and D respectively. In addition to the 12V power supply line, each camera also has a network cable for communication, and each camera is equipped with a lens of a fixed focal length. The above-mentioned high-speed cameras 23 form a binocular system in pairs and obtain binocular images, the pixel position of the luminous sonar transmitting module 12 in a large field of view, and the three-dimensional position of the luminous sonar module relative to the sonar receiving end 21;

[0059] A communication power supply unit 3, which includes a power supply subunit for supplying power to each device in the underwater sealed box 2, a communication subunit for communicating with each device in the underwater sealed box 2, and an NTP network timer for timing the high-speed camera 23. The communication subunit arranges the binocular images in a timestamp order and obtains an image sequence. The NTP network timer is externally connected to an antenna for receiving satellite signals and the NTP network timer accurately times the high-speed camera 23.

[0060] More specific:

[0061] The above-mentioned power supply sub-unit includes a power supply module and a light source controller. The power supply module supplies power to the switch, light source controller, camera, NTP network timer, sonar receiving end 21, sonar and other equipment. The power supply module itself can provide four voltage values ​​of 24V, 19V, 12V and 5V; one end of the above-mentioned light source controller is connected to the switch and the other end is connected to each light source device, which is responsible for power supply and on-off control.

[0062] The communication subunit includes a switch, which controls and communicates with each high-speed motor, light source, and sonar receiving end 21.

[0063] The host computer 4 includes a communication part for communicating with various devices, a control part for controlling various devices, a calibration part for underwater calibration under a large field of view, and a parsing part for obtaining parameter files based on image sequences and positioning tag positions.

[0064] The light source part 6 and the four high-speed cameras 23 are each equipped with a fill light source to fill the underwater field of view. Only the power line is connected to the light source part, and the controllers of the four light sources are integrated inside the power supply and communication unit on land.

[0065] A method for measuring underwater high-speed motion with a large field of view comprises the following steps:

[0066] S1, install and fix the underwater sealing box 2, and reserve a power supply interface and a communication interface inside the underwater sealing box 2;

[0067] S2. Multi-camera setup, i.e., setup of high-speed camera 23A and high-speed cameras 23B, C, and D. The distance between two adjacent cameras corresponds to the distance between each camera and the observation field of view. In the present embodiment, the distance between the two adjacent cameras is close to the distance between each camera and the observation field of view.

[0068] S3, installation and fixation of fill light source, sonar receiver 21, and radar;

[0069] S4, the sonar receiving end 21 communicates and coordinates with the luminous sonar transmitting module and calculates the position of the luminous tag online;

[0070] For step 4, the position solution method of the sonar receiving end 21 adopts a three-subarray positioning sub-method. Specifically, the three-subarray positioning sub-method uses three sonar receiving ends 21 to continuously detect the signal of the active sonar, and uses the reception time to measure the distance between the active sonar and each sonar receiving end 21. According to the determined distance to three fixed points in space, the relative position of the active sonar is cross-determined.

[0071] The three-subarray positioning method simplifies the underwater acoustic channel. The three-subarray system is positioned in the same plane. It does not consider the vertical distribution of the channel sound velocity, nor the multipath effect of the channel. However, this positioning method has a simple algorithm and can achieve high accuracy in locating close-range sound sources. It has been widely used in engineering.

[0072] S5, multi-camera high-speed shooting, NTP network timing module works normally to ensure that the NTP network timing module accurately synchronizes the time of each high-speed camera 23, and the system is ready;

[0073] S6, the underwater drone 1 is equipped with a luminous sonar transmitting module and moves in multiple directions in the field of view of the underwater multi-camera, the multi-camera takes pictures synchronously, and the motion measurement of the target is performed using a multi-target stator method;

[0074] Specifically, the above multi-objective stator method is as follows:

[0075] According to the pinhole imaging model, point P in the world coordinate system NEU (X, Y, Z) and the image p(u, v) formed in the upper and lower high-speed cameras 23 satisfy the following relationship

[0076]

[0077] Among them, K, R and T are the internal parameters, rotation matrix and translation matrix of the camera respectively. The real-time images of the high-speed camera 23A and the high-speed camera 23B obtained by this system can calculate the point p(u,v) where the drone is located. The active sonar of this system transmits the signal to the three sonar receiving end 21 devices, and the point P in the established coordinate system where the sonar receiving end 21 is located can be calculated. NEU (X,Y,Z). is the projection matrix of the camera. Camera calibration is to determine the parameters of the matrix P. P contains 12 parameters. 23 Normalized to 1, only 11 unknown parameters need to be solved.

[0078] The pinhole imaging model is expanded into the following equation

[0079]

[0080] Therefore, one world coordinate system point can provide two equations, and at least 6 world coordinate system points are needed to solve 11 unknown parameters, thereby obtaining the projection matrix. When the number of collected luminous sonar transmitting module coordinate points increases, the projection matrix will be gradually corrected to the best equation solution. Experiments have confirmed that when the number of collected luminous sonar transmitting module points exceeds 30 pairs, the calibration error will be reduced to a smaller range. Multi-eye camera calibration is actually the joint calibration of multiple binocular cameras. When the target height is between the two nearest high-speed cameras 23, the joint calibration file of the two nearest high-speed cameras 23 is used to measure the motion of the target.

[0081] In step 6, conventional drone pixel extraction requires three stages: manager discovery, tracking, and screening. Specifically:

[0082] Discovery: Found the drone target. Use the frame difference method to process the first 10 frames of the image sequence and segment the moving target in the video. Due to the presence of interference factors such as fish and image noise, there is more than one moving target segmented, but these interference sources are different from the real drone target in terms of target size, motion law, brightness level and other dimensions. This difference can be used to extract the real drone target;

[0083] Tracking: Track the drone target. Using the drone target extracted in step 1 as a template, the optical flow method is used to continuously track the drone target to obtain the target's coordinate sequence pl and pr, and confidence sequence Cl and Cr;

[0084] Screening: Setting the confidence threshold C T , filter out the coordinate sequences pl′ and pr′ whose left and right confidences are greater than the threshold.

[0085] Generally speaking, the conventional extraction method mentioned above is difficult to achieve by extracting feature corner points for stable tracking when the number of pixels occupied by the drone in the image is small (generally no more than 30 pixels).

[0086] In view of this, the method added by the present invention at this time is to add brightness information, extract the grayscale histogram of the luminous drone to extract pixel coordinates and stably track, wherein the grayscale histogram is calculated by counting the grayscale value of the drone area and the number of its occurrences to form a grayscale histogram, and find coordinates matching the histogram in the graphics captured by the multi-eye camera. During real-time tracking, in order to improve the tracking speed, when traversing each local pixel area (such as a 20×20 pixel grid), first determine the ratio between the highest brightness value and the average grayscale value. If it exceeds 3 times, the histogram is matched. If the ratio has not reached 3 times, it is directly ignored and the next local pixel grid is traversed.

[0087] For step 6, the underwater image is greatly affected by suspended particles and water flow. Based on this, a pulse coupled neural network (PCNN) sub-method is used in step 6 to filter the collected image.

[0088] Specific, special reference Figure 4 PCNN first accurately locates the noise pixels in the image, and then determines whether it is a noise pixel. If it is useful information, it will not be processed at all. If it is noise, it will be filtered in a median-like method. In this way, the useful information pixels can be retained to the maximum extent, so that the image can only gradually move towards the original image after filtering. In this way, noise pollution is filtered out as much as possible while keeping the useful information from being considered destroyed, thereby greatly improving the filtering performance.

[0089] The above noise judgment noise judgment: first select and excite a PCNN with the same size as the image, establish a firing matrix (FTM, Firing Time Map), and then run it with a (2m+1)×(2m+1) grid. The time corresponding to the neuron (0,0) at the center point in the window is recorded as T00, and the other times in the window are Tij. Let S1 be the number of T00-Tij greater than 1, and S-1 be the number of T00-Tij less than -1. When each neuron in the encircled area is connected to only four adjacent neurons, if S1≥3 or S-1>3, then the pixel corresponding to the neuron can be determined as a noise pixel, otherwise, the corresponding pixel is a non-noise pixel.

[0090] The filtering of the above noise pixels: When a certain pixel is determined to be a noise pixel in the image, the following median filtering method is given. First, a window with the noise pixel as the center is established. If S-1 is greater than half of the number of pixels in the window, the median of all the pixel grayscale values ​​in the window with ΔTij < -1 is taken as the grayscale value of the center pixel (0, 0) of the window; If S1 is greater than half of the number of pixels in the window, the median of all the pixel grayscale values ​​in the window with ΔTij > +1 is taken as the grayscale value of the center pixel (0, 0) of the window.

[0091] S7, the power supply communication unit arranges the multi-camera synchronized images in the order of timestamps, and records the underwater position of the luminous sonar transmitting module relative to the base station according to the timestamp, and synchronizes the binocular image with the positioning information by using the synchronization sub-method between the binocular image timestamp and the positioning information timestamp;

[0092] It should be noted that special reference Figure 3 In this embodiment, the synchronization sub-method between the binocular image timestamp and the positioning information timestamp is as follows:

[0093] After the binocular camera receives the NTP network timing device, it adopts the Beidou satellite reference time and receives the PPS signal of pulses per second. After receiving the trigger signal, the high-speed cameras 23A, B, C, and D start to collect images at high speed from the next full second of that moment; at the same time, the active sonar in the underwater drone 1 transmits an active signal, and outputs the relative position data of the luminous sonar transmitting module every 100ms starting from the full second, such as (30.5m, 26.3m, -89.6m). In this way, for example, if the frame rate of the high-speed camera 23 is set to 100fps, then in the collected images of the high-speed camera 23, there will always be timestamp images of 100ms, 200ms, 300ms... full 100 milliseconds. The above method is used to achieve the synchronization of the time of both parties.

[0094] S8, the host computer 4 receives the image sequence and the location of the positioning tag in real time, combines them with the sequence of timestamps through the image stitching sub-method, performs pairing, analysis and calculates the parameter file for calibration;

[0095] based on Figure 5 , the above image stitching sub-methods are discussed in detail:

[0096] Image stitching technology includes three steps: feature point extraction and matching, image registration, and image fusion.

[0097] In this system, since the real-time nature of stitching is emphasized for motion measurement, the feature point extraction and matching algorithm used is the ORB algorithm. Solving the transformation matrix H in image registration is the core of image registration, and the algorithm flow for solving it is as follows.

[0098] 1) Detect feature points in each image.

[0099] 2) Calculate the matching between feature points.

[0100] 3) Calculate the initial value of the transformation matrix between images.

[0101] 4) Iteratively refine the H transformation matrix.

[0102] 5) Guided matching: Use the estimated H to define the search area near the epipolar line to further determine the correspondence of feature points.

[0103] 6) Repeat iterations 4) and 5) until the number of corresponding points is simultaneously stable.

[0104] Assuming that the transformation between image sequences is a projection transformation, the eight degree-of-freedom parameters hi = (i = 0, 1, ..., 7) of the H matrix can be calculated using four sets of optimal matches and used as initial values.

[0105] Specifically, the calculation method of the above H is as follows:

[0106] Assuming that the corresponding points in the two images are aligned with the secondary coordinates (x', y', 1) and (x, y, 1), the homography matrix H is defined as:

[0107]

[0108] Then there is

[0109] After the matrix is ​​expanded, there are 3 equations. Substituting the third equation into the first two equations, we get:

[0110]

[0111] That is, one point pair corresponds to two equations.

[0112] The H matrix has 9 parameters, from h11 to h33. In fact, its degree of freedom is not 9, because the homogeneous coordinate system is used here, which means that it can be scaled to any scale. For example, if we multiply hij by any non-zero constant k, we get the following equation:

[0113]

[0114] The above equation results in the following:

[0115]

[0116] So actually the homography matrix H has only 8 degrees of freedom. The calculation process of H under 8 degrees of freedom adds constraints to H and changes the H matrix modulus to 1, as follows:

[0117]

[0118]

[0119] We will replace the following equation (including the ||H||=1 constraint):

[0120]

[0121] Multiply by the denominator and expand to get:

[0122] (h 31 x+h 32 y+h 33 )x ′ =h 11 x+h 12 y+h 13

[0123] (h 31 x+h 32 y+h 33 )y ′ =h 21 x+h 22 y+h 23

[0124] After sorting, we get:

[0125] h 11 x+h 12 y+h 13 -h 31 xx ′ -h 32 yx ′ -h 33 x ′ =0

[0126] h 11 x+h 12 y+h 13 -h 31 xy ′ -h 32 yy ′ -h 33 y ′ =0

[0127] If we get the corresponding N point pairs (feature point matching pairs) in the two pictures, we can get the following linear equations:

[0128]

[0129] Write it in matrix form:

[0130]

[0131] Since the homography matrix H contains the constraint ||H|| = 1, according to the linear equations in the above figure, for an 8-degree-of-freedom H, we need at least 4 pairs of corresponding points to calculate the homography matrix. However, the above is only a theoretical derivation. In real application scenarios, the point pairs we calculate will all contain noise. For example, the position of the points deviates by several pixels, or even the phenomenon of incorrect matching of feature point pairs occurs. If only 4 point pairs are used to calculate the homography matrix, there will be a large error. Therefore, in order to make the calculation more accurate, generally far more than 4 point pairs are used to calculate the homography matrix.

[0132] In order to improve the accuracy of image registration, this system uses the Random Sample Consensus (RANSAC) algorithm to solve and refine the image transformation matrix to achieve a better image stitching effect. The image fusion method adopted in this system is that according to the transformation matrix H between images, the corresponding images can be transformed to determine the overlapping area between the images, and the images to be fused are mapped onto a new blank image to form a stitched image. It should be noted that since ordinary cameras will automatically select exposure parameters when taking pictures, there will be a brightness difference between the input images, resulting in obvious light and dark changes at both ends of the stitching line of the stitched image. Therefore, during the fusion process, the stitching line needs to be processed. There are many methods for processing the stitching line of image stitching, such as color interpolation and multi-resolution spline techniques, etc. This system uses a fast and simple weighted smoothing algorithm to process the stitching seam problem. The main method of this algorithm is: the gray value Pixel of the pixel points in the overlapping area of the images is obtained by weighted averaging the gray values Pixel_L and _R of the corresponding points in the two images, that is, Pixel = k × Pixel_L + (1 - k) × Pixel_R, where k is an adjustable factor.

[0133] Usually 0 < k < 1, that is, in the overlapping area, along the direction from image 1 to image 2, k gradually changes from 1 to 0, so as to achieve smooth stitching of the overlapping area. To make the points in the overlapping area of the images have a greater correlation with the two images, let k = d1 / (d1 + d2), where d1 and d2 respectively represent the distances from the points in the overlapping area to the left and right boundaries of the overlapping area of the two images. That is, use the formula Pixel = × Pixel_L + × Pixel_R to process the stitching line.

[0134] S9. Use the calibrated parameter file to perform real-time monitoring underwater, track the target in real time, and complete the motion measurement of the target.

[0135] It is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modification, equivalent substitution or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A system for underwater high-speed motion measurement with a large field of view, characterized in that: Including underwater drone, underwater sealing box, communication power supply unit and host computer; An underwater drone, the underwater drone is located within the field of view of the underwater multi-camera and a luminous sonar transmitting module is mounted below the underwater drone; Underwater sealed box, the following equipment is placed inside the underwater sealed box: A sonar receiving end, which receives the signal from the light-emitting sonar transmitting module and is electrically connected to the processing module to transmit the external sound wave signal converted into an electrical signal to the processing module; A processing module, wherein the processing module is provided with a receiving unit for receiving a signal from a sonar receiving end and an analyzing unit for analyzing a signal transmitting position; High-speed cameras, the number of which is 4, the above-mentioned high-speed cameras form a binocular camera in pairs and obtain binocular images, the pixel position of the luminous sonar transmitting module in the large field of view, and the three-dimensional position of the luminous sonar module relative to the sonar receiving end; A communication power supply unit, which includes a power supply subunit for supplying power to various devices in the underwater sealed box, a communication subunit for communicating with various devices in the underwater sealed box, and an NTP network timer for timing the high-speed camera. The communication subunit arranges the binocular images in a timestamp order and obtains an image sequence. The NTP network timer is externally connected to an antenna for receiving satellite signals and the NTP network timer times the high-speed camera. The host computer includes a communication part for communicating with various devices, a control part for controlling various devices, a calibration part for underwater calibration under a large field of view, and a parsing part for obtaining parameter files based on image sequences and positioning tag positions.

2. A system for measuring underwater high-speed motion with a large field of view according to claim 1, characterized in that: Also includes: A light source part corresponds one-to-one with the high-speed camera and is electrically connected to the communication power supply unit.

3. A system for measuring underwater high-speed motion with a large field of view according to claim 2, characterized in that: The sonar transmitting module comprises a positioning tag submodule, a light emitting submodule and a sound wave conversion submodule.

4. A system for measuring underwater high-speed motion with a large field of view according to claim 3, characterized in that: The power supply subunit includes a power supply module and a light source controller, one end of the light source controller is connected to the switch and the other end is connected to each light source device; The communication subunit includes a switch, which controls and communicates with each high-speed motor, light source and sonar receiving end.

5. A system for measuring underwater high-speed motion with a large field of view according to claim 4, characterized in that: The above-mentioned underwater drone is connected to the ground remote controller via wire.

6. A system for measuring underwater high-speed motion with a large field of view according to claim 5, characterized in that: The number of sonar receivers is three.

7. A method for measuring underwater high-speed motion with a large field of view according to any one of claims 1 to 6, characterized in that: The following steps are involved: S1. Install and fix the underwater sealed box, and reserve a power supply interface and a communication interface inside the underwater sealed box; S2, multi-camera setup; S3, installation and fixation of fill light source, sonar receiver and radar; S4, the sonar receiving end communicates and coordinates with the luminous sonar transmitting module and calculates the position of the luminous tag online; S5. Multi-camera high-speed shooting, NTP network timing module time synchronization for each high-speed camera, the system is ready; S6, the underwater drone is equipped with a luminous sonar transmitting module and moves in multiple directions in the field of view of the underwater multi-camera, the multi-camera takes pictures synchronously, and the motion measurement of the target is performed using the multi-target stator method; S7, the power supply communication unit arranges the multi-camera synchronized images in the order of timestamps, and records the underwater position of the luminous sonar transmitting module relative to the base station according to the timestamp, and synchronizes the binocular image with the positioning information by using the synchronization sub-method between the binocular image timestamp and the positioning information timestamp; S8, the host computer receives the image sequence and the location of the positioning tag in real time, combines the image sequence with the timestamp sequence through the image stitching sub-method, performs pairing, analysis and calculates the parameter file for calibration; S9. Use the calibrated parameter file to conduct real-time underwater monitoring, track the target in real time and complete the motion measurement of the target.

8. The method for measuring underwater high-speed motion with a large field of view according to claim 7, characterized in that: In step S2, the distance between two adjacent cameras corresponds to the distance between each camera and the observation field of view.

9. A system and method for large-field-of-view underwater high-speed motion measurement according to claim 8, characterized in that: In step S4, the sonar receiving end position solution method adopts a three-subarray positioning method.

10. A system and method for large-field-of-view underwater high-speed motion measurement according to claim 9, characterized in that: Step S6 also includes using the high-speed camera to extract the grayscale histogram of the drone using the brightness information addition sub-method to extract and track pixel coordinates, and using the pulse coupled neural network sub-method to filter the collected image.

Images