Long-distance target ranging method, device, computer equipment and storage medium

Abstract

The present invention relates to the field of distance measurement technology, and discloses a method, device, computer equipment and storage medium for distance measurement of long-distance targets, including: using an image acquisition device to collect image data of a target object, using inertial navigation data synchronized with the image data by a carrier inertial navigation; preprocessing the image data to obtain preprocessed image data; determining the spatial position and direction of each frame of the image; performing feature matching on each frame of the image to obtain the same feature points of each frame of the image; and calculating the distance of the target object based on the same feature points of each frame of the image and the inertial navigation data. The present invention achieves high-precision target distance estimation and motion trajectory inference by combining a monocular camera and an inertial navigation system, thereby solving the limitations of a monocular camera in depth measurement, and can effectively improve the distance measurement accuracy of a single camera for a target, especially a long-distance target. It can be combined with the movement of the carrier in a dynamic scene to provide a more accurate target distance calculation.

Description

Technical Field

[0001] The present invention relates to the field of distance measurement technology, and in particular to a method, device, computer equipment and storage medium for long-distance target distance measurement. Background Art

[0002] When intelligent unmanned vehicles and ships are driving, they often face targets that need to be located at a long distance. Currently common methods include camera plus laser ranging, lidar ranging, triangulation ranging using binocular cameras, and other ranging methods. They each have their own advantages and disadvantages.

[0003] In the related technologies, laser ranging is difficult to accurately locate small targets at long distances, the cost of laser radar is relatively high, the binocular camera only relies on triangulation to measure distant targets with very low accuracy, and the monocular camera has strong limitations in depth measurement, making it difficult to achieve high-precision target distance estimation and motion trajectory inference. Summary of the invention

[0004] In view of this, the present invention provides a long-distance target distance measurement method, device, computer equipment and storage medium to solve the problem that it is difficult to achieve high-precision long-distance target distance estimation.

[0005] In a first aspect, the present invention provides a method for measuring the distance of a long-distance target, the method comprising:

[0006] The image data of the target object is collected by using an image acquisition device, and the inertial navigation data synchronized with the image data is obtained by using the carrier inertial navigation, and the inertial navigation data is used to characterize the motion data of the image acquisition device;

[0007] Preprocessing the image data to obtain preprocessed image data;

[0008] Based on the inertial navigation data, determine the spatial position and direction of each frame of image;

[0009] Based on the preprocessed image data, feature matching is performed on each frame of the image to obtain the same feature points of each frame of the image;

[0010] Based on the same feature points in each frame of the image and the inertial navigation data, the distance to the target object is calculated.

[0011] In the present invention, by combining the image data obtained by the monocular camera with the motion information provided by the inertial navigation system, the time-for-space method is adopted to greatly improve the visual angle difference of the ranging triangle. Compared with the traditional monocular or binocular ranging system, high-precision target distance estimation and motion trajectory inference are achieved, thereby solving the limitations of the monocular camera in distance measurement, and effectively improving the ranging accuracy of the single camera for the target, especially the long-distance target. At the same time, it can be combined with the movement of the carrier in dynamic scenes to provide more accurate target distance calculation.

[0012] In an optional implementation, the inertial navigation data includes: three-axis angular velocity data, three-axis acceleration data, and a relative position relationship between the image acquisition device and the inertial navigation system; based on the inertial navigation data, determining the spatial position and direction of each frame of the image includes:

[0013] The three-axis angular velocity data and the three-axis acceleration data of the image acquisition device are calculated according to the three-axis angular velocity data, the three-axis acceleration data and the relative position relationship between the image acquisition device and the inertial navigation system;

[0014] The three-axis angular velocity data and three-axis acceleration data of the image acquisition device are integrated on a time scale to obtain the position relationship between the image acquisition device at the current moment and the next moment, and determine the spatial position and direction of the image data at the current moment and the next moment.

[0015] In this method, the inertial navigation data collected by the inertial navigation system is used to calculate the position relationship of the camera at any two moments with a relatively small time span, that is, to determine the spatial coordinates and orientation of the images collected at these two moments, providing support for the calculation of the camera position and attitude.

[0016] In an optional implementation, based on the preprocessed image data, feature matching is performed on each frame of the image to obtain the same feature points of each frame of the image, including:

[0017] Perform feature point detection on each frame of image, identify the feature points of each frame of image, and generate feature descriptors corresponding to the feature points. The feature descriptors are used to characterize the local features of the feature points.

[0018] Based on the feature descriptor, the feature points of the current frame image are matched with the feature points of the previous frame image, and matching point pairs between the current frame image and the previous frame image are generated to obtain the same feature points of each frame image.

[0019] In this method, in order to measure the distance of the target object, it is necessary to find the same target in the images at any two moments obtained in the above steps. This can be achieved by matching the feature points of consecutive frame images to provide a data basis for subsequent distance calculation.

[0020] In an optional implementation, performing feature point detection on each frame of image to identify feature points of each frame of image includes:

[0021] Calculate the gradient information of each pixel in each frame of the image;

[0022] Based on the gradient information, the corner points whose gradient information difference between adjacent pixels is greater than a preset threshold are screened and determined as feature points.

[0023] In this method, by finding easy-to-track feature points in the image, the corner points of the high-contrast, texture-rich areas in the image are used as feature points, which makes it easier to identify the target feature points in the image and provides a data basis for subsequent distance calculations.

[0024] In an optional implementation, the distance of the target object is calculated based on the same feature points and inertial navigation data of each frame of the image, including:

[0025] Based on the same feature points in each frame of image, determine the imaging position of the target object in each frame of image;

[0026] Screening out a first imaging point of the target object in the first image data corresponding to the first moment and a second imaging point of the target object in the second image data corresponding to the second moment, and calculating a first pixel number from the first imaging point to the image center and a second pixel number from the second imaging point to the image center; the distance between the first imaging point and the second imaging point is greater than a preset threshold;

[0027] Calculate a first angle between a first straight line between the target object and the optical center of the image acquisition device at a first moment and a preset coordinate system, and a second angle between a second straight line between the target object and the optical center of the image acquisition device at a second moment and the preset coordinate system; the origin of the preset coordinate system is the optical center of the image acquisition device at the first moment, the X-axis of the preset coordinate system is the normal of the image acquisition device at the first moment, and the Y-axis of the preset coordinate system is parallel to the long side of the camera sensor corresponding to the image acquisition device;

[0028] Based on the first angle and the second angle and in combination with the inertial navigation data, the coordinates of the target object are calculated, and the distance of the target object is obtained.

[0029] In an optional implementation, based on the first angle and the second angle, combined with inertial navigation data, the coordinates of the target object are calculated to obtain the distance of the target object, including:

[0030] Establishing a first equation of a first straight line in a preset coordinate system;

[0031] Establishing a second equation of a second straight line in a preset coordinate system according to the inertial navigation data at the second moment and the second angle;

[0032] The first equation and the second equation are combined to calculate the coordinates of the target object;

[0033] Based on the coordinates of the target object and the inertial navigation data at the second moment, the distance between the target object and the image acquisition device is calculated.

[0034] In this method, the displacement changes of the same feature points photographed by the image acquisition device at different positions are combined with the camera's inertial navigation data to perform triangulation calculations to calculate the distance of the feature points, thereby calculating the distance to the target object.

[0035] In a second aspect, the present invention provides a long-distance target ranging device, the device comprising:

[0036] A data acquisition module is used to acquire image data of a target object using an image acquisition device, and to acquire inertial navigation data synchronized with the image data using a carrier inertial navigation, wherein the inertial navigation data is used to characterize motion data of the image acquisition device;

[0037] An image preprocessing module is used to preprocess image data to obtain preprocessed image data;

[0038] The attitude calculation module is used to determine the spatial position and direction of each frame of image based on inertial navigation data;

[0039] A feature matching module is used to perform feature matching on each frame of the image based on the preprocessed image data to obtain the same feature points in each frame of the image;

[0040] The distance calculation module is used to calculate the distance of the target object based on the same feature points and inertial navigation data of each frame image.

[0041] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the long-distance target ranging method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0042] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the long-distance target ranging method of the first aspect or any corresponding embodiment thereof.

[0043] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the long-distance target ranging method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 4 is a flow chart of a long-distance target ranging method according to an embodiment of the present invention.

[0046] Figure 2 The present invention is a flowchart of a distance measurement method based on a monocular camera and an inertial navigation system according to an embodiment of the present invention.

[0047] Figure 3 The figure is a schematic diagram of an application scenario of a distance measurement system based on a monocular camera and an inertial navigation system according to an embodiment of the present invention.

[0048] Figure 4 4 is a flow chart of another long-distance target ranging method according to an embodiment of the present invention.

[0049] Figure 5 4 is a flow chart of another long-distance target ranging method according to an embodiment of the present invention.

[0050] Figure 6 is a schematic diagram of target distance calculation according to an embodiment of the present invention.

[0051] Figure 7 4 is a structural block diagram of a long-distance target distance measuring device according to an embodiment of the present invention.

[0052] Figure 8 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0054] In the related technologies, laser ranging is difficult to accurately locate small targets at long distances, the cost of laser radar is relatively high, the binocular camera only relies on triangulation to measure distant targets with very low accuracy, and the monocular camera has strong limitations in depth measurement, making it difficult to achieve high-precision target distance estimation and motion trajectory inference.

[0055] In order to solve the above problems, a long-distance target ranging method is provided in an embodiment of the present invention, which is used in a computer device. It should be noted that its execution subject can be a long-distance target ranging device, which can be implemented as part or all of the computer device through software, hardware or a combination of software and hardware. The computer device can be a terminal or a client or a server. The server can be a server or a server cluster composed of multiple servers. The terminal in the embodiment of the present application can be a smart phone, a personal computer, a tablet computer and other intelligent hardware devices. In the following method embodiments, the execution subject is a computer device as an example for explanation.

[0056] The computer device in this embodiment is suitable for the use scenario of using a monocular camera to perform high-precision target distance estimation on a small target at a long distance. The present invention provides a long-distance target ranging method, which combines the image data obtained by the monocular camera with the motion information provided by the inertial navigation system, and adopts the method of exchanging time for space to greatly improve the viewing angle difference of the ranging triangle. Compared with the traditional monocular or binocular ranging system, it can achieve high-precision target distance estimation and motion trajectory speculation, thereby solving the limitations of the monocular camera in depth measurement, and can effectively improve the ranging accuracy of a single camera for a target, especially a long-distance target. At the same time, it can be combined with the movement of the carrier in a dynamic scene to provide a more accurate target distance calculation.

[0057] According to an embodiment of the present invention, an embodiment of a method for measuring the distance between a long-distance target is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0058] In this embodiment, a long-distance target distance measurement method is provided, which can be used for the above-mentioned computer device. Figure 1 is a flow chart of a long-distance target ranging method according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0059] Step S101, using an image acquisition device to acquire image data of a target object, and using a carrier inertial navigation to acquire inertial navigation data synchronized with the image data.

[0060] In the embodiment of the present invention, the inertial navigation data is used to represent the motion data of the image acquisition device.

[0061] In one example, image acquisition and inertial navigation data acquisition include: using a monocular camera to acquire image data of a target, and using an inertial navigation system to acquire motion information such as acceleration, angular velocity, and displacement of the camera. The inertial navigation system usually includes an accelerometer and a gyroscope, which can acquire the motion trajectory and attitude change information of the camera.

[0062] Step S102, preprocessing the image data to obtain preprocessed image data.

[0063] In one example, the captured image is preprocessed to perform operations such as denoising, image enhancement, and target detection. The preprocessing method is not limited in the present invention.

[0064] Step S103, determining the spatial position and direction of each frame of image based on the inertial navigation data.

[0065] In one example, based on the three-axis angular velocity and three-axis acceleration output by the inertial navigation system, and the relative position relationship between the camera and the inertial navigation system installed in the system, the three-axis angular velocity and three-axis acceleration of the camera can be calculated. By integrating its angular velocity and acceleration on a time scale, the position relationship between any two moments with a relatively small time span can be obtained, that is, the spatial coordinates and orientations of the images captured at these two moments can be measured.

[0066] Step S104: Based on the preprocessed image data, feature matching is performed on each frame of the image to obtain the same feature points of each frame of the image.

[0067] In one example, in order to measure the distance of a target object, it is necessary to find the same target in the images at any two moments in the above steps, and this requirement can be achieved by matching feature points of consecutive frame images.

[0068] Step S105, calculating the distance of the target object based on the same feature points of each frame image and the inertial navigation data.

[0069] In one example, based on the feature matching results in the two images and the camera position and posture data obtained at the time of capturing the two images in the above steps, a ranging triangle is constructed to perform geometric analysis or a deep learning model is used to calculate the actual distance of the target.

[0070] In one implementation scenario, Figure 2 is a workflow diagram of a distance measurement method based on a monocular camera and an inertial navigation system according to an embodiment of the present invention, such as Figure 2As shown, the ranging method based on a monocular camera and an inertial navigation system includes: installing an optical camera and an inertial navigation device on a movable platform; collecting images, obtaining the position and posture data of the camera at the same time, and preprocessing the images; performing feature matching on the images, constructing a ranging triangle, calculating the distance of the target object and outputting it.

[0071] Specifically, the ranging method based on a monocular camera and an inertial navigation system includes: 1) image acquisition and inertial navigation data acquisition, using a monocular camera to acquire image data of the target, and using the inertial navigation system to acquire motion information such as acceleration, angular velocity and displacement of the camera.

[0072] 2) Image preprocessing: preprocess the collected images and perform operations such as denoising, image enhancement, and target detection.

[0073] 3) Feature matching: by matching the feature points of consecutive frame images, the feature matching results are obtained and the same feature points of each frame image are determined.

[0074] 5) Distance calculation: Based on feature matching results and inertial navigation data, the actual distance of the target is calculated through geometric analysis or deep learning model.

[0075] 6) Result output and display: output the calculated distance information to a display device or storage system.

[0076] Figure 3 is a schematic diagram of an application scenario of a distance measurement system based on a monocular camera and an inertial navigation system according to an embodiment of the present invention, such as Figure 3 As shown, a camera and an inertial navigation device are installed on a movable platform to calculate the target position information of the target object. The ranging system based on a monocular camera and an inertial navigation system includes: 1. A camera module, which is a monocular camera responsible for collecting target images; 2. An inertial navigation module, which includes an accelerometer and a gyroscope, for obtaining the motion information of the camera in real time; 3. An image processing module, which includes an image preprocessing unit and a feature extraction unit; 4. A depth estimation and distance calculation module, which uses an attitude correction and geometric analysis method based on inertial navigation data to calculate the distance of the target; 5. An output module, which displays and records the distance information and the motion trajectory information of the target. The system also includes a posture calculation module, which is used to continuously calculate the position and attitude of the camera based on the inertial navigation data and the installation position relationship between the inertial navigation and the camera.

[0077] The long-distance target ranging method provided in this embodiment combines the image data obtained by the monocular camera with the motion information provided by the inertial navigation system, and adopts the method of exchanging time for space to greatly improve the visual angle difference of the ranging triangle. Compared with the traditional monocular or binocular ranging system, it can achieve high-precision target distance estimation and motion trajectory inference, thereby solving the limitations of the monocular camera in depth measurement, and can effectively improve the ranging accuracy of the single camera for the target, especially the long-distance target. At the same time, it can provide more accurate target distance calculation in dynamic scenes in combination with the movement of the carrier.

[0078] In this embodiment, a long-distance target distance measurement method is provided, which can be used for the above-mentioned computer device. Figure 4 FIG. 4 is a flow chart of another long-distance target distance measurement method according to an embodiment of the present invention. Figure 4 As shown, the process includes the following steps:

[0079] Step S401, using an image acquisition device to acquire image data of a target object, and using a carrier inertial navigation to acquire inertial navigation data synchronized with the image data. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0080] Step S402: preprocess the image data to obtain preprocessed image data. Figure 1 Step S102 of the illustrated embodiment will not be described in detail here.

[0081] Step S403: determining the spatial position and direction of each frame of image based on the inertial navigation data.

[0082] Specifically, the inertial navigation data includes: three-axis angular velocity data, three-axis acceleration data and the relative position relationship between the image acquisition device and the inertial navigation system; the above step S403 includes:

[0083] Step S4031, according to the three-axis angular velocity data, the three-axis acceleration data and the relative position relationship between the image acquisition device and the inertial navigation system, the three-axis angular velocity data and the three-axis acceleration data of the image acquisition device are calculated.

[0084] Step S4032, integrating the three-axis angular velocity data and the three-axis acceleration data of the image acquisition device on a time scale, obtaining the position relationship between the image acquisition device at the current moment and the next moment, and determining the spatial position and direction of the image data at the current moment and the next moment.

[0085] In one example, the three-axis angular velocity and three-axis acceleration of the camera can be calculated based on the three-axis angular velocity and three-axis acceleration output by the inertial navigation system, as well as the relative positional relationship between the camera and the inertial navigation system installed in the system. By integrating the angular velocity and acceleration on a time scale, the positional relationship between the camera at any two moments with a relatively small time span can be obtained, that is, the spatial coordinates and orientation of the images collected at these two moments can be measured. In order to obtain higher measurement accuracy for targets at a longer distance, the displacement values ​​of the camera at the two moments used to measure the distance must be large enough to obtain a larger viewing angle difference. On the other hand, when using inertial navigation data to calculate the camera displacement and attitude, the longer the integration time, the greater the error. Therefore, it is necessary to comprehensively consider the position calculation error and the viewing angle difference requirements to obtain better measurement accuracy.

[0086] Step S404: Based on the preprocessed image data, feature matching is performed on each frame of the image to obtain the same feature points of each frame of the image. Figure 1 Step S104 of the illustrated embodiment will not be described in detail here.

[0087] Step S405: Calculate the distance of the target object based on the same feature points and inertial navigation data of each frame of image. Figure 1 Step S105 of the illustrated embodiment will not be described in detail here.

[0088] In this method, the inertial navigation data collected by the inertial navigation system is used to calculate the position relationship of the camera at any two moments with a relatively small time span, that is, to determine the spatial coordinates and orientation of the images collected at these two moments, providing support for the calculation of the camera position and attitude.

[0089] The long-distance target ranging method provided in this embodiment calculates the position relationship of the camera at any two moments with a relatively small time span through the inertial navigation data collected by the inertial navigation system, that is, determines the spatial coordinates and orientation of the images collected at these two moments, and provides support for the calculation of the camera position and attitude.

[0090] In this embodiment, a long-distance target distance measurement method is provided, which can be used for the above-mentioned computer device. Figure 5 FIG. 4 is a flow chart of another long-distance target ranging method according to an embodiment of the present invention. Figure 5 As shown, the process includes the following steps:

[0091] Step S501, using an image acquisition device to acquire image data of a target object, and using a carrier inertial navigation to acquire inertial navigation data synchronized with the image data. Figure 4 Step S401 of the illustrated embodiment will not be described in detail here.

[0092] Step S502: preprocess the image data to obtain preprocessed image data. Figure 4 Step S402 of the illustrated embodiment will not be described in detail here.

[0093] Step S503: Determine the spatial position and direction of each frame of image based on the inertial navigation data. Figure 4 Step S403 of the illustrated embodiment will not be described in detail here.

[0094] Step S504: Based on the preprocessed image data, feature matching is performed on each frame of the image to obtain the same feature points of each frame of the image.

[0095] Specifically, the above step S504 includes:

[0096] Step S1, perform feature point detection on each frame of image, identify the feature points of each frame of image, and generate feature descriptors corresponding to the feature points.

[0097] In the embodiment of the present invention, the feature descriptor is used to characterize the local features of the feature points.

[0098] In some optional implementations, the above step S1 includes:

[0099] Step a1, calculate and obtain the gradient information of each pixel in each frame image.

[0100] Step a2: Based on the gradient information, corner points whose gradient information difference between adjacent pixels is greater than a preset threshold are screened, and the corner points are determined as feature points.

[0101] In one example, easy-to-track feature points are found in an image. These points are usually high-contrast, texture-rich areas in the image, such as corners, edges, etc. Common feature detection algorithms include Harris corner detection, SIFT (scale-invariant feature transform) and ORB (accelerated robust feature). By calculating the gradient information of each pixel in the image, looking for areas with large changes, corners are detected as feature points.

[0102] Generate a local window with the feature point as the center, analyze the window area, and extract the gradient direction or brightness difference. Match the feature points of the two frames to find the same object or scene area. Calculate the gradient magnitude m(x,y) and direction θ(x,y) for the pixels around the key point. ：

[0103]

[0104] Among them, m(x,y) represents the gradient magnitude, which indicates the intensity change amplitude of the image at the pixel (x,y) position, and is used to measure the size of the pixel gradient. L(x,y) represents the grayscale value of the image after Gaussian blur processing, which is used to reduce the influence of noise. L(x+1,y)-L(x-1,y) represents the pixel grayscale difference in the horizontal direction, and the rate of change of the image in the x direction is calculated by the grayscale values ​​of adjacent pixels. L(x,y+1)-L(x,y-1) represents the pixel grayscale difference in the vertical direction, which indicates the rate of change of the image in the y direction.

[0105] In this method, by finding easy-to-track feature points in the image, the corner points of the high-contrast, texture-rich areas in the image are used as feature points, which makes it easier to identify the target feature points in the image and provides a data basis for subsequent distance calculations.

[0106] Step S2, based on the feature descriptor, matching the feature points of the current frame image with the feature points of the previous frame image, generating matching point pairs between the current frame image and the previous frame image, and obtaining the same feature points of each frame image.

[0107] In one example, in order to measure the distance of a target object, it is necessary to find the same target in the images at any two moments in the above steps. This requirement can be achieved by matching feature points of consecutive frame images, including:

[0108] Step 1: First, find easy-to-track feature points in the image. These points are usually high-contrast, texture-rich areas in the image, such as corners, edges, etc. Common feature detection algorithms include Harris corner detection, SIFT (scale-invariant feature transform) and ORB (accelerated robust feature). By calculating the gradient information of each pixel in the image, looking for areas with large changes, corners are detected as feature points.

[0109] Step 2: After the feature points are detected, a unique "feature descriptor" needs to be generated for each feature point. The descriptor is a local feature of the feature point, such as direction, scale, etc., which can be used to uniquely identify the feature point. Algorithms such as SIFT and SURF provide methods for generating descriptors.

[0110] Step 3: Match the feature points of the consecutive frame with the feature points of the previous frame. Find the matching point pairs between the two frames by calculating the similarity of the feature descriptors (such as Euclidean distance), usually using the nearest neighbor search or K nearest neighbor matching method.

[0111] Step 4: Based on the principle of geometry, the displacement changes of the same feature points photographed by the camera at different positions are used, combined with the camera's motion information (such as rotation and translation) to perform triangulation calculations to calculate the distance of the feature points.

[0112] Step 5: Using the acceleration and angular velocity data provided by the inertial navigation system (INS), the Kalman filter (or extended Kalman filter EKF) is used to fuse the IMU and GPS data, correct the IMU drift, and reduce the error. The relative position and attitude relationship of the camera at any two different moments with a short time span can be calculated.

[0113] In this method, in order to measure the distance of the target object, it is necessary to find the same target in the images at any two moments obtained in the above steps. This can be achieved by matching the feature points of consecutive frame images to provide a data basis for subsequent distance calculation.

[0114] Step S505: Calculate the distance of the target object based on the same feature points of each frame of image and the inertial navigation data.

[0115] Specifically, the above step S505 includes:

[0116] Step S5051, based on the same feature points in each frame of image, determine the imaging position of the target object in each frame of image.

[0117] Step S5052, screening out the first imaging point of the target object in the first image data corresponding to the first moment and the second imaging point of the target object in the second image data corresponding to the second moment, and calculating the first pixel number from the first imaging point to the image center and the second pixel number from the second imaging point to the image center.

[0118] In the embodiment of the present invention, the distance between the first imaging point and the second imaging point is greater than a preset threshold.

[0119] Step S5053, calculating a first angle between a first straight line between the target object and the optical center of the image acquisition device at the first moment and the preset coordinate system, and a second angle between a second straight line between the target object and the optical center of the image acquisition device at the second moment and the preset coordinate system.

[0120] In an embodiment of the present invention, the origin of the preset coordinate system is the optical center of the image acquisition device at the first moment, the X-axis of the preset coordinate system is the normal of the image acquisition device at the first moment, and the Y-axis of the preset coordinate system is parallel to the long side of the camera sensor corresponding to the image acquisition device.

[0121] Step S5054, based on the first angle and the second angle, combined with the inertial navigation data, the coordinates of the target object are calculated to obtain the distance of the target object.

[0122] In an optional implementation, step S5054 includes:

[0123] Step b1, establishing a first equation of a first straight line in a preset coordinate system.

[0124] Step b2, establishing a second equation of the second straight line in the preset coordinate system according to the inertial navigation data at the second moment and the second angle.

[0125] Step b3, combining the first equation and the second equation to calculate the coordinates of the target object.

[0126] Step b4, based on the coordinates of the target object and the inertial navigation data at the second moment, calculate the distance between the target object and the image acquisition device.

[0127] In one example, Figure 6 is a schematic diagram of target distance calculation according to an embodiment of the present invention, in which:

[0128] 1——Image acquisition device (camera for short);

[0129] 2 – Camera lens;

[0130] 3——Measured target;

[0131] 4 – Camera sensor;

[0132] 5——Picture collected at time T1;

[0133] 6——Picture collected at time T2;

[0134] V1——image distance of the camera at time T1;

[0135] U1——object distance of the camera at time T1;

[0136] V2——image distance of the camera at time T2;

[0137] A——The optical center position of the camera lens at time T1;

[0138] B——The optical center position of the camera lens at time T2, and its coordinates x2, y2 are obtained from the camera position and posture data at time T2;

[0139] P——the edge point of the target being measured (assuming it is a point with strong contrast);

[0140] PA——The imaging point of point P on the camera sensor at time T1;

[0141] PB——The imaging point of point P on the camera sensor at time T2;

[0142] PA1——imaging of point P on the image collected at time T1;

[0143] n1——The number of pixels from PA1 to the center of the image;

[0144] PB1——imaging of point P on the image collected at time T2;

[0145] n2——the number of pixels from PA2 to the center of the image;

[0146] α1——Angle between the measured target point and the optical center line AP at time T1 and the X-axis

[0147] α2——Angle between the measured target point and the optical center line BP at time T2 and the X-axis

[0148] γ2——The angle between the normal line of the lens optical center and the line connecting the optical center and the target at time T2

[0149] β2——the angle between the lens normal and the X-axis at time T2, which is obtained from the camera position and attitude data at time T2;

[0150] O——the origin of the established coordinate system;

[0151] w——sensor pixel width;

[0152] like Figure 6 As shown in the figure, assuming that the pixel width of the camera sensor is w, the distance to the target is calculated by triangulation method using image feature point matching combined with the position and posture of the camera during image acquisition obtained in the above steps:

[0153] (1) Perform feature matching in the acquired images according to the aforementioned method to confirm the imaging position of the target P in each frame of the image;

[0154] (2) Select the collected image data and camera position data at time T1 and time T2 with a sufficiently large viewing angle difference (i.e., the difference in the pixel positions corresponding to point P on the image is sufficiently large) as the data source for distance calculation;

[0155] (3) Calculate the number of pixels n1 and n2 from the image center of points PA1 and PB1 respectively based on the result of feature matching;

[0156] (4) Establish a coordinate system with the optical center of the lens at time T1 as the origin, the lens normal as the X-axis, and the straight line parallel to the long side of the sensor passing through the optical center as the Y-axis, with the origin being O;

[0157] (5) According to the image distance V1 at time T1 and the position of the image of point P on the sensor, calculate the angle α1 between the straight line AP and the X-axis:

[0158] α1=arctan(n1*w / V1

[0159] (6) Establish the equation of line AP in the coordinate system:

[0160] yu=xu×tan(α1)

[0161] (7) According to the image distance V2 at time T2, the position of the image of point P on the sensor, and the camera deflection angle (the angle between the lens normal and the X-axis) β2 at time T2, calculate the angle α2 between the straight line BP and the X-axis:

[0162] α2=γ2-β2=arctan(n2*w / V2)-β2

[0163] (8) Establish the equation of the straight line BP in the coordinate system based on the camera position data (x2, y2) at time T2 and α2;

[0164] yu=y2+(xu-x2)×tan(α2)(9) Combining the equations of line AP and BP, we can calculate the coordinates of point P (xu, yu):

[0165]

[0166] (10) Thus, the distance |BP| between point P and the camera is obtained:

[0167]

[0168] In this method, the displacement changes of the same feature points photographed by the image acquisition device at different positions are combined with the camera's inertial navigation data to perform triangulation calculations to calculate the distance of the feature points, thereby calculating the distance to the target object.

[0169] The long-distance target distance measurement method provided in this embodiment needs to find the same target in the images at any two moments obtained in the above steps in order to measure the distance of the target object. This can be achieved by matching the feature points of consecutive frame images, providing a data basis for subsequent distance calculations. By finding easy-to-track feature points in the image, the corner points of high-contrast and texture-rich areas in the image are used as feature points, which facilitates the identification of target feature points in the image and provides a data basis for subsequent distance calculations. The displacement changes of the same feature points captured by the image acquisition device at different positions are combined with the inertial navigation data of the camera to perform triangulation calculations, and the distance of the feature points is calculated, thereby realizing the calculation of the distance of the target object.

[0170] In the present embodiment, a long-distance target distance measuring device is also provided, and this device is used to realize the above-mentioned embodiment and preferred implementation mode, and the description that has been made is not repeated. As used below, the term "module" can realize the combination of software and / or hardware of predetermined function. Although the device described in the following embodiments is preferably realized with software, the realization of hardware, or the combination of software and hardware is also possible and conceived.

[0171] This embodiment provides a long-distance target distance measurement device, such as Figure 7 As shown, including:

[0172] The data acquisition module 701 is used to acquire the image data of the target object using the image acquisition device, and to acquire the inertial navigation data synchronized with the image data using the carrier inertial navigation. The inertial navigation data is used to characterize the motion data of the image acquisition device. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0173] The image preprocessing module 702 is used to preprocess the image data to obtain preprocessed image data. Figure 1 Step S102 of the illustrated embodiment will not be described in detail here.

[0174] The attitude calculation module 703 is used to determine the spatial position and direction of each frame of image based on the inertial navigation data. Figure 1 Step S103 of the illustrated embodiment will not be described in detail here.

[0175] The feature matching module 704 is used to perform feature matching on each frame of the image based on the preprocessed image data to obtain the same feature points of each frame of the image. Figure 1 Step S104 of the illustrated embodiment will not be described in detail here.

[0176] The distance calculation module 705 is used to calculate the distance of the target object based on the same feature points and inertial navigation data of each frame of the image. Figure 1 Step S105 of the illustrated embodiment will not be described in detail here.

[0177] In some optional implementations, the inertial navigation data includes: three-axis angular velocity data, three-axis acceleration data and the relative position relationship between the image acquisition device and the inertial navigation system; the attitude calculation module 703 includes:

[0178] The first calculation unit is used to calculate the three-axis angular velocity data and the three-axis acceleration data of the image acquisition device according to the three-axis angular velocity data, the three-axis acceleration data and the relative position relationship between the image acquisition device and the inertial navigation system.

[0179] The position and direction determination unit is used to integrate the three-axis angular velocity data and three-axis acceleration data of the image acquisition device on a time scale to obtain the position relationship between the image acquisition device at the current moment and the next moment, and determine the spatial position and direction of the image data at the current moment and the next moment.

[0180] In some optional implementations, the feature matching module 704 includes:

[0181] The feature point recognition unit is used to detect feature points of each frame of image, identify the feature points of each frame of image, and generate feature descriptors corresponding to the feature points. The feature descriptors are used to characterize the local features of the feature points.

[0182] The matching point pair generating unit is used to match the feature points of the current frame image with the feature points of the previous frame image based on the feature descriptor, generate matching point pairs between the current frame image and the previous frame image, and obtain the same feature points of each frame image.

[0183] In some optional implementations, the feature point recognition unit includes:

[0184] The gradient information calculation subunit is used to calculate the gradient information of each pixel in each frame of the image.

[0185] The feature point determination subunit is used to screen corner points whose gradient information difference between adjacent pixel points is greater than a preset threshold based on gradient information, and determine the corner points as feature points.

[0186] In some optional implementations, the distance calculation module 705 includes:

[0187] The imaging position determination unit is used to determine the imaging position of the target object in each frame of image based on the same feature points in each frame of image.

[0188] A pixel number calculation unit is used to screen a first imaging point of a target object in first image data corresponding to a first moment and a second imaging point of the target object in second image data corresponding to a second moment, and calculate a first pixel number from the first imaging point to the image center and a second pixel number from the second imaging point to the image center; the distance between the first imaging point and the second imaging point is greater than a preset threshold.

[0189] An angle calculation unit is used to calculate a first angle between a first straight line between a target object and an optical center of an image acquisition device at a first moment and a preset coordinate system, and a second angle between a second straight line between the target object and the optical center of the image acquisition device at a second moment and the preset coordinate system; the origin of the preset coordinate system is the optical center of the image acquisition device at the first moment, the X-axis of the preset coordinate system is the normal of the image acquisition device at the first moment, and the Y-axis of the preset coordinate system is parallel to the long side of the camera sensor corresponding to the image acquisition device.

[0190] The distance calculation unit is used to calculate the coordinates of the target object based on the first angle and the second angle in combination with the inertial navigation data, and calculate the distance of the target object.

[0191] In some optional implementations, the distance calculation unit includes:

[0192] The first equation establishing subunit is used to establish a first equation of the first straight line in a preset coordinate system.

[0193] The second equation establishing subunit is used to establish a second equation of the second straight line in the preset coordinate system according to the inertial navigation data at the second moment and the second angle.

[0194] The coordinate calculation subunit is used to solve the first equation and the second equation to calculate the coordinates of the target object.

[0195] The distance calculation subunit is used to calculate the distance between the target object and the image acquisition device based on the coordinates of the target object and the inertial navigation data at the second moment.

[0196] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0197] The long-distance target ranging device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0198] The embodiment of the present invention also provides a computer device having the above Figure 7 The long-distance target ranging device shown.

[0199] See also Figure 8 , Figure 8 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 8 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 8 A processor 10 is taken as an example.

[0200] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0201] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0202] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0203] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0204] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 8 The example of connecting through bus is taken in the following.

[0205] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0206] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0207] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0208] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for measuring the distance of a long-distance target, characterized in that: The method comprises: Using an image acquisition device to acquire image data of a target object, and using a carrier inertial navigation to acquire inertial navigation data synchronized with the image data, wherein the inertial navigation data is used to characterize motion data of the image acquisition device; Preprocessing the image data to obtain preprocessed image data; Based on the inertial navigation data, determining the spatial position and direction of each frame of image; Based on the preprocessed image data, feature matching is performed on each frame of the image to obtain the same feature points of each frame of the image; The distance of the target object is calculated based on the same feature points of each frame of the image and the inertial navigation data.

2. The method according to claim 1, characterized in that The inertial navigation data includes: three-axis angular velocity data, three-axis acceleration data and the relative position relationship between the image acquisition device and the inertial navigation system; the spatial position and direction of each frame of image is determined based on the inertial navigation data, including: Calculate the three-axis angular velocity data and the three-axis acceleration data of the image acquisition device according to the three-axis angular velocity data, the three-axis acceleration data and the relative position relationship between the image acquisition device and the inertial navigation system; The three-axis angular velocity data and three-axis acceleration data of the image acquisition device are integrated on a time scale to obtain the positional relationship between the image acquisition device at the current moment and the next moment, and determine the spatial position and direction of the image data at the current moment and the next moment.

3. The method according to claim 2, characterized in that The performing feature matching on each frame of image based on the preprocessed image data to obtain the same feature points of each frame of image includes: Perform feature point detection on each frame of image, identify feature points of each frame of image, and generate feature descriptors corresponding to the feature points, wherein the feature descriptors are used to characterize local features of the feature points; Based on the feature descriptor, feature points of the current frame image and feature points of the previous frame image are matched to generate matching point pairs between the current frame image and the previous frame image, and obtain the same feature points of each frame image.

4. The method according to claim 3, characterized in that The detecting of feature points of each frame of image to identify the feature points of each frame of image includes: Calculate the gradient information of each pixel in each frame of image; Based on the gradient information, corner points whose gradient information difference between adjacent pixels is greater than a preset threshold are screened, and the corner points are determined to be the feature points.

5. The method according to claim 1, characterized in that The calculating the distance of the target object based on the same feature points of each frame of the image and the inertial navigation data includes: Based on the same feature points in each frame of image, determining the imaging position of the target object in each frame of image; Screening out a first imaging point of the target object in the first image data corresponding to the first moment and a second imaging point of the target object in the second image data corresponding to the second moment, and calculating a first pixel number from the first imaging point to the image center and a second pixel number from the second imaging point to the image center; the distance between the first imaging point and the second imaging point is greater than a preset threshold; Calculate a first angle between a first straight line between the target object and the optical center of the image acquisition device at the first moment and a preset coordinate system, and a second angle between a second straight line between the target object and the optical center of the image acquisition device at the second moment and the preset coordinate system; the origin of the preset coordinate system is the optical center of the image acquisition device at the first moment, the X-axis of the preset coordinate system is the normal of the image acquisition device at the first moment, and the Y-axis of the preset coordinate system is parallel to the long side of the camera sensor corresponding to the image acquisition device; Based on the first angle and the second angle, in combination with the inertial navigation data, the coordinates of the target object are calculated to obtain the distance of the target object.

6. The method according to claim 5, characterized in that The calculating the coordinates of the target object based on the first angle and the second angle in combination with the inertial navigation data to obtain the distance of the target object includes: Establishing a first equation of the first straight line in the preset coordinate system; Establishing a second equation of the second straight line in the preset coordinate system according to the inertial navigation data at the second moment and the second angle; The first equation and the second equation are combined to calculate the coordinates of the target object; Based on the coordinates of the target object and the inertial navigation data at the second moment, the distance between the target object and the image acquisition device is calculated.

7. A long-distance target distance measuring device, characterized in that: The device comprises: A data acquisition module, used to acquire image data of a target object using an image acquisition device, and to acquire inertial navigation data synchronized with the image data using a carrier inertial navigation, wherein the inertial navigation data is used to characterize motion data of the image acquisition device; An image preprocessing module, used to preprocess the image data to obtain preprocessed image data; An attitude calculation module, used to determine the spatial position and direction of each frame of image based on the inertial navigation data; A feature matching module is used to perform feature matching on each frame of the image based on the preprocessed image data to obtain a feature matching result; The distance calculation module is used to calculate the distance of the target object based on the feature matching result and the inertial navigation data.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the long-distance target ranging method according to any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the long-distance target ranging method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The method comprises computer instructions, wherein the computer instructions are used to cause a computer to execute the long-distance target ranging method according to any one of claims 1 to 6.

Images