Image space target ranging algorithm based on deep calibration learning

Abstract

The invention belongs to the field of image space target ranging algorithms, and particularly relates to an image space target ranging algorithm based on deep calibration learning, which comprises picture input, target detection, calibration function, parameter solution and a space target distance calculator. The invention provides an image space target ranging algorithm based on deep calibration learning, on one hand, the position of a target is detected through a target detection model, and the absolute position and the relative position of the target in an image are calculated; on the other hand, the real distance between the two targets in the real physical space is measured at the same time. And then establishing a corresponding relation between an image distance and a real distance through a large amount of annotation data in a real physical space.

Description

Technical Field

[0001] The present invention relates to the technical field of image space target ranging algorithms, and specifically provides an image space target ranging algorithm based on depth calibration learning. Background Art

[0002] (1) With the rapid development of computer vision technology, image space target ranging technology has been widely used in multiple fields. Especially in scenarios such as security monitoring, autonomous driving, robot navigation, and augmented reality (AR), the accurate measurement of the target distance has become a key requirement. For example, in security monitoring: In public places or industrial environments, real-time measurement of the distance between targets in the monitoring screen can provide important data support for security warning, behavior analysis, etc. In autonomous driving: Autonomous vehicles need to accurately perceive the surrounding environment and measure the distance between the vehicle and pedestrians, as well as between the vehicle and other vehicles to ensure driving safety. In robot navigation: When a robot navigates in a complex environment, it needs to measure the distance to obstacles to plan the optimal path. In augmented reality (AR): In AR applications, the measurement of the distance between virtual objects and real objects can enhance the realism of the user experience. However, traditional image ranging methods have many deficiencies in terms of accuracy, robustness, and intelligence, and it is difficult to meet the ranging requirements of high precision, real-time performance, and complex environments in the above scenarios.

[0003] (2) Traditional image space target ranging methods mainly rely on geometric models and feature matching techniques, and have the following technical pain points: 1. Low accuracy: Traditional methods are limited by factors such as image resolution, noise, and illumination changes, and it is difficult to achieve high-precision distance measurement. In complex scenarios, problems such as occlusion and deformation between targets further reduce the ranging accuracy. 2. Poor robustness: Traditional methods are sensitive to environmental factors such as illumination changes, background interference, and target occlusion, and have poor robustness. In dynamic scenarios, the rapid movement or deformation of the target will cause the ranging result to be unstable. 3. Lack of intelligence: Traditional methods usually require manual intervention or complex calibration processes, and it is difficult to achieve automation and intelligence. In multi-target scenarios, traditional methods are difficult to simultaneously process the distance measurement of multiple targets and cannot establish the mapping relationship between image distance and real distance. 4. Limited applicability: Traditional methods are usually designed for specific scenarios and are difficult to generalize to different application scenarios. In complex environments (such as at night, rainy or snowy weather, etc.), the performance of traditional methods drops significantly. Summary of the Invention

[0004] The purpose of the present invention is to provide an image space target ranging algorithm based on depth calibration learning, which solves the problems of low accuracy, poor robustness, and lack of intelligence in traditional image ranging methods, and provides a high-precision and intelligent distance measurement solution for scenarios such as security monitoring, autonomous driving, and robot navigation.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] An image space target ranging algorithm based on deep calibration learning, which respectively includes an input picture, target detection, a calibration function, parameter solution, and a space target distance calculator;

[0007] The main steps for implementing the algorithm are designed as follows:

[0008] (1) Target detection: Considering the complexity of the real scene and the multi-scale attributes of the detection target, YOLOV5 is selected as the target detection algorithm, and the model is trained with real scene data;

[0009] (2) Calibration function: The image space distance d(k), the camera resolution parameter α, and the distortion parameter β are used as the independent variables of the calibration function, and the physical space distance is used as the dependent variable of the calibration function. The calibration function is shown in the following formula (1);

[0010]

[0011] (3) Space ranging: The central points of the closer sides of the two targets are used as the starting and ending points of the measurement; The physical ranging represents The absolute value of the distance is the midpoint connection of the vertical lines of the inner edges of the two human body targets; Therefore, from the perspective of a two-dimensional image, the image ranging d(k) needs to construct a right triangle equation based on the target detection coordinate information; For example, the upper left and lower right coordinate information of two human body targets are respectively and According to the coordinate information of the two, its absolute position d(k) in the image can be calculated as shown in formula (2):

[0012]

[0013] In the formula, w represents the horizontal distance between the central points of the two targets, and h represents the vertical distance; represents the distance between the upper left and lower right corners of the first target, represents the distance between the upper left and lower right corners of the second target; Considering that the target distance is independent of the direction, therefore, w and h take absolute values;

[0014] (4) Solving deep network parameters: According to the real physical distance corresponding to the target in the provided batch of images and the image space distance obtained through the target detection model, parameter deep fitting is performed through the calibration function; First, determine the camera resolution parameter α and the distortion parameter β; Then specify the batch data n for training the calibration function; On the one hand, use a high-precision measurement tool to measure the real distance of the real physical space target Meanwhile, the captured image is sent into the target detection model to obtain coordinate information, and the image space distances d1(k),... d n (k) are calculated through formula (2); then, based on the above-known data, a calibration function is fitted, and a deep neural network is used as the fitting function; finally, by taking d(k), α, and β as the model inputs, as the model outputs, the neural network parameters are iteratively trained on the batch dataset n to obtain the parameter weight values;

[0015] (5) Spatial target distance calculator: Step (4) determines to use the deep neural network as the calibration function, and based on the batch data, the model parameters are trained. The model parameters are loaded as weights into the calibration function to form the spatial target distance calculator. This calculator can automatically calculate the distance between any two targets in the real physical space in the image according to the camera resolution, distortion parameters, and image input by the user.

[0016] Preferably, the calibration function is jointly composed of two modules: image ranging and real-world ranging.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0018] The present invention proposes an image spatial target ranging algorithm based on deep calibration learning. On the one hand, the target position is detected through the target detection model, and the absolute and relative positions of the target in the image are calculated. On the other hand, the real distances between two targets in the real physical space are measured simultaneously. Then, through a large number of labeled data in the real physical space, the corresponding relationship between the image distance and the real distance is established. And according to the deep calibration learning strategy, the distance mapping function between the distance of the target in the image and the real physical space is solved. At the same time, the system errors caused by factors such as camera resolution and distortion are added as correction factors to the distance mapping function to ensure that the designed distance solver has generalization and accuracy. This method has strong robustness and generalization ability and can meet the ranging requirements in complex environments such as light changes and occlusions. This technology has broad application prospects and important practical significance. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is the implementation flowchart of the present invention;

[0020] Figure 2 is the process diagram for solving the parameters of the present invention;

[0021] Figure 3 is the effect diagram of applying the algorithm of the present invention to the measurement and analysis of the safe operation distance of the power grid. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0023] Please refer to Figures 1-3 , an image space target ranging algorithm based on depth calibration learning, which respectively includes input pictures, target detection, calibration function, parameter solving, and space target distance calculator;

[0024] The main steps for implementing the algorithm are designed as follows:

[0025] (1) Target detection: Considering the complexity of the real scene and the multi-scale attributes of the detection target, YOLOV5 is selected as the target detection algorithm, and the model is trained on real scene data;

[0026] (2) Calibration function: Through a large number of experiments, this method proves that the target distance in the image space is the main calibration relationship for measuring the target distance in the physical space. In addition, the camera resolution and camera distortion parameters will affect the calibration accuracy. Taking the image space distance d(k), camera resolution parameter α, and distortion parameter β as the independent variables of the calibration function, and the physical space distance as the dependent variable of the calibration function, the calibration function is shown in the following formula (1);

[0027]

[0028] (3) Space ranging: The center points of the closer sides of the two targets are used as the starting and ending points for measurement; Physical ranging means The absolute value of the distance is the midpoint connection of the vertical lines of the inner edges of the two human targets; Therefore, from a two-dimensional image perspective, the image ranging d(k) needs to construct a right triangle equation based on the target detection coordinate information; For example, the upper left and lower right coordinate information of two human targets are respectively and According to the coordinate information of the two, the absolute position d(k) in the image can be calculated as shown in formula (2):

[0029]

[0030] In the formula, w represents the horizontal distance between the center points of the two targets, and h represents the vertical distance; represents the distance between the upper left and lower right corners of the first target, Indicates the distance between the upper left corner and the lower right corner of the second target; considering that the target distance is independent of direction, therefore, w and h take absolute values;

[0031] (4) Solving the deep network parameters: According to the real physical distance corresponding to the target in the provided batch of images and the image space distance obtained through the target detection model, perform parameter depth fitting through the calibration function; first, determine the camera resolution parameter α and the distortion parameter β; then specify the batch data n for training the calibration function; on the one hand, measure the real distance of the target in the real physical space with a high-precision measurement tool At the same time, send the captured image into the target detection model to obtain coordinate information, and calculate the image space distances d1(k),...d n (k) through formula (2); then fit the calibration function according to the above-known data, and use a deep neural network as the fitting function; finally, by taking d(k), α, and β as the model inputs, as the model output, iterate and train the neural network parameters on the batch data set n to obtain the parameter weight values;

[0032] (5) Spatial target distance calculator: Step (4) determines to use the deep neural network as the calibration function, and trains the model parameters according to the batch data, and loads the model parameters as weights into the calibration function to form a spatial target distance calculator. This calculator can automatically calculate the distance between any two targets in the real physical space in the image according to the camera resolution, distortion parameters, and image input by the user.

[0033] Among them, the calibration function is jointly composed of two modules: image ranging and real-world ranging.

[0034] The general process of implementing this algorithm is as follows:

[0035]

[0036] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An image space target ranging algorithm based on deep calibration learning, characterized in that: They respectively include an input image, object detection, a calibration function, parameter solving, and a spatial object distance calculator; The main steps of the implementation algorithm are designed as follows: (1) Object detection: Considering the complexity of the real scene and the multi-scale attributes of the detection targets, YOLOV5 is selected as the object detection algorithm, and the model is trained with real scene data; (2) Calibration function: Taking the image space distance d(k), the camera resolution parameter α, and the distortion parameter β as the independent variables of the calibration function, and the physical space distance as the dependent variable of the calibration function, the calibration function is shown in the following formula (1); (3)Spatial ranging: Use the central points on the closer sides of the two targets as the starting and ending points for measurement; Physical ranging means The absolute value of the distance is the line connecting the midpoints of the perpendicular lines from the inner edges of the two human targets; Therefore, from a two-dimensional image perspective, the image ranging d(k) needs to construct a right triangle equation based on the target detection coordinate information; For example, the upper left and lower right coordinate information of the two human targets are respectively and The absolute position d(k) of the object in the image can be calculated based on the coordinate information of the two, as shown in Equation (2): In the figure, w represents the horizontal distance between the centers of two targets, and h represents the vertical distance; represents the distances from the upper left corner to the lower right corner of the first target, represents the distances from the upper left corner to the lower right corner of the second target; considering that the target distance is independent of direction, therefore, the absolute values of w and h are taken; (4) Solving deep network parameters: According to the true physical distance corresponding to the target in the provided batch of images and the image space distance obtained through the target detection model, perform parameter depth fitting through the calibration function; first determine the camera resolution parameter α and the distortion parameter β; then specify the batch data n for training the calibration function; on the one hand, measure the true distance of the target in the true physical space with a high-precision measurement tool At the same time, send the captured image into the target detection model to obtain coordinate information, and calculate the image space distances d1(k),...d n (k) through formula (2); then fit the calibration function according to the above known data, and use a deep neural network as the fitting function; finally, by taking d(k), α, β as the model input, as the model output, iterate and train the neural network parameters on the batch dataset n to obtain the parameter weight values; (5) Spatial object distance calculator: In step (4), the deep neural network is determined as the calibration function, and the model parameters are trained based on batch data. The model parameters are loaded as weights into the calibration function to form the spatial object distance calculator. This calculator can automatically calculate the distance between any two targets in the image in the real physical space according to the camera resolution, distortion parameters, and image input by the user.

2. The image space target ranging algorithm based on deep calibration learning according to claim 1, wherein: The calibration function is jointly composed of two modules: image ranging and real-world ranging.

Images