Methods, systems, and media for measuring true height using a monocular camera

Abstract

This application provides a method, system, and medium for measuring true height based on a monocular camera, relating to the field of height measurement technology. The method includes: first, acquiring camera intrinsic parameters, including distortion parameters and principal point coordinates; second, acquiring a target image using a camera module and obtaining pitch angle data from an IMU component; then, preprocessing the image based on the intrinsic parameters to generate a denoised image and an edge image; next, inputting the processed image into an inference neural network composed of a backbone network, a feature pyramid, and a detection head to accurately locate the target's ground point and vertex coordinates; finally, based on these coordinates, pitch angle data, and intrinsic parameters, calculating the true physical height of the target object using a geometric calculation module. Implementing the method in this application avoids the limitations of traditional methods that rely on additional reference points, significantly reducing the cost of the measurement device.

Description

Technical Field

[0001] This application relates to the field of height measurement technology, and in particular to a method, system and medium for measuring real height based on a monocular camera. Background Technology

[0002] In the field of computer vision and measurement technology, target height measurement technology based on monocular cameras is widely used in various scenarios such as industrial inspection, security monitoring, and topographic mapping due to its cost-effectiveness and ease of deployment.

[0003] Existing height measurement methods based on monocular cameras revolve around parameter calibration, feature mapping, and geometric calculation. When the calibration object is deployed in a standardized manner and the prior knowledge of the target is clear (such as in a laboratory environment or a standardized production line), high measurement accuracy can be achieved (the error can be controlled at the millimeter level). Moreover, the algorithm logic is relatively mature and easy to implement in engineering.

[0004] However, such methods have significant limitations. The measurement of the target height must rely on specific calibration objects and prior knowledge of the target, making them unsuitable for accurate measurement in unconstrained scenarios. Summary of the Invention

[0005] This application provides a method, system, and medium for measuring real height based on a monocular camera, which addresses the problems of high equipment cost and reliance on additional reference objects in traditional height measurement methods.

[0006] Firstly, this application provides a method for measuring true height based on a monocular camera. The method includes: acquiring the camera's intrinsic parameters, which at least include radial distortion parameters, tangential distortion parameters, and the abscissa of the principal point in the image coordinate system. and ordinate The system acquires a single-frame image of the target object via a camera module and obtains the angle data from the IMU inertial navigation component in the most recent input module. The pitch angle data of the single-frame image is then calculated. Based on the camera's intrinsic parameters, the single-frame image is preprocessed to obtain a first processed image and a second processed image. The first processed image contains an image region after random noise removal, and the second processed image is an edge image region containing the key target edge. The first and second processed images are input into a pre-set inference neural network in the image feature calculation module to obtain the grounding point pixel coordinates and vertex pixel coordinates of the target object on the single-frame image. This inference neural network is pre-trained and contains a three-level structure: a backbone network, a feature pyramid, and a detection head. Based on the grounding point pixel coordinates, vertex pixel coordinates, pitch angle data, and camera intrinsic parameters, the physical height of the target object is calculated using a geometric calculation module.

[0007] By employing the above-described technical solution, a monocular camera and an IMU inertial navigation system are integrated to achieve high-precision object height measurement without any additional reference points. The neural network accurately locates the target's ground point and vertices, avoiding the limitations of traditional methods that rely on reference points of known dimensions. Pre-calibration of the camera's intrinsic parameters ensures the accuracy of the measurement results, while the pitch angle data provided by the IMU solves the camera attitude estimation problem. This combination of technologies significantly reduces the complexity and cost of the measurement device while maintaining high measurement accuracy. Compared with traditional binocular vision, LiDAR, and other technologies, this method simplifies the hardware configuration, improves system stability, and expands application scenarios.

[0008] Secondly, this application provides a height measurement system, characterized in that it includes: a camera intrinsic parameter acquisition module, used to acquire camera intrinsic parameters of a monocular camera, wherein the camera intrinsic parameters include at least radial distortion parameters, tangential distortion parameters, and the abscissa of the principal point in the image coordinate system. and ordinate The system includes: a pitch angle data acquisition module for acquiring a single-frame image of the target through the camera module and acquiring the angle data of the IMU inertial navigation component most recently input into the module, and calculating the pitch angle data of the single-frame image; an image region determination module for preprocessing the single-frame image of the target based on the camera's intrinsic parameters to obtain a first processed image and a second processed image, wherein the first processed image contains an image region after removing random noise and the second processed image is an edge image region containing the edge of the key target; a pixel coordinate acquisition module for inputting the first processed image and the second processed image into a preset inference neural network in the image feature calculation module to obtain the ground point pixel coordinates and vertex pixel coordinates of the target object on the single-frame image, wherein the inference neural network is pre-trained and contains a three-level structure of a backbone network, a feature pyramid, and a detection head; and a true height determination module for calculating the physical true height of the target object based on the ground point pixel coordinates, the vertex pixel coordinates, the pitch angle data, and the camera's intrinsic parameters through the geometric calculation module.

[0009] Thirdly, this application provides a height measurement system, characterized in that the height measurement system includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the height measurement system to perform the method described in the first aspect and any possible implementation thereof.

[0010] Fourthly, this application provides a computer-readable storage medium including instructions that, when executed on a height measurement system, cause the height measurement system to perform the method described in the first aspect and any possible implementation thereof. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of an application framework of the height measurement system in this application embodiment;

[0012] Figure 2 This is a general flowchart of a method for measuring the true height based on a monocular camera in the embodiments of this application;

[0013] Figure 3 This is a flowchart illustrating a method for measuring the true height based on a monocular camera in an embodiment of this application.

[0014] Figure 4 This is a schematic diagram of a network structure of an inference neural network in an embodiment of this application;

[0015] Figure 5 This is a schematic diagram of a positioning correction scenario in an embodiment of this application;

[0016] Figure 6 This is a schematic diagram of another application scenario of the real height measurement method based on a monocular camera in the application embodiments;

[0017] Figure 7 This is a schematic diagram of the physical device structure of a height measurement system in an embodiment of this application. Detailed Implementation

[0018] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to and includes any or all possible combinations of one or more of the listed items.

[0019] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0020] For ease of understanding, the height measurement system of this application will first be described below. Please refer to... Figure 1 This is a schematic diagram of an application framework for the height measurement system in this application embodiment.

[0021] Figure 1 This diagram illustrates a height measurement system using a monocular camera and an IMU (Inertial Measurement Unit). The system consists of two main hardware components: a monocular camera (labeled "camera" in the diagram) and an IMU mounted on top of the camera. The upper diagram shows a side view of the system, including: a monocular camera device with an IMU; the camera capturing a target (a bird silhouette in the diagram); a horizontal dashed line representing the camera's horizontal line of sight; the label "Camera Height" on the left indicating the vertical distance from the camera to the ground; and the label "Ground" at the bottom indicating the reference plane. A downward-facing laser rangefinder can be fixed to a pre-defined area around the camera (either on the left or right sides adjacent to the camera). When the laser beam is emitted towards the ground and reflects back, it returns to the receiving end of the laser rangefinder. By calculating the time difference between laser emission and reception, or through other optical measurement principles, the laser rangefinder can very accurately calculate the vertical distance from the laser rangefinder to the ground. This vertical distance value actually represents the height of the camera itself above the ground. Because the laser rangefinder is fixed to the camera, we obtain accurate data on the camera's installation height.

[0022] The lower figure is a schematic diagram of a coordinate system within a box, showing: a three-dimensional coordinate system (x, y, z axes); the positions of the IMU and camera within the coordinate system; the measurement axes in each direction; and the camera pointing towards the z-axis. This figure visually illustrates how the device, through the cooperation of a single camera and IMU sensor, can measure the height of a target object.

[0023] The following diagram, based on the application framework diagram above, illustrates the general flow of this application. Please refer to [link / reference]. Figure 2 , Figure 2 This is a general flowchart illustrating a method for measuring the true height based on a monocular camera in an embodiment of this application.

[0024] exist Figure 2 In terms of the input module, the process begins with three key inputs: input camera intrinsic parameters, including focal length, pixel size, distortion coefficient, and other internal camera parameters obtained through pre-calibration; input camera height, the vertical height of the measuring device above the ground; and input IMU data, angle data obtained from the inertial measurement unit, especially the pitch angle. After processing these input data, the system will identify the target to be detected and determine the object whose height needs to be measured.

[0025] Regarding the image preprocessing module, after receiving an image frame, the system performs a series of preprocessing steps: Image preprocessing: distortion correction of the original image; Extraction of region of interest: extraction and expansion of the target region to be detected based on user markings or automatic recognition; Guided filtering and edge extraction: noise removal using a guided filter while preserving edge information, generating two images: the denoised image and the edge image.

[0026] In terms of the feature detection and calculation module, this stage uses neural networks to detect target feature points: the neural network detects target vertices and ground points: the input is a preprocessed image, and the target is located through a multi-scale feature fusion network; the output is vertex information and ground point information: the network achieves sub-pixel-level accurate positioning through heatmap regression and coordinate offset prediction techniques; confidence evaluation: the reliability of the detection results is evaluated.

[0027] In terms of the geometric calculation module, the actual height of the target is calculated based on geometric relationships: Height calculation: Using the detected vertex and ground point coordinates, combined with the camera height and the pitch angle provided by the IMU, the actual physical height is calculated using a geometric model; Output results: Displaying the final calculated target height value; The unique feature of this system is that it can measure the true height of an object without the need for reference objects or prior knowledge of the target, using only a monocular camera and IMU data. This solves the problems of traditional methods, such as high equipment requirements, poor environmental adaptability, and the need for reference objects, and can be applied to mobile devices such as inspection robots and drones for real-time measurement.

[0028] The following provides a more detailed process description of the method provided in this implementation. Please refer to [link / reference]. Figure 3 This is a flowchart illustrating a method for measuring the true height based on a monocular camera in an embodiment of this application.

[0029] S301. Obtain the camera parameters of the monocular camera;

[0030] A monocular camera is a camera device with a single lens. Camera intrinsic parameters refer to the set of parameters describing the imaging geometry of the camera. These typically include: radial distortion parameters, which describe the image bending caused by the spherical structure of the lens, such as stretching or shrinking at the edges of the image; tangential distortion parameters, which describe the image shift or tilt caused by misalignment of the camera lens; and principal point x and y coordinates, which refer to the actual position of the intersection of the optical axis and the imaging plane on the image sensor, usually given in pixels. These parameters determine how to correctly map points in the real world to pixel positions on the image and are the basis for all subsequent geometric calculations.

[0031] First, the monocular camera is pre-calibrated to calculate its intrinsic parameters. The camera's intrinsic parameters do not change with the external environment; once the camera's optical system is established, its intrinsic parameters are determined. We only need to use algorithms to obtain the camera's intrinsic parameter matrix. and the radial distortion parameters of the lens , , and tangential distortion parameters , It has been calibrated. Among them, It represents the equivalent focal length (in pixels) in the x-axis direction of the image coordinate system. Its physical meaning is the physical focal length (mm) divided by the physical size of a single pixel of the image sensor in the x-axis direction (mm / pixel). It determines the imaging scaling ratio and field of view in the x-direction. This represents the equivalent focal length (in pixels) along the y-axis of the image coordinate system. Physically, it is the physical focal length (mm) divided by the physical size of a single pixel on the image sensor along the y-axis (mm / pixel). It determines the imaging scaling and field of view in the y-direction. It should be noted that... and They are usually very close (if the pixel is square), but if the sensor pixel is rectangular or the image has anisotropic distortion, the two values ​​will differ. This represents the x-axis coordinate (in pixels) of the principal point in the image pixel coordinate system. Physically, it is the x-coordinate of the intersection of the camera's optical axis and the imaging plane (the projection of the optical center onto the image). It is the y-axis coordinate (in pixels) of the principal point in the image pixel coordinate system. Physically, it is the vertical coordinate of the intersection of the camera optical axis and the imaging plane (the projection point of the optical center on the image).

[0032] Acquiring the intrinsic parameters of a monocular camera typically occurs during the initial system deployment phase or after lens replacement, and is considered part of the camera "calibration" process. During this phase, the data processing module can guide the user to photograph a calibration board with known dimensions and patterns (such as a checkerboard) using the camera, acquiring a series of images from multiple angles and distances. Then, it can directly use calibration functions from MATLAB toolboxes or OpenCV to calibrate the camera and directly calculate the intrinsic parameters. , , , and radial distortion parameters , , and tangential distortion parameters , .

[0033] S302. Acquire a single-frame image of the target through the camera module, and acquire the angle data of the IMU inertial navigation component in the most recent input module, and calculate the pitch angle data of the single-frame image of the target.

[0034] Here, pitch angle data refers to the angle value describing the degree of tilt of the device in the vertical direction, that is, the angle between the vertical direction of the camera and the horizontal plane; calculation refers to the process of processing and converting the raw sensor data, specifically converting the IMU's angular data into the required pitch angle data; pitch angle data describes the angle of rotation of the camera around the horizontal axis, used to characterize the degree of tilt of the camera relative to the horizontal plane, with positive values ​​indicating upward tilt and negative values ​​indicating downward tilt. The module receives an image frame from the camera. The IMU's data refresh rate is much higher than that of the image frame, so the most recent IMU angular data is obtained, and the calculated pitch angle is used as the pitch angle θ of the current image frame.

[0035] More specifically, after the system starts up and enters measurement mode, when it is necessary to measure the height of a target object, the system's data processing module first controls the camera module to capture a single-frame image of the target object. Since the data refresh rate of the IMU inertial navigation component (typically above 100Hz) is much higher than the image acquisition frequency of the camera module (typically 30-60Hz), the data processing module extracts the most recent IMU angular data from the input module at the moment the single-frame image of the target was captured. This data includes the angular velocity (in rad / s) and acceleration (in m / s²) measured by the IMU. 2 The following steps are then performed: First, the data is processed using an attitude calculation algorithm (such as complementary filtering). The angular velocity is integrated to obtain the angle change, and the angle is calibrated using acceleration data (mainly the gravitational acceleration component) to eliminate integration drift and obtain Euler angles (including pitch, roll, and yaw angles). Second, since there may be discrepancies between the IMU and camera installation positions (e.g., the IMU is 5cm above the camera), the pitch angle needs to be transformed from the IMU coordinate system to the camera coordinate system using a preset coordinate transformation matrix (based on the mechanical calibration parameters during installation) to ensure that the attitude data is consistent with the actual attitude of the camera. Third, time synchronization matching is performed by comparing the image capture timestamp with the IMU data timestamp and selecting the pitch angle data with the smallest time difference (usually required to be less than 10ms) to avoid attitude errors caused by data asynchrony. Finally, the matched pitch angle data is used as the attitude parameters of the target single-frame image. For example, when a drone is shooting a utility pole, the camera module acquires a single frame image of the utility pole while the IMU outputs angular data. The data processing module calculates that the pitch angle of the drone (camera) during the shooting is 3°, that is, the camera is tilted upward by 3°. This data will be used to correct the imaging distortion of the utility pole in the subsequent image.

[0036] S303. Based on the camera's intrinsic parameters, preprocess the single-frame image of the target to obtain a first processed image and a second processed image. The first processed image contains an image region after removing random noise, and the second processed image is an edge image region containing the edge of the key target.

[0037] Preprocessing refers to the preliminary processing operations performed on the original image, including distortion correction, region extraction, and filtering, with the aim of optimizing image quality. The first processed image is the image region after denoising, preserving the main information of the target while suppressing noise; the image region after removing random noise refers to the part of the first processed image where random noise such as sensor noise and environmental interference has been eliminated; the second processed image is the image region that highlights the edge features of the target; the edge image region of the key target edge refers to the region in the second processed image that contains the boundary between the target and the background, and between different parts within the target, and is an important basis for locating key points of the target.

[0038] After obtaining the target single-frame image, the image needs to be preprocessed: First, distortion correction is performed on the image using the previously obtained distortion coefficients. For each pixel (x, y) in the image (with the image center as the origin), correction is performed according to the correction formulas for radial distortion (image edge bending caused by the spherical characteristics of the lens) and tangential distortion (image tilting caused by lens assembly deviation). The correction method is as follows:

[0039]

[0040] ;

[0041]

[0042] in, , It refers to the coordinates of any pixel point with the center of the target single-frame image as the origin. The coordinates of the arbitrary pixel point ( The distance from the center of the target single-frame image. , These are intermediate correction parameters. The corrected coordinates. It needs to be mapped to the actual pixel position through the camera intrinsic parameter matrix: u and v are the correct positions of x and y on the graph. , ) is the coordinate of the arbitrary pixel point ( The correct coordinates of the points on the image after distortion correction. This represents the equivalent focal length along the x-axis in the image coordinate system. This represents the equivalent focal length along the y-axis in the image coordinate system.

[0043] After correction, the Region of Interest (ROI) of the target object is located using a target recognition algorithm. To avoid loss of edge information, the ROI is enlarged by a set factor, such as 1.5 times the original size (e.g., if the original ROI is 100×200 pixels, it becomes 150×300 pixels), resulting in a target region image that includes the target and part of the background. Next, a guided filter is applied to the target region image: the guided filter uses the target region image as a guide image and employs a local linear model to smooth noise while preserving edges (such as a clear boundary between the target and the background), resulting in the first processed image after denoising. Then, the difference between the target region image and the first processed image is calculated. = The difference image highlights edge information, and then it is subjected to weighted truncation (e.g., assigning a weight of 1.2 times to edge pixel values ​​and a weight of 0.8 times to non-edge pixels, truncating pixel values ​​outside the range of 0-255), resulting in the second processed image. For example, when capturing images of containers with noise, distortion correction can correct the container's barrel distortion, guided filtering removes random noise to make the container surface smoother, and interpolation and weighted truncation highlight the container's outline edges, providing clear features for subsequent positioning of the container's top and bottom.

[0044] The derivation of the correction formula is described below, mainly based on the physical model of lens distortion and the principle of coordinate mapping:

[0045] In an ideal situation (without distortion), spatial points Coordinates projected onto the normalized image plane for:

[0046]

[0047] Define the square of the radius (Distance from the point to the center of light).

[0048] Due to manufacturing and assembly errors, actual lenses exhibit two types of distortion:

[0049] The first type is radial distortion, which occurs because the lens curvature is uneven; light rays refract more strongly the farther they are from the optical center. Its mathematical model relates to the offset and radius. Related, and is an odd function (symmetric distortion):

[0050]

[0051] in Radial distortion coefficient ( (Dominant, corrects barrel distortion when the value is negative).

[0052] The second type is tangential distortion, caused by the physical fact that the lens is not parallel to the imaging plane (tilted assembly). Its mathematical model's offset depends on the product of coordinates (asymmetric distortion).

[0053]

[0054] This is the tangential distortion coefficient (describing the tilt of the lens plane).

[0055] Based on the above, the total distortion correction formula is derived, and the distortion offset is superimposed onto the ideal coordinates:

[0056]

[0057] Substituting the specific expression, we get the formula:

[0058]

[0059] In summary, the main difference lies in the radial portion through the multiplier. Adjust the scaling ratio (scaling is stronger towards the edges). There is scaling in the tangential portion. This is used to correct shear distortion caused by lens tilt (such as parallelogram distortion); and also... This is used to compensate for centrifugal offset (optical center asymmetry offset).

[0060] S304. Input the first processed image and the second processed image into the preset inference neural network in the image feature calculation module to obtain the ground point pixel coordinates and vertex pixel coordinates of the target object on the target single frame image. The inference neural network is pre-trained and includes a three-level structure of backbone network, feature pyramid and detection head.

[0061] The pre-trained inference neural network is a deep learning model used to locate key points of the target from the input image. The target object is the object to be measured, i.e., the object whose height needs to be acquired. The ground contact pixel coordinates are the pixel positions of the target's contact points with the ground in the image. The vertex pixel coordinates are the pixel positions of the highest point of the target in the image. The backbone network is the core network structure in the inference neural network responsible for basic feature extraction. It consists of multiple convolutional layers, pooling layers, and activation functions, used to extract features at different levels from the input image layer by layer. The feature pyramid is a network structure used to fuse multi-scale features, adapting to targets of different sizes by constructing feature maps at different levels. The detection head of the inference neural network is the output part of the network, responsible for converting the fused feature map into the coordinate information of the target's key points.

[0062] We pre-design a neural network to detect the ground points and vertices of the target in the image, and then use the previously obtained data... As input to the inference neural network, the inference neural network infers and calculates the detection of the target ground point and the target vertex. Its specific network structure is as follows: Figure 4 As shown, please refer to Figure 4 This is a schematic diagram of a network structure of an inference neural network in an embodiment of this application:

[0063] Network architecture typically employs a three-tier structure: a backbone network, a feature pyramid network (FPN), and a head. The backbone network extracts features progressively through convolutional layers: shallow layers preserve high-resolution spatial details such as edges and textures, while deeper layers capture high-level semantic information. However, single-scale features are insufficient to handle differences in target size; therefore, a feature pyramid network (FPN) is introduced to construct a bidirectional path, both top-down and bottom-up.

[0064] Spatial Adaptive Feature Fusion Network (CAT) as shown above Figure 4 As shown, a self-attention mechanism is employed to dynamically assign weights and facilitate cross-level interactions, achieving efficient fusion of these heterogeneous features. Its core lies in utilizing query, key, and value triples to model long-range dependencies between features. The similarity score formula is as follows:

[0065]

[0066] in, In this context, T represents the matrix transpose operation. This represents the feature dimension. The ultimate goal of multi-scale feature fusion is to assign adaptive weights to features at different levels. Self-attention mechanisms quantify the correlation between features across different levels. For example, deep semantic features can be used as queries (Q) to guide the model to filter local information that is strongly correlated with the target semantics from shallow detail features (K / V), thereby enhancing semantic consistency while preserving spatial details.

[0067] After constructing the inference neural network, the system's data processing module concatenates the first processed image (denoised image) and the second processed image (edge ​​image) into a dual-channel input, which is then fed into the backbone network. The backbone network extracts features through layer-by-layer convolutional operations: for example, shallow convolutional layers use 3×3 small convolutional kernels with a stride of 1, combined with the ReLU activation function, to preserve high-resolution feature maps. The output shallow detail features include the edge contours of the target (such as the vertical lines of a telephone pole and the texture of a pedestrian's clothing). Then, deep convolutional layers gradually compress the feature map size by increasing the number of convolutional kernels and pooling operations. At the same time, deep semantic features are captured through multi-layer nonlinear transformations, such as the category information and overall structure of targets like "telephone pole" and "pedestrian." For example, when processing an image containing pedestrians, shallow features can clearly present the texture of the pedestrian's hair and the wrinkles in their clothing, while deep features can clearly distinguish the pedestrian from the trees and vehicles in the background.

[0068] The second level is feature pyramid multi-scale fusion. To address the detection challenges caused by changes in target size (e.g., a distant pedestrian may occupy only 10×30 pixels in an image, while a nearby pedestrian may occupy 100×200 pixels), the feature pyramid fuses features through a two-way path: bottom-up and top-down. The bottom-up path transmits feature maps of different scales (e.g., C1 to C5, with sizes of 320×320, 160×160, 80×80, 40×40, and 20×20, respectively) output from the backbone network to each layer of the pyramid. The top-down path upsamples the deep feature map (e.g., C5) by a factor of 2 to match its size with the shallow feature map above (e.g., C4). After adjusting the number of channels through a 1×1 convolution, it is added element-wise to the C4 feature map to obtain the fused feature P4. Similarly, after upsampling P4, it is fused with C3 to obtain P3, and so on to generate multi-scale fused features (P3 to P5, with sizes of 80×80, 40×40, and 20×20). This fusion approach ensures that each level of features contains both shallow, detailed information (such as P3 retaining more edge details) and deep, semantic information (such as P5 enhancing target category features).

[0069] The third level is spatial adaptive feature fusion. Multi-scale fused features (P3 to P5) are input into the spatial adaptive feature fusion network, which dynamically adjusts feature weights based on a self-attention mechanism: First, the feature maps at each scale are converted into query, key, and value matrices (e.g., Q, K, and V are all 80×80×64) through convolutional layers; then, the similarity between the transposes of Q and K is calculated to obtain an attention weight map, where a higher weight value indicates a stronger correlation between the feature at that location and the target semantics; finally, the weight map is multiplied by the V matrix to achieve weighted fusion of features, resulting in a fused feature map of size 80×80×64. For example, for an image containing utility poles and billboards, the attention mechanism will assign higher weights to the edge features of the utility poles and weaken the interfering information of the billboards, making the fused feature map more focused on the target.

[0070] The fourth level involves locating key points for the detection head. After the fused feature map is input into the detection head, sub-pixel-level precise capture of the target's upper and lower vertices is achieved through the fusion of heatmap regression and coordinate offset prediction techniques. Its core principle lies in combining the spatial probability distribution advantages of the heatmap with the sub-pixel correction capability of the offset, forming a collaborative mechanism of "coarse positioning + fine-tuning," significantly improving the positioning accuracy and occlusion robustness of vertex coordinates. The process is as follows: Figure 5 As shown, Figure 5 This is a schematic diagram of a positioning correction scenario in an embodiment of this application:

[0071] Heatmap regression provides coarse-grained localization for each vertex by generating a probability distribution map aligned with the spatial dimensions of the input image. Its implementation involves three steps:

[0072] The first step is to generate the predicted heatmap. This begins by using multiple real heatmaps as training samples. These training sample images must contain precisely labeled real vertex coordinates and ground point coordinates. The data processing module generates realistic heatmaps based on these labeled coordinate training sample images, primarily using a Gaussian kernel function. , of which Real vertex coordinates Controlling the probability diffusion range. The probability of each pixel in the image belonging to a keypoint is calculated using a Gaussian kernel function, resulting in a probability distribution where the peak is located at the true coordinates and gradually decreases towards the surrounding area. When the value is set to 2, the probability of a location 1 pixel away from the true coordinates is approximately 0.88, and at 2 pixels it is approximately 0.61. This distribution preserves the spatial location information of key points while tolerating a certain degree of annotation error and localization deviation through a reasonable diffusion range. During training, the network calculates the difference between the predicted heatmap and the true heatmap using a loss function (e.g., calculating the squared error pixel by pixel), and adjusts the parameters of the backbone network, feature pyramid, and detector head (e.g., convolutional layer weights and bias terms) through backpropagation, so that the peak of the predicted heatmap gradually approaches the true coordinates, ultimately learning to identify the probability distribution pattern of key points from the fused feature map.

[0073] When the fused feature map (containing multi-scale fused features of the target) is input into the detection head, the number of channels is adjusted by a convolutional layer (e.g., outputting 2 channels corresponding to vertices and ground points respectively), and then the feature values ​​are mapped to the [0, 1] interval by a Sigmoid activation function to obtain the predicted heatmap. This predicted heatmap has been trained under supervision using real heatmaps and can form obvious peaks at target keypoint locations (e.g., 0.95 for vertices, 0.91 for ground points, and 0.05-0.2 for the background region), directly reflecting the probability distribution of the keypoints. The predicted heatmap is a probability distribution map aligned with the input image space; the value of each pixel represents the probability that the location is a ground point or a vertex, with higher values ​​indicating greater likelihood.

[0074] The second step is to determine the coarse estimated coordinates. Since the predicted heatmap may contain multiple local peaks (such as noise peaks at the target edge or spurious peaks from similar objects), non-maximum suppression (NMS) is used for filtering. Specifically, the data processing module first sets a probability threshold (e.g., 0.5) to filter out pixels below the threshold. Then, it performs local maximum detection on the remaining pixels. For each pixel, it compares its probability value with other pixels in its 8-neighborhood, retaining only the local maximum as candidate peak points. Finally, it selects the two points with the highest probability from the candidate peak points (corresponding to the vertex and ground point, respectively), and their coordinates are the coarse estimated coordinates. For example, when processing a heatmap containing pedestrians, NMS filters out spurious peaks in the background around the pedestrians, retaining the peak points at the head (vertex) and feet (ground point), thus obtaining the coarse estimated coordinates, i.e., locating the probability peak points. and As coarse estimations of the top-left and bottom-right vertices, this process effectively avoids multi-peak interference, ensuring the uniqueness and reliability of the coarse localization results. Non-maximum suppression is an algorithm used to filter local maxima. By suppressing non-peak pixels within a certain range around the peak point, it avoids the occurrence of multiple adjacent high-probability points. The peak point refers to the pixel with the highest probability value in the predicted heatmap, representing the potential location of a grounding point or vertex.

[0075] The third step is subpixel-level offset vector prediction. To eliminate the resolution loss of the heatmap, the coordinate offset prediction branch introduces a subpixel-level correction vector: the coordinate offset prediction branch network receives local features (usually 3×3 or 5×5 feature regions centered on peak points) corresponding to the coarsely estimated coordinates in the fused feature map, and outputs the subpixel-level offset vector (Δx, Δy) corresponding to the coarsely estimated coordinates through 2-3 convolutional layers and fully connected layers. The training process of this branch network requires a large number of sample images labeled with subpixel-level real coordinates: first, thousands of images containing different targets (such as vehicles, pedestrians, and utility poles) are collected, and the coordinates of real vertices and ground points are obtained through manual annotation or precision instrument measurement (accuracy up to 0.1 pixels); then, the sample images are input into the network, and the L1 loss between the predicted offset and the real offset (real coordinates - coarsely estimated coordinates) is calculated; the network parameters are iteratively adjusted through the backpropagation algorithm until the loss function converges (e.g., the loss value is less than 0.01 pixels). After training, the network has the ability to predict subtle offsets based on local features. For example, for a coarsely estimated coordinate (320, 100), the network can output an offset vector (0.3, -0.2), indicating that the true vertex is offset 0.3 pixels to the right and 0.2 pixels upward from the coarse coordinates. Figure 5 The offset is calculated in the middle. and These are the residual vectors from the top left and bottom right coarse positioning points to the actual vertices, respectively.

[0076] After coarse-grained localization, the next step is precise coordinate calculation. The data processing module weights and sums the coarsely estimated coordinates with the sub-pixel-level offset vector according to preset weights to obtain the final vertex and ground point pixel coordinates:

[0077]

[0078]

[0079]

[0080]

[0081] in, , , , This refers to the output image coordinates of the vertex and ground point. The weight settings need to balance the stability of the coarsely estimated coordinates and the correction capability of the offset vector, and are usually determined through cross-validation. For example, the weight of the coarsely estimated coordinates is set to 0.6, and the weight of the offset vector is set to 0.4. =0.6× +0.4× , =0.6× +0.4× This weighted method retains the overall positioning of the coarsely estimated coordinates while achieving sub-pixel-level fine adjustment through offset vectors, enabling the final coordinate positioning accuracy to reach the 0.1 pixel level, providing high-precision input data for subsequent physical height calculations.

[0082] Through the coordinated processing of the above four steps, this step successfully transforms the abstract features in the fused feature map into precise key point pixel coordinates, overcoming the limitations of a single localization method. It can still maintain high localization accuracy in complex scenes and is one of the core steps to achieve accurate target height measurement.

[0083] S305. Based on the pixel coordinates of the grounding point, the pixel coordinates of the vertex, the pitch angle data, and the camera intrinsic parameters, the physical true height of the target object to be measured is calculated through the geometric calculation module.

[0084] The position of the camera's optical center on the image is obtained through the input module. Grounding point pixel coordinates vertex pixel coordinates Camera focal length Camera height Based on spatial geometric constraints, and according to the principles of 3D projection and similar triangles, the following calculations are performed in the geometric calculation module:

[0085] (1)

[0086] ; (2)

[0087] ; (3)

[0088] ; (4)

[0089] in This shows the distance from the optical center to the grounding point of the measured target. It's pitch angle data. It is a functional relationship about camera pitch. It is the actual physical height of the target object being measured.

[0090] The following is combined Figure 6 The content introduces the detailed derivation of the formula in step S305:

[0091] Step 1: Define the coordinate system and point coordinates (3D projection basics)

[0092] World coordinate system: Origin is the projection of the camera's optical center onto the ground (simplified). The y-axis is upward, the x-axis is horizontal, and the z-axis is depth (as shown in the geometric constraint diagram). World coordinates of the camera's optical center: Target vertex 3D world coordinates: .in, It is the depth (from the optical center to the ground point) The distance (assuming the grounding point and the vertex are aligned in the depth direction). Target grounding point 3D world coordinates: (Height is 0).

[0093] Camera coordinate system: originating from the optical center, with the optical axis as the z-axis. Due to the pitch angle... The camera coordinate system is rotated relative to the world coordinate system (the camera does not yaw or roll, only rotation around the x-axis is considered). Rotation matrix: the transformation of a point from world coordinates to camera coordinates:

[0094]

[0095] The adjustment here is because the optical center height difference is already included in the coordinates: for the target point, It is the absolute height, but the height difference is in the components of the vector pointing from the optical center to the point. (Optical center height is used as a reference).

[0096] For the target vertex World coordinates: .

[0097] Camera coordinates (by rotation):

[0098]

[0099]

[0100] Step 2: Project onto the image plane (principle of similar triangles)

[0101] Projection model: The pinhole camera model projects 3D points onto the image plane. Figure 1 The imaging plane). Vertical image coordinates (v-coordinates) are calculated based on similar triangles: ,in It is the position of a point in the image (e.g., the v coordinate). It is the position of the optical center in the image (i.e. (vertical component) It is the focal length (distance from the imaging plane). and From the camera coordinate system.

[0102] Substitute the target vertex Camera coordinates:

[0103]

[0104] here, and Image location ( Figure 2 (indicated by the Chinese annotation), so It is a scalar (vertical direction).

[0105] Step 3: Introduce camera height and reference point (geometric constraints)

[0106] The diagram illustrates that the optical center, the imaging plane, and the three-dimensional point form a triangle:

[0107] For the target vertex The angle between the ray from the optical center to the point and the optical axis is related.

[0108] Similar triangle ratio: The position of an image point is proportional to the height difference and depth of the three-dimensional point, but the scaling of the pitch angle needs to be considered.

[0109] Re-expressing the projection formula, including Reference for photocentric model:

[0110]

[0111] in It is an adjustment item, and Related. When the point is on the ground ( =0), the projection should be related to the camera height.

[0112] Therefore, use points Reference:

[0113] When the point is at the horizontal plane of the camera height ( = ),Location Approximately (When z is large). But for general points, the projection is adjusted as follows:

[0114]

[0115] in and It takes into account the height difference and depth component after pitching.

[0116] After substituting the point coordinates, the algebraic structure is simplified as follows:

[0117]

[0118] However, the denominator may be small or negative, and there is no... Therefore, it needs to be corrected.

[0119] Step 4: Algebraic simplification and addition of absolute values ​​(to ensure consistency)

[0120] Add to Item: Camera Height An offset is introduced because it affects the reference position of the ground point. In the expression, the constant term... Used to compensate for coordinate system offsets (such as height difference references).

[0121] deal with Symbols: Use Ensure the denominator is positive when the pitch angle changes. Figure 2 (This simplifies the process by eliminating yaw and roll). In the formula... It only appears in the relevant items.

[0122] Rewrite the projection formula:

[0123]

[0124] Then, by multiplying the numerator and denominator together... Adjustment:

[0125]

[0126] But the denominator has And trigonometric identities, after simplification: the denominator Can be combined definition.

[0127] A more accurate derivation is based on the proportions of similar triangles:

[0128] Similar triangle ratio: Image displacement ratio is the three-dimensional height difference divided by the depth, but with pitch scaling.

[0129] Finally, after algebraic operations (including increments) (Compensation item), resulting in:

[0130]

[0131] The above is the derivation process of formula (1).

[0132] The derivation of formula (2) is as follows:

[0133] Step 1: Establish 3D geometric relationships (coordinate system definition), world coordinate system (simplified 1D model):

[0134] Camera optical center coordinates: (Optical center is above the ground) ).

[0135] Target grounding point coordinate: (The grounding point is located on the ground at a height of 0;) It is the depth from the optical center to the grounding point.

[0136] Pitch angle Impact: Camera coordinate system around world coordinate system Axis rotation .

[0137] Transformation matrix from world coordinates to camera coordinates:

[0138]

[0139] Note: Vector Indicates the direction from the optical center to the grounding point. Direction (grounding point lower than optical center) ).

[0140] Step 2: Calculate the coordinates in the camera coordinate system

[0141] Unfold the rotation matrix: Coordinates in the camera coordinate system:

[0142] It is the offset component in the vertical direction.

[0143] It is the depth component (along the optical axis).

[0144] Step 3: Project onto the image plane (principle of similar triangles)

[0145] Pinhole camera model: The position of image points is determined by the proportions of similar triangles. in It is the image point relative to the optical center. Vertical offset.

[0146] grounding point Image location:

[0147] Step 4: Fusion of Algebraic Reconstruction and Geometric Constraints

[0148] Unified denominator: Maintain consistency with the denominator of formula (1), i.e. .

[0149] Molecular Reconstruction: Introducing a Reference Point (The position of the camera's horizontal plane in the image):

[0150] When the point is on the horizontal plane at the camera's height (height) The image location is .

[0151] Through algebraic recombination, molecules are added and The compensation item (detailed derivation is below).

[0152] Refactoring process:

[0153] Will The expression is rewritten as:

[0154] Merging molecules:

[0155] Key adjustments:

[0156] Using reference points Relationship ( (Camera height on the horizontal plane).

[0157] By adding and cleverly combining items (Identical transformation) to make the molecular form consistent with formula (2): Note: This derivation involves algebraic identity transformations (such as...). Items are retained directly. replace , and add Compensation for geometric offset.

[0158] Step 5: Unify and simplify the denominator

[0159] Denominator handling: When (Camera overhead view) The denominator simplifies to: Consistent with the projection model.

[0160] Ensure the denominator is The physical meaning remains the same when the sign changes. Figure 2 emphasize).

[0161] Final form:

[0162]

[0163] molecular:

[0164] :depth Coupled with pitch angle.

[0165] : Camera height compensation term (geometric constraint core).

[0166] Denominator: Consistent with formula (1), reflecting the scaling effect of pitch angle on projection.

[0167] Formula (3) is the solution to the target height from formulas (1) and (2). An approximate expression for .

[0168] Step 1: Establish the core relationship between formulas (1) and (2)

[0169] Reviewing formulas (1) and (2), the denominators of both are :

[0170]

[0171] Under the long-distance assumption ( ), The second dominant factor:

[0172]

[0173] Will Substituting the approximate values ​​into formulas (1) and (2), we simplify to:

[0174] Simplification of Formula (2) (Dock Location) projection)

[0175]

[0176] Note: Numerator constant term exist It is often overlooked.

[0177] Simplification of Formula (1) (for vertices) projection)

[0178]

[0179] Combination can Expressed as:

[0180]

[0181] Organize key relationships:

[0182]

[0183] Step 2: Introduce image distance ratio

[0184] The goal is to Related to the distance ratio in the image. According to the simplified formula:

[0185] ,therefore:

[0186] It consists of two items:

[0187]

[0188] At long distance ( The first term is the largest, and the second term is the smallest. Therefore:

[0189]

[0190] Construction distance ratio:

[0191] Depend on Equation:

[0192]

[0193] Substitution :

[0194]

[0195] in (Eliminate the absolute value by squaring). Divide both numerator and denominator. :

[0196]

[0197] Step 3: Eliminate And introduce

[0198] The remaining ones need to be eliminated Variables. Extracted from the original expression of formula (2). Relationship:

[0199] From simplified formula (2):

[0200] Its molecules Substituting the projection relationship:

[0201]

[0202] However, the actual proportion includes Contributions (see) Formula). From a geometrical essence:

[0203] Pitch angle scaling function Physical origin:

[0204] In three-dimensional space, the target height The projection length in the image is proportional to: in It is the radial distance of the target ( ).

[0205] Due to pitch angle The angle between the optical axis and the ground plane has changed, so the scaling factor needs to be compensated for.

[0206]

[0207] Combined image distance ratio:

[0208]

[0209] Through algebraic matching ( Figure 3 (omitting the specific process), the optimal match was found to be:

[0210]

[0211]

[0212] Implicit Symbol adaptation (works for both overhead and low-angle views), actual calculations take Ensure the value is positive.

[0213] In this embodiment, by employing techniques such as pre-calibrating camera intrinsic parameters to lay a precise geometric foundation, acquiring and calculating precise pitch angles through an IMU inertial navigation component to correct camera attitude, performing distortion correction and noise reduction edge-preserving preprocessing on images to optimize input data, and utilizing an inference neural network including a backbone network, feature pyramid, and detection head to achieve sub-pixel-level precise positioning of target ground points and vertices, the system can collaboratively eliminate problems such as image distortion interference, attitude estimation errors, and insufficient key point positioning accuracy in traditional measurements. This effectively solves the problems of relying on additional reference objects, complex and expensive measurement equipment, sensitivity to camera attitude, and unstable target feature extraction in existing technologies. Consequently, it achieves the technical effects of completing high-precision target height measurement without any known reference objects, significantly reducing the cost and complexity of measurement devices, and improving the system's adaptability and measurement accuracy in unconstrained scenarios.

[0214] The application scenario of step S305 in this embodiment will be described below. Please refer to... Figure 6 , Figure 6 This is a schematic diagram illustrating another application scenario of the real height measurement method based on a monocular camera in this application:

[0215] Figure 6 The left side shows a schematic of the actual physical space, and the upper right corner shows its projection in a two-dimensional imaging coordinate system (image plane). The entire process demonstrates how to use a monocular camera, IMU, and image processing technology to map the actual three-dimensional height into measurable two-dimensional image features, and then reconstruct the target height through geometric calculations. Specifically, the camera is positioned at a certain height, and the camera height given by the IMU device is known to be... Looking down at the ground, the angle between the optical axis and the horizontal plane is θ (pitch angle). The camera's optical center is the geometric reference point for imaging; the imaging plane is the projection plane in space, on which the object forms an image. The target's focal point in the image has been predicted relatively accurately using a neural network; let's call it... And the vertices in the target image are set as Under the scenario and imaging geometry constraints shown in this figure, the principle of a monocular camera measuring the height of a target object can be intuitively demonstrated. We assume that the camera has been calibrated during installation, with zero yaw and roll angles, and only a pitch angle exists, meaning the camera's optical axis has only one angle θ around the horizontal direction. This simplification makes the spatial relationships of the entire measurement system clearer.

[0216] The images captured by the camera have been calibrated using intrinsic parameters, and the coordinates of the principal point, i.e., the position of the optical center in the image, have been obtained. And the focal length f, these parameters are obtained automatically through checkerboard calibration and software tools (such as OpenCV, Matlab). On the imaging plane, the principal point It serves as the reference point for the entire geometric mapping; all projections of targets onto the image are based on this reference. In real-world scenarios, target objects such as streetlights, pedestrians, and pole-shaped objects have a physical height of [missing information]. The top and bottom projections of the target object in the image are respectively... (Vertex pixel coordinates) and (Ground point pixel coordinates), these two points have already been obtained in previous steps. The image also shows... The horizontal plane, representing the camera's altitude, is positioned in the image. This virtual horizontal line corresponds to the projection onto the imaging plane when the camera's optical center is parallel to the ground. The black line in the middle of the image represents the imaginary imaging plane. During imaging, there is a strict geometric relationship between the actual height of the object and the projected distances between its vertex and ground point on the imaging plane. Using known camera height, focal length, principal point, pitch angle θ, and the pixel positions of the target vertex and ground point, distances in two-dimensional image space can be reconstructed into actual heights in three-dimensional physical space using trigonometric geometry and the principles of similar triangles.

[0217] In this embodiment, by combining a monocular camera with an IMU inertial navigation component and introducing a key point detection and three-dimensional geometric calculation model based on deep learning, the physical height of any target in the image can be automatically and accurately measured without the need for known reference objects or prior knowledge of the target.

[0218] The height measurement system in the embodiments of this invention is described below from the perspective of hardware processing. Please refer to [link / reference]. Figure 7 This is a schematic diagram of the physical device structure of a height measurement system in an embodiment of this application.

[0219] It should be noted that, Figure 7 The structure of the height measurement system shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0220] like Figure 7 As shown, the altitude measurement system includes a Central Processing Unit (CPU) 701, which can perform various appropriate actions and processes based on a program stored in Read-Only Memory (ROM) 702 or a program loaded from storage section 708 into Random Access Memory (RAM) 707, such as performing the methods described in the above embodiments. The RAM 707 also stores various programs and data required for system operation. The CPU 701, ROM 702, and RAM 707 are interconnected via a bus 704. An Input / Output (I / O) interface 705 is also connected to the bus 704.

[0221] The following components are connected to I / O interface 705: input section 706 including audio input devices, push-button switches, etc.; output section 707 including a liquid crystal display (LCD) and audio output devices, indicator lights, etc.; storage section 708 including a hard disk, etc.; and communication section 709 including a network interface card such as a LAN (Local Area Network) card, modem, etc. Communication section 709 performs communication processing via a network such as the Internet. Drive 710 is also connected to I / O interface 705 as needed. Removable media 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 710 as needed so that computer programs read from them can be installed into storage section 708 as needed.

[0222] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing computer programs for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 709, and / or installed from removable medium 711. When the computer program is executed by central processing unit (CPU) 701, it performs the various functions defined in the present invention.

[0223] It should be noted that specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0224] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, program segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those shown in the drawings.

[0225] Specifically, the height measurement system of this embodiment includes a processor and a memory. The memory stores a computer program. When the computer program is executed by the processor, it implements the real height measurement method based on a monocular camera provided in the above embodiment.

[0226] In another aspect, the present invention also provides a computer-readable storage medium, which may be included in the height measurement system described in the above embodiments; or it may exist independently and not assembled into the height measurement system. The storage medium carries one or more computer programs that, when executed by a processor of the height measurement system, cause the height measurement system to implement the real height measurement method based on a monocular camera provided in the above embodiments.

[0227] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0228] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as meaning "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".

[0229] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for measuring true height based on a monocular camera, characterized in that, The method includes: Obtain the camera intrinsic parameters of a monocular camera, wherein the camera intrinsic parameters include at least radial distortion parameters, tangential distortion parameters, and the x-coordinate of the principal point in the image coordinate system. and ordinate ; The camera module acquires a single-frame image of the target and obtains the angle data of the IMU inertial navigation component in the most recent input module. The pitch angle data of the single-frame image of the target is then calculated. Based on the camera's intrinsic parameters, the target single-frame image is preprocessed to obtain a first processed image and a second processed image. The first processed image contains an image region after removing random noise, and the second processed image is an edge image region containing the edges of key targets. The first processed image and the second processed image are input into the preset inference neural network in the image feature calculation module to obtain the ground point pixel coordinates and vertex pixel coordinates of the target object on the target single frame image. The inference neural network is pre-trained and includes a three-level structure of backbone network, feature pyramid and detection head. Based on the ground point pixel coordinates, the vertex pixel coordinates, the pitch angle data, and the camera intrinsic parameters, the physical true height of the target object is calculated through the geometric calculation module. The step of calculating the physical true height of the target object using a geometric calculation module based on the grounding point pixel coordinates, the vertex pixel coordinates, the pitch angle data, and the camera intrinsic parameters includes: The position of the camera's optical center on the image is obtained through the input module. The grounding point pixel coordinates The vertex pixel coordinates Camera focal length Camera height The camera height The measurement was obtained by using a laser rangefinder connected to the camera. Based on spatial geometric constraints, and according to the principles of 3D projection and similar triangles, the following calculations are performed in the geometric calculation module: ； ； Where z This shows the distance from the optical center to the grounding point of the measured target. The pitch angle data is as described. It is the actual physical height of the target object being measured; Combining the formulas, we get: ； in, It is a functional relationship with respect to camera pitch, and the corresponding calculation formula is: 。 2. The method according to claim 1, characterized in that, The step of preprocessing the target single-frame image based on the camera's intra-camera parameters to obtain a first processed image and a second processed image includes: Radial distortion parameters are obtained through the input module. , , and tangential distortion parameters , ; The target single-frame image is subjected to distortion correction based on the radial distortion parameters and the tangential distortion parameters. The correction method is as follows: ； ； ； ； in, , It refers to the coordinates of any pixel point with the center of the target single-frame image as the origin. The coordinates of the arbitrary pixel point ( The distance from the center of the target single-frame image. , It is an intermediate correction parameter. , ) is the coordinate of the arbitrary pixel point ( The correct coordinates of the points on the image after distortion correction. This represents the equivalent focal length along the x-axis in the image coordinate system. This represents the equivalent focal length along the y-axis in the image coordinate system.

3. The method according to claim 2, characterized in that, After the step of performing distortion correction on the target single-frame image based on the radial distortion parameters and the tangential distortion parameters, the method further includes: The target object to be tested is located in the spatial region by a target recognition algorithm. The area space is expanded by a set factor to obtain a target area image, which includes the target object to be tested and some background information. The target region image is filtered using a guided filter to obtain the first processed image; The difference between the target region image and the first processed image is calculated, and a weighted truncation process is performed to obtain the second processed image.

4. The method according to claim 1, characterized in that, The step of inputting the first processed image and the second processed image into a preset inference neural network in the image feature calculation module to obtain the ground point pixel coordinates and vertex pixel coordinates of the target object on the target single frame image includes: Feature extraction is performed on the first processed image and the second processed image through a backbone network to obtain shallow detail features and deep semantic features. The shallow detail features preserve high-resolution spatial details, and the deep semantic features capture high-level semantic information. The shallow detail features and the deep semantic features are fused using a bidirectional path constructed from top to bottom and bottom to top through the feature pyramid to obtain multi-scale fused features. The multi-scale fusion features are fused using a spatial adaptive feature fusion network to obtain a fused feature map. The fused feature map is input into the detection head of the inference neural network, and the coordinates of the grounding point pixel and the vertex pixel are determined by the heatmap regression algorithm and the coordinate offset prediction algorithm.

5. The method according to claim 4, characterized in that, The steps of determining the grounding point pixel coordinates and the vertex pixel coordinates using a heatmap regression algorithm and a coordinate offset prediction algorithm include: Based on the fused feature map, a predicted heatmap is generated through an activation function. Each pixel value in the predicted heatmap represents the probability that a ground point or vertex exists at that location. Non-maximum suppression is applied to the predicted heatmap to determine the peak point with the highest probability, which is used as a coarse estimate of the coordinates of the vertex and ground point of the target object on the target single-frame image; The subpixel-level offset vector of the coarsely estimated coordinates is determined by a coordinate offset prediction branch network, wherein the coordinate offset prediction branch network is constructed in advance by iterative training based on multiple training sample images labeled with real vertex coordinates and ground point coordinates. The coarsely estimated coordinates and the sub-pixel offset vector are weighted and summed according to preset weights to obtain the vertex pixel coordinates and the ground point pixel coordinates.

6. The method according to claim 1, characterized in that, The steps of acquiring a single-frame image of the target through the camera module, acquiring the angle data of the IMU inertial navigation component in the most recent input module, and calculating the pitch angle data of the single-frame image of the target specifically include: After acquiring a single-frame image containing the target, determine the most recent angular velocity and acceleration data of the IMU inertial navigation component; The attitude calculation algorithm converts the angular velocity and acceleration data into Euler angles. Extract the pitch angle data from the Euler angle representation and transform the pitch angle data from the IMU coordinate system to the camera coordinate system; The converted pitch angle data is time-synchronized and matched with the target single-frame image; The matched pitch angle data is used as the attitude parameter of the target single-frame image.

7. A height measurement system applied to the method of claim 1, characterized in that, include: The camera intrinsic parameter acquisition module is used to acquire the camera intrinsic parameters of the monocular camera. These parameters include at least radial distortion parameters, tangential distortion parameters, and the x-coordinate of the principal point in the image coordinate system. and ordinate ; The pitch angle data acquisition module is used to acquire a single frame image of the target through the camera module and acquire the angle data of the IMU inertial navigation component in the most recent input module, and obtain the pitch angle data of the single frame image of the target by calculation. The image region determination module is used to preprocess the target single-frame image based on the camera intra-parameters to obtain a first processed image and a second processed image. The first processed image contains an image region after removing random noise, and the second processed image is an edge image region containing the edge of a key target. The pixel coordinate acquisition module is used to input the first processed image and the second processed image into the preset inference neural network in the image feature calculation module to obtain the ground point pixel coordinates and vertex pixel coordinates of the target object on the target single frame image. The inference neural network is pre-trained and includes a three-level structure of backbone network, feature pyramid and detection head. The true height determination module is used to calculate the physical true height of the target object to be measured through the geometric calculation module based on the ground point pixel coordinates, the vertex pixel coordinates, the pitch angle data, and the camera intrinsic parameters.

8. A height measurement system, characterized in that, The height measurement system includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the height measurement system to perform the method as described in any one of claims 1-6.

9. A computer-readable storage medium comprising instructions, characterized in that, When the instruction is executed on the altitude measurement system, it causes the altitude measurement system to perform the method as described in any one of claims 1-6.

Images