Monocular camera ranging method and device, storage medium and electronic equipment

Abstract

The application discloses a monocular camera ranging method and device, a storage medium and an electronic device. The method comprises the following steps: acquiring a target image by using a monocular camera, and inputting the target image into a preset target detection model; in the target detection model, determining a target object in the target image and a target frame corresponding to the target object; determining an image height of the target object in the target image according to two-dimensional coordinates of the target frame; and determining a target distance between the target object and the monocular camera according to a focal length of the monocular camera, an actual height of the target object and the image height. The method solves the problems of the existing monocular camera ranging method, such as complexity and high hardware requirement.

Description

Technical Field

[0001] This application relates to the field of autonomous driving, and in particular to a monocular camera ranging method and apparatus, storage medium and electronic device. Background Technology

[0002] Perception technology plays a crucial role in autonomous driving systems. Various sensors installed on driverless vehicles detect the surrounding environment and identify potential obstacles, acting as the "eyes" of the vehicle. Commonly used perception sensors include LiDAR, millimeter-wave radar, and cameras. Among these, cameras collect rich feature information and are relatively inexpensive; however, existing ranging methods using camera-collected information are complex and require high-performance hardware. For example, CN202210810678.0 discloses a monocular ranging system and method for underground unmanned vehicles, based on Kalman filtering for prediction and relying on multi-target fusion algorithms, resulting in high algorithm complexity and demanding high parameter tuning capabilities and hardware performance during implementation. CN202210456355.6 discloses a monocular real-time ranging method based on deep learning target detection, which automatically estimates the mapping relationship between the target's three-dimensional coordinates and the corresponding two-dimensional coordinates of the camera image using ground reference points. This method requires accurate identification of the actual coordinates of reference points such as ground lane lines, corners or contours of traffic signs, etc., which places high demands on the accuracy of the recognition algorithm. CN202210897606.4 proposes a monocular ranging method and system based on adaptive target detection network and license plate detection. The accuracy of the ranging depends on the accuracy of the contour. However, the recognition effect of the contour varies greatly under different poses and lighting conditions. Therefore, the robustness of using this method for monocular ranging is poor and the accuracy cannot be guaranteed. Summary of the Invention

[0003] In view of this, this application provides a monocular camera ranging method and device, storage medium and electronic device, which solves the problems of complexity and high hardware requirements of existing monocular camera ranging methods while ensuring accuracy.

[0004] According to one aspect of this application, a monocular camera ranging method is provided, comprising:

[0005] A target image is acquired using a monocular camera, and the target image is input into a preset target detection model;

[0006] In the target detection model, the target object in the target image and the target bounding box corresponding to the target object are determined;

[0007] Based on the two-dimensional coordinates of the target bounding box, determine the image height of the target object in the target image;

[0008] The target distance between the target object and the monocular camera is determined based on the focal length of the monocular camera, the actual height of the target object, and the image height.

[0009] Optionally, determining the target object in the target image and the target bounding box corresponding to the target object includes:

[0010] Extract image features from the target image and generate a feature map based on the image features;

[0011] The feature map is divided into a preset number of sub-regions, and each sub-region is detected using a predefined detection box to obtain at least one candidate box and the confidence level of the candidate box, wherein the candidate box contains a candidate object.

[0012] Based on the preset threshold of the target detection model and the confidence level corresponding to the candidate boxes, a target box is determined in the candidate boxes, and the candidate objects contained in the target box are determined to be the target objects.

[0013] Optionally, the preset threshold includes a confidence threshold and an intersection-union ratio (IU) threshold;

[0014] Accordingly, determining the target box from the candidate boxes based on the preset threshold of the target detection model and the confidence level corresponding to the candidate boxes includes:

[0015] In the at least one candidate box, candidate boxes with a confidence level less than the confidence threshold are removed to obtain a candidate set;

[0016] The candidate box with the highest confidence in the candidate set is determined as the base box. The base box is removed from the candidate set and added to the base box set.

[0017] Determine the intersection-union ratio (IUR) between each candidate box in the candidate set and the base box, remove candidate boxes whose IUR is greater than the IUR threshold, and return to the step of determining the candidate box with the highest confidence in the candidate set as the base box, until the candidate set is an empty set;

[0018] The candidate boxes in the set of base frames are determined as the target boxes.

[0019] Optionally, determining the image height of the target object in the target image based on the two-dimensional coordinates of the target bounding box includes:

[0020] Based on the two-dimensional coordinates of the target bounding box, the number of pixels in the target bounding box along the y-axis is determined, and the image height is determined based on the number of pixels.

[0021] Optionally, the method further includes:

[0022] Multiple preset categories are pre-defined, and object samples corresponding to each preset category are collected respectively;

[0023] The average height of the object samples corresponding to each preset category is determined as the category height corresponding to the preset category;

[0024] Accordingly, before determining the target distance between the target object and the monocular camera based on the focal length of the monocular camera, the actual height of the target object, and the image height, the method further includes:

[0025] In the target detection model, the target category of the target object is determined, and the category height corresponding to the target category is determined as the actual height of the target object.

[0026] Optionally, before inputting the target image into a preset target detection model, the method includes:

[0027] The target detection model is trained using pre-labeled training images, and the preset threshold of the target detection model is adjusted.

[0028] Optionally, before training the object detection model using pre-labeled training images, the method further includes:

[0029] Acquire multiple original training images, wherein the original training images contain objects to be labeled, and the number of objects to be labeled corresponding to each preset category is greater than a first preset number threshold.

[0030] The number of pixels occupied by each of the objects to be labeled is determined. If the number of pixels is greater than a second preset threshold, the object to be labeled is labeled to obtain the pre-labeled training image.

[0031] According to another aspect of this application, a monocular camera ranging device is provided, the device comprising:

[0032] The imaging module is used to acquire target images using a monocular camera and input the target images into a preset target detection model.

[0033] The detection module is used to determine the target object in the target image and the target bounding box corresponding to the target object in the target detection model;

[0034] The calculation module is used to determine the image height of the target object in the target image based on the two-dimensional coordinates of the target box; and to determine the target distance between the target object and the monocular camera based on the focal length of the monocular camera, the actual height of the target object, and the image height.

[0035] Optionally, the detection module is used for:

[0036] Extract image features from the target image and generate a feature map based on the image features;

[0037] The feature map is divided into a preset number of sub-regions, and each sub-region is detected using a predefined detection box to obtain at least one candidate box and the confidence level of the candidate box, wherein the candidate box contains a candidate object.

[0038] Based on the preset threshold of the target detection model and the confidence level corresponding to the candidate boxes, a target box is determined in the candidate boxes, and the candidate objects contained in the target box are determined to be the target objects.

[0039] Optionally, the preset threshold includes a confidence threshold and an intersection-union ratio (IU) threshold;

[0040] Accordingly, the detection module is used for:

[0041] In the at least one candidate box, candidate boxes with a confidence level less than the confidence threshold are removed to obtain a candidate set;

[0042] The candidate box with the highest confidence in the candidate set is determined as the base box. The base box is removed from the candidate set and added to the base box set.

[0043] Determine the intersection-union ratio (IUR) between each candidate box in the candidate set and the base box, remove candidate boxes whose IUR is greater than the IUR threshold, and return to the step of determining the candidate box with the highest confidence in the candidate set as the base box, until the candidate set is an empty set;

[0044] The candidate boxes in the set of base frames are determined as the target boxes.

[0045] Optionally, the computing module is used for:

[0046] Based on the two-dimensional coordinates of the target bounding box, the number of pixels in the target bounding box along the y-axis is determined, and the image height is determined based on the number of pixels.

[0047] Optionally, the device further includes an initialization module for:

[0048] Multiple preset categories are pre-defined, and object samples corresponding to each preset category are collected respectively;

[0049] The average height of the object samples corresponding to each preset category is determined as the category height corresponding to the preset category;

[0050] Accordingly, the detection module is further configured to: determine the target category of the target object in the target detection model;

[0051] The calculation module is also used to: determine the category height corresponding to the target category as the actual height of the target object.

[0052] Optionally, the initialization module is used for:

[0053] The target detection model is trained using pre-labeled training images, and the preset threshold of the target detection model is adjusted.

[0054] Optionally, the initialization module is further configured to:

[0055] Acquire multiple original training images, wherein the original training images contain objects to be labeled, and the number of objects to be labeled corresponding to each preset category is greater than a first preset number threshold.

[0056] The number of pixels occupied by each of the objects to be labeled is determined. If the number of pixels is greater than a second preset threshold, the object to be labeled is labeled to obtain the pre-labeled training image.

[0057] According to another aspect of this application, a storage medium is provided that stores a program or instructions thereon, which, when executed by a processor, implement the above-described monocular camera ranging method.

[0058] According to another aspect of this application, an electronic device is provided, including a storage medium and a processor, wherein the storage medium stores a computer program, and the processor executes the computer program to implement the above-described monocular camera ranging method.

[0059] By employing the above technical solution, this application utilizes an object detection algorithm to detect target objects in images, featuring low hardware requirements and high object detection accuracy. Furthermore, the ranging algorithm uses a relatively simple geometric model method, which is simpler and offers better real-time computation compared to machine learning-based ranging algorithms. Additionally, the target bounding box corresponding to the target object and its distance can be displayed on the screen in real time, allowing users to intuitively and instantly understand the distances between the camera and objects in the surrounding environment.

[0060] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0061] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0062] Figure 1 A flowchart illustrating a monocular camera ranging method provided in an embodiment of this application is shown.

[0063] Figure 2 A flowchart illustrating another monocular camera ranging method provided in an embodiment of this application is shown;

[0064] Figure 3 A flowchart illustrating another monocular camera ranging method provided in an embodiment of this application is shown;

[0065] Figure 4 A flowchart illustrating another monocular camera ranging method provided in an embodiment of this application is shown;

[0066] Figure 5 This invention provides a schematic diagram showing the coordinates of the grounding point of a target object in another monocular camera ranging method according to an embodiment of this application.

[0067] Figure 6 This illustration shows a schematic diagram of a monocular ranging application of another monocular camera ranging method provided in an embodiment of this application;

[0068] Figure 7 A structural block diagram of a monocular camera ranging device provided in an embodiment of this application is shown. Detailed Implementation

[0069] The present application will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of the present application can be combined with each other.

[0070] This embodiment provides a monocular camera ranging method, such as... Figure 1 As shown, the method includes the following steps:

[0071] Step 101: Acquire the target image using a monocular camera and input the target image into a preset target detection model;

[0072] Step 102: In the target detection model, determine the target object in the target image and the target bounding box corresponding to the target object;

[0073] Step 103: Determine the image height of the target object in the target image based on the two-dimensional coordinates of the target bounding box;

[0074] Step 104: Determine the target distance between the target object and the monocular camera based on the focal length of the monocular camera, the actual height of the target object, and the image height.

[0075] The monocular camera ranging method provided in this application determines the distance between a target object in an image and the camera based on the image captured by the camera. Specifically, this application mainly includes two steps: target detection and monocular ranging. The target detection step is shown in steps 101-102. First, a target image is acquired using a monocular camera. Then, a preset target detection model is input, and the target image is detected using the target detection model to obtain the target object in the target image.

[0076] The object detection model can be the YOLOv5s algorithm. YOLO is an open-source object detection model that uses a neural network to predict object categories and bounding boxes. YOLOv5s is a version of YOLOv5, and it has the smallest depth and the smallest feature map width in the YOLOv5 series.

[0077] The monocular ranging steps are shown in steps 103-104. After obtaining the target object, the height of the target object in the image and its actual height in the actual scene are determined. Combined with the focal length of the camera, the target distance between the target object and the camera is calculated based on a geometric method.

[0078] Specifically, after identifying the target object using the object detection model and outputting the corresponding bounding box, the grounding point of the target object can be determined using geometric methods. For example, based on the two-dimensional coordinates of the bounding box, the center of the bounding box at its lowest vertical coordinate can be considered the grounding point of the target object. Using the grounding point as the measurement position of the target object, the distance between the grounding point of the target object and the optical center of the camera can be estimated in subsequent calculations, and this distance can be used as the target distance between the target object and the camera.

[0079] The target distance can be calculated using the following formula:

[0080] Where d represents the distance from the target object to the camera, f represents the focal length of the camera, H is the actual height of the target object, and h is the image height of the target object in the target image.

[0081] This embodiment uses an object detection algorithm to detect target objects in images, featuring low hardware requirements and high object detection accuracy. Furthermore, the ranging algorithm employs a relatively simple geometric model method, which is simpler and offers better real-time computation compared to machine learning-based ranging algorithms. Additionally, the target bounding box corresponding to the target object and its distance can be displayed on the screen in real time, allowing users to intuitively understand the distances between the camera and objects in the surrounding environment.

[0082] Further, in step 103, the image height of the target object in the target image is determined based on the two-dimensional coordinates of the target bounding box, including the following steps:

[0083] Step 1031: Determine the number of pixels in the target box along the y-axis based on the two-dimensional coordinates of the target box, and determine the image height based on the number of pixels.

[0084] In step 1031, it is understood that, in the same image, the more pixels a target occupies in the vertical direction, the higher its height. Based on this, this embodiment determines the image height according to the number of pixels in the target bounding box in the y-axis direction, and this number of pixels can be directly used as the image height of the target object.

[0085] Furthermore, as a refinement and extension of the specific implementation of the above embodiments, and to fully illustrate the specific implementation process of this embodiment, another monocular camera ranging method is provided, such as... Figure 2 As shown, the method includes the following steps:

[0086] Step 201: Acquire the target image using a monocular camera and input the target image into a preset target detection model;

[0087] Step 202: In the target detection model, extract image features from the target image and generate a feature map based on the image features;

[0088] Step 203: In the target detection model, the feature map is divided into a preset number of sub-regions, and each sub-region is detected using a predefined detection box to obtain at least one candidate box and the confidence level of the candidate box, wherein the candidate box contains a candidate object.

[0089] Step 204: In the target detection model, based on the preset threshold of the target detection model and the confidence level corresponding to the candidate boxes, the target box is determined in the candidate boxes, and the candidate objects contained in the target box are determined as the target objects.

[0090] In steps 202-204, the target image is input into the feature extraction module of the target detection model. This module extracts image features and generates a feature map, which expresses the deep semantic features of the image. The feature map is then divided into multiple sub-regions. Assuming that a target object may exist within each sub-region, a predefined detection box is used for detection. The output shows the possible target objects and their corresponding detection boxes, i.e., candidate boxes and candidate objects. Simultaneously, the confidence score for each candidate box is also output, indicating the probability that a target object exists within the candidate box. Finally, based on the preset threshold of the target detection model and the confidence score of each candidate box, candidate boxes with lower probabilities are eliminated to obtain the target boxes. The candidate objects contained within these target boxes are then identified as the target objects.

[0091] This embodiment utilizes an object detection model to obtain the target object in the image and the bounding box containing the target object. Since the bounding box in the object detection model is the bounding box of the target object, the height of the target object in the image can be approximately determined in subsequent steps based on the two-dimensional coordinates of the bounding box, and then the distance can be estimated based on this height.

[0092] Preferably, in the embodiments of this application, the preset threshold includes a confidence threshold and an intersection-union ratio (IU) threshold.

[0093] Understandably, confidence level indicates the probability of an event occurring; in YOLO, it indicates the likelihood of an object being detected in the current bounding box. Therefore, a smaller confidence threshold results in more detected objects but lower accuracy; conversely, a larger confidence threshold results in fewer detected objects but higher accuracy. The confidence threshold can be set based on practical application needs, historical experience, and test results to achieve a balance between the number of detected objects and accuracy. In practical applications, a confidence threshold between 0.6 and 0.85 is recommended.

[0094] The Intersection over Union (IoU) threshold is the ratio of the intersection to the union of two detected bounding boxes. In this embodiment, the overlap rate of two candidate boxes can be determined based on their IoU values. If the IoU values ​​of two candidate boxes are greater than the threshold, it is considered that the two candidate boxes have identified the same object. Therefore, a larger IoU threshold is less likely to be zero, resulting in more detected objects, but also a higher probability of duplicate detected objects; a smaller IoU threshold results in fewer detected objects and a lower probability of duplication. The IoU threshold can be set according to actual application needs, historical experience, and test results to achieve a balance between the number of detected objects and the duplication rate. In practical applications, the IoU threshold can be set to a value between 0.4 and 0.6.

[0095] Accordingly, such as Figure 3As shown, in step 204, the target box is determined from the candidate boxes based on the preset threshold of the target detection model and the confidence level corresponding to the candidate boxes, including the following steps:

[0096] Step 2041: In at least one candidate box, remove candidate boxes with a confidence level less than the confidence level threshold to obtain a candidate set;

[0097] Step 2042: Determine the candidate box with the highest confidence in the candidate set as the base box, remove the base box from the candidate set, and add the base box to the base box set;

[0098] Step 2043: Determine the intersection-union ratio (IUR) between each candidate box and the base box in the candidate set, remove candidate boxes with IUR greater than the IUR threshold, and return to the step of determining the candidate box with the highest confidence in the candidate set as the base box, until the candidate set is empty;

[0099] Step 2044: Determine the candidate boxes in the base box set as the target boxes.

[0100] In steps 2041-2044, target boxes are determined from the candidate boxes based on the confidence threshold and the intersection-union ratio (IU) threshold. First, candidate boxes unlikely to contain a target object are eliminated based on the confidence threshold. Specifically, the relationship between the confidence level of each candidate box and the confidence threshold is determined. If the confidence level is less than the confidence threshold, the candidate box is considered to have a low probability of containing a target object and is therefore eliminated. The remaining candidate boxes form a selection set, where the candidate boxes in the selection set have a higher probability of containing a target object.

[0101] For example, consider the following five candidate boxes: candidate box A has a confidence level of 0.8, candidate box B has a confidence level of 0.7, candidate box C has a confidence level of 0.7, candidate box D has a confidence level of 0.6, and candidate box E has a confidence level of 0.3. If the confidence threshold is 0.6, then candidate box E is removed, and the remaining candidate boxes ABCD form the set of candidate boxes.

[0102] Then, duplicate candidate boxes for the target object are removed based on the Intersection over Union (IoU) threshold. Specifically, the candidate box with the highest confidence in the candidate set is taken as the base box. The base box is removed from the candidate set, and the IoU between each remaining candidate box and the base box is determined. If the IoU is greater than the IoU threshold, the candidate box is considered to be the same target object identified by the base box, and the candidate box can be suppressed to avoid duplication. The suppressed candidate box is removed from the candidate set, and the remaining candidate boxes in the candidate set can be regarded as detection boxes that do not overlap with the target object identified by the base box. Therefore, these candidate boxes can be used for the next round of selection. After multiple iterations, until the candidate set is empty, all candidate boxes in the candidate set have been judged. The candidate boxes in the base box set at this time are taken as target boxes. There may be one or more target boxes.

[0103] For example, with an intersection-over-union (IoU) threshold of 0.5, in the candidate box set consisting of candidate boxes ABCD, the candidate box with the highest confidence (i.e., candidate box A) is selected as the base box and added to the base box set. Candidate box A is removed from the candidate set, leaving candidate boxes BCD. The IoU ratios between candidate boxes BCD and base box A are determined. If the IoU ratio between candidate box B and A is 0.8, the IoU ratio between candidate box C and A is 0.3, and the IoU ratio between candidate box D and A is 0.2, then candidate box B is suppressed. Candidate box B is removed from the candidate set, leaving candidate boxes CD, and the next round of selection is performed. In the new round of selection, the candidate box with the highest confidence (i.e., candidate box C) is selected as the base box and added to the base box set. Candidate box C is removed from the candidate set, leaving candidate box D. If the IoU ratio between candidate box D and base box C is greater than 0.5, then candidate box D is suppressed. Remove candidate box D from the candidate set. At this point, the candidate set is empty, and the loop ends. Now, the base box set contains two candidate boxes A and C, which are the target candidate boxes.

[0104] Preferably, in this embodiment of the application, the method further includes the following steps:

[0105] Multiple preset categories are set in advance, and object samples corresponding to each preset category are collected respectively; the average height of the object samples corresponding to each preset category is determined as the category height corresponding to the preset category.

[0106] Accordingly, before determining the target distance between the target object and the monocular camera based on the focal length of the monocular camera, the actual height of the target object, and the image height, the method further includes:

[0107] In the object detection model, the target category of the target object is determined, and the category height corresponding to the target category is determined as the actual height of the target object.

[0108] Specifically, since the actual height of a target object is difficult to measure, its actual height can be set based on its category. Specifically, multiple target object categories to be measured are pre-defined, and several sets of samples are collected for each category. The average height of these samples is then taken as the actual height of the target object in that category. Correspondingly, in the target detection step, the target detection model can classify the detected target objects and then determine their actual height based on their category.

[0109] Preferably, before inputting the target image into a preset target detection model, the method includes:

[0110] The target detection model is trained using pre-labeled training images, and the preset threshold of the target detection model is adjusted.

[0111] In this embodiment, specifically, the training images are run in the training code of the YOLO-V5S algorithm model. After training, a new model file is obtained, which is placed in the read directory specified by the object detection module code. After training the object detection model, preset thresholds are adjusted, including confidence thresholds and cross-ratio thresholds. The adjustment methods and principles are as described above and will not be repeated here.

[0112] Preferably, the annotation method for training images includes the following steps:

[0113] Acquire multiple original training images, wherein the original training images contain objects to be labeled, and the number of objects to be labeled corresponding to each preset category is greater than a first preset threshold.

[0114] The number of pixels occupied by each object to be labeled is determined. If the number of pixels is greater than the second preset threshold, the object to be labeled is labeled, and a pre-labeled training image is obtained.

[0115] In this embodiment, original training images are acquired, and the objects to be labeled in the original training images are labeled to obtain pre-labeled training images, which are then input into the object detection model to train the object detection model. Multiple original training images can be acquired to ensure that the number of objects to be labeled for each preset category is greater than a first preset threshold, thereby improving the detection accuracy of objects in that category. Furthermore, objects that are too small in the image, i.e., those occupying less than a second preset threshold of pixels, can be ignored, and only those occupying a certain number of pixels are labeled.

[0116] Figure 4 A flowchart illustrating another monocular camera ranging method is shown. This embodiment is applied to an electronic device including the following hardware: a 6mm focal length camera, an industrial computer, a monitor, a power supply, etc. The 6mm camera has a resolution of 1920×1080 and uses a USB 3.0 interface for power supply and data transmission; the industrial computer is configured with an i7 CPU, a 2070 Super GPU (8GB VRAM), and 32GB RAM.

[0117] like Figure 4 As shown, the method includes the following steps:

[0118] Step 301: Data labeling;

[0119] In this step, labelImg annotation software is used for annotation. To minimize the probability of false detection, small targets at a distance (e.g., targets occupying less than 10 pixels) are not annotated. To ensure that targets are not missed, at least 100 images containing the same type of target are taken and annotated.

[0120] Step 302: Model training;

[0121] In this step, the training model uses the YOLOv5 algorithm, and the original neural network model uses the smaller YOLOv5s algorithm. To speed up training and enable the algorithm to converge faster, theoretically, the batch_size (the amount of data required for one network parameter update) should be set as large as possible. Due to the limitations of graphics card memory, the batch_size can be set to 32 in this embodiment.

[0122] Step 303: Target detection;

[0123] In this step, the trained model is first copied to a specified path; second, the size of some parameters is adjusted so that some false detection targets can be filtered out, mainly through the "confidence threshold" and "non-maximum suppression" parameters. In this embodiment, the "confidence threshold" is set to 0.65 and the "non-maximum suppression threshold" is set to 0.45; finally, the default model type is set to the YOLO-V5S algorithm model to ensure that the program calls the relevant network parameters of the YOLO-V5S algorithm model when it is executed.

[0124] Step 304: Grounding point determination;

[0125] In this step, such as Figure 5 As shown, the coordinates of the four vertices of the target box are represented as (x, y ... A ,y A ),(x B ,y B ),(x C ,y C ),(x D ,y D And the coordinates of the four vertices satisfy the following relationship:

[0126] y A =y B y C =y D x A =x D x B =x C

[0127] Grounding point P(x) P ,yP The calculation method for ) can be expressed as:

[0128] x P =0.5*(x C +x D )

[0129] y P =y C =y D

[0130] For example, if the coordinates of the four vertices of the target bounding box detected by a certain target are (124, 64), (232, 64), (124, 166), and (232, 166), then the coordinates of the grounding point of the target are (178, 166).

[0131] Step 305: Monocular ranging.

[0132] In this step, such as Figure 6 As shown, a camera P is mounted on an unmanned vehicle A. A pedestrian B and a vehicle C are in front of the unmanned vehicle. HA is the mounting height of the camera on A, Z1 is the distance from pedestrian B to camera P, and Z2 is the distance from vehicle C to camera P. Let the focal length of camera P be f. The upper left corner shows a magnified view of the image inside camera P. y1 and y2 are the images of the grounding points of pedestrian B and vehicle C on the camera's photosensitive element, respectively. The coordinates of y1 are (x...). B ,y B The coordinates of y2 are (x...). C ,y C The specific steps for performing the monocular ranging procedure are as follows:

[0133] Step 3051: Collect height data for 100 vehicles of this category and height data for 100 pedestrians, and measure their average value as the true height of the target in this category. The height of the pedestrians is represented as H. p The height of the vehicle is represented by H. v ;

[0134] Step 3052: The coordinates of the ground contact points of pedestrian B and vehicle C in the image can be calculated using the formula from step 304 above;

[0135] Step 3053: According to Figure 6 Furthermore, the camera imaging principle can be used to obtain the relationship between the height of pedestrian B and vehicle C on the imaging plane and the distance from the target to the camera;

[0136] pedestrian: vehicle:

[0137] Step 3054: Since the heights of pedestrian B and vehicle C on the imaging plane are known and the focal length of the camera can be obtained from intrinsic parameter calibration or data provided by the camera manufacturer, the distances from pedestrian B and vehicle C to the camera can be obtained according to the formula in step 3053 above:

[0138] Distance from pedestrian to camera:

[0139] Distance from vehicle to camera:

[0140] For example, in this embodiment, the camera's focal length is 6mm, the average height of pedestrians is 1.65 meters, and the average height of vehicles is 2 meters. When a pedestrian's height in the camera coordinate system is detected to be 0.002 meters, and a vehicle's height in the camera coordinate system is detected to be 0.0015 meters, the distance from the pedestrian to the camera can be calculated as follows: meters; the distance from the vehicle to the camera is: rice.

[0141] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0142] Furthermore, as a specific implementation of the above-mentioned monocular camera ranging method, this application embodiment provides a monocular camera ranging device, such as... Figure 7 As shown, the device includes: an imaging module, a detection module, and a computing module.

[0143] The imaging module is used to acquire target images using a monocular camera and input the target images into a preset target detection model.

[0144] The detection module is used in the target detection model to determine the target objects and their corresponding bounding boxes in the target image.

[0145] The calculation module is used to determine the image height of the target object in the target image based on the two-dimensional coordinates of the target bounding box; and to determine the target distance between the target object and the monocular camera based on the focal length of the monocular camera, the actual height of the target object, and the image height.

[0146] In specific application scenarios, optionally, the detection module is used for:

[0147] Extract image features from the target image and generate a feature map based on the image features;

[0148] The feature map is divided into a preset number of sub-regions, and each sub-region is detected using a predefined detection box to obtain at least one candidate box and the confidence level of the candidate box. The candidate box contains the candidate object.

[0149] Based on the preset threshold of the target detection model and the confidence level corresponding to the candidate boxes, the target box is determined from the candidate boxes, and the candidate objects contained in the target box are determined to be the target objects.

[0150] In specific application scenarios, the preset thresholds may optionally include a confidence threshold and an intersection-union ratio threshold;

[0151] Accordingly, the detection module is used for:

[0152] In at least one candidate box, remove candidate boxes with a confidence level lower than the confidence threshold to obtain the candidate set;

[0153] The candidate box with the highest confidence in the candidate set is identified as the base box. The base box is removed from the candidate set and added to the base box set.

[0154] Determine the intersection-union ratio (IUR) between each candidate box and the base box in the candidate set, remove candidate boxes with IUR greater than the IUR threshold, and return to the step of determining the candidate box with the highest confidence in the candidate set as the base box, until the candidate set is empty;

[0155] Identify the candidate boxes in the base frame set as the target boxes.

[0156] In specific application scenarios, optionally, the computing module is used for:

[0157] Based on the two-dimensional coordinates of the target bounding box, determine the number of pixels in the y-axis direction of the target bounding box, and then determine the image height based on the number of pixels.

[0158] In specific application scenarios, the device may optionally further include an initialization module, used for:

[0159] Multiple preset categories are set in advance, and object samples corresponding to each preset category are collected respectively;

[0160] Determine the average height of the object samples corresponding to each preset category, and use it as the category height corresponding to the preset category;

[0161] Accordingly, the detection module is also used to: determine the target category of the target object in the target detection model;

[0162] The calculation module is also used to: determine the category height corresponding to the target category as the actual height of the target object.

[0163] In specific application scenarios, the initialization module may optionally be used for:

[0164] The target detection model is trained using pre-labeled training images, and the preset threshold of the target detection model is adjusted.

[0165] In specific application scenarios, the initialization module may optionally also be used for:

[0166] Acquire multiple original training images, wherein the original training images contain objects to be labeled, and the number of objects to be labeled corresponding to each preset category is greater than a first preset threshold.

[0167] The number of pixels occupied by each object to be labeled is determined. If the number of pixels is greater than the second preset threshold, the object to be labeled is labeled, and a pre-labeled training image is obtained.

[0168] It should be noted that other corresponding descriptions of the functional modules involved in the monocular camera ranging device provided in this application embodiment can be found in the corresponding descriptions in the above method, and will not be repeated here.

[0169] Based on the above method, the present application also provides a storage medium storing a computer program that, when executed by a processor, implements the above monocular camera ranging method.

[0170] Based on this understanding, the technical solution of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, or portable hard drive) and includes several instructions to cause an electronic device (such as a personal computer, server, or network device) to execute the methods described in the various implementation scenarios of this application.

[0171] Based on the above, Figures 1 to 6 The method shown, and Figure 7 To achieve the above objectives, this application also provides an electronic device, specifically a personal computer, server, network device, etc., as shown in the virtual device embodiment. The electronic device includes a storage medium and a processor; the storage medium stores a computer program; the processor executes the computer program to achieve the above-described objectives. Figures 1 to 6 The method for ranging using a monocular camera is shown.

[0172] Optionally, the electronic device may also include a user interface, a network interface, a camera, radio frequency (RF) circuitry, sensors, audio circuitry, a Wi-Fi module, etc. The user interface may include a display screen, input units such as a keyboard, etc., and optional user interfaces may also include USB interfaces, card reader interfaces, etc. The network interface may optionally include standard wired interfaces, wireless interfaces (such as Bluetooth interfaces, Wi-Fi interfaces), etc.

[0173] Those skilled in the art will understand that the electronic device structure provided in this embodiment does not constitute a limitation on the electronic device, and may include more or fewer components, or combine certain components, or have different component arrangements.

[0174] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages and stores the hardware and software resources of the electronic device, supporting the operation of information processing programs and other software and / or programs. The network communication module is used to enable communication between various controls within the storage medium, as well as communication with other hardware and software within the physical device.

[0175] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platform, or it can be implemented by hardware.

[0176] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the units or processes shown in the drawings are not necessarily essential for implementing this application. Those skilled in the art will understand that the units in the apparatus of the embodiment can be distributed within the apparatus of the embodiment as described, or can be located in one or more apparatuses different from this embodiment, with corresponding changes. The units of the above-described embodiment can be combined into one unit, or further divided into multiple sub-units.

[0177] The serial numbers in this application are for descriptive purposes only and do not represent the superiority or inferiority of any particular implementation scenario. The above disclosures are merely a few specific implementation scenarios of this application; however, this application is not limited thereto, and any variations conceived by those skilled in the art should fall within the protection scope of this application.

Claims

1. A method for ranging using a monocular camera, characterized in that, The method includes: A target image is acquired using a monocular camera, and the target image is input into a preset target detection model; In the target detection model, the target object in the target image and the target bounding box corresponding to the target object are determined; Based on the two-dimensional coordinates of the target bounding box, determine the image height of the target object in the target image; The target distance between the target object and the monocular camera is determined based on the focal length of the monocular camera, the actual height of the target object, and the image height. The method further includes: Multiple preset categories are pre-defined, and object samples corresponding to each preset category are collected respectively; The average height of the object samples corresponding to each preset category is determined as the category height corresponding to the preset category; Accordingly, before determining the target distance between the target object and the monocular camera based on the focal length of the monocular camera, the actual height of the target object, and the image height, the method further includes: In the target detection model, the target category of the target object is determined, and the category height corresponding to the target category is determined as the actual height of the target object.

2. The method according to claim 1, characterized in that, Determining the target object in the target image and the target bounding box corresponding to the target object includes: Extract image features from the target image and generate a feature map based on the image features; The feature map is divided into a preset number of sub-regions, and each sub-region is detected using a predefined detection box to obtain at least one candidate box and the confidence level of the candidate box, wherein the candidate box contains a candidate object. Based on the preset threshold of the target detection model and the confidence level corresponding to the candidate boxes, a target box is determined in the candidate boxes, and the candidate objects contained in the target box are determined to be the target objects.

3. The method according to claim 2, characterized in that, The preset thresholds include a confidence threshold and an intersection-over-union (IoU) threshold; Accordingly, determining the target box from the candidate boxes based on the preset threshold of the target detection model and the confidence level corresponding to the candidate boxes includes: In the at least one candidate box, candidate boxes with a confidence level less than the confidence threshold are removed to obtain a candidate set; The candidate box with the highest confidence in the candidate set is determined as the base box. The base box is removed from the candidate set and added to the base box set. Determine the intersection-union ratio (IUR) between each candidate box in the candidate set and the base box, remove candidate boxes whose IUR is greater than the IUR threshold, and return to the step of determining the candidate box with the highest confidence in the candidate set as the base box, until the candidate set is an empty set; The candidate boxes in the set of base frames are determined as the target boxes.

4. The method according to claim 1, characterized in that, Determining the image height of the target object in the target image based on the two-dimensional coordinates of the target bounding box includes: Based on the two-dimensional coordinates of the target bounding box, the number of pixels in the target bounding box along the y-axis is determined, and the image height is determined based on the number of pixels.

5. The method according to claim 1, characterized in that, Before inputting the target image into the preset target detection model, the method includes: The target detection model is trained using pre-labeled training images, and the preset threshold of the target detection model is adjusted.

6. The method according to claim 5, characterized in that, Before training the object detection model using pre-labeled training images, the method further includes: Multiple original training images are acquired, wherein the original training images contain objects to be labeled, and the number of objects to be labeled corresponding to each preset category is greater than a first preset number threshold. The number of pixels occupied by each of the objects to be labeled is determined. If the number of pixels is greater than a second preset threshold, the object to be labeled is labeled to obtain the pre-labeled training image.

7. A monocular camera ranging device, characterized in that, The device includes: The imaging module is used to acquire target images using a monocular camera and input the target images into a preset target detection model. The detection module is used to determine the target object in the target image and the target bounding box corresponding to the target object in the target detection model; The calculation module is used to determine the image height of the target object in the target image based on the two-dimensional coordinates of the target box; and to determine the target distance between the target object and the monocular camera based on the focal length of the monocular camera, the actual height of the target object, and the image height. An initialization module is used to pre-set multiple preset categories and collect object samples corresponding to each preset category; and determine the average height of the object samples corresponding to each preset category as the category height corresponding to the preset category. The detection module is also used to determine the target category of the target object in the target detection model; The calculation module is also used to determine that the category height corresponding to the target category is the actual height of the target object.

8. A storage medium having a program or instructions stored thereon, characterized in that, When the program or instructions are executed by the processor, they implement the method as described in any one of claims 1 to 6.

9. An electronic device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the program, it implements the method of any one of claims 1 to 6.

Images