Camera height estimation method, apparatus, readable storage medium and electronic device

Abstract

Embodiments of the present disclosure disclose a camera height estimation method and device, a computer readable storage medium and an electronic device. The method comprises: obtaining a to-be-identified image of a target lane photographed by a camera on a mobile device at a target position; determining reference lane line information corresponding to the target lane in a preset map according to the target position; extracting perceived lane line information from the to-be-identified image; determining a real-time reference width of the target lane based on the reference lane line information; determining a real-time perceived width of the target lane based on the perceived lane line information; determining a camera height estimation ratio based on the real-time perceived width and the real-time reference width; and determining a current height of the camera based on the camera height estimation ratio and a preset initial height of the camera. Embodiments of the present disclosure can estimate the camera height in real time based on a visual perception manner, improve the accuracy of camera height estimation, and further help to improve the accuracy of environment perception and vehicle positioning using the camera height.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a camera height estimation method, apparatus, computer-readable storage medium, and electronic device. Background Technology

[0002] In the field of autonomous driving, when autonomous vehicles are driving on roads and detecting road conditions, they rely on intrinsic and extrinsic parameters such as camera height to project features such as lane lines from captured images into the Vehicle Coordinate System (VCS). This VCS is then matched with corresponding features in a high-precision map to determine the vehicle's current position. However, since camera extrinsic parameters do not change after factory calibration, the camera height in a real-world vehicle varies depending on factors such as the number of passengers, tire pressure, and suspension height. This results in a scaling effect between the image information projected using a fixed camera height extrinsic parameter and the map, affecting the accuracy and stability of matching real-time perceived features with features recorded in the map, and consequently impacting vehicle positioning accuracy. Summary of the Invention

[0003] To address the aforementioned technical problems, embodiments of this disclosure provide a camera height estimation method, apparatus, computer-readable storage medium, and electronic device to achieve dynamic estimation of camera height and improve the accuracy of environmental perception and vehicle positioning based on images captured by the camera.

[0004] Embodiments of this disclosure provide a camera height estimation method, which includes: acquiring an image of a target lane captured by a camera on a mobile device at a target location; determining reference lane line information corresponding to the target lane in a preset map based on the target location; extracting perceived lane line information from the image to be identified; determining a real-time reference width of the target lane based on the reference lane line information; determining a real-time perceived width of the target lane based on the perceived lane line information; determining a camera height estimation ratio based on the real-time perceived width and the real-time reference width; and determining the current height of the camera based on the camera height estimation ratio and a preset initial height of the camera.

[0005] According to another aspect of the present disclosure, a camera height estimation apparatus is provided. The apparatus includes: an acquisition module for acquiring an image to be identified captured by a camera on a mobile device at a target location of a target lane; a first determination module for determining reference lane line information corresponding to the target lane in a preset map based on the target location; an extraction module for extracting perceived lane line information from the image to be identified; a second determination module for determining a real-time reference width of the target lane based on the reference lane line information; a third determination module for determining a real-time perceived width of the target lane based on the perceived lane line information; a fourth determination module for determining a camera height estimation ratio based on the real-time perceived width and the real-time reference width; and a fifth determination module for determining the current height of the camera based on the camera height estimation ratio and a preset initial height of the camera.

[0006] According to another aspect of the present disclosure, a computer-readable storage medium is provided that stores a computer program for execution by a processor to perform the camera height estimation method described above.

[0007] According to another aspect of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; and a processor for reading executable instructions from the memory and executing the instructions to implement the camera height estimation method described above.

[0008] According to another aspect of the present disclosure, a computer program product is provided, the computer program product including computer program instructions that, when executed by an instruction processor, perform the camera height estimation method proposed in the present disclosure.

[0009] Based on the camera height estimation method, apparatus, computer-readable storage medium, and electronic device provided in the above embodiments of this disclosure, the real-time perceived width and real-time reference width are obtained by calculating the lane width of the perceived lane line information and the reference lane line information in the map obtained in real time. Based on the real-time perceived width and real-time reference width, a camera height estimation ratio is determined. Finally, based on the camera height estimation ratio and the initial height of the camera, the current height of the camera is determined. The embodiments of this disclosure achieve real-time estimation of camera height based on visual perception and fully utilize existing reference lane line information, improving the accuracy of camera height estimation and thus contributing to improving the accuracy of environmental perception and vehicle positioning using camera height.

[0010] The technical solutions of this disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0011] Figure 1 This is the system diagram to which this disclosure applies;

[0012] Figure 2 This is a schematic flowchart of a camera height estimation method provided in an exemplary embodiment of this disclosure;

[0013] Figure 3 This is a flowchart illustrating a camera height estimation method provided in another exemplary embodiment of this disclosure;

[0014] Figure 4 This is a flowchart illustrating a camera height estimation method provided in another exemplary embodiment of this disclosure;

[0015] Figure 5 This is a flowchart illustrating a camera height estimation method provided in another exemplary embodiment of this disclosure;

[0016] Figure 6 This is a flowchart illustrating a camera height estimation method provided in another exemplary embodiment of this disclosure;

[0017] Figure 7A This is a schematic diagram illustrating the distance between a reference sampling point and a reference lane line provided in another exemplary embodiment of this disclosure;

[0018] Figure 7B This is a schematic diagram illustrating the distance between a sensing sampling point and a sensing lane line provided in another exemplary embodiment of this disclosure;

[0019] Figure 8 This is a flowchart illustrating a camera height estimation method provided in another exemplary embodiment of this disclosure;

[0020] Figure 9 This is a flowchart illustrating a camera height estimation method provided in another exemplary embodiment of this disclosure;

[0021] Figure 10 This is a flowchart illustrating a camera height estimation method provided in another exemplary embodiment of this disclosure;

[0022] Figure 11 This is a flowchart illustrating a camera height estimation method provided in another exemplary embodiment of this disclosure;

[0023] Figure 12 This is a schematic diagram of the structure of a camera height estimation device provided in an exemplary embodiment of the present disclosure;

[0024] Figure 13 This is a schematic diagram of the structure of a camera height estimation device provided in another exemplary embodiment of this disclosure;

[0025] Figure 14 This is a structural diagram of an electronic device provided in an exemplary embodiment of this disclosure. Detailed Implementation

[0026] To explain this disclosure, exemplary embodiments of the disclosure will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the disclosure, and not all of them. It should be understood that the disclosure is not limited to exemplary embodiments.

[0027] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0028] Application Overview

[0029] Current autonomous driving technology lacks a solution for dynamically estimating camera height. This results in a scaling of the image information obtained by projecting it using a fixed camera height extrinsic parameter compared to existing map information. This affects the accuracy and stability of matching visual perception features with the map, thereby impacting vehicle positioning accuracy.

[0030] This disclosure aims to solve the problem by proposing a camera height estimation method. By determining the perceived width and reference width of the lane line, a camera height estimation ratio is determined based on the perceived width and reference width. The real-time height of the camera is then determined based on the camera height ratio, which helps to accurately perform road recognition and vehicle positioning using the real-time determined camera height.

[0031] Exemplary System

[0032] Figure 1 An exemplary system architecture 100 is shown that can be applied to camera height estimation methods or camera height estimation devices according to embodiments of the present disclosure.

[0033] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102, a server 103, and a mobile device 104. The terminal device 101 may be mounted on the mobile device 104. The network 102 serves as the medium for providing a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0034] Users can use terminal device 101 to interact with server 103 via network 102 to receive or send messages, etc. Various communication client applications, such as map applications and navigation applications, can be installed on terminal device 101.

[0035] Terminal device 101 can be various electronic devices, including but not limited to in-vehicle terminals, mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), etc. Terminal device 101 is typically located on mobile device 104.

[0036] The mobile device 104 can be any type of device such as a car, bicycle, or drone, and is equipped with a camera 105, which can capture images of the environment surrounding the mobile device 104.

[0037] Server 103 can be a server that provides various services, such as a background image server that recognizes images uploaded by terminal device 101. The background image server can store preset maps, use the received images to estimate the camera height, and feed the camera height back to terminal device 101.

[0038] It should be noted that the spatial information prediction method provided in the embodiments of this disclosure can be executed by the server 103 or by the terminal device 101. Accordingly, the spatial information prediction device can be set in the server 103 or in the terminal device 101.

[0039] It should be understood that Figure 1 The number of terminal devices 101, network 102, server 103, and mobile devices 104 in the diagram is merely illustrative. Depending on implementation needs, any number of terminal devices 101, network 102, server 103, and mobile devices 104 can be included. If images do not need to be acquired remotely, the above system architecture may exclude the network and server, including only mobile devices 104, terminal devices 101, and camera 105.

[0040] Exemplary methods

[0041] Figure 2 This is a schematic flowchart of a camera height estimation method provided in an exemplary embodiment of this disclosure. This embodiment can be applied to electronic devices (such as...). Figure 1 On the terminal device 101 or server 103 shown, such as Figure 2 As shown, the method includes the following steps:

[0042] Step 201: Obtain the image to be identified taken by the camera on the mobile device at the target location of the target lane.

[0043] The aforementioned mobile devices can be, for example, Figure 1 The mobile device 104 shown above contains an image to be identified, which is composed of... Figure 1The image was captured by camera 105. The target lane can be any designated lane, typically the lane the mobile device is currently in. The camera on the mobile device captures images of the road ahead in real time, thus obtaining an image to be identified that includes the target lane. The target location can be the real-time location of the mobile device, which can be obtained through a positioning device on the mobile device.

[0044] Step 202: Based on the target location, determine the reference lane line information corresponding to the target lane in the preset map.

[0045] The preset map can be a pre-built electronic map, which can be a high-precision map containing various information such as roads, buildings, and road signs. The electronic device can determine from the map the lane line information representing the target lane within the area corresponding to the target location. Typically, lane line information can include a series of coordinate values; the curve formed by these coordinate values ​​represents the lane line.

[0046] Step 203: Extract perceived lane line information from the image to be identified.

[0047] Specifically, electronic devices can use image-based object detection methods (such as neural network-based object detection methods) to detect lane lines in the image to be recognized, thereby obtaining perceived lane line information. The perceived lane line information can also include a series of coordinate values, and the curve formed by these coordinate values ​​represents the lane line.

[0048] Step 204: Determine the real-time reference width of the target lane based on the reference lane line information.

[0049] Specifically, electronic devices can determine the distance between the curves constituting the target lane, as represented by reference lane line information, and determine the real-time reference width of the target lane. For example, the average distance between multiple points included in any one of the two lane lines and the other lane line can be calculated as the real-time reference width.

[0050] Step 205: Determine the real-time perceived width of the target lane based on the perceived lane line information.

[0051] Specifically, the real-time perceived width of the target lane can be determined using the same method as in step 204.

[0052] Step 206: Determine the camera height estimation ratio based on the real-time perceived width and the real-time reference width.

[0053] Specifically, the ratio of the real-time perceived width to the real-time reference width can be determined as the camera height estimation ratio. Optionally, to improve the accuracy of the camera height estimation ratio, the average of the multiple camera height estimation ratios corresponding to the multiple images included in the sliding window can be determined as the current camera height estimation ratio based on a sliding window method.

[0054] Step 207: Determine the current height of the camera based on the camera height estimation ratio and the initial height of the camera.

[0055] Specifically, since there is a linear relationship between the size of objects in real-time captured images and the size of real objects, the aforementioned camera height estimation ratio can be used as a correction ratio between the actual camera height and the initial height, which is obtained through pre-calibration of the camera. That is, the estimated camera height Z can be calculated using the following formula:

[0056] Z = Z calib *K

[0057] Among them, Z calib Let K be the initial height, and K be the ratio of the above camera height estimation.

[0058] The method provided in the above embodiments of this disclosure calculates the lane width from the perceived lane line information obtained in real time and the reference lane line information in the map, respectively, to obtain the real-time perceived width and the real-time reference width. Based on the real-time perceived width and the real-time reference width, a camera height estimation ratio is determined. Finally, based on the camera height estimation ratio and the camera's initial height, the current height of the camera is determined. The embodiments of this disclosure achieve real-time estimation of camera height based on visual perception and fully utilize existing reference lane line information, improving the accuracy of camera height estimation and thus contributing to improved accuracy in environmental perception and vehicle positioning using camera height.

[0059] In some alternative implementations, such as Figure 3 As shown, step 201 includes:

[0060] Step 2011: Obtain the original image of the target lane taken by the camera on the mobile device.

[0061] The original image is the image generated directly after the camera takes the picture.

[0062] Step 2012: Determine the image acquisition status information corresponding to the original image.

[0063] The image acquisition status information corresponding to the original image indicates the movement status of the mobile device at the time the original image was captured. Optionally, the image acquisition status information includes, but is not limited to, at least one of the following: the current positioning status information of the mobile device, the speed of the mobile device, the turning angle, the time interval between two valid images to be identified, and the movement length of the mobile device being greater than a preset length (e.g., 10 meters). The positioning status information can be determined by satellite positioning signals. For example, when the electronic device cannot receive a positioning signal, positioning status information indicating a positioning anomaly is generated; when the positioning signal received by the electronic device has a large error compared to the previously received positioning signal, positioning status information indicating a positioning anomaly is generated.

[0064] Step 2013: In response to the image acquisition status information meeting the preset recognition conditions, the image to be recognized is obtained based on the original image.

[0065] The above recognition conditions are those under which the camera can capture a valid image to be recognized. As an example, the recognition conditions include, but are not limited to, at least one of the following: the positioning status information indicates normal positioning; the mobile device's speed is within a preset speed range (e.g., greater than or equal to 50 km / h); and the turning angle is within a preset angle range (e.g., less than or equal to 0.02 rad). If the image acquisition status information meets the recognition conditions, the original image is identified as the image to be recognized.

[0066] This embodiment judges the image acquisition status information to obtain images that meet the recognition conditions, thereby filtering out high-quality images for recognition, improving the accuracy of extracting perceived lane line information, and thus improving the accuracy of estimating camera height.

[0067] In some alternative implementations, such as Figure 4 As shown, step 203 includes:

[0068] Step 2031: Extract the initial lane line information representing the target lane from the image to be identified.

[0069] The initial lane line information is the lane line information extracted directly from the image to be identified using object detection methods.

[0070] Step 2032: Perform lane line quality detection on the initial lane line information to obtain the quality detection results.

[0071] The lane line quality detection method may include, but is not limited to, at least one of the following: parallelism detection, curvature detection, and matching degree detection of lane lines with reference lane lines.

[0072] Step 2033: In response to the quality detection result meeting the lane line quality conditions, generate perceived lane line information based on the initial lane line information.

[0073] Lane line quality conditions are the conditions under which lane lines can be used for real-time reference width calculation. For example, the quality inspection result is determined to meet the lane line quality conditions when at least one of the following is true: the parallelism detection result of the lane line is within a preset parallelism value range, the lane line area is within a preset curvature range, and the length matching the reference lane line is greater than or equal to a preset length.

[0074] This embodiment obtains high-quality perceived lane line information by performing quality inspection on the initial lane line information, thereby filtering out high-quality perceived lane line information, improving the accuracy of determining the real-time perceived width of the target lane, and thus improving the accuracy of estimating the camera height.

[0075] In some alternative implementations, such as Figure 5 As shown, step 2032 may include at least one of the following sub-steps:

[0076] Step 20321: Determine the parallelism information of the two lane lines represented by the initial lane line information, and determine the quality inspection result of the initial lane line information based on the parallelism information.

[0077] Optionally, parallelism information can be obtained as follows: For either of the two lane lines, determine the maximum and minimum distances between each point on that lane line and the other lane line; these maximum and minimum values ​​constitute the parallelism information. If the difference between the maximum and minimum values ​​is less than or equal to a first preset distance threshold (e.g., 1 meter), a quality detection result indicating that the initial lane line information meets the parallelism condition is generated. Optionally, if the curvature of the initial lane line information is greater than or equal to a preset curvature (e.g., 0.4), the first preset distance threshold can be relaxed to 2 meters.

[0078] Step 20322: Determine the matching degree information between the initial lane line information and the reference lane line information. Based on the matching degree information, determine the quality inspection result of the initial lane line information.

[0079] The matching degree information indicates whether the initial lane line information matches the reference lane line information. Optionally, the matching degree information can be obtained as follows: if the effective matching length in front of the mobile device is greater than or equal to a preset length (e.g., 30 meters), a quality detection result indicating that the initial lane line information matches the reference lane line information is generated. The condition for the above effective matching can be: the distance between the lane lines represented by the corresponding initial lane line information and the lane lines represented by the reference lane line information is less than or equal to a second preset distance threshold.

[0080] Step 20323: Determine the lane line curvature based on the initial lane line information, and determine the quality inspection result of the initial lane line information based on the lane line curvature and curvature reference value.

[0081] Specifically, if the lane curvature is less than or equal to the curvature reference value (e.g., 10) -4 This generates a quality inspection result indicating that the initial lane line information meets the curvature condition.

[0082] It should be noted that at least one of the three quality inspection results obtained from steps 20321-20323 can be selected as the final quality inspection result. Accordingly, if at least one of the three quality inspection results meets the corresponding lane line quality conditions, the initial lane line information is determined as the perceived lane line information.

[0083] This embodiment provides multiple methods for determining quality detection results, which can accurately judge the quality of the sensed lane line information from multiple dimensions, thereby helping to further filter out high-quality sensed lane line information and improve the accuracy of estimating camera height.

[0084] In some alternative implementations, such as Figure 6 As shown, step 204 includes:

[0085] Step 2041: Determine the corresponding first reference sampling point and second reference sampling point from the reference lane line information.

[0086] The first reference sampling point and the second reference sampling point are located on the first reference lane line and the second reference lane line represented by the reference lane line information, respectively. Typically, samples are taken from the two lane lines constituting the target lane at preset intervals, and two sampling points with the same longitudinal (i.e., directly in front of the mobile device) coordinates on the two lane lines are determined as the first reference sampling point and the second reference sampling point. For example... Figure 7A As shown, 701 is the first reference lane line, 702 is the second reference lane line, and the longitudinal coordinates of the first reference sampling point 7011 and the second reference sampling point 7021 are the same.

[0087] Step 2042: Determine the first distance between the first reference sampling point and the second reference lane line, and determine the second distance between the second reference sampling point and the first reference lane line.

[0088] like Figure 7A As shown, d meas_l and d meas_r The first distance and the second distance are the distances from the first reference sampling point 7011 and the second reference sampling point 7021 to the corresponding reference lane lines, respectively. Typically, the perception curve is constructed using a cubic curve equation, i.e., y = c0 + c1x + c2x. 2 +c3x 3 Therefore, the linear equation y = c0 + c1x, which is approximated by the perception curve, can be used to calculate the distance from the reference sampling point to the straight line as the distance from the reference sampling point to the lane line.

[0089] Step 2043: Determine the real-time reference width based on the first distance and the second distance.

[0090] Optionally, the average of the first and second distances can be used as the real-time reference width.

[0091] This embodiment simplifies the process of determining the real-time reference width by identifying reference sampling points from reference lane line information and using these points to determine the real-time reference width, thereby improving the efficiency of estimating camera height.

[0092] In some alternative implementations, such as Figure 8 As shown, step 205 includes:

[0093] Step 2051: Determine the corresponding first and second sensing sampling points from the perceived lane line information.

[0094] The first and second sensing sampling points are located on the first and second sensing lane lines represented by the sensing lane line information, respectively. The method for obtaining the first and second sensing sampling points is basically the same as step 2041 above, and will not be repeated here. Figure 7B As shown, 703 and 704 are two lane lines represented by the perceived lane line information, and the longitudinal coordinates of the first perception sampling point 7031 and the second perception sampling point 7041 are the same.

[0095] Step 2052: Determine the third distance between the first sensing sampling point and the second sensing lane line, and determine the fourth distance between the second sensing sampling point and the first sensing lane line.

[0096] The method for obtaining the distance between the sensing sampling point and the sensing lane line is basically the same as step 2042 above, and will not be repeated here. Figure 7B As shown, d map_l and d map_r The third and fourth distances are the distances from the first sensing sampling point 7031 and the second sensing sampling point 7041 to the relative sensing lane line, respectively.

[0097] Step 2053: Determine the real-time sensing width based on the third and fourth distances.

[0098] Optionally, the average of the third and fourth distances can be used as the real-time reference width.

[0099] This embodiment simplifies the process of determining the real-time sensing width by identifying sensing sampling points from the perceived lane line information and using these sampling points to determine the real-time sensing width, thereby helping to improve the efficiency of estimating camera height.

[0100] In some alternative implementations, such as Figure 9 As shown, step 206 includes:

[0101] Step 2061: Obtain the historical perceived width set and historical reference width set corresponding to the first distance traveled by the mobile device.

[0102] The first path can be a distance already traveled by the mobile device, ending at the device's current location, such as the most recently traveled 5 kilometers. The historical perceived width set and the historical reference width set are sets of multiple perceived widths and multiple reference widths obtained based on the captured frames of images during the first path's travel. Since the steps provided in this embodiment are executed in real-time during the mobile device's movement, the image sequence corresponding to the first path is considered as an image sequence included in a sliding window. As the mobile device moves, the image sequence included in the sliding window is continuously updated, adding the currently captured image to the sliding window and removing the oldest captured image from the sliding window.

[0103] Step 2062: Determine whether the real-time perceived width and the real-time reference width meet the preset width deviation conditions.

[0104] The width deviation condition is that the deviation of the real-time reference width must meet the conditions for calculating the camera height ratio. For example, a width deviation threshold (e.g., 0.15 meters) can be set. If the deviations of both the real-time perceived width and the real-time reference width from the preset baseline width are less than or equal to the width deviation threshold, then the real-time perceived width and the real-time reference width are determined to meet the width deviation condition.

[0105] Step 2063: If the real-time sensing width and the real-time reference width meet the width deviation condition, determine the first estimated sensing width based on the historical sensing width set and the real-time sensing width, and determine the first estimated reference width based on the historical reference width set and the real-time reference width.

[0106] Optionally, the average value, median value, etc. of the historical sensing width and real-time sensing width included in the historical sensing width set can be calculated, and the calculated value can be determined as the first estimated sensing width. Similarly, the average value, median value, etc. of the historical reference width and real-time reference width included in the historical reference width set can be calculated, and the calculated value can be determined as the first estimated reference width.

[0107] Step 2064: Determine the camera height estimation ratio based on the first estimated perception width and the first estimated reference width.

[0108] Typically, the ratio of the first estimated sensing width to the first estimated reference width can be determined as the camera height ratio. Alternatively, the ratio of the first estimated reference width to the first estimated sensing width can also be determined as the camera height ratio.

[0109] This embodiment filters the real-time perceived width and real-time reference width by setting width deviation conditions, obtaining real-time perceived width and real-time reference width with higher accuracy. It also combines the historical perceived width set and the historical reference width set to determine the first estimated perceived width and the first estimated reference width, which can improve the accuracy of lane width estimation during mobile device operation, and thus improve the accuracy of determining the camera height ratio.

[0110] In some alternative implementations, such as Figure 10 As shown, step 2062 includes:

[0111] Step 20621: Extract the target historical perception width subset from the historical perception width set, and extract the target historical reference width subset from the historical reference width set.

[0112] Here, the target history perceived width subset and the target history reference width subset correspond to the second distance traveled. The second distance, like the first distance, can end at the current location of the mobile device. Typically, to improve processing speed, the second distance can be shorter than the first distance, for example, the most recently traveled 2 kilometers. The image sequence corresponding to this second distance can be included as another image sequence within a shorter sliding window.

[0113] Step 20622: Determine the second estimated sensing width based on the target historical sensing width subset and the real-time sensing width, and determine the second estimated reference width based on the target historical reference width subset and the real-time reference width.

[0114] Optionally, the average value, median value, etc. of the historical sensing width and real-time sensing width included in the historical sensing width set can be calculated, and the calculated value can be determined as the second estimated sensing width. Similarly, the average value, median value, etc. of the historical reference width and real-time reference width included in the historical reference width set can be calculated, and the calculated value can be determined as the second estimated reference width.

[0115] Step 20623: In response to the fact that the deviation between the real-time perceived width and the second estimated perceived width is within the preset perceived width deviation, and the real-time reference width and the second estimated reference width are within the preset reference width deviation, it is determined that the real-time perceived width and the real-time reference width meet the preset width deviation conditions.

[0116] This embodiment statistically analyzes the target historical perceived width subset and the target historical reference width subset corresponding to the second route, and judges the width deviation based on the statistically obtained second estimated perceived width and second estimated reference width. This can avoid the influence of abnormal data during the driving process, thereby improving the accuracy of the lane width estimated by the mobile device during driving, and thus improving the accuracy of determining the camera height ratio.

[0117] In some alternative implementations, such as Figure 11 As shown, step 2064 includes:

[0118] Step 20641: Based on the first estimated perceptual width and the first estimated reference width, determine the real-time camera height ratio corresponding to the image to be identified.

[0119] Typically, the ratio of the first estimated sensing width to the first estimated reference width can be determined as the camera height ratio. Alternatively, the ratio of the first estimated reference width to the first estimated sensing width can also be determined as the camera height ratio.

[0120] Step 20642: Obtain the set of historical camera height ratios corresponding to the first route.

[0121] The historical camera height ratio set is the set of camera height ratios determined at corresponding moments in each frame of an image during the first leg of the mobile device's journey.

[0122] Step 20643: Determine the distribution range of camera height ratios in the real-time camera height ratio set and the historical camera height ratio set.

[0123] The range of camera height ratio distribution here can be the difference between the maximum and minimum values ​​in the real-time camera height ratio and the historical camera height ratio set, that is, the difference between the maximum and minimum values ​​of each camera height ratio contained in the sliding window corresponding to the first path.

[0124] Step 20644: Determine the estimated camera height ratio based on the camera height ratio distribution range and the real-time camera height ratio.

[0125] As an example, when the difference between the maximum and minimum values ​​is less than or equal to a preset height range, the real-time camera height ratio can be determined as the camera height estimation ratio used for camera height estimation.

[0126] This embodiment determines the estimated camera height ratio by statistically analyzing a set of historical camera height ratios and based on the distribution range of these ratios. This can reduce the error in camera height ratios caused by lane line recognition deviations.

[0127] In some alternative implementations, step 20644 can be performed as follows:

[0128] In response to the camera height ratio distribution range conforming to the height ratio distribution condition, the real-time camera height ratio is determined as the estimated camera height ratio.

[0129] Typically, the height ratio distribution condition can be defined as the convergence of the camera height ratio distribution range, meaning that the difference between the maximum and minimum values ​​of the various camera height ratios contained in the sliding window corresponding to the first path is less than or equal to a preset value. When the camera height ratio distribution range converges, it indicates that the camera height ratio is being output stably. Therefore, the currently obtained real-time camera height ratio can be used as the estimated camera height ratio used to determine the camera height.

[0130] In response to the fact that the range of camera height ratio distribution does not meet the height ratio distribution conditions, the previously determined historical camera height estimation ratio will be used as the camera height estimation ratio.

[0131] When the range of camera height ratio distribution does not meet the height ratio distribution conditions, it means that the range of camera height ratio distribution does not converge, that is, the camera height ratio obtained in the recent period has changed significantly. Therefore, the historical camera height estimation ratio obtained in the most recent converged state can be used as the camera height estimation ratio used to determine the camera height.

[0132] This embodiment can accurately determine the changes in the camera height ratio obtained in a recent period by setting height ratio distribution conditions. The estimated camera height ratio output based on the changes can accurately reflect the current camera height, thereby improving the accuracy of real-time camera height estimation.

[0133] Exemplary device

[0134] Figure 12 This is a schematic diagram of a camera height estimation device provided in an exemplary embodiment of this disclosure. This embodiment can be applied to electronic devices, such as... Figure 12 As shown, the camera height estimation device includes: an acquisition module 1201, used to acquire an image of the target lane captured by a camera on a mobile device at a target location; a first determination module 1202, used to determine reference lane line information corresponding to the target lane in a preset map based on the target location; an extraction module 1203, used to extract perceived lane line information from the image to be identified; a second determination module 1204, used to determine the real-time reference width of the target lane based on the reference lane line information; a third determination module 1205, used to determine the real-time perceived width of the target lane based on the perceived lane line information; a fourth determination module 1206, used to determine the camera height estimation ratio based on the real-time perceived width and the real-time reference width; and a fifth determination module 1207, used to determine the current height of the camera based on the camera height estimation ratio and the camera's preset initial height.

[0135] In this embodiment, the acquisition module 1201 can acquire the image to be identified taken by the camera on the mobile device at the target location of the target lane.

[0136] The aforementioned mobile devices can be, for example, Figure 1 The mobile device 104 shown above contains an image to be identified, which is composed of... Figure 1 The image was captured by camera 105. The target lane can be any designated lane; typically, it is the lane the mobile device is currently in. The camera on the mobile device captures images of the road ahead in real time, thus obtaining an image to be identified that includes the target lane. The target location can be the real-time location of the mobile device, which can be obtained through a positioning device on the mobile device.

[0137] In this embodiment, the first determining module 1202 can determine the reference lane line information corresponding to the target lane in a preset map based on the target location.

[0138] The preset map can be a pre-built electronic map, which can be a high-precision map containing various information such as roads, buildings, and road signs. The electronic device can determine from the map the lane line information representing the target lane within the area corresponding to the target location. Typically, lane line information can include a series of coordinate values; the curve formed by these coordinate values ​​represents the lane line.

[0139] In this embodiment, the extraction module 1203 can extract perceived lane line information from the image to be identified.

[0140] Specifically, the extraction module 1203 can use an image-based object detection method (e.g., a neural network-based object detection method) to detect lane lines in the image to be identified, thereby obtaining perceived lane line information. The perceived lane line information may also include a series of coordinate values, and the curve formed by these coordinate values ​​represents the lane line.

[0141] In this embodiment, the second determining module 1204 can determine the real-time reference width of the target lane based on the reference lane line information.

[0142] Specifically, the second determining module 1204 can determine the distance between the curves constituting the target lane as represented by the reference lane line information, and determine the real-time reference width of the target lane. For example, the average distance between multiple points included in any one of the two lane lines and the other lane line can be calculated as the real-time reference width.

[0143] In this embodiment, the third determining module 1205 can determine the real-time perceived width of the target lane based on the perceived lane line information.

[0144] Specifically, the real-time perceived width of the target lane can be determined using the same method as the second determining module 1204.

[0145] In this embodiment, the fourth determining module 1206 can determine the camera height estimation ratio based on the real-time perceived width and the real-time reference width.

[0146] Specifically, the ratio of the real-time perceived width to the real-time reference width can be determined as the camera height estimation ratio. Optionally, to improve the accuracy of the camera height estimation ratio, the average of the multiple camera height estimation ratios corresponding to the multiple images included in the sliding window can be determined as the current camera height estimation ratio based on a sliding window method.

[0147] In this embodiment, the fifth determining module 1207 can determine the current height of the camera based on the camera height estimation ratio and the initial height of the camera.

[0148] Specifically, since there is a linear relationship between the size of objects in real-time captured images and the size of real objects, the aforementioned camera height estimation ratio can be used as a correction ratio between the actual camera height and the initial height, which is obtained through pre-calibration of the camera. That is, the estimated camera height can be calculated using the following formula:

[0149] Z = Z calib *K

[0150] Among them, Z calib Let K be the initial height, and K be the ratio of the above camera height estimation.

[0151] Reference Figure 13 , Figure 13 This is a schematic diagram of the structure of a camera height estimation device provided in another exemplary embodiment of this disclosure.

[0152] In some optional implementations, the acquisition module 1201 includes: a first acquisition unit 12011, used to acquire the original image of the target lane captured by the camera on the mobile device; a first determination unit 12012, used to determine the image acquisition status information corresponding to the original image; and a first generation unit 12013, used to obtain the image to be recognized based on the original image in response to the image acquisition status information meeting the preset recognition conditions.

[0153] In some optional implementations, the extraction module 1203 includes: an extraction unit 12031, used to extract initial lane line information representing the target lane from the image to be identified; a detection unit 12032, used to perform lane line quality detection on the initial lane line information to obtain a quality detection result; and a second generation unit 12033, used to generate perceived lane line information based on the initial lane line information in response to the quality detection result satisfying the lane line quality condition.

[0154] In some optional implementations, the detection unit 12032 includes: a first determining subunit 120321, used to determine the parallelism information of the two lane lines represented by the initial lane line information; and based on the parallelism information, to determine the quality detection result of the initial lane line information; and / or, a second determining subunit 120322, used to determine the matching degree information between the initial lane line information and the reference lane line information; and based on the matching degree information, to determine the quality detection result of the initial lane line information; and / or, a third determining subunit 120323, used to determine the lane line curvature according to the initial lane line information; and based on the lane line curvature and the curvature reference value, to determine the quality detection result of the initial lane line information.

[0155] In some optional implementations, the second determining module 1204 includes: a second determining unit 12041, configured to determine corresponding first and second reference sampling points from the reference lane line information, wherein the first and second reference sampling points are located on the first and second reference lane lines represented by the reference lane line information, respectively; a third determining unit 12042, configured to determine a first distance between the first reference sampling point and the second reference lane line, and to determine a second distance between the second reference sampling point and the first reference lane line; and a fourth determining unit 12043, configured to determine a real-time reference width based on the first and second distances.

[0156] In some optional implementations, the third determining module 1205 includes: a fifth determining unit 12051, used to determine corresponding first sensing sampling points and second sensing sampling points from the perceived lane line information, wherein the first sensing sampling points and second sensing sampling points are located on the first sensing lane line and the second sensing lane line represented by the perceived lane line information, respectively; a sixth determining unit 12052, used to determine a third distance between the first sensing sampling point and the second sensing lane line, and to determine a fourth distance between the second sensing sampling point and the first sensing lane line; and a seventh determining unit 12053, used to determine the real-time sensing width based on the third distance and the fourth distance.

[0157] In some optional implementations, the fourth determining module 1206 includes: a second acquiring unit 12061, used to acquire a historical sensing width set and a historical reference width set corresponding to the first distance traveled by the mobile device; an eighth determining unit 12062, used to determine whether the real-time sensing width and the real-time reference width meet a preset width deviation condition; a ninth determining unit 12063, used to determine a first estimated sensing width based on the historical sensing width set and the real-time sensing width if the real-time sensing width and the real-time reference width meet the width deviation condition, and to determine a first estimated reference width based on the historical reference width set and the real-time reference width; and a tenth determining unit 12064, used to determine a camera height estimation ratio based on the first estimated sensing width and the first estimated reference width.

[0158] In some optional implementations, the eighth determining unit 12062 includes: an extraction subunit 120621, used to extract a target historical perceived width subset from a historical perceived width set and a target historical reference width subset from a historical reference width set, wherein the target historical perceived width subset and the target historical reference width subset correspond to the second distance traveled; a fourth determining subunit 120622, used to determine a second estimated perceived width based on the target historical perceived width subset and the real-time perceived width, and to determine a second estimated reference width based on the target historical reference width subset and the real-time reference width; and a fifth determining subunit 120623, used to determine that the real-time perceived width and the real-time reference width meet a preset width deviation condition in response to the deviation between the real-time perceived width and the second estimated perceived width being within a preset perceived width deviation, and the real-time reference width being within a preset reference width deviation.

[0159] In some optional implementations, the tenth determining unit 12064 includes: a sixth determining subunit 120641, used to determine the real-time camera height ratio corresponding to the image to be identified based on the first estimated perceptual width and the first estimated reference width; an acquisition subunit 120642, used to acquire the set of historical camera height ratios corresponding to the first path; a seventh determining subunit 120643, used to determine the camera height ratio distribution range of the real-time camera height ratio and the set of historical camera height ratios; and an eighth determining subunit 120644, used to determine the estimated camera height ratio based on the camera height ratio distribution range and the real-time camera height ratio.

[0160] In some optional implementations, the eighth determining subunit 120644 is further configured to: determine the real-time camera height ratio as the camera height estimation ratio in response to the camera height ratio distribution range meeting the height ratio distribution conditions; and determine the previously determined historical camera height estimation ratio as the camera height estimation ratio in response to the camera height ratio distribution range not meeting the height ratio distribution conditions.

[0161] The camera height estimation device provided in the above embodiments of this disclosure calculates the lane width from the perceived lane line information obtained in real time and the reference lane line information in the map, respectively, to obtain the real-time perceived width and the real-time reference width. Based on the real-time perceived width and the real-time reference width, a camera height estimation ratio is determined. Finally, based on the camera height estimation ratio and the initial height of the camera, the current height of the camera is determined. The embodiments of this disclosure achieve real-time estimation of camera height based on visual perception and fully utilize existing reference lane line information, improving the accuracy of camera height estimation and thus contributing to improving the accuracy of environmental perception and vehicle positioning using camera height.

[0162] Exemplary electronic devices

[0163] Below, for reference Figure 14 To describe an electronic device according to embodiments of the present disclosure. The electronic device may be as follows: Figure 1 The terminal device 101 and server 103 shown, or either one or both, or a standalone device independent of them, can communicate with the terminal device 101 and server 103 to receive the collected input signals from them.

[0164] Figure 14 A block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0165] like Figure 14 As shown, the electronic device 1400 includes one or more processors 1401 and memory 1402.

[0166] The processor 1401 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 1400 to perform desired functions.

[0167] The memory 1402 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1401 may execute the program instructions to implement the camera height estimation methods of the various embodiments of this disclosure described above and / or other desired functions. Various contents such as input signals, signal components, and noise components may also be stored in the computer-readable storage medium.

[0168] In one example, the electronic device 1400 may also include an input device 1403 and an output device 1404, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0169] For example, when the electronic device is a terminal device 101 or a server 103, the input device 1403 can be a camera, mouse, keyboard, or other devices used to input images, various commands, etc. When the electronic device is a standalone device, the input device 1403 can be a communication network connector used to receive input images, various commands, etc. from the terminal device 101 and the server 103.

[0170] The output device 1404 can output various information to the outside, including the determined current height of the camera. The output device 1404 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0171] Of course, for the sake of simplicity, Figure 14 Only some of the components of the electronic device 1400 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 1400 may include any other suitable components depending on the specific application.

[0172] Exemplary computer program products and computer-readable storage media

[0173] In addition to the methods and apparatus described above, embodiments of this disclosure may also provide a computer program product, including computer program instructions that, when executed by a processor, cause the processor to perform the steps in the camera height estimation methods of the various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0174] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of embodiments of this disclosure. These programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0175] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the camera height estimation methods of the various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0176] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, but is not limited to, systems, apparatuses, or devices that are electrical, magnetic, optical, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0177] The basic principles of this disclosure have been described above with reference to specific embodiments. However, the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0178] Various modifications and variations can be made to this disclosure without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, this disclosure is also intended to include such modifications and variations.

Claims

1. A camera height estimation method, comprising: Acquire the image of the target lane captured by the camera on the mobile device at the target location; Based on the target location, determine the reference lane line information corresponding to the target lane in the preset map; Extract lane line information from the image to be identified; The real-time reference width of the target lane is determined based on the reference lane line information; Based on the perceived lane line information, the real-time perceived width of the target lane is determined; Based on the real-time perceived width and the real-time reference width, the camera height estimation ratio is determined; Based on the camera height estimation ratio and the camera's initial height, the current height of the camera is determined; The step of determining the camera height estimation ratio based on the real-time perceived width and the real-time reference width includes: Obtain the historical perceived width set and historical reference width set corresponding to the first distance traveled by the mobile device; Determine whether the real-time perceived width and the real-time reference width meet the preset width deviation conditions; If the real-time sensing width and the real-time reference width meet the width deviation condition, a first estimated sensing width is determined based on the historical sensing width set and the real-time sensing width, and a first estimated reference width is determined based on the historical reference width set and the real-time reference width. Based on the first estimated perception width and the first estimated reference width, the camera height estimation ratio is determined.

2. The method according to claim 1, wherein, The acquisition of the image to be identified, captured by the camera on the mobile device at the target location of the target lane, includes: Acquire the original image of the target lane captured by the camera on the mobile device; Determine the image acquisition status information corresponding to the original image; In response to the image acquisition status information meeting preset recognition conditions, the image to be recognized is obtained based on the original image.

3. The method according to claim 1, wherein, The step of extracting perceived lane line information from the image to be identified includes: Extract initial lane line information representing the target lane from the image to be identified; The initial lane line information is subjected to lane line quality detection to obtain the quality detection results; In response to the quality detection result satisfying the lane line quality condition, the perceived lane line information is generated based on the initial lane line information.

4. The method according to claim 3, wherein, The process of performing lane line quality detection on the initial lane line information to obtain quality detection results includes: Determine the parallelism information of the two lane lines represented by the initial lane line information; Based on the parallelism information, determine the quality inspection result of the initial lane line information; and / or, Determine the matching degree information between the initial lane line information and the reference lane line information; Based on the matching degree information, determine the quality detection result of the initial lane line information; and / or, Based on the initial lane line information, determine the lane line curvature; Based on the lane line curvature and curvature reference value, the quality inspection result of the initial lane line information is determined.

5. The method according to claim 1, wherein, Determining the real-time reference width of the target lane based on the reference lane line information includes: A corresponding first reference sampling point and a second reference sampling point are determined from the reference lane line information, wherein the first reference sampling point and the second reference sampling point are respectively located on the first reference lane line and the second reference lane line represented by the reference lane line information; Determine a first distance between the first reference sampling point and the second reference lane line, and determine a second distance between the second reference sampling point and the first reference lane line; The real-time reference width is determined based on the first distance and the second distance.

6. The method according to claim 1, wherein, Determining the real-time perceived width of the target lane based on the perceived lane line information includes: The corresponding first sensing sampling point and second sensing sampling point are determined from the sensing lane line information, wherein the first sensing sampling point and the second sensing sampling point are respectively located on the first sensing lane line and the second sensing lane line represented by the sensing lane line information; Determine a third distance between the first sensing sampling point and the second sensing lane line, and determine a fourth distance between the second sensing sampling point and the first sensing lane line; The real-time sensing width is determined based on the third distance and the fourth distance.

7. The method according to claim 1, wherein, Determining whether the real-time perceived width and the real-time reference width meet the preset width deviation condition includes: Extract a target historical perception width subset from the historical perception width set, and extract a target historical reference width subset from the historical reference width set, wherein the target historical perception width subset and the target historical reference width subset correspond to the second distance already traveled; A second estimated sensing width is determined based on the target historical sensing width subset and the real-time sensing width, and a second estimated reference width is determined based on the target historical reference width subset and the real-time reference width; In response to the fact that the deviation between the real-time perceived width and the second estimated perceived width is within a preset perceived width deviation, and the real-time reference width and the second estimated reference width are within a preset reference width deviation, it is determined that the real-time perceived width and the real-time reference width meet the preset width deviation condition.

8. The method according to claim 7, wherein, The step of determining the camera height estimation ratio based on the first estimated sensing width and the first estimated reference width includes: Based on the first estimated perception width and the first estimated reference width, the real-time camera height ratio corresponding to the image to be identified is determined; Obtain the set of historical camera height ratios corresponding to the first route; Determine the distribution range of the camera height ratio of the real-time camera height ratio and the set of historical camera height ratios; The estimated camera height ratio is determined based on the camera height ratio distribution range and the real-time camera height ratio.

9. The method according to claim 8, wherein, The step of determining the estimated camera height ratio based on the camera height ratio distribution range and the real-time camera height ratio includes: In response to the fact that the camera height ratio distribution range meets the height ratio distribution condition, the real-time camera height ratio is determined as the estimated camera height ratio; In response to the fact that the range of camera height ratio distribution does not meet the height ratio distribution conditions, the previously determined historical camera height estimation ratio is determined as the camera height estimation ratio.

10. A camera height estimation device, comprising: The acquisition module is used to acquire the image of the target lane captured by the camera on the mobile device at the target location; The first determining module is used to determine the reference lane line information corresponding to the target lane in a preset map based on the target location; The extraction module is used to extract perceived lane line information from the image to be identified; The second determining module is used to determine the real-time reference width of the target lane based on the reference lane line information; The third determining module is used to determine the real-time perceived width of the target lane based on the perceived lane line information; The fourth determining module is used to determine the camera height estimation ratio based on the real-time perceived width and the real-time reference width; The fifth determining module is used to determine the current height of the camera based on the camera height estimation ratio and the camera's preset initial height; The fourth determining module includes: The second acquisition unit is used to acquire the historical perceived width set and the historical reference width set corresponding to the first distance traveled by the mobile device. The eighth determining unit is used to determine whether the real-time perceived width and the real-time reference width meet the preset width deviation conditions; The ninth determining unit is configured to determine a first estimated sensing width based on the historical sensing width set and the real-time sensing width if the real-time sensing width and the real-time reference width meet the width deviation condition, and to determine a first estimated reference width based on the historical reference width set and the real-time reference width. The tenth determining unit is used to determine the camera height estimation ratio based on the first estimated perception width and the first estimated reference width.

11. A computer-readable storage medium storing a computer program for execution by a processor to implement the method of any one of claims 1-9.

12. An electronic device, the electronic device comprising: processor; Memory for storing the executable instructions of the processor; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the method described in any one of claims 1-9.

Images