Readable storage medium, landing point correction method, image ranging method and system

Abstract

A landing point correction method, the landing point being a feature point on the bottom edge of a target frame in a pixel coordinate system, the method comprising: acquiring trajectory information of the target frame (S100), the trajectory information at least comprising a difference between a landing point ordinate and a vanishing point ordinate, and a timestamp; obtaining an ordinate difference-time linear fitting equation based on the timestamp within the first preset time and the difference between the ordinate of the landing point and the ordinate of the vanishing point, and calculating the difference between the ordinate of the landing point and the ordinate of the vanishing point at the current moment according to the ordinate difference-time linear fitting equation so as to correct the pixel ordinate of the landing point (S200), therefore, the distance measurement error caused by the change of the landing point in the vehicle movement process is effectively reduced. The invention also provides an image ranging method, an image ranging system and a readable storage medium.

Description

Readable storage medium, landing site correction method, image ranging method and system TECHNICAL FIELD

[0001] The present application belongs to the field of intelligent driving, and particularly relates to a landing site correction method, an image ranging method, an image ranging system and a readable storage medium. BACKGROUND

[0002] With the continuous upgrading and iteration of software and hardware configurations of intelligent systems of automobiles, it is common for automobiles to be equipped with intelligent vehicle lamps with interactive functions, and it is very crucial for the interactive functions of intelligent vehicle lamps to accurately measure the positions of perceived targets relative to vehicle lamps, such as achieving the ADB shielding function of a matrix LED headlamp on a target vehicle and the projection function of a digital projection headlamp DLP, and if accurate perception measurement cannot be achieved, the interaction of intelligent headlamps will be a poor experience. At present, considering cost and practicability, a monocular measurement method is mainly used to achieve the ranging of perceived targets, that is, the position of a target on an image is recognized by using a perception algorithm based on a picture captured by a monocular camera, and the actual spatial coordinate position of the target is mapped according to the imaging principle of the camera.

[0003] Monocular ranging mainly relies on the geometric information of objects in an image to calculate the distance. A commonly used method is triangulation, as shown in FIG. 1, A is the host vehicle, B and C are the front vehicles, P is the camera, the focal length of the camera is f, the height of the camera is H, the horizontal straight-line distances of the camera and the front vehicles B and C are Z1 and Z2 respectively, and the projections of the detection boxes corresponding to B and C on the image are y1 and y2 respectively, so according to similar triangles, y1 = f x H / Z1 and y2 = f x H / Z2 can be obtained.

[0004] Converting the above formula, Z1 = f x H / y1 and Z2 = f x H / y2 can be obtained.

[0005] Therefore, the actual spatial distance needs to be calculated by knowing the coordinates of the landing site of the detection box corresponding to the object in the image. However, in the actual driving process, due to the inevitable objective factors such as vehicle bumping, shaking, target shielding and the like, the target on the camera imaging picture will change accordingly, resulting in a change in the position of the landing site, and the converted spatial distance has an error, thereby affecting the interactive experience of the intelligent headlamp.

[0006] SUMMARY

[0007] The technical problem to be solved by the present application is to provide a landing site correction method, an image ranging method, an image ranging system and a readable storage medium, which can correct the coordinates of the landing site and effectively reduce the ranging error caused by the change of the landing site during the movement of the vehicle.

[0008] To solve the above technical problems, the present application provides a landing point correction method, the landing point being a feature point on the bottom edge of a target frame in a pixel coordinate system, the method comprising:

[0009] obtaining trajectory information of the target frame, wherein the trajectory information at least includes a difference value between a longitudinal coordinate of the landing point and a vanishing point, and a timestamp;

[0010] correcting a pixel longitudinal coordinate of the landing point based on the timestamp and the difference value between the longitudinal coordinate of the landing point and the vanishing point within a first preset time.

[0011] As a specific embodiment, the correction of the pixel longitudinal coordinate of the landing point based on the timestamp and the difference value between the longitudinal coordinate of the landing point and the vanishing point within the first preset time comprises:

[0012] in response to a trajectory length of the target frame within the first preset time being greater than a first preset length, deriving a longitudinal coordinate difference-time linear fitting equation based on the timestamp and the difference value between the longitudinal coordinate of the landing point and the vanishing point, and calculating the difference value between the longitudinal coordinate of the landing point and the vanishing point at the current time according to the longitudinal coordinate difference-time linear fitting equation to correct the pixel longitudinal coordinate of the landing point; or,

[0013] in response to the trajectory length of the target frame within the first preset time being less than or equal to the first preset length, sorting the difference value between the longitudinal coordinate of the landing point and the vanishing point and taking a median value, and correcting the pixel longitudinal coordinate of the landing point based on the median value.

[0014] As a specific embodiment, the method further comprises:

[0015] in response to the trajectory length of the target frame within the first preset time being greater than the first preset length and less than or equal to a second preset length, performing a far-end judgment on the landing point;

[0016] in response to the landing point being a far-end, sorting the difference value between the longitudinal coordinate of the landing point and the vanishing point and taking a median value, and correcting the pixel longitudinal coordinate of the landing point based on the median value;

[0017] in response to the landing point not being a far-end, deriving a longitudinal coordinate difference-time linear fitting equation based on the timestamp and the difference value between the longitudinal coordinate of the landing point and the vanishing point, and calculating the difference value between the longitudinal coordinate of the landing point and the vanishing point at the current time according to the longitudinal coordinate difference-time linear fitting equation to correct the pixel longitudinal coordinate of the landing point;

[0018] wherein the second preset length is greater than the first preset length.

[0019] As a specific embodiment, the remote determination of the landing position comprises:

[0020] obtaining a first distance from the vanishing point to the bottom of the image and a second distance from the landing position to the bottom of the image;

[0021] in response to the second distance being less than or equal to a preset percentage of the first distance, determining that the landing position is not remote;

[0022] in response to the second distance being greater than a preset percentage of the first distance, determining that the landing position is remote.

[0023] As a specific embodiment, the trajectory information of the target frame further comprises the pixel coordinate width and / or the pixel coordinate height of the target frame, and the method further comprises:

[0024] obtaining a vertical coordinate difference-time linear fitting equation based on the time stamp and the difference between the vertical coordinate of the landing position and the vertical coordinate of the vanishing point, and performing parameter stability analysis on the vertical coordinate difference-time linear fitting equation according to the maximum difference method;

[0025] in response to the parameter stability of the vertical coordinate difference-time linear fitting equation, calculating the difference between the vertical coordinate of the landing position and the vertical coordinate of the vanishing point at the current time according to the vertical coordinate difference-time linear fitting equation to correct the pixel vertical coordinate of the landing position;

[0026] in response to the parameter instability of the vertical coordinate difference-time linear fitting equation, obtaining a difference stability value of the difference between the vertical coordinate of the landing position and the vertical coordinate of the vanishing point within a second preset time, obtaining a pixel coordinate width-time linear fitting equation and / or a pixel coordinate height-time linear fitting equation based on the time stamp within a first preset time and the pixel coordinate width and / or the pixel coordinate height of the target frame, and calculating the size change ratio of the target frame at the current time and the previous time according to the pixel coordinate width-time linear fitting equation and / or the pixel coordinate height-time linear fitting equation, so as to correct the pixel vertical coordinate of the landing position according to the size change ratio and the difference stability value;

[0027] wherein the second preset time is less than the first preset time.

[0028] As a specific embodiment, the obtaining of the difference stability value of the difference between the vertical coordinate of the landing position and the vertical coordinate of the vanishing point within the second preset time comprises:

[0029] sorting the difference between the vertical coordinate of the landing position and the vertical coordinate of the vanishing point within the second preset time and taking a plurality of groups of data located in the middle section;

[0030] calculating the average value of the plurality of groups of data, and taking the average value as the difference stability value.

[0031] As a specific embodiment, the landing point is a midpoint on the bottom side of the target frame.

[0032] The second aspect of the present application provides an image ranging method, the image is captured by a camera, the method comprising:

[0033] identifying a target frame in the image captured by the camera, and generating trajectory information of the target frame by using a tracking algorithm;

[0034] locating a feature point on the bottom of the target frame as a landing point, and correcting a pixel vertical coordinate of the landing point by using the landing point correction method described above;

[0035] based on the pixel coordinate of the corrected landing point and the pixel coordinate width and pixel coordinate height of the target frame at the current time, obtaining pixel coordinates of four corner points of the target frame;

[0036] based on the pixel coordinate of the corrected landing point and the pixel coordinates of the four corner points of the target frame, obtaining world coordinates of the landing point and the four corner points of the target frame in a world coordinate system;

[0037] obtaining ranging coordinates of a target object in the world coordinate system according to the world coordinates of the landing point, and obtaining the width and height of the target object according to the world coordinates of the four corner points of the target frame.

[0038] As a specific embodiment, the obtaining of the world coordinates of the landing point and the four corner points of the target frame in the world coordinate system based on the pixel coordinate of the corrected landing point and the pixel coordinates of the four corner points of the target frame comprises:

[0039] calibrating the internal parameters, external parameters and distortion parameters of the camera;

[0040] obtaining a mapping conversion equation between the pixel coordinate system, the image coordinate system, the camera coordinate system and the world coordinate system according to the internal parameters, external parameters and distortion parameters of the camera;

[0041] based on the pixel coordinate of the corrected landing point, the pixel coordinates of the four corner points of the target frame and the mapping conversion equation, obtaining the world coordinates of the landing point and the four corner points of the target frame in the world coordinate system.

[0042] As a specific embodiment, the image ranging method is applied to vehicle image ranging, the camera is a vehicle-mounted camera, and the world coordinate system is a vehicle body coordinate system.

[0043] The third aspect of the present application provides an image ranging system, comprising:

[0044] a camera for collecting images;

[0045] a trajectory generation module for identifying a target frame in the image collected by the camera and generating trajectory information of the target frame by using a tracking algorithm;

[0046] a landing site correction module for positioning a feature point on the bottom of the target frame as a landing site and correcting the pixel vertical coordinate of the landing site by using the above landing site correction method;

[0047] a coordinate conversion module for obtaining world coordinates of the landing site and four corner points of the target frame in a world coordinate system based on the pixel coordinates of the corrected landing site and the pixel coordinates of the four corner points of the target frame;

[0048] a target object ranging module for obtaining ranging coordinates of a target object in a world coordinate system according to the world coordinates of the landing site and obtaining the width and height of the target object according to the world coordinates of the four corner points of the target frame.

[0049] The fourth aspect of the present application provides a readable storage medium, and instructions of the storage medium are executed by a processor of an electronic device, and the electronic device can execute the above landing site correction method or the above image ranging method.

[0050] Through the above method, the present application has the following beneficial effects:

[0051] The present application can obtain the trajectory information of the target frame, and then obtain the vertical coordinate difference-time linear fitting equation based on the time stamp within the first preset time and the difference between the vertical coordinates of the landing site and the vanishing point, and further calculate the difference between the vertical coordinates of the landing site and the vanishing point at the current time according to the vertical coordinate difference-time linear fitting equation to correct the pixel vertical coordinate of the landing site. Even if the vehicle is subjected to inevitable objective factors such as bumping, shaking, target blocking and the like during movement, the ranging error caused by the change of the landing site can still be reduced. BRIEF DESCRIPTION OF DRAWINGS

[0052] Fig. 1 is a schematic diagram of the principle of triangulation;

[0053] Fig. 2 is a step diagram of the landing site correction method of the present application;

[0054] Fig. 3 is a step diagram of the image ranging method of the present application;

[0055] Fig. 4 is a flowchart of a specific embodiment of the landing site correction method of the present application;

[0056] Fig. 5 is a schematic diagram of the principle of camera pinhole imaging;

[0057] Fig. 6 is a schematic diagram of the framework of the image ranging system of the present application.

[0058] Reference Signs List

[0059] 100: camera; 200: trajectory generation module; 300: landing site correction module; 400: coordinate conversion module; 500: target ranging module. DETAILED DESCRIPTION

[0060] The specific embodiments of the present application are described in detail below with reference to the accompanying drawings, and it should be understood that the specific embodiments described herein are merely intended to illustrate and explain the present application, and the protection scope of the present application is not limited to the specific embodiments described below.

[0061] The first aspect of the present application provides a landing site correction method, wherein the landing site is a feature point on the bottom edge of the target frame in the pixel coordinate system, which can be any set point on the bottom edge of the target frame, for example, the feature point can be the left end point, the right end point, the bottom middle point, etc., which is selected according to the actual setting, and preferably, the landing site is the middle point on the bottom edge of the target frame, thereby improving the reliability of the data. Specifically, referring to FIG. 2, the landing site correction method of the present application comprises:

[0062] S100, obtaining trajectory information of the target frame, wherein the trajectory information at least includes a difference value between the vertical coordinate of the landing site and the vertical coordinate of the vanishing point, and a timestamp;

[0063] It should be noted that the vanishing point is the intersection point of parallel lines in three-dimensional space on a two-dimensional image, that is, it can be regarded as the projection of a point at infinity on the projection plane, and in the landing site correction method of the present application, the projection of the road vanishing point of the road where the target object corresponding to the target frame is located on the image can be taken as the vanishing point.

[0064] S200, correcting the pixel vertical coordinate of the landing site based on the timestamp within the first preset time and the difference value between the vertical coordinate of the landing site and the vertical coordinate of the vanishing point. Specifically, the vertical coordinate difference-time linear fitting equation can be derived based on the timestamp within the first preset time and the difference value between the vertical coordinate of the landing site and the vertical coordinate of the vanishing point, and the vertical coordinate difference of the landing site at the current time is calculated according to the vertical coordinate difference-time linear fitting equation to correct the pixel vertical coordinate of the landing site. The linear fitting algorithm used in the linear fitting equation is various, for example, the least square method, the gradient descent method and the regularization linear regression can be used, which is not limited here.

[0065] In normal driving, the vehicle speed can be regarded as uniform speed, and the relative speed of the camera and the target can also be regarded as uniform speed, and the distance formula between the two is: Dist(dt)=Dist now -V*dt (1)

[0066] From the principle of pinhole imaging, we know that:

[0067] where F is the focal length of the camera; K is a parameter with a fixed value; W wrd is the actual width of the target, and is a constant; Dist is the distance of the target from the camera, W img is the pixel width of the target in the camera imaging, and both are variable. Transforming equation (2) gives: Dist = (F / K) K*W wrd / W img (3)

[0068] Let F / K = C, and equation (3) is substituted into equation (1) to obtain:

[0069] Similarly, the actual height H wrd of the target and the pixel height H img of the target in the camera imaging, and the distance Dist of the target from the camera satisfy:

[0070] The distance between the camera and the target can be fitted as a linear function of time. In the monocular measurement process, the actual spatial coordinate position of the target is mapped through the landing site, and the actual width and height of the target are used for distance measurement. According to the reasoning of equations (1)-(5) above and the principle of triangulation, under the condition that the camera height is constant, the vertical coordinate value of the landing site on the target frame corresponding to the target on the image also changes linearly. Therefore, the landing site correction method of the present application linearly fits the multiple measurement data of the difference between the vertical coordinates of the landing site and the vanishing point, so as to obtain a relatively accurate linear function. When the vehicle is jolted during driving, the landing site changes more in the up and down direction, that is, the vertical coordinate of the landing site changes more obviously, and the change of the vertical coordinate of the landing site is also the up and down jumping of the corresponding linear function in the stable state, and finally tends to the vertical coordinate value corresponding to the stable state. The linear fitting equation of the vertical coordinate difference-time can effectively correct the vertical coordinate value of the landing site, provide reliable data for subsequent distance measurement, and reduce the distance measurement error caused by the change of the landing site.

[0071] It should be noted that in the landing site correction method of the present application, the target frame obtained can be one or multiple, the trajectory information of each target frame is stored separately, and the landing site corresponding to each target frame is processed separately. In addition, in the process of identifying the target frame, if the same target disappears and then appears in the collected image, in order to avoid misidentification, the target frame corresponding to the reappearing target is taken as a new target frame, and the trajectory information of the target frame is stored again. The trajectory information of the target frame stored before is independent of each other.

[0072] In step S200, the first preset time can be set according to actual sampling requirements. For example, taking the sampling frame rate of the camera as 30 frames per second, in order to ensure a sufficient number of sampling frames for data analysis, the first preset time can be set as two seconds, that is, the difference between the landing point vertical coordinate and the vanishing point vertical coordinate in the last 60 frames is obtained to fit a more accurate vertical coordinate difference-time linear fitting equation. In addition, since only the time stamp and the difference between the landing point vertical coordinate and the vanishing point vertical coordinate within the first preset time are needed when performing linear fitting, only the trajectory information of the target box within the first preset time is stored, thereby avoiding unnecessary data storage.

[0073] In the identification and detection process of the target box, the target corresponding to the target box can be detected for the first time, and the time can be less than the first preset time, so that the length of the trajectory of the target box within the first preset time does not meet the data requirement for the linear fitting equation. In this regard, considering the delay and multiple frames, in some embodiments of the present application, a first preset length is also provided. In response to the length of the trajectory of the target box within the first preset time being greater than the first preset length, a vertical coordinate difference-time linear fitting equation is derived based on the time stamp and the difference between the landing point vertical coordinate and the vanishing point vertical coordinate, and the difference between the landing point vertical coordinate and the vanishing point vertical coordinate at the current time is calculated according to the vertical coordinate difference-time linear fitting equation to correct the pixel vertical coordinate of the landing point. In response to the length of the trajectory of the target box within the first preset time being less than or equal to the first preset length, the median value of the difference between the landing point vertical coordinate and the vanishing point vertical coordinate is sorted and used to correct the pixel vertical coordinate of the landing point.

[0074] It should be noted that the target corresponding to the target box in the world coordinate system is far away from the camera, and in this case, the target box detected on the camera image is too small, that is, the landing point of the target in this case is at the far end and is easily affected by interference, so more historical data is needed for fitting and prediction. In this regard, in some embodiments of the present application, a second preset length is also provided, which is less than the first preset length. In response to the length of the trajectory of the target box within the first preset time being greater than the first preset length and less than or equal to the second preset length, a far-end judgment is performed on the landing point. In response to the landing point being at the far end, it is indicated that the data amount under the trajectory length is not enough for fitting and prediction, so the median value of the difference between the landing point vertical coordinate and the vanishing point vertical coordinate is sorted and used to correct the pixel vertical coordinate of the landing point. In response to the landing point not being at the far end, a vertical coordinate difference-time linear fitting equation is derived based on the time stamp and the difference between the landing point vertical coordinate and the vanishing point vertical coordinate, and the difference between the landing point vertical coordinate and the vanishing point vertical coordinate at the current time is calculated according to the vertical coordinate difference-time linear fitting equation to correct the pixel vertical coordinate of the landing point.

[0075] Specifically, the remote determination of the landing position comprises: obtaining a first distance from the vanishing point to the bottom of the image and a second distance from the landing position to the bottom of the image; in response to the second distance being less than or equal to a preset percentage of the first distance, determining that the landing position is not remote; and in response to the second distance being greater than the preset percentage of the first distance, determining that the landing position is remote. The preset percentage is set according to design requirements, for example, if the second distance is less than or equal to 1 / 4 of the first distance, it is indicated that the distance between the target frame corresponding to the target object and the camera is relatively close, and linear fitting can be performed.

[0076] In response to the length of the trajectory of the target frame within the first preset time being greater than the second preset length, the trajectory information of the target frame is sufficient, the amount of data for linear fitting is sufficient, the vertical coordinate difference-time linear fitting equation can be directly obtained based on the timestamp and the difference between the vertical coordinate of the landing position and the vertical coordinate of the vanishing point, and the difference between the vertical coordinate of the landing position and the vertical coordinate of the vanishing point at the current time is calculated based on the vertical coordinate difference-time linear fitting equation to correct the pixel vertical coordinate of the landing position. The preset trajectory length can be expressed by time or frame number, for example, with a sampling frame rate of 30 frames / s, the first trajectory length can be 15 frames or 0.5 s, and the second trajectory length can be 30 frames or 1 s.

[0077] It should be noted that during the movement of the vehicle, due to unavoidable factors, the change of the landing position is relatively complex, and there may be data in the data for linear fitting that interferes with the correction accuracy of the landing position coordinates. In order to further improve the reliability of the corrected landing position coordinates, in some embodiments of the present application, the trajectory information of the target frame further comprises the pixel coordinate width and / or the pixel coordinate height of the target frame, and the landing position correction method of the present application further comprises:

[0078] The vertical coordinate difference-time linear fitting equation is obtained based on the timestamp and the difference between the vertical coordinate of the landing position and the vertical coordinate of the vanishing point, and the parameter stability of the vertical coordinate difference-time linear fitting equation is analyzed according to the maximum-minimum difference method. The maximum and minimum values in the data for linear fitting equation are found, and the stability is judged by comparing the absolute value of the difference between the maximum and minimum values with a comparison value, which can be set with reference to the empirical average value of the data. If the absolute value of the difference is less than or equal to the comparison value, it is determined that the parameters are stable, and if the absolute value of the difference is greater than the comparison value, it is determined that the parameters are unstable.

[0079] In response to the parameter stability of the vertical coordinate difference-time linear fitting equation, the difference between the vertical coordinate of the landing position and the vertical coordinate of the vanishing point at the current time is calculated based on the vertical coordinate difference-time linear fitting equation to correct the pixel vertical coordinate of the landing position.

[0080] In response to instability of parameters of the ordinate difference-time linear fitting equation, a difference stable value of the difference between the ordinate of the landing point and the ordinate of the vanishing point within a second preset time is obtained, a pixel coordinate width-time linear fitting equation and / or a pixel coordinate height-time linear fitting equation is derived based on the time stamp within the first preset time and the pixel coordinate width and / or the pixel coordinate height of the target frame, and a size change ratio of the target frame at the current time and the last time is calculated according to the pixel coordinate width-time linear fitting equation and / or the pixel coordinate height-time linear fitting equation, so as to correct the pixel ordinate of the landing point according to the size change ratio and the difference stable value. When the pixel coordinate width-time linear fitting equation and the pixel coordinate height-time linear fitting equation are both used to calculate, the average of two values calculated according to the two linear fitting equations can be used as the size change ratio. The second preset time is less than the first preset time, that is, when the parameters of the ordinate difference-time linear fitting equation are unstable, data within a shorter time is selected for calculation, so as to ensure the reliability of the data.

[0081] In some embodiments of the application, obtaining the difference stable value of the difference between the ordinate of the landing point and the ordinate of the vanishing point within the second preset time includes: sorting the difference between the ordinate of the landing point and the ordinate of the vanishing point within the second preset time, and taking a plurality of groups of data located in the middle section; calculating the average value of the plurality of groups of data, and taking the average value as the difference stable value. The specific range of the middle section should be set according to actual needs, for example, the average value of the values of 5 frames of data located in the middle section after sorting can be selected as the difference stable value according to the actual simulation situation and experience.

[0082] It should be noted that the parameter stability analysis of the ordinate difference-time linear fitting equation according to the extreme value difference method can be reasonably selected according to different trajectory lengths, for example, when the trajectory length of the target frame is between the first trajectory length and the second trajectory length, the parameter stability analysis is not required, and the pixel ordinate of the landing point is directly corrected according to the linear fitting equation; when the trajectory length of the target frame exceeds the second trajectory length, the parameter stability analysis is performed to ensure the reliability of the data.

[0083] In order to better understand the technical scheme of the landing point correction method of the application, the sampling frequency of the camera is 30 frames / s, and the landing point is the midpoint of the bottom edge of the target frame, which is described in combination with the flowchart shown in FIG. 4.

[0084] 1. Obtain the trajectory information of the target frame and store the trajectory information of the target frame within the first preset time (2s). The trajectory information includes the pixel coordinate width Wi of the target frame, the pixel coordinate height Hi of the target frame, the difference Y~i between the landing site vertical coordinate and the vanishing point vertical coordinate (Y~i = BotYi - VanYi), and the timestamp ts. Wherein, BotYi is the landing site vertical coordinate, and VanYi is the vanishing point vertical coordinate.

[0085] 2. When the trajectory length of the target frame is greater than or equal to the first preset length (0.5s or 15 frames), linearly fit the equation of Wi, Hi, and Y~i with respect to ts, to obtain the pixel coordinate width-time linear fitting equation LS1, the pixel coordinate height-time linear fitting equation LS2, and the vertical coordinate difference-time linear fitting equation LS3, and store the trajectory equation for the first preset time (2s).

[0086] 3. Landing site correction

[0087] Considering the delay and multiple frame results, in order to maximize the accuracy of the results, the landing site correction step follows the following rules:

[0088] 1) When the trajectory length of the target frame is less than or equal to the first preset length (0.5s or 15 frames), sort the difference Y~i between the landing site vertical coordinate and the vanishing point vertical coordinate by size to take the median value of Y~i. If the number of stored trajectory information frames of the target frame is even, take the average value between the two middle values as the median value of Y~i. Then, according to the formula Y~i = BotYi - VanYi, the corrected pixel vertical coordinate BotY is obtained.

[0089] 2) When the trajectory length of the target frame is greater than the first preset length (0.5s or 15 frames) and less than or equal to the second preset length (1s or 30 frames), the far end of the landing site is determined based on the current time Y~i.

[0090] a) If the second distance from the current time landing site to the bottom of the image is greater than one fourth of the first distance from the vanishing point to the bottom of the image, the landing site is far end. Sort the difference Y~i between the landing site vertical coordinate and the vanishing point vertical coordinate by size to take the median value of Y~i. If the number of stored trajectory information frames of the target frame is even, take the average value between the two middle values as the median value of Y~i. Then, according to the formula Y~i = BotYi - VanYi, the corrected pixel vertical coordinate BotY is obtained.

[0091] b) if a second distance from the landing point to the bottom of the image is less than or equal to one fourth of a first distance from the vanishing point to the bottom of the image, the landing point is not a far point, a Y i ~ of a current time TsCur is calculated according to a vertical coordinate difference-time linear fitting equation LS3, and a corrected pixel vertical coordinate BotY of the landing point is calculated according to a formula Y i ~ = BotY i - VanY i ;

[0092] 3) when a track length of the target frame is greater than a second preset length (1s or 30 frames), a vertical coordinate difference Yi~ of the landing point in a second preset time (1s) is extracted, the vertical coordinate difference Yi~ is sorted, 5 frames of data in a middle section are selected, an average value of the 5 frames of data is calculated as a difference stable value TsB, and parameter stability analysis is performed according to the maximum difference value method based on the vertical coordinate difference-time linear fitting equation LS3 in the first preset time;

[0093] a) when the parameter of the vertical coordinate difference-time linear fitting equation LS3 is stable, a Y i ~ of a current time TsCur is calculated according to the equation LS3, and a corrected pixel vertical coordinate BotY of the landing point is calculated according to a formula Y i ~ = BotY i - VanY i ;

[0094] b) when the parameter of the vertical coordinate difference-time linear fitting equation LS3 is unstable, a pixel coordinate width at the current time and a pixel coordinate width at a previous time are calculated according to a pixel coordinate width-time linear fitting equation LS1, a width size change ratio of the two times is calculated, a pixel coordinate height at the current time and a pixel coordinate height at the previous time are calculated according to a pixel coordinate height-time linear fitting equation LS2, a height size change ratio of the two times is calculated, an average value of the width size change ratio and the height size change ratio is taken as a size change ratio Scale of the two times, a vertical coordinate difference Yi~ B of the landing point at the current time TsCur is calculated as Yi~ B = TsB * Scale, and a corrected pixel vertical coordinate BotY of the landing point is calculated according to a formula Y i ~ = BotY i - VanY i.

[0095] The second aspect of the present application also provides an image ranging method, and the image is collected by a camera, as shown in FIG. 3, the image ranging method comprises the following steps:

[0096] T100, identifying a target frame in an image collected by a camera, and generating track information of the target frame by using a tracking algorithm;

[0097] The tracking algorithm is an image recognition algorithm, which can extract and identify target features in the collected image, so as to determine whether the target in the front and rear frames of the image is the same target, thereby tracking and identifying the same target to generate the track information of the target frame corresponding to the target;

[0098] T200, positioning the feature point on the bottom of the target frame as the landing point, which is preferably the midpoint of the bottom edge of the target frame, and correcting the pixel vertical coordinate of the landing point by using the landing point correction method provided in the first aspect of the application;

[0099] T300, obtaining the world coordinates of the landing point and the four corner points of the target frame based on the pixel coordinates of the corrected landing point and the pixel coordinates of the four corner points of the target frame;

[0100] T400, obtaining the ranging coordinates of the target object in the world coordinate system according to the world coordinates of the landing point, and obtaining the width and height of the target object according to the world coordinates of the four corner points of the target frame.

[0101] It should be noted that the conversion of camera image coordinates to information in the world coordinate system is converted to the camera coordinate system and then to the image coordinate system. In the process of converting to the pixel coordinate system, the world coordinate system is the absolute coordinate system of the objective three-dimensional world, which can be represented by (Xw, Yw, Zw). The position of the digital camera and any other object placed in the three-dimensional environment is described by this reference coordinate system. The camera coordinate system is the optical center coordinate system, which takes the optical center of the camera as the coordinate origin, and the X-axis and Y-axis are parallel to the X-axis and Y-axis of the image coordinate system, respectively. The optical axis of the camera is the Z-axis, and its coordinate value is represented by (Xc, Yc, Zc). The image coordinate system takes the center of the CCD (charge coupled device) image plane as the coordinate origin, and the X-axis and Y-axis are parallel to the two perpendicular edges of the image plane, respectively. The coordinate value is represented by (x, y), which represents the position of the pixel in the image in physical units (such as millimeters). The pixel coordinate system takes the top left corner of the image plane as the origin, and the X-axis and Y-axis are parallel to the X-axis and Y-axis of the image coordinate system, respectively. The coordinate value is represented by (u, v). The camera calibration formula is as follows:

[0102] Where [R T] is the external parameter of the camera to be calibrated, i.e. to solve a matrix relationship from the actual world coordinate system to the camera coordinate system; [f x , f y , u0, v0] are the internal parameters of the camera to be calibrated, i.e. to solve a matrix relationship from the camera coordinate system to the pixel coordinate system, which includes two parts: from the camera coordinate system to the image coordinate system, and from the image coordinate system to the pixel coordinate system. f x =f / d x , f y =f / d y , where f is the physical focal length of the camera, d x , d y represents the physical size of 1 imaging pixel on the X-axis and Y-axis of the CCD image plane.

[0103] During the camera calibration process described above, the camera's external and internal parameters can be solved. In the process of calibrating the internal parameters, the camera's distortion parameters (k1, k2, p1, p2, p3) can also be obtained, where k1 and k2 are the radial distortion coefficients of the lens, and p1, p2, and p3 are the tangential distortion coefficients of the lens.

[0104] In some embodiments of the present invention, step T300 includes:

[0105] T301. Calibrate the internal parameters, external parameters, and distortion parameters of the camera;

[0106] T302. Based on the camera's intrinsic, extrinsic, and distortion parameters, derive the mapping transformation equations between the pixel coordinate system, image coordinate system, camera coordinate system, and world coordinate system; specifically, define the rotation matrix R, translation matrix T, and inverse rotation matrix R⁻¹. C = {k1,k2,p1,p2,k3} R -1 =1 / R

[0107] The relationship between the camera coordinate system and the world coordinate system is represented by Euler angles, where r is the roll angle, y is the yaw angle, p is the pitch angle, and x is the yaw angle. cam y cam z cam The coordinates of the camera mounting point in the world coordinate system are given.

[0108] T303. Based on the corrected pixel coordinates of the landing point, the pixel coordinates of the four corner points of the target box, and the mapping transformation equation, the world coordinates of the landing point and the four corner points of the target box in the world coordinate system are obtained.

[0109] Specifically as follows:

[0110] Get the pixel coordinates P of the landing point at the current moment. img (x img y img ), and correct y according to the landing point correction algorithm. img ;

[0111] The distortion correction algorithm is used to obtain point P. img The actual normalized coordinates P on the CCD norm (x norm y norm, 1), that is, the coordinates in the image coordinate system, the de-distortion algorithm can use the de-distortion algorithm in Matlab software or OpenCv (Open Computer Vision Library, cross-platform computer vision library) to process the coordinates and the camera internal parameters to obtain the actual imaging pixel coordinates after distortion correction, as follows, u = x norm *f x + u0 v = y norm *f y + v0

[0112] The P norm point in the image coordinate system is counter-rotated to obtain the normalized coordinates P c_norm (X c_norm , Y c_norm , 1) in the camera coordinate system, as follows,

[0113] Referring to FIG. 5, the camera coordinates P c (X c , Y c , Z c ) of the target object can be obtained by using the triangular similarity of the camera pinhole imaging, as follows, fScale = H / Y c_norm == Z c / 1.0 = X c / X c_norm = Y c / Y c_norm Z c = 1.0*H / Y c_norm X c = X c_norm *H / Y c_norm Y c = Y c_norm *H / Y c_norm

[0114] wherein fScale is the distance between the camera and the target object, and H is the installation height of the camera from the ground;

[0115] The camera coordinates of the target object are converted into the world coordinate system by the camera translation operation, so as to obtain the world coordinates P wrd (X wrd , Y wrd , Z wrd ) corresponding to the landing position of the target frame on the image, as follows,

[0116] The fScale obtained in the process of calculating the landing position corresponding to the world coordinates is used as a normalization factor of the target distance to calculate the world coordinates of the four corner points (the upper left point, the lower left point, the upper right point, and the lower right point).

[0117] It should be noted that, in the application of the image ranging method to vehicle image ranging, the camera is a vehicle-mounted camera, and in order to facilitate the implementation of corresponding control, the world coordinate system can adopt a vehicle body coordinate system, so as to provide stable and accurate interaction effects for intelligent headlamps.

[0118] The third aspect of the present application also provides an image ranging system corresponding to the above-mentioned image ranging method, as shown in FIG. 6, the system comprises:

[0119] a camera 100, configured to collect images;

[0120] a trajectory generation module 200, configured to identify a target frame in the image collected by the camera and generate trajectory information of the target frame by using a tracking algorithm;

[0121] a landing position correction module 300, configured to position a feature point on the bottom of the target frame as a landing position, and correct a pixel longitudinal coordinate of the landing position by using the landing position correction method provided in the first aspect of the present application;

[0122] a coordinate conversion module 400, configured to obtain world coordinates of the landing position and four corner points of the target frame in a world coordinate system based on pixel coordinates of the landing position after correction and pixel coordinates of the four corner points of the target frame;

[0123] a target object ranging module 500, configured to obtain ranging coordinates of a target object in the world coordinate system according to the world coordinates of the landing position, and obtain a width and a height of the target object according to the world coordinates of the four corner points of the target frame.

[0124] The image ranging system provided by the present application can correct the pixel coordinate position of the imaging landing position of the perceived target in the monocular camera by using the above-mentioned landing position correction method, effectively eliminates the monocular ranging error caused by violent motion, bumping and shielding during vehicle driving, provides more stable perception measurement results for the shielding and projection of intelligent vehicle lamps, and further provides stable and accurate interaction effects for intelligent headlamps.

[0125] The fourth aspect of the present application provides a readable storage medium, and instructions of the storage medium are executed by a processor of an electronic device, so that the electronic device can execute the landing position correction method provided in the first aspect of the present application or the image ranging method provided in the second aspect of the present application, thus having all the beneficial effects thereof, which will not be repeated here.

[0126] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart and / or block diagram block or blocks.

[0127] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart and / or block diagram block or blocks.

[0128] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart and / or block diagram block or blocks.

[0129] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0130] The memory can include non-persistent memory and / or persistent memory, which can be volatile and / or non-volatile memory, such as read-only memory (ROM), random access memory (RAM), and / or flash memory, among others. The memory is an example of computer readable media.

[0131] Computer readable media includes permanent and non-permanent, moveable and non- moveable media that can be implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data. The memory is an example of computer readable media.

[0132] Examples of storage media include, but are not limited to, phase-change RAM (PRAM), static RAM (SRAM), dynamic RAM (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile discs (DVDs) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transitory medium that can be used to store information for access by a computing device. In context with the present disclosure, computer readable medium does not encompass transitory media, such as modulated data signals and carrier waves.

[0133] It is also to be noted that the term "comprising" or "including" or any other variation thereof is intended to cover a non-exclusive inclusion, such that processes, methods, articles, or apparatuses that comprise a list of elements are not necessarily limited to those elements, but can include other elements not expressly listed or inherent to such processes, methods, articles, or apparatuses.

[0134] The embodiments of the present application only illustrate the technical solutions of the present application and are not intended to limit the present application. The present application can have various modifications and changes for those skilled in the art. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the scope of claims of the present application.

Claims

1. A method of correcting a landing point, the landing point being a feature point on a bottom edge of a target frame in a pixel coordinate system, characterized by, The method comprises: acquiring trajectory information of the target frame, wherein the trajectory information at least comprises a difference between a landing site vertical coordinate and a vanishing point vertical coordinate, and a timestamp; correcting a pixel vertical coordinate of the landing site based on the timestamp and the difference between the landing site vertical coordinate and the vanishing point vertical coordinate within a first preset time.

2. The method of claim 1, wherein, The correction of the pixel vertical coordinate of the landing site based on the timestamp and the difference between the landing site vertical coordinate and the vanishing point vertical coordinate within the first preset time comprises: in response to a trajectory length of the target frame within the first preset time being greater than a first preset length, deriving a vertical coordinate difference-time linear fitting equation based on the timestamp and the difference between the landing site vertical coordinate and the vanishing point vertical coordinate, and calculating the difference between the landing site vertical coordinate and the vanishing point vertical coordinate at a current time according to the vertical coordinate difference-time linear fitting equation to correct the pixel vertical coordinate of the landing site; or in response to the trajectory length of the target frame within the first preset time being less than or equal to the first preset length, sorting the difference between the landing site vertical coordinate and the vanishing point vertical coordinate and taking a median value to correct the pixel vertical coordinate of the landing site based on the median value.

3. The fall location correction method according to claim 2, characterized by, The method further comprises: in response to the trajectory length of the target frame within the first preset time being greater than the first preset length and less than or equal to a second preset length, performing a remote judgment on the landing site; in response to the landing site being remote, sorting the difference between the landing site vertical coordinate and the vanishing point vertical coordinate and taking a median value to correct the pixel vertical coordinate of the landing site based on the median value; in response to the landing site not being remote, deriving a vertical coordinate difference-time linear fitting equation based on the timestamp and the difference between the landing site vertical coordinate and the vanishing point vertical coordinate, and calculating the difference between the landing site vertical coordinate and the vanishing point vertical coordinate at a current time according to the vertical coordinate difference-time linear fitting equation to correct the pixel vertical coordinate of the landing site; wherein the second preset length is greater than the first preset length.

4. The fall location correction method according to claim 3, characterized by, The remote judgment on the landing site comprises: acquiring a first distance from the vanishing point to the bottom of the image and a second distance from the landing site to the bottom of the image; in response to the second distance being less than or equal to a preset percentage of the first distance, deriving that the landing site is not remote; in response to the second distance being greater than the preset percentage of the first distance, deriving that the landing site is remote.

5. The fall location correction method according to claim 3, characterized by, The trajectory information of the target frame further comprises a pixel coordinate width and / or a pixel coordinate height of the target frame, and the method further comprises: deriving a vertical coordinate difference-time linear fitting equation based on the timestamp and the difference between the landing site vertical coordinate and the vanishing point vertical coordinate, and performing parameter stability analysis on the vertical coordinate difference-time linear fitting equation according to a maximum-minimum value difference method; in response to the parameter stability of the vertical coordinate difference-time linear fitting equation, calculating the difference between the landing site vertical coordinate and the vanishing point vertical coordinate at a current time according to the vertical coordinate difference-time linear fitting equation to correct the pixel vertical coordinate of the landing site. In response to instability of parameters of the ordinate difference-time linear fitting equation, a difference steady value of the difference between the ordinate of the landing point and the ordinate of the vanishing point within a second preset time is obtained, a pixel coordinate width-time linear fitting equation and / or a pixel coordinate height-time linear fitting equation is derived based on the time stamp within the first preset time and the pixel coordinate width and / or the pixel coordinate height of the target frame, and a size change ratio of the target frame at the current time and the last time is calculated according to the pixel coordinate width-time linear fitting equation and / or the pixel coordinate height-time linear fitting equation, so as to correct the pixel ordinate of the landing point according to the size change ratio and the difference steady value. The second preset time is less than the first preset time.

6. The fall location correction method according to claim 5, characterized by, The method for obtaining the difference steady value of the difference between the ordinate of the landing point and the ordinate of the vanishing point within the second preset time comprises: sorting the difference between the ordinate of the landing point and the ordinate of the vanishing point within the second preset time, and taking a plurality of groups of data in the middle section as the difference steady value; calculating an average value of the plurality of groups of data, and taking the average value as the difference steady value.

7. The impact point correction method according to claim 1, characterized by, The landing point is a midpoint on the bottom edge of the target frame.

8. A method of image ranging, the image being captured by a camera, characterized in that, The method comprises: identifying the target frame in the image collected by the camera, and generating trajectory information of the target frame by using a tracking algorithm; positioning a feature point on the bottom of the target frame as a landing point, and correcting the pixel ordinate of the landing point by using the landing point correction method in any one of claims 1-7; deriving pixel coordinates of four corner points of the target frame based on the pixel coordinates of the corrected landing point and the pixel coordinate width and the pixel coordinate height of the target frame at the current time; deriving world coordinates of the landing point and the four corner points of the target frame in a world coordinate system based on the pixel coordinates of the corrected landing point and the pixel coordinates of the four corner points of the target frame; deriving ranging coordinates of the target object in the world coordinate system according to the world coordinates of the landing point, and deriving the width and the height of the target object according to the world coordinates of the four corner points of the target frame.

9. The image ranging method of claim 8, wherein, The method for deriving the world coordinates of the landing point and the four corner points of the target frame in the world coordinate system based on the pixel coordinates of the corrected landing point and the pixel coordinates of the four corner points of the target frame comprises: calibrating internal parameters, external parameters and distortion parameters of the camera; deriving a mapping conversion equation between a pixel coordinate system, an image coordinate system, a camera coordinate system and a world coordinate system according to the internal parameters, the external parameters and the distortion parameters of the camera; deriving the world coordinates of the landing point and the four corner points of the target frame in the world coordinate system based on the pixel coordinates of the corrected landing point, the pixel coordinates of the four corner points of the target frame and the mapping conversion equation.

10. The image ranging method according to claim 8 or 9, which is applied to vehicle image ranging, the camera being a vehicle-mounted camera, characterized in that, The world coordinate system is a vehicle body coordinate system.

11. An image ranging system characterized by, The system comprises: a camera (100) configured to collect an image; a trajectory generation module (200) configured to identify a target frame in the image collected by the camera, and generate trajectory information of the target frame by using a tracking algorithm; The landing point correction module (300) is configured to locate a feature point on the bottom of the target frame as a landing point, and correct a pixel longitudinal coordinate of the landing point by using the landing point correction method in any one of claims 1-7; The coordinate conversion module (400) is configured to obtain world coordinates of the landing point and four corner points of the target frame in a world coordinate system based on pixel coordinates of the corrected landing point and pixel coordinates of the four corner points of the target frame; The target object ranging module (500) is configured to obtain ranging coordinates of a target object in the world coordinate system according to the world coordinates of the landing point, and obtain a width and a height of the target object according to the world coordinates of the four corner points of the target frame.

12. A readable storage medium, characterized by, The instructions of the storage medium are executed by the processor of the electronic device, and the electronic device can execute the landing point correction method in any one of claims 1-7 or the image ranging method in any one of claims 8-10.

Images