Information processing method, ranging method and device

Abstract

Embodiments of the present application provide an information processing method, a ranging method and device, the method comprising: processing a target image by an edge detection operator to obtain edge information of an object in the target image, the target image being obtained by a target camera; extracting a first straight line in an image corresponding to the edge information by a straight line detection segmentation algorithm, the first straight line corresponding to a line segment length greater than a preset threshold; performing partition processing on the target image according to the first straight line to obtain a target region image; and determining a target vanishing point according to the target region image, the target vanishing point being used to correct a pitch angle and a horizontal angle of the target camera and a world coordinate system. The method provided by the embodiments of the present application can overcome the problem of low ranging accuracy in the prior art.

Description

Technical Field

[0001] This application relates to the field of ranging technology, and in particular to an information processing method, a ranging method and an apparatus. Background Technology

[0002] With the development of technology, there are numerous distance-based applications and demand scenarios, and these are highly dependent on distance. Examples include autonomous driving and indoor robots.

[0003] Currently, common ranging methods include pinhole imaging ranging, inverse perspective transformation ranging, and PNP ranging. Among these, the accuracy of the camera parameters is crucial. However, pinhole imaging ranging requires knowledge of the object's length and width, as well as the camera's intrinsic parameters and distortion coefficients. The model used is an approximation, making it extremely sensitive to noise. Furthermore, due to the use of similar triangles, the object must be parallel to the image plane. Inverse perspective transformation ranging suffers from calibration errors, which increase with distance. In real-world scenarios, such as autonomous driving, vehicle movement can cause camera vibrations, deviating from the calibrated homography matrix and further amplifying the ranging error. PNP ranging has stringent prerequisites, requiring 3D-2D point pairs. However, in real-world applications, such as autonomous driving and indoor robots, obtaining precise 3D points is often impossible, and even relying solely on a monocular camera may not yield accurate 3D points.

[0004] Therefore, existing ranging methods suffer from complex ranging conditions and large ranging errors, which makes it impossible to accurately correct the shooting camera parameters, resulting in low ranging accuracy. Summary of the Invention

[0005] This application provides an information processing method, a ranging method, and an apparatus to overcome the problem in the prior art that the shooting camera parameters cannot be accurately corrected, resulting in low ranging accuracy.

[0006] In a first aspect, embodiments of this application provide an information processing method, including:

[0007] The edge information of objects in the target image is obtained by processing the target image with an edge detection operator. The target image is captured by a target camera.

[0008] The first straight line in the image corresponding to the edge information is extracted by a straight line detection and segmentation algorithm, and the length of the line segment corresponding to the first straight line is greater than a preset threshold.

[0009] Based on the first straight line, the target image is partitioned to obtain a target region image;

[0010] Based on the target area image, the target vanishing point is determined, and the target vanishing point is used to correct the pitch and horizontal angles of the target camera relative to the world coordinate system.

[0011] Secondly, embodiments of this application provide a ranging method, including:

[0012] Obtain the coordinates of the target camera in the world coordinate system and the focal length of the target camera;

[0013] The target vanishing point is extracted from the target image formed by the target point to be ranged by the target camera. The target vanishing point is used to correct the pitch angle and horizontal angle of the target camera with respect to the world coordinate system.

[0014] Obtain the projected coordinates of the target point on the target image;

[0015] Based on the target camera's coordinates in the world coordinate system, the target camera's focal length, the target image's size, and the projected coordinates, the distance from the target point to the target camera is determined using the pitch and horizontal angles corrected by the target vanishing point.

[0016] The target extinction point is determined by the information processing method described above.

[0017] Thirdly, embodiments of this application provide an information processing apparatus, including:

[0018] The first processing module is used to process the target image using an edge detection operator to obtain the edge information of the objects in the target image, wherein the target image is obtained by a target camera.

[0019] The second processing module is used to extract the first straight line in the image corresponding to the edge information through a straight line detection and segmentation algorithm, wherein the length of the line segment corresponding to the first straight line is greater than a preset threshold.

[0020] The third processing module is used to partition the target image according to the first straight line to obtain a target region image;

[0021] The fourth processing module is used to determine the target vanishing point based on the target area image. The target vanishing point is used to correct the pitch angle and horizontal angle of the target camera relative to the world coordinate system.

[0022] Fourthly, embodiments of this application provide a ranging device, including:

[0023] The acquisition module is used to acquire the coordinates of the target camera in the world coordinate system and the focal length of the target camera;

[0024] The vanishing point extraction module is used to extract the target vanishing point from the target image formed by the target point to be measured captured by the target camera. The target vanishing point is used to correct the pitch angle and horizontal angle of the target camera with respect to the world coordinate system.

[0025] The acquisition module is further configured to acquire the projection coordinates of the target point on the target image;

[0026] The ranging module is used to determine the distance from the target point to the target camera based on the target camera's coordinates in the world coordinate system, the target camera's focal length, the size of the target image, and the projected coordinates, using the pitch angle and horizontal angle corrected by the target vanishing point.

[0027] Fifthly, embodiments of this application provide an electronic device, including: at least one processor and a memory;

[0028] The memory stores computer-executed instructions;

[0029] The at least one processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the methods described in the first and second aspects above.

[0030] Sixthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the methods described in the first and second aspects above.

[0031] In a seventh aspect, embodiments of this application provide a computer program product, including a computer program, characterized in that, when the computer program is executed by a processor, it implements the methods described in the first and second aspects above.

[0032] The information processing method, ranging method, and apparatus provided in this embodiment first process the target image using an edge detection operator to obtain the edge information of objects in the target image, which is obtained by a target camera. Then, a line detection and segmentation algorithm is used to extract the first straight line in the image corresponding to the edge information, where the length of the line segment corresponding to the first straight line is greater than a preset threshold. Based on the first straight line, the target image is partitioned to obtain a target region image. Based on the target region image, the vanishing point of the target is determined to correct the pitch and horizontal angles of the target camera relative to the world coordinate system. This eliminates the need for complex and stringent ranging conditions to correct camera parameters. Furthermore, the method of using the vanishing point extraction to correct the pitch and horizontal angles of the target camera relative to the world coordinate system is highly accurate, thus avoiding the problem of large ranging errors. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This is an application scenario diagram of the information processing method provided in the embodiments of this application;

[0035] Figure 2A A schematic flowchart illustrating the ranging method provided in the embodiments of this application;

[0036] Figure 2B A flowchart illustrating the information processing method provided in this application embodiment;

[0037] Figure 3 This is an application scenario diagram of the information processing method provided in another embodiment of this application;

[0038] Figure 4 A flowchart illustrating an information processing method provided in yet another embodiment of this application;

[0039] Figure 5 An application scenario diagram of an information processing method provided in another embodiment of this application;

[0040] Figure 6 A flowchart illustrating an information processing method provided in yet another embodiment of this application;

[0041] Figure 7 A flowchart illustrating an information processing method provided in another embodiment of this application;

[0042] Figure 8 An application scenario diagram of the information processing method provided in another embodiment of this application;

[0043] Figure 9 An application scenario diagram of an information processing method provided in another embodiment of this application;

[0044] Figure 10A This is a schematic diagram of the structure of the information processing device provided in the embodiments of this application;

[0045] Figure 10B This is a schematic diagram of the structure of the ranging device provided in the embodiments of this application;

[0046] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0048] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0049] Currently, common ranging methods include pinhole imaging ranging, inverse perspective transformation ranging, and PNP ranging. Among these, the accuracy of the camera parameters is crucial. However, pinhole imaging ranging requires knowledge of the object's length and width, intrinsic parameters, and distortion coefficients. The model used is approximate and extremely sensitive to noise. Furthermore, due to the use of similar triangles, the object must be parallel to the image plane. Inverse perspective transformation ranging suffers from calibration errors, which increase with distance. In real-world scenarios, such as autonomous driving, vehicle movement can cause camera vibrations, making the calibration homography matrix less accurate and increasing the ranging error. PNP ranging has stringent prerequisites, requiring 3D-2D point pairs. However, in real-world applications such as autonomous driving and indoor robots, obtaining precise 3D points is often impossible, and even relying solely on a monocular camera for ranging may not yield accurate 3D points. Therefore, existing ranging methods suffer from complex ranging conditions and large ranging errors, which makes it impossible to accurately correct the shooting camera parameters, resulting in low ranging accuracy.

[0050] Therefore, to address the aforementioned problems, the technical concept of this application is to use the vanishing point to solve for the pitch angle and yaw angle between the camera and the world coordinate system. Due to vehicle bumps, camera vibrations, and other reasons, the fixed pitch and yaw angles will change, resulting in a larger ranging error as the distance increases. Therefore, the pitch and yaw angles solved in real time can be used for adaptive ranging, which can reduce ranging error and improve ranging accuracy. The conditions for using the ranging method are relatively simple. The distance from the point to the camera can be solved by obtaining the coordinates of the camera in the world coordinate system, the focal length of the target camera, the vanishing point, and the projection coordinates of the point on the ground to be measured on the image.

[0051] In practical applications, the camera coordinate system, world coordinate system, and image plane coordinate system are first established. The establishment of coordinate systems can be flexibly handled according to different situations. For example, in an autonomous driving system, the origin of the world coordinate system is the camera's projection onto the ground, and its direction is the same as that of the camera coordinate system. (See [link to relevant documentation]). Figure 1 As shown, Using the world coordinate system, For the camera coordinate system, The camera's coordinates in the world coordinate system are: Indoors, the origin of the world coordinate system can be fixed; only the camera's coordinates in the world coordinate system need to be known. That's it. Then, the LSD algorithm (i.e., line detection and segmentation algorithm) is used to extract line segments. Vertical line segments are used to segment the image captured by the camera and remove line segments. The vanishing point is selected by voting using the length, angle, and distance from the vanishing point to the line, and the optimal vanishing point can be determined in real time. The pitch and yaw angles are then calculated using the vanishing point. Simultaneously, the distance from the target camera to the camera can be calculated using the focal length f of the target camera and the projected coordinates (u, v) of the point on the ground to be measured on the image.

[0052] The technical solutions of this application will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0053] Figure 2A This is a flowchart illustrating the ranging method provided in an embodiment of this application. The method may include:

[0054] S101. Obtain the coordinates of the target camera in the world coordinate system and the focal length of the target camera.

[0055] In this embodiment, the execution entity can be a ranging device. First, the camera coordinate system and the world coordinate system are established, wherein, see [reference needed]. Figure 3As shown, in the camera coordinate system: O is the optical center, which is the origin of the camera coordinate system. The z-axis passes through the image plane, the x-axis is to the right, and the y-axis is downward. P0 is the origin of the image plane coordinate system, the u-axis is to the right, and the v-axis is downward. The world coordinate system can be established in different ways depending on the situation. For example, in an autonomous driving system, the origin of the world coordinate system is the projection of the camera onto the ground, the z-axis is upward, the x-axis is to the right, and the y-axis is forward. Indoors, the origin of the world coordinate system can be fixed. Only one condition needs to be met: when measuring distance, the coordinates of the camera (here referring to the target camera; the following examples will use the camera as an example) in the world coordinate system must be known. See also Figure 1 As shown.

[0056] S102. Extract the target vanishing point from the target image formed by capturing the target point to be measured through the target camera. The target vanishing point is used to correct the pitch angle and horizontal angle of the target camera with respect to the world coordinate system.

[0057] In this embodiment, see Figure 4 As shown, the vanishing point extraction process is as follows: First, edge extraction is performed on the target image. Straight lines are detected on the edge-extracted image. Then, all vertical lines are extracted, and the image is divided into regions. Lines in the ground and middle regions are selected, and vertical lines are discarded. Then, a vote is held among the regions defined by the ground and middle regions. The point with the highest score is the vanishing point, and then the vanishing point extraction ends. The vanishing point can be used to calculate the pitch angle and the yaw angle.

[0058] S103. Obtain the projection coordinates of the target point on the target image.

[0059] In this example, the projected coordinates of the target point on the image are the coordinates in the image plane coordinate system, such as (u, v).

[0060] S104. Based on the coordinates of the target camera in the world coordinate system, the focal length of the target camera, the size of the target image, and the projection coordinates, the distance from the target point to the target camera is determined using the pitch angle and horizontal angle corrected by the target vanishing point.

[0061] In this embodiment, when the vanishing point coordinates and the object's projection on the image are obtained... First, the pitch and yaw angles are calculated using the vanishing point. Then, based on the pitch and yaw angles and the initial coordinates of the target camera in the world coordinate system, the corrected angles are calculated. and That is, the actual coordinates of the target camera in the world coordinate system, and finally using the corrected coordinates. and And the yaw angle is solved and Complete the solution for IPM. and These are the x-axis and y-axis distances from the point being solved to the camera, respectively. In other words, the distance from the target point to the target camera.

[0062] The ranging method provided in this embodiment obtains the coordinates of the target camera in the world coordinate system and the focal length of the target camera; then extracts the target vanishing point from the target image formed by the target point to be measured captured by the target camera; obtains the projection coordinates of the target point on the target image; and determines the distance from the target point to the target camera based on the coordinates of the target camera in the world coordinate system, the focal length of the target camera, the size of the target image, the target vanishing point, and the projection coordinates. This method does not require complex and demanding ranging conditions, avoids the problem of large ranging errors, and improves the accuracy of ranging.

[0063] Figure 2B This is a flowchart illustrating an information processing method provided in an embodiment of this application. The method may include:

[0064] S201. The target image is processed by an edge detection operator to obtain the edge information of the objects in the target image.

[0065] In this embodiment, the vanishing point is caused by the perspective effect in the image, so small, irregular textures cause interference and need to be removed. Specifically, the Canny operator (i.e., the edge detection operator) is used to process the image to obtain the edge information of objects in the image, with the aim of removing interference caused by color and some irregular textures.

[0066] S202. Using a line detection and segmentation algorithm, extract the first straight line in the image corresponding to the edge information, wherein the length of the line segment corresponding to the first straight line is greater than a preset threshold.

[0067] In this embodiment, after obtaining the edge information of objects in the image, a straight line detection algorithm, such as the LSD algorithm, can be used to extract straight lines in the image after edge extraction, and a threshold τ (i.e., a preset threshold) is set to select straight lines with a length greater than τ.

[0068] S203. Based on the first straight line, the target image is partitioned to obtain a target region image.

[0069] The determination of the target region image can be achieved through the following steps:

[0070] Step b1: Obtain a vertical line perpendicular to the horizontal axis in the phase plane coordinate system from the first straight line, and divide the target image into multiple sub-images. Here, the vertical line is perpendicular to the u-axis.

[0071] Step b2: Obtain straight lines containing the vertical lines from each of the plurality of sub-images to form a group of straight lines. Each line segment in the group of straight lines is represented by two two-dimensional coordinate points, which are coordinate points in the phase plane coordinate system.

[0072] Step b3: Obtain the points with the maximum v-axis coordinate and the points with the minimum v-axis coordinate corresponding to each sub-image from the group of lines.

[0073] Step b4: Connect all points with the maximum ordinate to generate a first geometric line, and connect all points with the minimum ordinate to generate a second geometric line. The first and second geometric lines divide the target image into three regions.

[0074] Step b5: Select the region with a large vanishing point response in the target image from the three regions as the target region image.

[0075] For example, after obtaining the lines extracted by the LSD algorithm, all lines perpendicular to the u-axis can be selected. Simultaneously, the image with width W and height H can be divided into 20 smaller images (i.e., sub-images), each with a width of 0.05. W is the height, and H is the width. Select the lines perpendicular to the u-axis contained in each small image to form a group of lines. Since each line segment is represented by two points, There is a point with the largest v-axis value. The point with the smallest v-axis value Therefore, based on these 20 small images, 40 points can be obtained. Then, connect all the points with the largest v-axis values ​​(i.e., the points whose ordinates are the maximum for the sub-images), and simultaneously connect all the points with the smallest v-axis values ​​(i.e., the points whose ordinates are the minimum for the sub-images). At this point, the image is divided into three parts: sky, middle, and ground, as shown below. Figure 5 As shown. Since the line segments in the middle and on the ground have the greatest response to the vanishing point when solving for the vanishing point, while the line segments in the sky have almost no response, the sky line segments are removed using this segmentation region, and the remaining area is the target region image.

[0076] S204. Determine the target vanishing point based on the target area image.

[0077] The information processing method provided in this embodiment processes the target image using an edge detection operator to obtain the edge information of objects in the target image, which is obtained by a target camera. Then, a line detection and segmentation algorithm is used to extract the first straight line in the image corresponding to the edge information, where the length of the line segment corresponding to the first straight line is greater than a preset threshold. Based on the first straight line, the target image is partitioned to obtain a target region image. Based on the target region image, the vanishing point of the target is determined to correct the pitch and horizontal angles of the target camera relative to the world coordinate system. This eliminates the need for complex and demanding ranging conditions to correct camera parameters. Furthermore, the method of using the vanishing point extraction to correct the pitch and horizontal angles of the target camera relative to the world coordinate system is highly accurate, thus avoiding the problem of large ranging errors.

[0078] The target vanishing point can be determined through the following steps:

[0079] Step c1: Divide the first straight line contained in the target region image to obtain at least two groups of straight lines, wherein each group of straight lines corresponds to an angle range, and the angle range is the range of the angle between each straight line in each group and the positive direction of the vertical coordinate axis in the target region image.

[0080] In this embodiment, vanishing points are extracted by establishing a voting region. Specifically, a straight line is first selected:

[0081] The detected straight lines are divided into two groups, S1 and S2, according to the following formula (1):

[0082] (1)

[0083] in It is the selected straight line, for example, the first straight line contained in the target region image. It is a straight line The angle between the image and the positive direction of the u-axis.

[0084] Step c2: Based on the at least two sets of straight lines, repeat the following steps to obtain a point set: obtain one line segment from each of the at least two sets of straight lines; if at least two line segments satisfy the first preset condition, calculate the intersection point of the at least two line segments.

[0085] In this embodiment, two line segments are selected. , ,judge and Do the two lines satisfy the following equation (2):

[0086] (2)

[0087] Where N is the image height. They are respectively and The angle between the image and the positive direction of the u-axis.

[0088] If satisfied, then the solution can be found. and The intersection point P is obtained by repeating this process to find all line pairs that satisfy the conditions of equation (2) above, and thus obtain the point set. .

[0089] Step c3: Based on the point set, determine the voting region in the target region image that contains the target vanishing point.

[0090] The voting area can be obtained through the following steps:

[0091] Step d1: Obtain the first point with the maximum x-coordinate, the second point with the minimum x-coordinate, the third point with the maximum y-coordinate, and the fourth point with the minimum y-coordinate from the set of points.

[0092] Step d2: Generate a rectangle based on the first point, the second point, the third point, and the fourth point, wherein the area of ​​the rectangle is the voting area.

[0093] Specifically, the point where the maximum value of the u-axis is obtained from this set of points (i.e., the first point). The point where the u-axis has its minimum value (i.e., the second point). The point where the v-axis reaches its maximum value (i.e., the third point). The point where the v-axis has its minimum value (i.e., the fourth point). Using these four points, a rectangle can be constructed. The four vertices of this rectangle are: , , and This rectangle is the voting area.

[0094] See Figure 6 As shown, lines in the sky area and vertical lines are removed from the detected line set (the first straight line and the vertical lines). The remaining straight lines are then stored separately in line sets S1 and S2 using the above formula (1). Then, straight line L1 (i.e. ) and L2 (i.e. ), determine whether L1 and L2 satisfy the above equation (2). If not, continue to select lines L1 and L2; if so, find the intersection point P of L1 and L2 and store it in the point set. (Right now Then determine if the line segment combinations have been exhausted. If not, continue selecting lines L1 and L2; if so, then in the point set... Select , , , This constitutes the voting area.

[0095] Step c4: Based on the voting area, vote on the pixels within the voting area to determine the target vanishing point.

[0096] The process of determining the target vanishing point based on the voting area can be achieved through the following steps:

[0097] Step e1: Obtain the voting line from the voting area, and obtain and initialize the candidate voting points from the voting line.

[0098] Step e2: Based on the candidate voting points, calculate the line segment weight, angle weight, and distance weight corresponding to the voting line.

[0099] Step e3: Determine the score of the candidate voting point based on the line segment weight, angle weight, and distance weight, and take the candidate voting point with the highest score as the target elimination point.

[0100] In this embodiment, the pixel with the highest score is selected as the vanishing point of the image by voting on the pixels in the voting area.

[0101] Specifically, voting strategies can be divided into three aspects:

[0102] 1) Line segment weights

[0103] Based on the line segment detection algorithm (i.e., the LSD algorithm), the longer the detected line segment, the more pixels there are in the same direction. Since longer line segments have more pixels in the same direction, they play an important role in the voting process. Therefore, the line segment length can affect the voting score, and a line segment weight corresponding to the line segment length is set.

[0104] Among them, the line segment weight The definition can be: The magnitude of the weight is related to the length of the line segment; the longer the line segment, the greater the weight. The larger, the smaller, The smaller it is.

[0105] 2) Angle weights

[0106] In practical applications, the directions of road edges and boundaries, and even tire tracks and ruts left by previous vehicles, often converge within a certain angular range. In photographs taken by cameras during vehicle movement, the directional angles of these lines passing through the vanishing point are often angled, such as road boundaries, lane lines, and tire tracks. These straight lines are more effective for voting. Therefore, a weight can be assigned to lines with different angles, known as directional weights. Furthermore, lines that are nearly horizontal or vertical are generally invalid, as they do not pass through the vanishing point. Therefore, the directional weights are defined as follows:

[0107] (1) Line segments that are perpendicular to or parallel to the width and height of the image are considered to have an angle weight of 0;

[0108] (2) For other line segments, the solution is obtained using equation (3):

[0109] =

[0110]

[0111] in, Let H be the angle between the line segment and the positive direction of the u-axis, H be the height of the image, and D be the length of the image diagonal. The angle weights can be solved using equation (3). .

[0112] 3) Distance weight

[0113] For vanishing points, when the error is zero, most lines in the image should intersect at that point; therefore, the voted vanishing point should be as close as possible to the line segment. In practical applications, during the voting process, different Gaussian values ​​need to be assigned to the candidate vanishing point (i.e., candidate voting point) based on its distance from the line. The purpose is to make the voting process for the line smoother, thereby improving the detection accuracy of the vanishing point.

[0114] Since this embodiment uses line segments to solve for vanishing points, the more line segments a candidate vanishing point has that point, the greater the probability that the point is a vanishing point. Therefore, the distance from the candidate vanishing point to each point is also a key factor. The definition of the distance weight is as follows:

[0115]

[0116] in, Let be the distance from the candidate vanishing point to the line. The distance weight from the vanishing point to the line segment can be calculated using the above equation (4). .

[0117] After solving for the three weights, the scores of the candidate vanishing points can be obtained using the following formula (5). The candidate vanishing point with the highest score is selected as the vanishing point to be solved, i.e., the target vanishing point.

[0118]

[0119] in, Score the candidate vanishing point. represents the pixel coordinates of the candidate vanishing point, and n represents the number of voting lines.

[0120] In practical applications, see Figure 7 As shown, first, the voting line is obtained, then candidate voting points are selected, and initialization is performed. Then, based on the candidate vanishing points, a straight line is selected, and three weights, namely W, are calculated. L W O W S Solve Determine if all lines have been voted on; if not, continue selecting lines based on candidate vanishing points and calculate three weights, namely W. L W O W S If so, then record. Then determine all points Have all been recorded? If not, continue selecting candidate voting points and initializing. If so, then find the largest. The point is taken as the vanishing point, i.e., the target vanishing point.

[0121] Wherein, the line segment weight is used to represent the weight associated with the line segment length; the angle weight is used to represent the weight associated with the line segment angle; and the distance weight is used to represent the weight associated with the distance from the candidate voting point to the line.

[0122] In one possible design, based on the above embodiments, a detailed explanation of how to measure the distance from a point to a camera is provided. Specifically, based on the target camera's coordinates in the world coordinate system, the target camera's focal length, the size of the target image, and the projected coordinates, the distance from the target point to the target camera is determined using the pitch and horizontal angles corrected by the target vanishing point. This can be achieved through the following steps:

[0123] Step f1: Obtain the initial pitch angle and initial horizontal angle corresponding to the target camera.

[0124] Step f2: Based on the target vanishing point, the focal length of the target camera, and the size of the target image, correct the initial pitch angle and the initial horizontal angle to determine the corrected pitch angle and the corrected horizontal angle corresponding to the target camera.

[0125] Step f3: Based on the corrected pitch angle and corrected horizontal angle corresponding to the target camera, correct the coordinates of the target camera in the world coordinate system to obtain the actual coordinates of the target camera in the world coordinate system.

[0126] Step f4: Determine the distance from the target point to the target camera based on the actual coordinates of the target camera in the world coordinate system and the horizontal angle; wherein, the distance from the target point to the target camera includes the distance from the target point to the target camera in the horizontal axis direction and the distance in the vertical axis direction.

[0127] In this embodiment, adaptive IPM ranging is used, which combines the calibration IPM algorithm with the adaptive IPM algorithm. The calibration IPM algorithm is used to determine whether the adaptive IPM algorithm is effective. By establishing an error model for the calibration IPM algorithm, if the positioning result of the adaptive IPM algorithm is within the range of this model, the positioning result is reliable; otherwise, it is unreliable.

[0128] Specifically, in combination Figure 1 As shown, assuming See Figure 8 As shown, let the projected coordinates of point m (i.e., the target point to be measured) on the image be... ,in ,in For camera master point The v-axis coordinates are given, the camera height is h, and the camera focal length is f. Let the initial pitch angle be the initial tilt angle. The effect of the initial pitch angle on IPM is as follows: The value of point m on the y-axis is:

[0129]

[0130] Similarly, such as Figure 9 As shown, the effect of the γ angle on IPM: where, ,in For camera master point If the u-axis coordinate is given, then the x-axis value of point m is:

[0131]

[0132] In practical applications, It is not necessarily 0. Based on the above equations (6) and (7), the adaptive IPM algorithm can be obtained:

[0133]

[0134] in, The pitch angle obtained from the vanishing point The horizontal angle is given, and the camera's coordinates in the world coordinate system are... The corrected camera coordinates in the world coordinate system can be obtained through equation (8).

[0135] When the camera is mounted or moved, the angle between it and the vertical plane is not necessarily 0, resulting in a constantly changing yaw angle. The relationship between this yaw angle and the 3D coordinates is as follows:

[0136]

[0137] and The angle (the corrected pitch obtained from the vanishing point) and the yaw angle are related to the vanishing point, image height and width, and focal length, see Equation (10).

[0138]

[0139] Where H and W are the image height and width, respectively. These are the vanishing point coordinates, i.e., the target vanishing point coordinates.

[0140] Specifically, when the target vanishing point coordinates and the object's projection on the image are obtained... First, use the vanishing point to solve for the pitch angle and yaw angle, then use equation (8) to solve for... and Finally, the solution is obtained using equation (9). and This completes the solution for IPM. and These are the distances from the point being solved to the camera in the x-direction (i.e., the distance from the target point to the target camera along the horizontal axis) and the distances in the y-direction (i.e., the distance from the target point to the target camera along the vertical axis), respectively.

[0141] The algorithm employs an optimized vanishing point detection method: It selects lines in the central and ground regions of the image that are not perpendicular to the image's u-axis and v-axis, and performs pairwise intersections to obtain a set of intersection points. These points are then clustered to identify the vanishing points. A vanishing point tracking method is then used: a tracking algorithm is employed to track the vanishing points. The calibration IPM algorithm is combined with an adaptive IPM algorithm, and the calibration IPM algorithm is used to determine the effectiveness of the adaptive IPM algorithm. An error model for the calibration IPM algorithm is established; if the localization result of the adaptive IPM algorithm falls within this model's range, the localization result is considered reliable; otherwise, it is unreliable.

[0142] Therefore, in this application, compared to pinhole imaging ranging, the distance to the camera can be calculated from any point on the ground that is projected onto the image. This eliminates the need to know the length and width of the object being measured, nor does it require the object to be parallel to the image plane. Furthermore, the pitch and yaw angles of the camera relative to the world coordinate system can be calculated in real-time using vanishing points. Because fixed pitch and yaw angles can change due to vehicle bumps, camera vibrations, etc., the ranging error increases with distance. Therefore, the real-time calculated pitch and yaw angles allow for adaptive ranging, resulting in higher accuracy. Moreover, during ranging, only image processing is needed to obtain 2D points on the image for measurement. In contrast, the PNP ranging method requires more than four 3D points and matching them with 2D points on the image, making its application conditions demanding. This application, however, has simpler conditions, requiring only the camera's coordinates in the world coordinate system. The focal length f of the target camera, and the vanishing point. And the projected coordinates of the point on the ground to be measured on the image. Then the distance from that point to the camera can be calculated.

[0143] To implement the aforementioned information processing method, this embodiment provides an information processing apparatus. See also... Figure 10A , Figure 10A This is a schematic diagram of the structure of an information processing device provided in an embodiment of this application. The information processing device 100 includes: a first processing module 1001, a second processing module 1002, a third processing module 1003, and a fourth processing module 1004. The first processing module 1001 is used to process a target image using an edge detection operator to obtain edge information of objects in the target image, wherein the target image is captured by a target camera. The second processing module 1002 is used to extract a first straight line in the image corresponding to the edge information using a straight line detection and segmentation algorithm, wherein the length of the line segment corresponding to the first straight line is greater than a preset threshold. The third processing module 1003 is used to partition the target image according to the first straight line to obtain a target region image. The fourth processing module 1004 is used to determine the target vanishing point according to the target region image, wherein the target vanishing point is used to correct the pitch angle and horizontal angle of the target camera relative to the world coordinate system.

[0144] In this embodiment, a first processing module 1001, a second processing module 1002, a third processing module 1003, and a fourth processing module 1004 are configured to process the target image using an edge detection operator to obtain the edge information of objects in the target image, which is captured by a target camera. Then, a line detection and segmentation algorithm is used to extract the first straight line in the image corresponding to the edge information, where the length of the line segment corresponding to the first straight line is greater than a preset threshold. Based on the first straight line, the target image is partitioned to obtain a target region image. Based on the target region image, the target vanishing point is determined to correct the pitch and horizontal angles of the target camera relative to the world coordinate system. This eliminates the need for complex and stringent ranging conditions to correct camera parameters. Furthermore, the method of using the vanishing point extraction to correct the pitch and horizontal angles of the target camera relative to the world coordinate system is highly accurate, thus avoiding the problem of large ranging errors.

[0145] The apparatus provided in this embodiment can be used to execute the technical solutions of the above method embodiments. Its implementation principle and technical effects are similar, and will not be described again here.

[0146] In one possible design, the third processing module is specifically used for: obtaining a vertical line perpendicular to the horizontal axis in the phase plane coordinate system from the first straight line, and dividing the target image to obtain multiple sub-images; obtaining straight lines from each of the multiple sub-images to form a group of straight lines, wherein each line segment in the group of straight lines is represented by a two-dimensional coordinate point, the two-dimensional coordinate point being a coordinate point in the phase plane coordinate system; obtaining the point with the maximum ordinate and the point with the minimum ordinate corresponding to each sub-image from the group of straight lines; connecting all the points with the maximum ordinate to generate a first geometric line, and connecting all the points with the minimum ordinate to generate a second geometric line, wherein the first geometric line and the second geometric line divide the target image into three regions; and selecting the region with a large vanishing point response in the target image from the three regions as the target region image.

[0147] In one possible design, the fourth processing module is specifically used to: divide the first straight line contained in the target region image to obtain at least two sets of straight lines, wherein each set of straight lines corresponds to an angle range, the angle range being the range of the angle between each straight line in each set and the positive direction of the vertical coordinate axis in the target region image; based on the at least two sets of straight lines, repeat the following steps to obtain a point set: obtain a line segment from each of the at least two sets of straight lines, and if at least two line segments satisfy a first preset condition, calculate the intersection point of the at least two line segments; based on the point set, determine the voting region in the target region image containing the target vanishing point; based on the voting region, vote on the pixels in the voting region to determine the target vanishing point.

[0148] In one possible design, the fourth processing module is specifically used to: obtain from the set of points a first point with the maximum horizontal coordinate, a second point with the minimum horizontal coordinate, a third point with the maximum vertical coordinate, and a fourth point with the minimum vertical coordinate; and generate a rectangle based on the first, second, third, and fourth points, wherein the area of ​​the rectangle is the voting area.

[0149] In one possible design, the fourth processing module is specifically used to obtain voting lines from the voting area, and obtain and initialize candidate voting points from the voting lines; calculate the line segment weight, angle weight, and distance weight corresponding to the voting lines based on the candidate voting points; determine the score of the candidate voting points based on the line segment weight, angle weight, and distance weight, and take the candidate voting point with the highest score as the target elimination point.

[0150] In one possible design, the line segment weight is used to represent the weight associated with the line segment length; the angle weight is used to represent the weight associated with the line segment angle; and the distance weight is used to represent the weight associated with the distance from the candidate voting point to the line.

[0151] To implement the aforementioned ranging method, this embodiment provides a ranging device. See also... Figure 10B , Figure 10BThis is a schematic diagram of the ranging device provided in an embodiment of this application. The ranging device 110 includes: an acquisition module 1101, a vanishing point extraction module 1102, and a ranging module 1103. The acquisition module 1101 is used to acquire the coordinates of the target camera in the world coordinate system and the focal length of the target camera. The vanishing point extraction module 1102 is used to extract the target vanishing point from the target image formed by the target camera capturing the target point to be measured. The target vanishing point is used to correct the pitch angle and horizontal angle of the target camera relative to the world coordinate system. The acquisition module 1101 is also used to acquire the projection coordinates of the target point on the target image. The ranging module 1103 is used to determine the distance from the target point to the target camera based on the coordinates of the target camera in the world coordinate system, the focal length of the target camera, the size of the target image, and the projection coordinates, using the pitch angle and horizontal angle corrected by the target vanishing point.

[0152] In this embodiment, by setting up an acquisition module 1101, a vanishing point extraction module 1102, and a ranging module 1103, the target camera's coordinates in the world coordinate system and its focal length are acquired. Then, the target vanishing point is extracted from the target image formed by the target point to be ranged, captured by the target camera. The projected coordinates of the target point on the target image are acquired, and the distance from the target point to the target camera is determined based on the target camera's coordinates in the world coordinate system, its focal length, the size of the target image, the target vanishing point, and the projected coordinates. This eliminates the need for complex and demanding ranging conditions, avoids the problem of large ranging errors, and improves the accuracy of ranging.

[0153] The apparatus provided in this embodiment can be used to execute the technical solutions of the above method embodiments. Its implementation principle and technical effects are similar, and will not be described again here.

[0154] In one possible design, the ranging module 1103 is specifically used for: acquiring the initial pitch angle and initial horizontal angle corresponding to the target camera; correcting the initial pitch angle and initial horizontal angle based on the target vanishing point, the focal length of the target camera, and the size of the target image to determine the corrected pitch angle and the corrected horizontal angle corresponding to the target camera; correcting the coordinates of the target camera in the world coordinate system based on the corrected pitch angle and the corrected horizontal angle corresponding to the target camera to obtain the actual coordinates of the target camera in the world coordinate system; and determining the distance from the target point to the target camera based on the actual coordinates of the target camera in the world coordinate system and the horizontal angle; wherein the distance from the target point to the target camera includes the distance from the target point to the target camera along the horizontal axis and the distance along the vertical axis.

[0155] To implement the methods of the above embodiments, this embodiment provides an electronic device. Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 11 As shown, the electronic device 120 of this embodiment includes a processor 1201 and a memory 1202; wherein, the memory 1202 is used to store computer execution instructions; the processor 1201 is used to execute the computer execution instructions stored in the memory to implement the various steps performed in the above embodiment. For details, please refer to the relevant descriptions in the foregoing method embodiments.

[0156] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the method described above.

[0157] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the method described above.

[0158] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms. Additionally, the functional modules in the various embodiments of this application may be integrated into one processing unit, or each module may exist physically separately, or two or more modules may be integrated into one unit. The above-mentioned modular units can be implemented in hardware or in the form of hardware plus software functional units.

[0159] The integrated modules implemented as software functional modules described above can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of this application. It should be understood that the processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0160] The memory may include high-speed RAM, and may also include non-volatile memory (NVM), such as at least one disk drive, and may also be a USB flash drive, external hard drive, read-only memory, disk, or optical disc. The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses in the accompanying drawings are not limited to a single bus or a single type of bus. The aforementioned storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, disk, or optical disc. The storage medium can be any available medium accessible to general-purpose or special-purpose computers.

[0161] An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor. Both the processor and the storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and storage medium can exist as discrete components in an electronic device or host device.

[0162] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0163] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. An information processing method, characterized in that, include: The edge information of objects in the target image is obtained by processing the target image with an edge detection operator. The target image is captured by a target camera. The first straight line in the image corresponding to the edge information is extracted by a straight line detection and segmentation algorithm, and the length of the line segment corresponding to the first straight line is greater than a preset threshold. Based on the first straight line, the target image is partitioned to obtain a target region image; Based on the target area image, the target vanishing point is determined, and the target vanishing point is used to correct the pitch angle and horizontal angle of the target camera relative to the world coordinate system; The step of partitioning the target image according to the first straight line to obtain a target region image includes: Obtain a vertical line perpendicular to the horizontal axis in the phase plane coordinate system from the first straight line, and divide the target image to obtain multiple sub-images; Straight lines are obtained from each of the plurality of sub-images to form a group of straight lines, wherein each line segment in the group of straight lines is represented by a two-dimensional coordinate point, the two-dimensional coordinate point being a coordinate point in the phase plane coordinate system; From the set of lines, obtain the points in each sub-image where the ordinate has the maximum value and the points where the ordinate has the minimum value; Connect all points with the maximum ordinate to generate a first geometric line, and connect all points with the minimum ordinate to generate a second geometric line. The first and second geometric lines divide the target image into three regions. The region with the largest vanishing point response in the target image is selected from the three regions and used as the target region image.

2. The method according to claim 1, characterized in that, Determining the target vanishing point based on the target region image includes: The first straight line contained in the target region image is divided to obtain at least two groups of straight lines, wherein each group of straight lines corresponds to an angle range, and the angle range is the range of the angle between each straight line in each group and the positive direction of the vertical coordinate axis in the target region image. Based on the at least two sets of straight lines, repeat the following steps to obtain a point set: obtain one line segment from each of the at least two sets of straight lines; if at least two line segments satisfy the first preset condition, calculate the intersection point of the at least two line segments. Based on the point set, determine the voting region in the target region image that contains the target vanishing point; Based on the voting area, the pixels within the voting area are voted on to determine the target vanishing point.

3. The method according to claim 2, characterized in that, The step of determining the voting region containing the target vanishing point in the target region image based on the point set includes: From the set of points, obtain the first point with the maximum x-coordinate, the second point with the minimum x-coordinate, the third point with the maximum y-coordinate, and the fourth point with the minimum y-coordinate; A rectangle is generated based on the first, second, third, and fourth points, wherein the area of ​​the rectangle is the voting area.

4. The method according to claim 2, characterized in that, The step of determining the target vanishing point by voting on pixels within the voting area includes: Obtain the voting line from the voting area, and obtain and initialize candidate voting points from the voting line; Based on the candidate voting points, the line segment weight, angle weight, and distance weight corresponding to the voting line are calculated. Based on the line segment weight, angle weight, and distance weight, the score of the candidate voting point is determined, and the candidate voting point with the highest score is taken as the target elimination point.

5. The method according to claim 4, characterized in that, The line segment weight is used to represent the weight associated with the line segment length; the angle weight is used to represent the weight associated with the line segment angle; and the distance weight is used to represent the weight associated with the distance from the candidate voting point to the line.

6. A distance measurement method, characterized in that, include: Obtain the coordinates of the target camera in the world coordinate system and the focal length of the target camera; The target vanishing point is extracted from the target image formed by the target point to be ranged by the target camera. The target vanishing point is used to correct the pitch angle and horizontal angle of the target camera with respect to the world coordinate system. Obtain the projected coordinates of the target point on the target image; Based on the target camera's coordinates in the world coordinate system, the target camera's focal length, the target image's size, and the projected coordinates, the distance from the target point to the target camera is determined using the pitch and horizontal angles corrected by the target vanishing point. The target vanishing point is determined by the information processing method as described in any one of claims 1-5.

7. The method according to claim 6, characterized in that, The step of determining the distance from the target point to the target camera based on the target camera's coordinates in the world coordinate system, the target camera's focal length, the target image's size, and the projected coordinates, using the pitch and horizontal angles corrected by the target vanishing point, includes: Obtain the initial pitch angle and initial horizontal angle corresponding to the target camera; Based on the target vanishing point, the focal length of the target camera, and the size of the target image, the initial pitch angle and the initial horizontal angle are corrected to determine the corrected pitch angle and the corrected horizontal angle corresponding to the target camera. Based on the corrected pitch angle and corrected horizontal angle corresponding to the target camera, the coordinates of the target camera in the world coordinate system are corrected to obtain the actual coordinates of the target camera in the world coordinate system; The distance from the target point to the target camera is determined based on the actual coordinates of the target camera in the world coordinate system and the horizontal angle.

8. An information processing device, characterized in that, include: The first processing module is used to process the target image using an edge detection operator to obtain the edge information of the objects in the target image, wherein the target image is obtained by a target camera. The second processing module is used to extract the first straight line in the image corresponding to the edge information through a straight line detection and segmentation algorithm, wherein the length of the line segment corresponding to the first straight line is greater than a preset threshold. The third processing module is used to partition the target image according to the first straight line to obtain a target region image; The fourth processing module is used to determine the target vanishing point based on the target area image. The target vanishing point is used to correct the pitch angle and horizontal angle of the target camera relative to the world coordinate system. The third processing module is specifically used for: Obtain a vertical line perpendicular to the horizontal axis in the phase plane coordinate system from the first straight line, and divide the target image to obtain multiple sub-images; Straight lines are obtained from each of the plurality of sub-images to form a group of straight lines, wherein each line segment in the group of straight lines is represented by a two-dimensional coordinate point, the two-dimensional coordinate point being a coordinate point in the phase plane coordinate system; From the set of lines, obtain the points in each sub-image where the ordinate has the maximum value and the points where the ordinate has the minimum value; Connect all points with the maximum ordinate to generate a first geometric line, and connect all points with the minimum ordinate to generate a second geometric line. The first and second geometric lines divide the target image into three regions. The region with the largest vanishing point response in the target image is selected from the three regions and used as the target region image.

9. A ranging device, characterized in that, include: The acquisition module is used to acquire the coordinates of the target camera in the world coordinate system and the focal length of the target camera; The vanishing point extraction module is used to extract the target vanishing point from the target image formed by the target point to be measured captured by the target camera. The target vanishing point is used to correct the pitch angle and horizontal angle of the target camera with respect to the world coordinate system. The acquisition module is further configured to acquire the projection coordinates of the target point on the target image; The ranging module is used to determine the distance from the target point to the target camera based on the coordinates of the target camera in the world coordinate system, the focal length of the target camera, the size of the target image, and the projected coordinates, using the pitch angle and horizontal angle corrected by the target vanishing point. The target vanishing point is determined by the information processing method as described in any one of claims 1-5.

Images