Single-tire-based heading detection method, device, equipment and storage medium

Abstract

The application relates to a single-tire-based heading detection method, device, equipment and storage medium. The heading detection method comprises the following steps: regressing an ellipse for representing the outer contour of a tire on an image, and four vertices of the ellipse, wherein the four vertices correspond to four end points of two mutually perpendicular diameters of the tire; using a tire center and a heading angle to represent a first constraint of the four end points; using a camera optical center and the four vertices to represent a second constraint of the four end points; and analyzing the first constraint and the second constraint to obtain the heading angle, so as to determine the heading of an obstacle to which the tire belongs. The heading detection method provided by the application can improve the problem that the heading detection accuracy of an obstacle in the prior art is not high.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus, device, and storage medium for heading detection based on a single tire. Background Technology

[0002] During the process of autonomous driving, the autonomous driving system needs to identify and detect information such as the heading of other obstacles in the surrounding environment in order to make more reasonable and effective path planning.

[0003] Currently, to determine a car's heading, the contact points of the two tires on the same side of the car can be detected first, and then the heading can be determined based on the direction of the line connecting the two tire contact points.

[0004] However, this method requires a high degree of image integrity, necessitating the simultaneous detection of at least two wheels of the car. It cannot be applied if only a single tire is detected. Therefore, existing technologies still suffer from low accuracy in obstacle heading detection. Summary of the Invention

[0005] Based on this, this application provides a heading detection method, apparatus, device, and storage medium based on a single tire, which improves the problem of low heading detection accuracy in the prior art.

[0006] In a first aspect, this application provides a heading detection method based on a single tire. The heading detection method includes: regressing an ellipse representing the outer contour of the tire from an image, and the four vertices of the ellipse, wherein the four vertices correspond to the four endpoints of two mutually perpendicular diameters of the tire; representing a first constraint on the four endpoints using the tire's center and a heading angle, wherein the heading angle is the angle between the tangent at the tire's contact point and the horizontal axis of the camera coordinate system, and the first constraint describes the tire shape's restriction on the four endpoints; representing a second constraint on the four endpoints using the camera's optical center and the four vertices, wherein the second constraint describes the mapping relationship's restriction on the four endpoints; and analyzing the heading angle based on the first and second constraints to determine the heading of the obstacle to which the tire belongs.

[0007] In conjunction with the first aspect, in the first possible implementation of the first aspect, the step of representing the first constraint of the four endpoints using the tire center and heading angle includes: setting parameters for the tire center, tire radius, heading angle, and yaw angle respectively, wherein the heading angle is the angle between the tangent of the tire's contact point and the horizontal axis of the camera coordinate system, and the yaw angle is the angle between the diameter of the tire's contact point and the vertical axis of the camera coordinate system; and using the parameters to represent the first constraint of the first endpoint, the first constraint of the second endpoint, the first constraint of the third endpoint, and the first constraint of the fourth endpoint among the four endpoints respectively.

[0008] In conjunction with the first possible implementation of the first aspect, in the second possible implementation of the first aspect, the step of representing the second constraint of the four endpoints using the camera optical center and the four vertices includes: representing a first plane based on the camera optical center and two major axis vertices among the four vertices, wherein the first endpoint and the second endpoint corresponding to the major axis vertex among the four endpoints are located on the first plane; and representing a second plane based on the camera optical center and two minor axis vertices among the four vertices, wherein the third endpoint and the fourth endpoint corresponding to the minor axis vertex among the four endpoints are located on the second plane.

[0009] In conjunction with the second possible implementation of the first aspect, in the third possible implementation of the first aspect, the step of obtaining the heading angle based on the analysis of the first constraint and the second constraint includes: substituting the first constraint and the first second constraint into the first plane respectively, and substituting the first third constraint and the first fourth constraint into the second plane to obtain four formulas; solving the four formulas simultaneously to obtain the heading angle.

[0010] In conjunction with the third possible implementation of the first aspect, in the fourth possible implementation of the first aspect, the mathematical expressions of the first constraint, the first second constraint, the first third constraint, and the first fourth constraint are respectively: (x0-Rsin(m)cos(n), y0+Rsin(m)sin(n), z0+Rcos(m)), (x0+Rsin(m)cos(n), y0-Rsin(m)sin(n), z0-Rcos(m)), (x0-Rcos(m)cos(n), y0+Rcos(m)sin(n), z0-Rsin(m)) and (x0+Rcos(m)cos(n), y0-Rcos(m)sin(n), z0+Rsin(m)); where (x0, y0, z0) represents the center of the tire, R represents the tire radius, n represents the heading angle, and m represents the yaw angle.

[0011] In conjunction with the fourth possible implementation of the first aspect, in the fifth possible implementation of the first aspect, the mathematical expressions for the first plane and the second plane are respectively: M1x+N1y+P1z+Q1=0 and M2x+N2y+P2z+Q2=0; wherein, M1, N1, P1, Q1, M2, N2, P2 and Q2 are known constants, and x, y and z are unknown parameters used to represent any point on the first plane and the second plane.

[0012] In conjunction with the fifth possible implementation of the first aspect, in the sixth possible implementation of the first aspect, the mathematical expression for the aforementioned heading angle is: or, Where H = M1M2 - N1N2, S = -(M1N2 + M2N1), T = -2P1P2 - M1M2 - N1N2.

[0013] Secondly, this application provides a heading detection device, which includes: an image unit for regressing an ellipse representing the outer contour of a tire and four vertices of the ellipse on an image, wherein the four vertices correspond to the four endpoints of two mutually perpendicular diameters of the tire; a construction unit for representing a first constraint of the four endpoints using the tire center and a heading angle, wherein the heading angle is the angle between the tangent at the tire's contact point and the horizontal axis of the camera coordinate system, and the first constraint describes the restriction of the tire shape on the four endpoints; the construction unit is further used to represent a second constraint of the four endpoints using the camera optical center and the four vertices, wherein the second constraint describes the restriction of the mapping relationship on the four endpoints; and an analysis unit for analyzing the heading angle based on the first and second constraints to determine the heading of the obstacle to which the tire belongs.

[0014] In conjunction with the second aspect, in the first possible implementation of the second aspect, the above-mentioned construction unit is specifically used to: set parameters for the tire center, tire radius, heading angle, and yaw angle, respectively, wherein the heading angle is the angle between the tangent of the tire's contact point and the horizontal axis of the camera coordinate system, and the yaw angle is the angle between the diameter of the tire's contact point and the vertical axis of the camera coordinate system; and use the parameters to represent the first constraint of the first endpoint, the first constraint of the second endpoint, the first constraint of the third endpoint, and the first constraint of the fourth endpoint among the four endpoints.

[0015] In conjunction with the first possible implementation of the second aspect, in the second possible implementation of the second aspect, the above-mentioned building unit is specifically used to: represent a first plane based on the camera optical center and two major axis vertices among the four vertices, wherein the first endpoint and the second endpoint corresponding to the major axis vertex among the four endpoints are located on the first plane; and represent a second plane based on the camera optical center and two minor axis vertices among the four vertices, wherein the third endpoint and the fourth endpoint corresponding to the minor axis vertex among the four endpoints are located on the second plane.

[0016] In conjunction with the second possible implementation of the second aspect, in the third possible implementation of the second aspect, the above-mentioned analysis unit is specifically used to: substitute the first constraint and the first second constraint into the first plane respectively, and substitute the first third constraint and the first fourth constraint into the second plane to obtain four formulas; solve the four formulas simultaneously to obtain the heading angle.

[0017] In conjunction with the third possible implementation of the second aspect, in the fourth possible implementation of the second aspect, the mathematical expressions of the first constraint, the first second constraint, the first third constraint, and the first fourth constraint are respectively: (x0-Rsin(m)cos(n), y0+Rsin(m)sin(n), z0+Rcos(m)), (x0+Rsin(m)cos(n), y0-Rsin(m)sin(n), z0-Rcos(m)), (x0-Rcos(m)cos(n), y0+Rcos(m)sin(n), z0-Rsin(m)) and (x0+Rcos(m)cos(n), y0-Rcos(m)sin(n), z0+Rsin(m)); where (x0, y0, z0) represents the center of the tire, R represents the tire radius, n represents the heading angle, and m represents the yaw angle.

[0018] In conjunction with the fourth possible implementation of the second aspect, in the fifth possible implementation of the second aspect, the mathematical expressions for the first plane and the second plane are respectively: M1x+N1y+P1z+Q1=0 and M2x+N2y+P2z+Q2=0; wherein, M1, N1, P1, Q1, M2, N2, P2 and Q2 are known constants, and x, y and z are unknown parameters used to represent any point on the first plane and the second plane.

[0019] In conjunction with the fifth possible implementation of the second aspect, in the sixth possible implementation of the second aspect, the mathematical expression for the aforementioned heading angle is: or, Where H = M1M2 - N1N2, S = -(M1N2 + M2N1), T = -2P1P2 - M1M2 - N1N2.

[0020] Thirdly, this application also provides a heading detection device, which includes a processor and a memory connected via a bus; the processor is used to execute multiple instructions; the memory is used to store multiple instructions, which are adapted to be loaded by the processor and executed as a heading detection method as described in the first aspect or any embodiment of the first aspect.

[0021] Fourthly, this application also provides a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor and executing a heading detection method as described in the first aspect or any embodiment of the first aspect.

[0022] In summary, this application provides a method, apparatus, device, and storage medium for heading detection based on a single tire. The heading detection apparatus / device establishes a first constraint relationship and a second constraint relationship on the four endpoints of the tire based on the tire's shape and mapping relationship. These two constraints are then used to analyze and obtain the tire's heading angle, thereby determining the heading of the obstacle to which the tire belongs. It is evident that this application has lower requirements for image completeness and can determine the obstacle's heading even when only a single tire of a car is detected, thus improving the problem of low accuracy in obstacle heading detection in existing technologies. Attached Figure Description

[0023] Figure 1 A schematic diagram including an image of an obstacle in one embodiment provided in this application;

[0024] Figure 2 A schematic diagram illustrating the mapping relationship between the tire and the ellipse in the image provided in this application;

[0025] Figure 3 A flowchart illustrating the heading detection method based on a single tire provided in this application;

[0026] Figure 4 A schematic block diagram of a heading detection device provided in this application;

[0027] Figure 5 A structural block diagram of a heading detection device provided in this application. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0029] Since the embodiments of this application involve a relatively large number of technical terms, for ease of understanding, the relevant terms and concepts that may be involved in the embodiments of this application will be introduced below.

[0030] 1. Vertex of the ellipse

[0031] The vertices of an ellipse are the points where its major and minor axes intersect the ellipse's boundary. The ellipse described in this application has four vertices, including the two points where the major axis intersects the ellipse (i.e., the vertices of the major axis) and the two points where the minor axis intersects the ellipse (i.e., the vertices of the minor axis). For example, as... Figure 2 As shown, the four vertices of the ellipse abcd include the major axis vertex a, the major axis vertex b, the minor axis vertex c, and the minor axis vertex d.

[0032] 2. Tire end point

[0033] The endpoints of a tire are the points where the tire's diameter intersects with the tire's boundary. The tire described in this application includes four endpoints, each corresponding to one of the four vertices of an ellipse. For example, as... Figure 2 As shown, the four endpoints include endpoint A, endpoint B, endpoint C, and endpoint D. It's important to note that the tire is a physical object in the camera coordinate system, and the ellipse described above is a mapping of the tire's outer contour to the pixel coordinate system. Specifically:

[0034] First, according to the principles of optical imaging, there is a one-to-one correspondence between the endpoints of the tire and the vertices of the ellipse. For example... Figure 2 As shown, vertex 'a' of the major axis of ellipse abcd corresponds to the first endpoint A on tire ABCD. Similarly, vertex 'b' of the major axis corresponds to the second endpoint B, vertex 'c' of the minor axis corresponds to the third endpoint C, and vertex 'd' of the minor axis corresponds to the fourth endpoint D. Furthermore, the two corresponding points on the tire and the ellipse lie on the same ray originating from the camera's optical center O. For example... Figure 2 As shown, A and a are on the ray originating from the optical center O, B and b are on the ray originating from the optical center O, C and c are on the ray originating from the optical center O, and D and d are on the ray originating from the optical center O.

[0035] Secondly, according to the principles of optical imaging, the diameters corresponding to the major and minor axes of an ellipse are perpendicular to each other, for example... Figure 2 As shown, the major axis ab and minor axis cd of the ellipse abcd correspond to the diameters AB and CD of the tire ABCD, respectively, and diameters AB and CD are perpendicular to each other. Although the tire ABCD is deformed after being projected onto the image, changing from a perfect circle to an ellipse abcd, the diameters AB and CD corresponding to the major axis ab and minor axis cd of the ellipse abcd remain perpendicular.

[0036] It should be noted that the heading detection equipment and device involved in this application may include, but are not limited to, dedicated heading detection equipment, autonomous driving systems, terminal equipment, computers, processors, etc., and may be a device integrated into a vehicle or a detachable independent device on the vehicle. The heading detection equipment and device can interact with cameras on the vehicle, such as acquiring images captured by the cameras including obstacles. The processor may include, but is not limited to, a central processing unit (CPU), a general-purpose processor, a coprocessor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The processor can implement the methods described in this application, such as analyzing the heading angle of the obstacle to which the tire belongs based on the first constraint and the second constraint, etc., which will not be elaborated further in this application.

[0037] It should also be noted that the illustrations provided in this embodiment are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show components related to the present invention and are not drawn according to the actual number, shape, and size of the components in implementation. In actual implementation, the form, quantity, and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex. The structures, proportions, sizes, etc., shown in the accompanying drawings of this specification are only used to complement the content disclosed in the specification for those skilled in the art to understand and read, and are not intended to limit the conditions under which the present invention can be implemented. Therefore, they have no substantial technical significance. Any modification to the structure, change in the proportional relationship, or adjustment of the size, without affecting the effects and objectives that the present invention can produce, should still fall within the scope of the technical content disclosed in the present invention. Furthermore, the orientations or positional relationships indicated by terms such as "upper," "lower," "left," "right," "middle," "longitudinal," "lateral," "horizontal," "inner," "outer," "radial," and "circumferential" used in this specification are based on the orientations or positional relationships shown in the accompanying drawings and are only for the purpose of simplifying the description. They do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Changes or adjustments to their relative relationships, without substantially altering the technical content, should also be considered within the scope of the present invention. Therefore, they should not be construed as limitations on the present invention. In addition, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0038] Currently, when only a single tire is detected as an obstacle, the accuracy of heading detection remains low because existing technology cannot accurately determine the obstacle's heading. For example, ... Figure 1 The image shown includes obstacle 110, the body of which is cropped so that tire 120 is fully visible, while tire 130 is only partially visible. In existing technology, the contact point b of tire 120 and the contact point h of tire 130 can be detected first, and then connected to determine the heading of obstacle 110. However, the contact point h of tire 130 is not in the image, and the heading detection equipment / device may mistake any point on tire 130 other than h as the contact point, leading to incorrect detection results. Therefore, existing technology suffers from low accuracy in obstacle heading detection.

[0039] To address this issue, this application proposes a heading detection method based on a single tire. This method can detect the heading of a car by performing image analysis on only the single tire when it is captured in an image. Specifically, the heading detection device / apparatus first regresses an ellipse representing the outer contour of the tire from the image, along with the four vertices of the ellipse. The four vertices correspond to the four endpoints of the tire's two mutually perpendicular diameters. Then, the tire's center is used to represent the first constraint of the four endpoints, and the camera's optical center and the four vertices are used to represent the second constraint of the four endpoints. The first and second constraints respectively describe the limitations imposed on the four endpoints by the tire's shape and mapping relationship. Finally, the heading angle is obtained based on the first and second constraints to determine the heading of the obstacle to which the tire belongs.

[0040] The heading detection device / applied in this application can first use neural network technology such as wheel detection to regress the ellipse abcd representing the outer contour of the tire from the image, and can determine the major and minor axes of the ellipse abcd and the four vertices a, b, c, and d of the ellipse. The endpoints of these four vertices in the camera coordinate system are A, B, C, and D, respectively. According to the principle of optical imaging, AB and CD are two mutually perpendicular diameters on the tire. Therefore, the first constraint of endpoints A, B, C, and D can be represented using the tire center E and the heading angle. In addition, according to the principle of optical imaging, there is a mapping relationship between the tire ABCD and the ellipse abcd based on the camera optical center O. Therefore, the second constraint of endpoints A, B, C, and D can be represented using the camera optical center O and the vertices a, b, c, and d. Finally, the heading angle n is obtained by solving the first and second constraints simultaneously. Figure 2 As shown, the heading angle n can be used to represent the heading of the obstacle 110 to which the tire belongs.

[0041] To better understand the implementation process of the above-described heading detection method based on a single tire, this application provides an embodiment. Next, this application will combine... Figure 1 Images of obstacles are included. Figure 2 A diagram illustrating the mapping relationship in the diagram and Figure 3 The flowchart in the diagram, with the heading detection equipment as the main implementer, provides a detailed explanation of the heading detection method based on a single tire in this application:

[0042] 301: Regress the ellipse representing the outer contour of the tire from the image, and the four vertices of the ellipse.

[0043] The four vertices correspond to the four endpoints of the tire's two mutually perpendicular diameters. The heading detection equipment obtains, for example... Figure 1 After the image shown, neural network technology such as wheel detection can be used to detect the tires of the obstacle 110 in the image to obtain the tire 120, and regress the major and minor axes and orientation of the tire 120. Then, the major and minor axes and orientation are used to represent the ellipse abcd of the outer contour of the tire 120. Finally, the extreme values ​​of the ellipse abcd are obtained to obtain the four vertices a, b, c and d of the ellipse abcd.

[0044] 302: The first constraint of the four endpoints is represented by the tire center and heading angle.

[0045] Wherein, the heading angle is the angle between the tangent at the tire's contact point B and the horizontal axis X of the camera coordinate system, for example... Figure 2 The included angle n. The first constraint describes the restrictions imposed by the tire shape on the four endpoints; therefore, the first constraint can also be understood as a tire physical model. When using the tire center to represent the first constraint on the four endpoints, the heading detection device can set the tire center E as (x0, y0, z0), and then, based on the property that the tire is a perfect circle and the four endpoints are the endpoints of two mutually perpendicular diameters on that perfect circle, use (x0, y0, z0) to represent the four endpoints.

[0046] 303: The second constraint is represented by the camera optical center and four vertices, which are used to represent the four endpoints.

[0047] The second constraint describes the restriction of the mapping relationship on the four endpoints; therefore, the second constraint can also be understood as a mapping relationship model. The second constraint of the four endpoints is represented using the camera's optical center and the four vertices. This is achieved by utilizing the characteristic that the corresponding points on the tire ABCD and the ellipse abcd lie on rays originating from the optical center O. For example, a ray Oa passing through point A is established to form the second constraint for point A; a ray Ob passing through point B is established to form the second constraint for point B; a ray Oc passing through point C is established to form the second constraint for point C; and a ray Od passing through point D is established to form the second constraint for point D. As another example, a plane Oab containing point A is established to form the second constraint for point A; a plane Oab containing point B is established to form the second constraint for point B; a plane Ocd containing point C is established to form the second constraint for point C; and a plane Ocd containing point D is established to form the second constraint for point D.

[0048] It should be noted that this application does not restrict the order in which the first and second constraints are formed; they can be executed sequentially or in parallel.

[0049] 304: The heading angle is obtained based on the analysis of the first and second constraints to determine the heading of the obstacle to which the tire belongs.

[0050] Since the four endpoints A, B, C and D simultaneously satisfy the first constraint and the second constraint, and the first constraint contains the heading angle of the tire, the heading detection device can obtain the heading angle n by substituting the first constraint into the second constraint for calculation, thereby determining the heading of the obstacle 110 to which the tire belongs.

[0051] In summary, the heading detection device / equipment establishes a first constraint relationship and a second constraint relationship on the four endpoints of the tire based on the tire shape and mapping relationship, and uses these two constraints to analyze and obtain the heading angle of the tire, thereby determining the heading of the obstacle to which the tire belongs. Therefore, the heading detection method provided in this application can improve the problem of low accuracy in obstacle heading detection in the prior art.

[0052] In another feasible approach, the step of representing the first constraint of the four endpoints using the tire center and heading angle includes: setting parameters for the tire center, tire radius, heading angle, and yaw angle, respectively, wherein the yaw angle is the angle between the diameter of the tire's contact point and the vertical axis of the camera coordinate system; and using the parameters to represent the first constraint of the first endpoint, the first constraint of the second endpoint, the first constraint of the third endpoint, and the first constraint of the fourth endpoint.

[0053] In this embodiment, to form the first constraint, the heading detection device sets the parameters of the tire center E, tire radius, heading angle, and yaw angle to (x0, y0, z0), R, n, and m, respectively. Combined with... Figure 3As shown, the tire center refers to the axis center (point E) of the actual tire in the camera coordinate system. The tire radius is the radius of the actual tire (half of AB or CD, i.e., AB = CD = 2R). The heading angle is the angle (angle n) between the tangent at the tire's contact point B and the X-axis of the camera coordinate system. The deflection angle is the angle (angle m) between the diameter of the tire's contact point and the Y-axis of the camera coordinate system. After setting the above parameters, the first constraint of the four endpoints is expressed based on these parameters.

[0054] For example, the mathematical expression for the first constraint corresponding to the first endpoint A is: (x0-Rsin(m)cos(n), y0+Rsin(m)sin(n), z0+Rcos(m)); the mathematical expression for the first constraint corresponding to the second endpoint B is: (x0+Rsin(m)cos(n), y0-Rsin(m)sin(n), z0-Rcos(m)); and the mathematical expression for the first constraint corresponding to the third endpoint C is: (x0-Rc The mathematical expression for the first constraint of the fourth endpoint D is: (x0+Rcos(m)cos(n), y0-Rcos(m)sin(n), z0+Rsin(m)); where (x0, y0, z0) represents the center of the tire, R represents the physical radius of the tire, n represents the heading angle, and m represents the yaw angle. The meaning of these parameters will not be elaborated further below.

[0055] In another possible implementation, the step of representing the second constraint of the four endpoints using the camera optical center and the four vertices includes: representing a first plane based on the camera optical center and two major axis vertices of the four vertices, wherein the first endpoint and the second endpoint corresponding to the major axis vertex of the four endpoints are located on the first plane; and representing a second plane based on the camera optical center and two minor axis vertices of the four vertices, wherein the third endpoint and the fourth endpoint corresponding to the minor axis vertex of the four endpoints are located on the second plane.

[0056] In this embodiment, based on the principle of optical imaging, the camera optical center O, the first vertex a, and the first endpoint A lie on one ray, while the camera optical center O, the first vertex b, and the first endpoint B lie on another ray. Therefore, these two rays can form a plane, with O, a, b, A, and B on this plane, and any three points therein can define the plane. Furthermore, the camera optical center O, the first vertex c, and the first endpoint C lie on one ray, while the camera optical center O, the first vertex d, and the first endpoint D lie on another ray. Therefore, these two rays can also form a plane, with O, c, d, C, and D on this plane, and any three points therein can define the plane. Therefore, to form the second constraint, the heading detection device uses the camera optical center O, the major axis vertex a, and the major axis vertex b to represent the first plane, and uses the camera optical center O, the minor axis vertex c, and the minor axis vertex d to represent the second plane. According to the principle of optical imaging, the first endpoint A and the second endpoint B are located on the first plane, and the third endpoint C and the fourth endpoint D are located on the second plane. Therefore, the first plane is the second constraint of the first endpoint A and the second endpoint B, and the second plane is the second constraint of the third endpoint C and the fourth endpoint D.

[0057] For example, the mathematical expressions for the first plane and the second plane are M1x+N1y+P1z+Q1=0 and M2x+N2y+P2z+Q2=0, respectively; where M1, N1, P1, Q1, M2, N2, P2, and Q2 are known constants, and x, y, and z are unknown parameters used to represent any point on the first and second planes. The meaning of these parameters will not be elaborated further below.

[0058] It should be noted that the camera's optical center O is a point located in the camera coordinate system, and the vertices of the ellipse are also points located in the camera coordinate system. Therefore, before forming the first and second planes mentioned above, it is necessary to first determine the coordinates of the vertices of the ellipse in the camera coordinate system. Specifically, since the pixel coordinates of any point on the image can be directly determined, the heading detection device can first obtain the pixel coordinates of vertices a, b, c, and d, then convert the pixel coordinates into image coordinates through methods such as proportional scaling, and finally convert the image coordinates back to camera coordinates using camera intrinsic parameters to obtain the positions of vertices a, b, c, and d in the camera coordinate system.

[0059] In another feasible approach, the steps of obtaining the heading angle based on the first and second constraints include: substituting the first and second constraints into the first plane, and substituting the first and fourth constraints into the second plane to obtain four equations; and solving the four equations simultaneously to obtain the heading angle.

[0060] In this embodiment, the first constraint includes a first constraint corresponding to the first endpoint, a first second constraint corresponding to the second endpoint, a first third constraint corresponding to the third endpoint, and a first fourth constraint corresponding to the third endpoint. The second constraint includes a first plane corresponding to the first and second endpoints, and a second plane corresponding to the third and fourth endpoints. Since both the first constraint and the first plane are used to restrict the first endpoint, substituting the first constraint into the first plane yields a first formula. Similarly, substituting the first second constraint into the first plane yields a second formula, substituting the first third constraint into the second plane yields a third formula, and substituting the first fourth constraint into the second plane yields a fourth formula.

[0061] For example, the mathematical expression for the four equations above is: M1x A +N1y A +P1z A +Q1=0、M1x B +N1y B +P1z B +Q1=0、M2x c +N2y c +P2z c +Q2=0 and M2x D +N2y D +P2z D +Q2=0. It should be noted that (x A y A , z A (x) represents the first constraint. B y B , z B (x) represents the first and second constraints. c y c , z c (x) represents the first and third constraints. D y D , z D () indicates the first fourth constraint. Specifically, x A =x0 - Rsin(m)cos(n), y A =y0+Rsin(m)sin(n), z A =z0 + Rcos(m); x B =x0 + Rsin(m)cos(n), y B =y0-Rsin(m)sin(n), z B = z0 - Rcos(m); x c =x0-Rcos(m)cos(n), y c =y0+Rcos(m)sin(n), z c = z0 - Rsin(m); x D=x0+Rcos(m)cos(n), y D =y0-Rcos(m)sin(n), z D = z0 + Rsin(m). The meaning of these parameters will not be elaborated further below.

[0062] In another feasible approach, the mathematical expression for the heading angle obtained through analysis can be: or, Where H = M1M2 - N1N2, S = -(M1N2 + M2N1), and T = -2P1P2 - M1M2 - N1N2. The meanings of these parameters will not be elaborated further below.

[0063] Next, this application will explain how to solve the four equations simultaneously to obtain the heading. Specifically:

[0064] Substituting the first constraint into the first plane, the first second constraint into the first plane, the first third constraint into the second plane, and the first third constraint into the second plane, we obtain the following four formulas: M1x A +N1y A +P1z A +Q1=0、M1x B +N1y B +P1z B +Q1=0、M2x c +N2y c +P2z c +Q2=0、M2x D +N2y D +P2z D +Q2=0, where, assuming M i N i P i (i = 1, 2, 3, or 4) is not equal to 0. Simplifying the above four expressions yields:

[0065] M1sin(m)cos(n)-N1sin(m)sin(n)-P1cos(m)=0......①

[0066] M2cos(m)cos(n)-N2cos(m)sin(n)+P2sin(m)=0......②.

[0067] Let M1cos(n)-N1sin(n)=F1(n) and M2cos(n)-N2sin(n)=F2(n), then simplifying ① and ②, we get:

[0068] F1(n)sin(m)-P1cos(m)=0......③

[0069] F2(n)cos(m)+P2sin(m)=0......④.

[0070] Simplifying ③, we get cos(m)=F1(n)sin(m) / P1, and substituting this formula into ④, we get:

[0071] (F2(n)F1(n) / P1+P2)sin(m)=0......⑤.

[0072] Analyzing ⑤, when sin(m)=0, i.e. m=0 or m=π, substituting into formula ①, the contradiction does not hold; when F1(n)F2(n) / P1+P2=0, that is, F1(n)F2(n)=-P1P2, expanding F1(n) and F2(n), we can obtain:

[0073] (M1cos(n)-N1sin(n))(M2cos(n)-N2sin(n))=-P1P2......⑥

[0074] Further simplification of ⑥ yields:

[0075] M1M2cos 2 (n)+N1N2sin 2 (n)-(M1N2+M2N1)sin(n)cos(n)=-P0P1...⑦

[0076] According to the double-angle formula in trigonometric functions, ⑦ can be converted to:

[0077]

[0078] Further simplification of ⑧ yields:

[0079] (M1M2-N1N2)*cos(2n)-(M1N2+M2N1)*sin(2n))=-2P1P2-M1M2-N1N2......⑨

[0080] Let H = M1M2 - N1N2, S = -(M1N2 + M2N1), T = -2P1P2 - M1M2 - N1N2, and further simplify ⑨ as:

[0081] Hcos(2n)+Ssin(2n)=T......⑩

[0082] According to the auxiliary angle formula, ⑩ can be transformed into:

[0083]

[0084] right Further simplification yields the heading n:

[0085] or,

[0086] Where H = M1M2 - N1N2, S = -(M1N2 + M2N1), T = -2P1P2 - M1M2 - N1N2, and M0, M1, N0, N1, P0 and P1 are known constants, so n can be calculated.

[0087] In another embodiment, the present invention also provides a heading detection device, see [link to previous document]. Figure 4 The embodiments of the present invention can divide the device into functional units according to the above method examples. For example, each function can be divided into separate functional units, or two or more functions can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in the embodiments of the present invention is illustrative and only represents one logical functional division; other division methods may be used in actual implementation. Figure 4 As shown, the heading detection device includes an image unit 410, a construction unit 420, and an analysis unit 430. Specifically: the image unit 410 is used to regress an ellipse representing the outer contour of the tire and its four vertices on the image, wherein the four vertices correspond to the four endpoints of the tire's two mutually perpendicular diameters; the construction unit 420 is used to represent a first constraint of the four endpoints using the tire's center and heading angle, wherein the heading angle is the angle between the tangent at the tire's contact point and the horizontal axis of the camera coordinate system, and the first constraint describes the tire shape's restriction on the four endpoints; the construction unit 420 is also used to represent a second constraint of the four endpoints using the camera's optical center and the four vertices, wherein the second constraint describes the mapping relationship's restriction on the four endpoints; the analysis unit 430 is used to analyze and obtain the heading angle based on the first and second constraints to determine the heading of the obstacle to which the tire belongs.

[0088] In another feasible manner, the aforementioned building unit 420 is specifically used to: set parameters for the tire center, tire radius, heading angle, and yaw angle, respectively, wherein the heading angle is the angle between the tangent of the tire's contact point and the horizontal axis of the camera coordinate system, and the yaw angle is the angle between the diameter of the tire's contact point and the vertical axis of the camera coordinate system; and use the parameters to represent the first constraint of the first endpoint, the first constraint of the second endpoint, the first constraint of the third endpoint, and the first constraint of the fourth endpoint among the four endpoints.

[0089] In another possible implementation, the aforementioned building unit 420 is specifically used to: represent a first plane based on the camera optical center and two major axis vertices among the four vertices, wherein the first and second endpoints corresponding to the major axis vertices among the four endpoints are located on the first plane; and represent a second plane based on the camera optical center and two minor axis vertices among the four vertices, wherein the third and fourth endpoints corresponding to the minor axis vertices among the four endpoints are located on the second plane.

[0090] In another feasible manner, the analysis unit 430 is specifically used to: substitute the first constraint and the first second constraint into the first plane, and substitute the first third constraint and the first fourth constraint into the second plane to obtain four equations; solve the four equations simultaneously to obtain the heading angle.

[0091] In another feasible implementation, the mathematical expressions for the first constraint, the first second constraint, the first third constraint, and the first fourth constraint are respectively: (x0-Rsin(m)cos(n), y0+Rsin(m)sin(n), z0+Rcos(m)), (x0+Rsin(m)cos(n), y0-Rsin(m)sin(n), z0-Rcos(m)), (x0-Rcos(m)cos(n), y0+Rcos(m)sin(n), z0-Rsin(m)) and (x0+Rcos(m)cos(n), y0-Rcos(m)sin(n), z0+Rsin(m)); where (x0, y0, z0) represents the tire center, R represents the tire radius, n represents the heading angle, and m represents the yaw angle.

[0092] In another possible implementation, the mathematical expressions for the first and second planes are M1x + N1y + P1z + Q1 = 0 and M2x + N2y + P2z + Q2 = 0, respectively; where M1, N1, P1, Q1, M2, N2, P2, and Q2 are known constants, and x, y, and z are unknown parameters used to represent any point on the first and second planes.

[0093] In another feasible approach, the mathematical expression for the heading angle is: n = or, Where H = M1M2 - N1N2, S = -(M1N2 + M2N1), T = -2P1P2 - M1M2 - N1N2.

[0094] In one embodiment, this application also provides a heading detection device, see [link to relevant documentation]. Figure 5The heading detection device can be, but is not limited to, various personal computers, laptops, smartphones, tablets, portable wearable devices, and servers. The server can be a standalone server or a server cluster consisting of multiple servers. As shown in the figure, the heading detection device in this embodiment may include a processor 510 and a memory 520. The processor 510 and the memory 520 are connected via a bus 530. The processor 510 is used to execute multiple instructions; the memory 520 is used to store multiple instructions, which are adapted to be loaded by the processor 510 and executed as in the heading detection method of the above embodiment.

[0095] The processor 510 can be an Electronic Control Unit (ECU), a Central Processing Unit (CPU), a general-purpose processor, a coprocessor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The processor 510 can also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc. In this embodiment, the processor 510 can be a microcontroller. By programming the microcontroller, various control functions can be implemented. For example, in this embodiment, the processor can acquire, process, and demodulate the heading angle of obstacles. The processor has the advantages of powerful computing capabilities and fast processing speed. Specifically: the processor 510 is used to execute the functions of the image unit 410, to regress an ellipse representing the outer contour of the tire on the image, and the four vertices of the ellipse, wherein the four vertices correspond to the four endpoints of the two mutually perpendicular diameters of the tire; it is also used to represent a first constraint of the four endpoints using the tire center and the heading angle, wherein the heading angle is the angle between the tangent at the tire's contact point and the horizontal axis of the camera coordinate system, and the first constraint is used to describe the restrictions of the tire shape on the four endpoints; it is also used to represent a second constraint of the four endpoints using the camera optical center and the four vertices, wherein the second constraint is used to describe the restrictions of the mapping relationship on the four endpoints; and it is also used to execute the functions of the analysis unit 430, to analyze and obtain the heading angle based on the first and second constraints, so as to determine the heading of the obstacle to which the tire belongs.

[0096] In another possible implementation, the processor 510 is specifically used to: set parameters for the tire center, tire radius, heading angle, and yaw angle, respectively, wherein the heading angle is the angle between the tangent of the tire's contact point and the horizontal axis of the camera coordinate system, and the yaw angle is the angle between the diameter of the tire's contact point and the vertical axis of the camera coordinate system; and use the parameters to represent the first constraint of the first endpoint, the first constraint of the second endpoint, the first constraint of the third endpoint, and the first constraint of the fourth endpoint among the four endpoints.

[0097] In another possible implementation, the processor 510 is specifically used to: represent a first plane based on the camera optical center and two major axis vertices among the four vertices, wherein the first and second endpoints corresponding to the major axis vertices among the four endpoints are located on the first plane; and represent a second plane based on the camera optical center and two minor axis vertices among the four vertices, wherein the third and fourth endpoints corresponding to the minor axis vertices among the four endpoints are located on the second plane.

[0098] In another possible implementation, the processor 510 is specifically used to: substitute the first constraint and the first second constraint into the first plane, and substitute the first third constraint and the first fourth constraint into the second plane to obtain four equations; solve the four equations simultaneously to obtain the heading angle.

[0099] In another feasible implementation, the mathematical expressions for the first constraint, the first second constraint, the first third constraint, and the first fourth constraint are respectively: (x0-Rsin(m)cos(n), y0+Rsin(m)sin(n), z0+Rcos(m)), (x0+Rsin(m)cos(n), y0-Rsin(m)sin(n), z0-Rcos(m)), (x0-Rcos(m)cos(n), y0+Rcos(m)sin(n), z0-Rsin(m)) and (x0+Rcos(m)cos(n), y0-Rcos(m)sin(n), z0+Rsin(m)); where (x0, y0, z0) represents the tire center, R represents the tire radius, n represents the heading angle, and m represents the yaw angle.

[0100] In another possible implementation, the mathematical expressions for the first and second planes are M1x + N1y + P1z + Q1 = 0 and M2x + N2y + P2z + Q2 = 0, respectively; where M1, N1, P1, Q1, M2, N2, P2, and Q2 are known constants, and x, y, and z are unknown parameters used to represent any point on the first and second planes.

[0101] In another feasible approach, the mathematical expression for the heading angle is: n = or, Where H = M1M2 - N1N2, S = -(M1N2 + M2N1), T = -2P1P2 - M1M2 - N1N2.

[0102] In one possible implementation, this application also provides a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor and executing the methods in any of the foregoing embodiments. A processor 510 is configured to execute the plurality of instructions; a memory 520 is configured to store the plurality of instructions adapted for loading by the processor 510 and executing the heading detection method as described in the foregoing embodiments.

[0103] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0104] The above embodiments merely illustrate several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A heading detection method based on a single tire, characterized in that, include: The image is reconstructed to represent the outer contour of the tire, and the four vertices of the ellipse are reconstructed, wherein the four vertices correspond to the four endpoints of the two mutually perpendicular diameters of the tire. The first constraint of the four endpoints is represented by the tire center and heading angle, including: setting parameters for the tire center, tire radius, heading angle, and deflection angle, wherein the deflection angle is the angle between the diameter of the tire's contact point and the vertical axis of the camera coordinate system; using the parameters to represent the first constraint of the first endpoint, the first constraint of the second endpoint, the first constraint of the third endpoint, and the first constraint of the fourth endpoint; wherein the heading angle is the angle between the tangent of the tire's contact point and the horizontal axis of the camera coordinate system, and the first constraint is used to describe the restriction of the tire shape on the four endpoints; The second constraint, representing the four endpoints using the camera optical center and the four vertices, includes: representing a first plane based on the camera optical center and two major axis vertices of the four vertices, wherein the first and second endpoints corresponding to the major axis vertices are located on the first plane; representing a second plane based on the camera optical center and two minor axis vertices of the four vertices, wherein the third and fourth endpoints corresponding to the minor axis vertices are located on the second plane; wherein the second constraint describes the restriction of the mapping relationship on the four endpoints; Substitute the first constraint and the first second constraint into the first plane, and substitute the first third constraint and the first fourth constraint into the second plane to obtain four equations; solve the four equations simultaneously to obtain the heading, and determine the heading of the obstacle to which the tire belongs.

2. The method according to claim 1, characterized in that, The mathematical expressions for the first constraint, the first second constraint, the first third constraint, and the first fourth constraint are as follows: ； ； ； ； in, R represents the center of the tire, R represents the tire radius, n represents the heading angle, and m represents the yaw angle.

3. The method according to claim 2, characterized in that, The mathematical expressions for the first plane and the second plane are as follows: and ; Among them, the , , , , , , and The x, y, and z are known constants, and are unknown parameters used to represent any point on the first and second planes.

4. The method according to claim 3, characterized in that, The mathematical expression for the heading angle is: ,or, ; in, , , .

5. A heading detection device, characterized in that, include: An image unit is used to regress an ellipse representing the outer contour of a tire on an image, and the four vertices of the ellipse, wherein the four vertices correspond to the four endpoints of two mutually perpendicular diameters of the tire. A construction unit is used to represent the first constraint of the four endpoints using the tire center and heading angle, including: setting parameters for the tire center, tire radius, heading angle, and yaw angle, wherein the yaw angle is the angle between the diameter of the tire's contact point and the vertical axis of the camera coordinate system; using the parameters to represent the first constraint of the first endpoint, the first constraint of the second endpoint, the first constraint of the third endpoint, and the first constraint of the fourth endpoint; wherein the heading angle is the angle between the tangent of the tire's contact point and the horizontal axis of the camera coordinate system, and the first constraint is used to describe the restriction of the tire shape on the four endpoints; The construction unit is further configured to represent a second constraint on the four endpoints using the camera optical center and the four vertices, including: representing a first plane based on the camera optical center and two major axis vertices among the four vertices, wherein the first endpoint and the second endpoint corresponding to the major axis vertex among the four endpoints are located on the first plane; representing a second plane based on the camera optical center and two minor axis vertices among the four vertices, wherein the third endpoint and the fourth endpoint corresponding to the minor axis vertex among the four endpoints are located on the second plane; wherein the second constraint is used to describe the restriction of the mapping relationship on the four endpoints; The analysis unit is used to substitute the first constraint and the first second constraint into the first plane, and substitute the first third constraint and the first fourth constraint into the second plane to obtain four formulas; the four formulas are solved simultaneously to obtain the heading, so as to determine the heading of the obstacle to which the tire belongs.

6. A heading detection device, characterized in that, The device includes a processor and a memory connected via a bus; the processor is used to execute multiple instructions; the storage medium is used to store the multiple instructions, which are adapted to be loaded by the processor and executed as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor and executing the heading detection method as described in any one of claims 1-4.

Images