Focal length estimation method, focal length estimation device, and storage medium

Abstract

The present disclosure relates to a focal length estimation method, a focal length estimation device, and a storage medium for computer vision. According to an embodiment of the present disclosure, the focal length estimation method includes: determining an image marking area corresponding to a true marking area of a road in an image captured by an imaging device equipped on a vehicle on the road in a real space; determining a virtual trapezoid in the image based on the image marking area; and estimating the focal length of the imaging device based on the upper base size of the virtual trapezoid, the lower base size of the virtual trapezoid, the first road surface marking size, and the second road surface marking size. The beneficial effects of the method, device, and storage medium of the present disclosure at least include: estimating the focal length using the known standard sizes of road markings, reducing the number of parameters to be detected in focal length estimation, and reducing the complexity of focal length estimation.

Description

Technical Field

[0001] The present disclosure generally relates to computer vision, and more specifically, to a focal length estimation method, a focal length estimation device, and a storage medium. Background Art

[0002] In recent years, with the improvement of the information processing performance of computers, computer vision technology has received increasing attention. Computer vision includes machine vision that uses cameras and computers to replace the human eye to identify, track, and measure targets, etc., and also includes further performing graphics processing to obtain images that are more suitable for human eye observation or transmission to instrument detection. For example, computer vision can be used for autonomous driving or assisted driving of vehicles. In computer vision processing, a camera is an important component. The focal length of the camera is an important parameter. For example, detecting the true size of an entity in an image captured by a camera or the distance between entities may require knowing the focal length of the camera. It is desirable for the computer to autonomously determine the focal length of the camera instead of manually providing it.

[0003] Conventional focal length estimation methods may require parameters such as the height of the camera, the physical length perpendicular to the road direction, the physical length parallel to the road direction, the swing angle, the tilt angle, and / or the rotation angle in addition to the captured image. Summary of the Invention

[0004] A brief overview of the present disclosure will be given below to provide a basic understanding of certain aspects of the present disclosure. It should be understood that this overview is not an exhaustive overview of the present disclosure. It is not intended to identify the key or important parts of the present disclosure, nor is it intended to limit the scope of the present disclosure. Its purpose is only to present certain concepts in a simplified form as a prelude to the more detailed description to be discussed later.

[0005] According to one aspect of the present disclosure, there is provided a focal length estimation method for computer vision. The focal length estimation method includes: determining an image marker area corresponding to a true marker area of a road in an image captured by a camera device equipped on a vehicle on the road in the real space; determining a virtual trapezoid in the image based on the image marker area; and estimating the focal length of the camera device based on the upper base size of the virtual trapezoid, the lower base size of the virtual trapezoid, the first road surface marker size, and the second road surface marker size; wherein the true marker area includes at least part of a true predetermined road surface marker; the first road surface marker size is the length of a virtual graphic corresponding to the virtual trapezoid in the real space in a first direction; the second road surface marker size is the length of the virtual graphic in a second direction perpendicular to the first direction; and the first direction is the extension direction of the road at the position where the image is captured.

[0006] According to one aspect of the present disclosure, a focal length estimation device for computer vision is provided. The focal length estimation device includes: a memory storing instructions thereon; and one or more processors capable of communicating with the memory to execute the instructions fetched from the memory, and the instructions cause the one or more processors to: determine an image marker area corresponding to a real marker area of a road in a real space in an image captured by a camera device equipped on a vehicle; determine a virtual trapezoid in the image based on the image marker area; and estimate the focal length of the camera device based on an upper base size of the virtual trapezoid, a lower base size of the virtual trapezoid, a first road surface marker size, and a second road surface marker size; wherein the real marker area includes at least part of real predetermined road surface markers; the first road surface marker size is a length of a virtual figure corresponding to the virtual trapezoid in a real space in a first direction; the second road surface marker size is a length of the virtual figure in a second direction perpendicular to the first direction; and the first direction is an extending direction of the road at a position where the image is captured.

[0007] According to another aspect of the present disclosure, a computer-readable storage medium storing a program is provided. The program causes a computer running the program to: determine an image marker area corresponding to a real marker area of a road in a real space in an image captured by a camera device equipped on a vehicle; determine a virtual trapezoid in the image based on the image marker area; and estimate the focal length of the camera device based on an upper base size of the virtual trapezoid, a lower base size of the virtual trapezoid, a first road surface marker size, and a second road surface marker size; wherein the real marker area includes at least part of real predetermined road surface markers; the first road surface marker size is a length of a virtual figure corresponding to the virtual trapezoid in a real space in a first direction; the second road surface marker size is a length of the virtual figure in a second direction perpendicular to the first direction; and the first direction is an extending direction of the road at a position where the image is captured.

[0008] The beneficial effects of the focal length estimation method, the focal length estimation device, and the storage medium of the present disclosure at least include: reducing the number of parameters to be detected in focal length estimation and reducing the complexity of focal length estimation. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Embodiments of the present disclosure will be described below with reference to the accompanying drawings, which will help to more easily understand the above and other objects, features, and advantages of the present disclosure. The drawings are only for showing the principles of the present disclosure. The dimensions and relative positions of the units do not have to be drawn to scale in the drawings. The same reference numerals may represent the same features. In the drawings:

[0010] Figure 1 An exemplary flowchart of a focal length estimation method according to an embodiment of the present disclosure is shown;

[0011] Figure 2Shows a schematic diagram of an image marking area according to an embodiment of the present disclosure;

[0012] Figure 3 Shows a schematic diagram of an image marking area according to an embodiment of the present disclosure;

[0013] Figure 4 Shows a schematic diagram of the boundary of a diamond mark in an image marking area according to an embodiment of the present disclosure;

[0014] Figure 5 Shows an exemplary method for identifying the outer boundary line of a diamond mark according to an embodiment of the present disclosure;

[0015] Figure 6 Shows a focal length estimation device according to an embodiment of the present disclosure;

[0016] Figure 7 Shows a focal length estimation device according to an embodiment of the present disclosure; and

[0017] Figure 8 Shows an exemplary block diagram of an information processing device according to an embodiment of the present disclosure. Detailed Embodiments

[0018] Hereinafter, exemplary embodiments of the present disclosure will be described with reference to the accompanying drawings. For clarity and conciseness, not all features of the actual embodiments are described in the specification. However, it should be understood that many embodiment-specific decisions may be made during the development of any such actual embodiment to achieve the specific goals of the developer, and these decisions may vary with different embodiments.

[0019] Here, it should also be noted that in order to avoid obscuring the present disclosure with unnecessary details, only the device structures closely related to the solution according to the present disclosure are shown in the drawings, while other details less related to the present disclosure are omitted.

[0020] It should be understood that the present disclosure is not limited to the described embodiments only due to the following description with reference to the drawings. Herein, where feasible, embodiments can be combined with each other, features can be replaced or borrowed between different embodiments, and one or more features can be omitted in one embodiment.

[0021] Computer program code for operating aspects of embodiments of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, Smalltalk, C++, etc., and also including conventional procedural programming languages such as the "C" programming language or similar programming languages.

[0022] The method of the present disclosure may be implemented by a circuit having a corresponding functional configuration. The circuit includes a circuit for a processor.

[0023] One aspect of the present disclosure provides a focal length estimation method for computer vision. More specifically, the focal length estimation method may be used for autonomous driving of vehicles, assisted driving of vehicles, size estimation of entities in the real space, and traffic condition monitoring. For example, using a known focal length, the distance between the host vehicle and the vehicle ahead can be estimated from the coordinates of the target in the traffic scene in the image.

[0024] The "focal length" here refers to the lens of the imaging device for taking images having a focal length f.

[0025] Generally, focal length estimation may be based on the pinhole camera model. Based on the pinhole camera model, various focal length estimation methods can be conceived according to the selected images and physical measurements.

[0026] The inventors noticed that there are road markings such as diamond-shaped crosswalk warning markings and dotted lane demarcation segments on the road (i.e., on the road surface) where the vehicle travels, and the sizes of the road markings conform to relevant standards, that is, the sizes of the road markings are known and have a standard unified size. Based on this, through analysis and research, the inventors conceived the technical solution of the present disclosure: using road markings to estimate the focal length. Specifically, based on the pinhole camera model, the principle of similar triangles is used to estimate the focal length of the imaging device. The focal length estimation scheme of the present disclosure can, for example: estimate the focal length using only two physical lengths without other parameters of the imaging device: one is the length perpendicular to the road direction, and the other is the length parallel to the road direction. These two physical lengths are related to the road markings, so they can both be obtained from the standards related to the sizes of the road markings or simply calculated. On the one hand, this can reduce the number of parameters to be detected in focal length estimation, reduce the complexity of focal length estimation, and improve the estimation speed.

[0027] The following refers to Figure 1 An exemplary illustration of the focal length estimation method for computer vision of the present disclosure is given.

[0028] Figure 1 An exemplary flowchart of a focal length estimation method 100 according to an embodiment of the present disclosure is shown.

[0029] In step S101, an image marking area corresponding to a real marking area on a road in a real space in an image Im captured by an imaging device equipped on a vehicle is determined. The imaging device is, for example, a camera. The real marking area of the road is an area including at least part of road markings. For example, an area including part of road markings, one road marking, multiple road markings, or a non-integer number of road markings. The road markings are, for example: diamond-shaped crosswalk warning markings (also simply referred to as "diamond markings"). The image marking area is, for example, a rectangular area in the image Im that includes road markings. For example, the image marking area is determined by extracting, from the image Im, an image marking area corresponding to the real marking area of the road through a neural network model (also referred to as a "traffic target extraction model") capable of recognizing predetermined road markings in the image. The neural network model may include a feature extraction part and a classification part. The data for training the neural network model is, for example, traffic image samples that mark the borders and categories of each target in a traffic scene. The image marking area is, for example, an area defined by the circumscribed box of a single diamond marking in the image Im. For cases where the image captured by the imaging device does not include road markings or the included road markings cannot be used for focal length estimation, such images can be discarded.

[0030] Reference Figure 2 is made to illustrate the image marking area. Figure 2 FIG. shows a schematic diagram of an image marking area 200 according to an embodiment of the present disclosure. For convenience of description, a plurality of symbols are added in the figure to identify each point: vanishing point O, a quadrilateral T with vertices T 0 , T 1 , T 2 , T 7 . The quadrilateral T 0 T 1 T 2 T 7 . The line segment T 3 T 4 is a line segment passing through point T 7 and parallel to the line segment T 1 T 2 . The line segment T 5 T 6 is a line segment passing through point T 0 and parallel to the line segment T 1 T 2 . The length of the line segment T 3 T 4 is w 34 ; the length of the line segment T 1 T 2 is w 12 ; the length of the line segment T 5 T 6 is w 56 . It should be noted that:Figure 2 The △D shown is not the line segment T 3 T 4 and the line segment T 1 T 2 but rather corresponds to the size of the first road marking in the real space. In the example of Figure 2 , △D is half of the longitudinal length of the diamond-shaped crosswalk warning marking in the real space. Here, the longitudinal direction, also referred to as the first direction d1, is the extension direction of the road at the position where the captured image Im is taken. The direction perpendicular to the first direction d1 is called the second direction d2. The image marking area 200 can be the image marking area in the image Im that contains road markings, that is, the image marking area 200 can be the image marking area containing road markings extracted from the image Im, where the quadrilateral T 0 T 1 T 2 T 7 T

[0031] In addition, reference is also made to Figure 3 to illustrate the image marking area. Figure 3 A schematic diagram of the image marking area 300 according to an embodiment of the present disclosure is shown, where, for convenience of description, a plurality of symbols are added in the figure to identify each point: the vanishing point O, the points T 7 , T 8 , T 5 , T 1 , T 2 , T 6 , T 3 and T 4 , the line segment T 7 T 8 (not shown), the line segment T 3 T 4 , the line segment T 1 T 2 and the line segment T 5 T 6 are parallel to each other. The length of the line segment T 3 T 4 is w 34 ; the length of the line segment T 1 T 2 is w 12 ; the length of the line segment T 5 T 6has a length of w 56 . Similarly, it should be noted that: Figure 3 △D shown in 3 T 4 is not the distance between the line segment T 1 T 2 and the line segment T, but the first road marking size in the real space. In the example of Figure 3 , △D is the distance between two adjacent dotted lane demarcation line segments in the longitudinal direction in the real space. Here, the longitudinal direction, also known as the first direction d1, is the extension direction of the road at the position of the captured image Im. The direction perpendicular to the first direction d1 is called the second direction d2. The image marking area 300 can be the image marking area in the image Im that contains road markings, that is, the image marking area 300 can be the image marking area containing road markings extracted from the image Im, where the line segment T 5 T 1 、T 6 T 2 、、T 3 T 7 、T 4 T 8 corresponds to a dotted lane demarcation line segment on the road in the real space. The vanishing point refers to the point where parallel lines in the actual space appear to intersect in the two-dimensional image. The vanishing point O is the point where two parallel lines (for example, two lines corresponding to the two edges of the road, a pair of long solid lines on the road, or a pair of dotted lane demarcation lines on the road) at the position of the captured image Im in the actual space appear to intersect in the two-dimensional image. The focal length can be estimated based on the image marking area.

[0032] The road markings in the present disclosure are not limited to dotted lane demarcation lines and diamond-shaped crosswalk warning markings.

[0033] In step S103, a virtual trapezoid in the image Im is determined based on the image marking area. The virtual trapezoid has an upper base, a lower base, and two waists. Here, "virtual" means that there is no trapezoid formed by road marking lines in the image marking area extracted from the image Im, but a trapezoid outlined artificially for estimating the focal length. For example, in the image marking area 200 shown in Figure 2 , the virtual trapezoid can be: trapezoid T 5 T 6 T 2 T 1 、trapezoid T 5 T 6 T 3 T 4 or trapezoid T 1 T 2 T 4 T 3 ; in Figure 3In the image marking area 300 shown, the virtual trapezoid can be: trapezoid T 5 T 6 T 2 T 1 、trapezoid T 5 T 6 T 3 T 4 、trapezoid T 3 T 4 T 8 T 7 or trapezoid T 1 T 2 T 4 T 3 。The virtual trapezoid can be determined by detecting the vertices of the road markings in the image marking area. Vertex detection can be achieved by various methods. For example, first, use edge detection algorithms such as Sobel edge detection, Canny edge detection, or Prewitt edge detection algorithms to detect the edges of the road markings; then use line detection algorithms such as Hough transform, EDLine line detection, or Line Segment Detector (LSD) algorithms to detect the lines or line segments on the edges. Finally, find the intersections or vertices of these lines and line segments.

[0034] In step S105, estimate the focal length f of the imaging device based on the upper base size lu of the virtual trapezoid, the lower base size l l 、the first road marking size L, and the second road marking size W. The first road marking size L is the length of the virtual figure corresponding to the virtual trapezoid in the real space in the first direction d1; the second road marking size W is the length of the virtual figure in the second direction d2 perpendicular to the first direction d1; the first direction d1 is the extension direction of the road at the position of the captured image Im.

[0035] For example, when the virtual trapezoid is Figure 2 the trapezoid T in 1 T 2 T 4 T 3 , L = D 12 -D 34 = △D, D 12 is, when capturing the image Im, the distance between the lens of the imaging device and the diamond-shaped crosswalk warning marking in the first direction d1 (i.e., the diamond T' in the real space corresponding to the quadrilateral T 0 T 1 T 7 T 2 in the image 0 T’ 1 T’ 7T’ 2 , where the points with the same subscript correspond to each other) of the line segment T’ 1 T’ 2 The distance between, D 34 is, when capturing the image Im, the distance between the lens of the imaging device and the diamond-shaped crosswalk warning marking (diamond T’ 0 T’ 1 T’ 7 T’ 2 ) of the line segment T’ 3 T’ 4 The distance between. Let W represent the width of the diamond T’ 0 T’ 1 T’ 7 T’ 2 in the second direction d2, that is, the length of the line segment T’ 1 T’ 2 is W. Based on the pinhole camera model, equations (1) and (2) can be obtained.

[0036]

[0037]

[0038] From equations (1) and (2), equation (3) can be obtained.

[0039]

[0040] From equation (3), equation (4) can be obtained.

[0041]

[0042] From L = D 12 -D 34 = △D, lu = w 34 , l l = w 12 Equation (5) can be obtained.

[0043]

[0044] According to the standards related to road markings, the first road marking size L and the second road marking size W can be determined. That is, L and W are known. The upper base size w 34 of the virtual trapezoid, the lower base size w 12 of the virtual trapezoid can be determined based on the coordinates of the corresponding vertices of the virtual trapezoid in the image Im. That is, the focal length of the imaging device can be estimated based on the upper base size of the virtual trapezoid, the lower base size of the virtual trapezoid, the first road marking size, and the second road marking size. From one-point perspective, the trapezoid T 1 T 2 T4 T 3 is a one - point perspective projection of a rectangle in the real space, that is, the virtual figure corresponding to the virtual trapezoid in the real space can be a rectangle. If the road direction is regarded as the length direction of this rectangle, the first road marking size is the length of this rectangle, and the second road marking size is the width of this rectangle. That is, the focal length of the imaging device can be estimated based on the upper - base size (lu) of the virtual trapezoid, the lower - base size (l l ), the length (L) and width (W) of the corresponding rectangle of the virtual trapezoid in the real space.

[0045] When the virtual trapezoid is selected as Figure 2 the trapezoid T in 5 T 6 T 2 T 1 or T 5 T 6 T 3 T 4 at this time, based on a similar derivation, the equation (5) can also be obtained.

[0046] When the virtual trapezoid is selected as Figure 3 the trapezoid T in 5 T 6 T 2 T 1 、T 5 T 6 T 3 T 4 、T 3 T 4 T 8 T 7 、T 1 T 2 T 4 T 3 or T 5 T 6 T 8 T 7 at this time, based on a similar derivation, the equation (5) can also be obtained.

[0047] Referring to Figure 2 it can be known that when the road marking used to estimate the focal length is selected as the diamond - shaped crosswalk warning marking, the real marking area includes at least part of the diamond - shaped crosswalk warning marking. For example, the real marking area does not include the part of the diamond - shaped crosswalk warning marking corresponding to the point T 7 the corresponding point T 7 ’. In one example, the real marking area includes a diamond - shaped crosswalk warning marking.

[0048] Referring to Figure 2It can be known that when the road marking used for estimating the focal length is selected as the diamond-shaped crosswalk warning marking, the virtual trapezoid and the outer contour of the pattern of the selected diamond-shaped crosswalk warning marking may have two, three or four common points.

[0049] In one embodiment, method 100 further includes determining a vanishing point for the road based on the image marking area. The vanishing point may be the vanishing point corresponding to the road at the captured image Im. The vanishing point can be used to determine the vertices of the virtual trapezoid. The determination of the vanishing point may include two parts: one is the detection of the road boundary, which can be implemented by methods such as stereo vision, color detection or Hough transform; the other is to find the vanishing point, and an exemplary implementation is to find the vanishing point by voting using texture features. Determining the vanishing point in the image is a conventional technique and will not be elaborated here.

[0050] The following describes how to optimize the selection of road markings. As Figure 2 、 3 shown, the image Im may include multiple optional virtual trapezoids. When it is determined that the image marking area contains multiple candidate road marking patterns corresponding to multiple real road markings, the candidate road marking pattern corresponding to the real road marking closest to the imaging device among the multiple real road markings is selected to determine the virtual trapezoid. That is, it is preferred to use the road marking closest to the imaging device to estimate the focal length. This can be confirmed by referring to the following derivation.

[0051] Referring to Equation (4), by taking the partial derivative, Equation (6) can be obtained.

[0052]

[0053] Let Equation (6) can be transformed into Equation (7).

[0054]

[0055] In the above formula, the larger the value of, the smaller the absolute values of both terms on the right side of the equal sign. Therefore, in order to reduce the error, the road marking with a larger value should be used to estimate the focal length, and it is preferred to use the road marking closest to the imaging device to estimate the focal length.

[0056] For example, in Figure 2 , it is preferred to use trapezoid T 5 T 6 T 2 T 1 to estimate the focal length. In Figure 3 , it is preferred to use trapezoid T 5 T 6 T 2 T 1 to estimate the focal length.

[0057] From an application perspective, a camera device (e.g., a dash cam) usually pays more attention to closer targets, so the targets in the foreground are generally captured more clearly. Therefore, selecting a road marking pattern that is closer can, to a greater extent, ensure that the vertices of the road marking pattern are accurately extracted, thereby improving the accuracy of focal length estimation.

[0058] The determination of the virtual trapezoid is further described below. In the national standard, the dimensions of the area enclosed by the outer boundary of the road marking are usually specified. Therefore, when estimating the focal length using the road surface marking, it is preferable to find the outer boundary (i.e., the outer edge) of the pattern from the detected straight strip pattern. After determining the outer boundary, the vertices of the line segments can be determined, and thus the virtual trapezoid can be determined based on the vertices. In one embodiment of the present disclosure, estimating the focal length of the camera device includes: identifying the outer edge of the diamond-shaped crosswalk warning marking. The method for detecting the outer boundary is exemplarily described below using the diamond-shaped marking as an example.

[0059] Figure 4 FIG. shows a schematic diagram of the boundary of the diamond-shaped marking in the image marking area according to an embodiment of the present disclosure, where 8 straight lines l determined by straight line detection are shown. 1 to l 8 , and the parametric equation of the straight line m of the plane rectangular coordinate system xoy and the straight line m is also shown by a dashed line in the upper left corner, where θ is the angle measured counterclockwise from the perpendicular line from the origin o to the straight line m to the x-axis, and ρ is the distance from the origin o to the straight line m. Note: Figure 4 In 1 to l 8 , each of the shown straight lines is a straight line extended with respect to the boundary of the diamond-shaped marking (i.e., the line segment corresponding to the edge of the diamond-shaped marking). As shown in Figure 4 , since each side of the diamond-shaped marking is not a straight line but a strip with a certain width, edge detection of the diamond-shaped marking in the image marking area can determine 8 straight lines (i.e., the parameters θ and ρ of each straight line) l 1 , l 2 , l 3 , l 4 , l 5 , l 6 , l 7 and l 8 , where the 4 straight lines l 1 , l 2 , l 3 , l 4 constituting the outer boundary are required for determining the virtual trapezoid. For the straight line m in the rectangular coordinate system, its equation can be written as an equation (8) including the parameters θ and ρ.

[0060]

[0061] In one example, four lines l 1 , l 2 , l 3 , l 4 that form the outer boundary among the eight lines can be identified according to eight parameter pairs (ρ, θ) of the eight lines. The parameters here can be given by a line detection algorithm. Figure 5 FIG. 500 shows an exemplary method for identifying the outer boundary lines of a diamond marker according to an embodiment of the present disclosure. In step 501, eight parameter pairs (ρ, θ) corresponding to eight lines are received (input). In step 502, based on whether θ in each parameter pair is greater than 90°, the eight lines are divided into two groups: a line group G2 where θ is greater than 90°, and a line group G1 composed of the remaining lines. In step 503, for the line group G2, based on whether each sin θ is greater than zero, the line group G2 is divided into a line group G22 where sin θ is greater than zero and a line group G21 composed of the remaining lines. In step 504, the line with the largest ρ in the line group G22 is identified as line l 1 . In step 505, the line with the largest ρ in the line group G21 is identified as line l 2 . In step 506, the line with the smallest ρ in the line group G1 is identified as line l 3 . In step 507, the line with the largest ρ in the line group G1 is identified as line l 4 .

[0062] Those skilled in the art can understand that the method for detecting the outer boundary of the present disclosure is not limited to the above exemplary manner.

[0063] The present disclosure also provides a focal length estimation device for computer vision. An exemplary description will be made below with reference to Figure 6 . Figure 6FIG. 600 shows a focal length estimation device 600 according to an embodiment of the present disclosure. The device 600 includes: a first determination unit 601, a second determination unit 603, and an estimation unit 605. The first determination unit 601 is configured to: determine an image marker area corresponding to a real marker area of a road in a real space in an image captured by a camera device equipped on a vehicle on the road. The second determination unit 603 is configured to: determine a virtual trapezoid in the image based on the image marker area. The estimation unit 605 is configured to: estimate the focal length of the camera device based on the upper base size of the virtual trapezoid, the lower base size of the virtual trapezoid, the first road surface marker size, and the second road surface marker size; wherein, the real marker area includes at least part of real predetermined road surface markers; the first road surface marker size is the length of a virtual graphic corresponding to the virtual trapezoid in a real space in a first direction; the second road surface marker size is the length of the virtual graphic in a second direction perpendicular to the first direction; and the first direction is the extending direction of the road at the position where the image is captured. The device 600 has a corresponding relationship with the method 100. For further configuration of the device 600, reference may be made to the description of the focal length estimation method in the present disclosure.

[0064] The present disclosure also provides a focal length estimation device. The following is an exemplary description with reference to Figure 7 FIGs. Figure 7 FIG. 700 shows a focal length estimation device 700 according to an embodiment of the present disclosure, which can be used to estimate the focal length of a camera device for computer vision. The focal length estimation device 700 includes: a memory 701 having instructions stored thereon; and one or more processors 703 that can communicate with the memory to execute instructions fetched from the memory, and the instructions cause the one or more processors to: determine an image marker area corresponding to a real marker area of a road in a real space in an image captured by a camera device equipped on a vehicle on the road; determine a virtual trapezoid in the image based on the image marker area; and estimate the focal length of the camera device based on the upper base size of the virtual trapezoid, the lower base size of the virtual trapezoid, the first road surface marker size, and the second road surface marker size; wherein, the real marker area includes at least part of real predetermined road surface markers; the first road surface marker size is the length of a virtual graphic corresponding to the virtual trapezoid in a real space in a first direction; the second road surface marker size is the length of the virtual graphic in a second direction perpendicular to the first direction; and the first direction is the extending direction of the road at the position where the image is captured. The device corresponds to the focal length estimation method of the present disclosure. For further configuration of the device, reference may be made to the description of the focal length estimation method in the present disclosure.

[0065] One aspect of the present disclosure provides a computer-readable storage medium storing a program. The program causes a computer running the program to: determine an image marking area corresponding to a real marking area on a road in a real space in an image captured by a camera device equipped on a vehicle; determine a virtual trapezoid in the image based on the image marking area; and estimate a focal length of the camera device based on an upper base size of the virtual trapezoid, a lower base size of the virtual trapezoid, a first road marking size, and a second road marking size; wherein the real marking area includes at least part of a real predetermined road marking; the first road marking size is a length of a virtual figure corresponding to the virtual trapezoid in a real space in a first direction; the second road marking size is a length of the virtual figure in a second direction perpendicular to the first direction; and the first direction is an extending direction of the road at a position where the image is captured. More configuration details of the program can be referred to the description of the focal length estimation method in the present disclosure.

[0066] According to one aspect of the present disclosure, an information processing device is further provided.

[0067] Figure 8 is an exemplary block diagram of an information processing device 800 according to an embodiment of the present disclosure. In Figure 8 it, a central processing unit (CPU) 801 performs various processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage section 808 into a random access memory (RAM) 803. In the RAM 803, data and the like required when the CPU 801 executes various processes are also stored as needed.

[0068] The CPU 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output interface 805 is also connected to the bus 804.

[0069] The following components are connected to the input / output interface 805: an input section 806 including a soft keyboard and the like; an output section 807 including a display such as a liquid crystal display (LCD) and a speaker; a storage section 808 such as a hard disk; and a communication section 809 including a network interface card such as a LAN card, a modem, and the like. The communication section 809 performs communication processing via a network such as the Internet, a local area network, a mobile network, or a combination thereof.

[0070] A driver 810 is also connected to the input / output interface 805 as needed. A removable medium 811 such as a semiconductor memory is installed on the driver 810 as needed, so that a program read from it is installed into the storage section 808 as needed.

[0071] The CPU 801 can run a program that can implement the functions of the focal length estimation method of the present disclosure.

[0072] The solution of the present disclosure includes estimating the focal length of a camera device by using the standard size of road markings on a road surface. This is beneficial in that: reducing the number of parameters to be detected in focal length estimation, reducing the complexity of focal length estimation, and improving the estimation speed.

[0073] As described above, according to the present disclosure, a principle for estimating the focal length of a camera device is provided. It should be noted that the effects of the solution of the present disclosure are not necessarily limited to the above effects, and any of the effects shown in this specification or other effects understandable from this specification can be achieved in addition to or instead of the effects described in the previous paragraphs.

[0074] Although the present invention has been disclosed above by the description of specific embodiments of the present invention, it should be understood that those skilled in the art can design various modifications (including, in the case of rows, combinations or substitutions of features between embodiments), improvements or equivalents to the present invention within the spirit and scope of the appended claims. These modifications, improvements or equivalents should also be considered to be included within the protection scope of the present invention.

[0075] It should be emphasized that the term "comprising / including" when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0076] In addition, the methods of the embodiments of the present invention are not limited to being executed in the time sequence described in the specification or shown in the drawings, and can also be executed in other time sequences, in parallel or independently. Therefore, the execution sequence of the methods described in this specification does not limit the technical scope of the present invention.

[0077] Supplementary Note

[0078] The present disclosure includes but is not limited to the following solutions.

[0079] 1. A focal length estimation method for computer vision, characterized by comprising:

[0080] Determining an image marking area corresponding to a real marking area of the road in an image captured by a camera device equipped on a vehicle on the road in real space;

[0081] Based on the image marking area, determining a virtual trapezoid in the image; and

[0082] Based on the upper base size of the virtual trapezoid, the lower base size of the virtual trapezoid, a first road marking size, and a second road marking size, estimating the focal length of the camera device;

[0083] Wherein, the real marking area includes at least part of real predetermined road markings;

[0084] The first road marking dimension is the length of the virtual figure corresponding to the virtual trapezoid in the real space in the first direction;

[0085] The second road marking dimension is the length of the virtual figure in the second direction perpendicular to the first direction; and

[0086] The first direction is the extending direction of the road at the position where the image is captured.

[0087] 2. The focal length estimation method according to Note 1, wherein the real marking area includes at least part of a diamond-shaped crosswalk warning marking.

[0088] 3. The focal length estimation method according to Note 2, wherein estimating the focal length of the imaging device includes: identifying the outer edge of the diamond-shaped crosswalk warning marking.

[0089] 4. The focal length estimation method according to Note 2, wherein in the image marking area, the virtual trapezoid and the outer contour of the pattern of the selected diamond-shaped crosswalk warning marking have two, three or four common points.

[0090] 5. The focal length estimation method according to Note 1, wherein the real marking area includes one diamond-shaped crosswalk warning marking.

[0091] 6. The focal length estimation method according to Note 1, wherein the real marking area includes multiple diamond-shaped crosswalk warning markings.

[0092] 7. The focal length estimation method according to Note 6, wherein the virtual trapezoid in the image is determined based on the marking pattern corresponding to the diamond-shaped crosswalk warning marking closest to the imaging device among the multiple diamond-shaped crosswalk warning markings in the real marking area.

[0093] 8. The focal length estimation method according to Note 1, wherein the real marking area includes at least a pair of dashed lane demarcation line segments.

[0094] 9. The focal length estimation method according to Note 1, wherein the real marking area includes two pairs of partial dashed lane demarcation line segments.

[0095] 10. The focal length estimation method according to Note 1, wherein determining the virtual trapezoid in the image based on the image marking area includes: when it is determined that the image marking area contains multiple candidate pavement marking patterns corresponding to multiple real road markings, selecting the candidate pavement marking pattern corresponding to the real road marking closest to the imaging device among the multiple real road markings to determine the virtual trapezoid.

[0096] 11. The focal length estimation method according to Note 1, wherein the virtual graphic includes a rectangle.

[0097] 12. The focal length estimation method according to Note 1, further comprising: determining a vanishing point for the road based on the image marking area.

[0098] 13. A focal length estimation device for computer vision, comprising:

[0099] A memory storing instructions thereon; and

[0100] One or more processors capable of communicating with the memory to execute the instructions fetched from the memory, and the instructions cause the one or more processors to:

[0101] Determine an image marking area in an image captured by a camera device equipped on a vehicle on a road in the real space, corresponding to a real marking area of the road;

[0102] Determine a virtual trapezoid in the image based on the image marking area; and

[0103] Estimate the focal length of the camera device based on an upper base size of the virtual trapezoid, a lower base size of the virtual trapezoid, a first road marking size, and a second road marking size;

[0104] Wherein the real marking area includes at least part of a real predetermined road marking;

[0105] The first road marking size is a length of a virtual graphic corresponding to the virtual trapezoid in the real space in a first direction;

[0106] The second road marking size is a length of the virtual graphic in a second direction perpendicular to the first direction; and

[0107] The first direction is an extending direction of the road at a position where the image is captured.

[0108] 14. A computer-readable storage medium storing a program thereon, wherein the program causes a computer running the program to:

[0109] Determine an image marking area in an image captured by a camera device equipped on a vehicle on a road in the real space, corresponding to a real marking area of the road;

[0110] Determine a virtual trapezoid in the image based on the image marking area; and

[0111] Estimate the focal length of the imaging device based on the upper base dimension of the virtual trapezoid, the lower base dimension of the virtual trapezoid, the first road marking dimension, and the second road marking dimension;

[0112] wherein the real marking area includes at least part of the real predetermined road markings;

[0113] The first road marking dimension is the length of the virtual figure corresponding to the virtual trapezoid in the real space in a first direction;

[0114] The second road marking dimension is the length of the virtual figure in a second direction perpendicular to the first direction; and

[0115] The first direction is the extending direction of the road at the position where the image is captured.

Claims

1. A focal length estimation method for computer vision, characterized in that, comprising: determining an image marking area corresponding to a real marking area of the road in an image captured by a camera device equipped on a vehicle on the real space; determining a virtual trapezoid in the image based on the image marking area; and Estimate the focal length f of the imaging device based on the upper base dimension lu of the virtual trapezoid and the lower base dimension l of the virtual trapezoid according to the following equation, the first road marking dimension L, and the second road marking dimension W; l ​ wherein, the real marking area includes at least part of real predetermined road markings; the first road marking size is the length of a virtual figure corresponding to the virtual trapezoid in the real space in a first direction; the second road marking size is the length of the virtual figure in a second direction perpendicular to the first direction; and the first direction is the extending direction of the road at the position where the image is captured.

2. The focal length estimation method according to claim 1, wherein, the real marking area includes at least part of a diamond-shaped crosswalk warning marking.

3. The focal length estimation method according to claim 2, wherein, in the image marking area, the virtual trapezoid and the outer contour of the pattern of a selected diamond-shaped crosswalk warning marking have two, three or four common points.

4. The focal length estimation method according to claim 2, wherein, the real marking area includes a plurality of diamond-shaped crosswalk warning markings.

5. The focal length estimation method according to claim 4, wherein, determining the virtual trapezoid in the image based on a marking pattern corresponding to the diamond-shaped crosswalk warning marking closest to the camera device among the plurality of diamond-shaped crosswalk warning markings in the real marking area.

6. The focal length estimation method according to claim 1, wherein, the real marking area includes at least a pair of dashed lane demarcation line segments.

7. The focal length estimation method according to claim 1, wherein, determining the virtual trapezoid in the image based on the image marking area includes: when determining that the image marking area contains a plurality of candidate road marking patterns corresponding to a plurality of real road markings, selecting a candidate road marking pattern corresponding to the real road marking closest to the camera device among the plurality of real road markings to determine the virtual trapezoid.

8. The focal length estimation method according to claim 1, wherein, the virtual figure includes a rectangle.

9. A focal length estimation device for computer vision, characterized in that, comprising: a memory storing instructions thereon; and one or more processors capable of communicating with the memory to execute the instructions obtained from the memory, and the instructions cause the one or more processors to: determine an image marking area corresponding to a real marking area of the road in an image captured by a camera device equipped on a vehicle on the real space; determine a virtual trapezoid in the image based on the image marking area; and Estimate the focal length f of the imaging device based on the upper base dimension lu of the virtual trapezoid, the lower base dimension l of the virtual trapezoid, the first road marking dimension L, and the second road marking dimension W according to the following equation; l ​ wherein, the real marking area includes at least part of real predetermined road markings; the first road marking size is the length of a virtual figure corresponding to the virtual trapezoid in the real space in a first direction; the second road marking size is the length of the virtual figure in a second direction perpendicular to the first direction; and The first direction is the extension direction of the road at the position where the image is captured.

10. A computer-readable storage medium storing a program, characterized in that the program causes a computer running the program to: determine an image marking area corresponding to a real marking area of a road in an image captured by an imaging device equipped on a vehicle on the road in the real space; determine a virtual trapezoid in the image based on the image marking area; and estimate a focal length f of the imaging device based on an upper base size lu of the virtual trapezoid, a lower base size lu of the virtual trapezoid, a first road marking size L, and a second road marking size W according to the following equation; wherein the real marking area includes at least part of a real predetermined road marking; the first road marking size is the length of a virtual graphic corresponding to the virtual trapezoid in the real space in the first direction; the second road marking size is the length of the virtual graphic in a second direction perpendicular to the first direction; and the first direction is the extension direction of the road at the position where the image is captured.

Images