A method, device and storage medium for visual ranging of a vehicle

Abstract

The present invention discloses a vehicle visual distance measurement method, device and storage medium, which relates to the technical field of vehicle distance measurement, and solves the problem of low accuracy when the visual distance measurement method is used to measure the distance between an obstacle and a vehicle. The method comprises: acquiring a captured image through a visual sensor, the captured image includes at least one object; determining a bounding box and a vanishing line of a target object in the captured image, the target object being any one of the at least one object; determining two corner points of the target object according to a lower frame in the bounding box used to indicate the ground; if each corner point meets a preset condition, determining two corresponding target points according to the two corner points and the target vanishing point on the vanishing line, the first distance between the target point and the visual sensor being less than the second distance between the corner point and the visual sensor; determining the target distance between the vehicle and the target object according to the two target points, the two corner points and the first preset value.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle distance measurement, and in particular to a vehicle visual distance measurement method, device and storage medium. Background Art

[0002] Distance measurement is a very important part of assisted driving technology. It can measure the distance between the vehicle and the surrounding obstacles to assist the driver in making accurate predictions. At present, commonly used distance measurement methods can include laser ranging, ultrasonic ranging, microwave ranging, infrared ranging or visual ranging. Among them, visual ranging has strong adaptability in terms of accuracy, speed, intelligence, etc., and has the advantages of high precision, good stability, and non-contact measurement. Therefore, it is widely used in assisted driving technology.

[0003] When the distance between the obstacle and the vehicle is far, the accuracy of the distance between the obstacle and the vehicle measured by the existing visual ranging method is low, which may cause the driver to misjudge and cause a traffic accident. Summary of the invention

[0004] The present invention provides a vehicle visual distance measurement method, device and storage medium, which solve the problem of low accuracy when the visual distance measurement method is used to measure the distance between an obstacle and a vehicle.

[0005] In order to achieve the above object, the present invention adopts the following technical scheme:

[0006] In a first aspect, the present invention provides a method for visual ranging of a vehicle, the method comprising:

[0007] Acquire a captured image by a visual sensor, where the captured image includes at least one object;

[0008] Determine a bounding box and a vanishing line of a target object in the captured image, wherein the target object is any one of the at least one object;

[0009] Determine two corner points of the target object according to the lower border of the bounding box, where the lower border is used to indicate the ground;

[0010] If each corner point meets the preset conditions, two corresponding target points are determined according to the two corner points and the target vanishing point on the vanishing line;

[0011] The first distance between the target point and the visual sensor is smaller than the second distance between the corner point and the visual sensor, one target point is located on a line connecting a target vanishing point and a corner point, and two target points are located in an area formed by a lower border and a lower edge of the captured image, and the lower edge is used to indicate the ground;

[0012] A target distance between the vehicle and the target object is determined according to the two target points, the two corner points and the first preset value.

[0013] When the visual ranging method of the vehicle provided by the present invention is used to measure the distance between the vehicle and the obstacle, a captured image is obtained by a visual sensor on the vehicle, and the captured image includes at least one object. The bounding box and vanishing line of any object in the at least one object are determined, and when the two corner points of the lower frame indicating the ground in the determined bounding box are far from the visual sensor, two target points that are close to the visual sensor and located in the area composed of the lower frame and the lower edge of the captured image can be obtained according to the two corner points and the target vanishing point on the vanishing line. Finally, the target distance between the vehicle and any object in the captured image can be determined according to the two target points, the two corner points and the first preset value. As can be seen from the above, in the present invention, two target points that are close to the visual sensor are determined by the vanishing line and the two corner points that are far from the visual sensor, and when the distance between the object and the visual sensor is close, the target distance between the object and the visual sensor can be calculated very accurately. That is to say, the present invention projects an obstacle located at a farther position to a position closer to the visual sensor, thereby accurately calculating the target distance between the visual sensor and the obstacle located at a farther position, thereby solving the problem of low accuracy when using the visual ranging method to measure the distance between the obstacle and the vehicle.

[0014] In a possible implementation, determining two corresponding target points according to the two corner points and the target vanishing point on the vanishing line includes:

[0015] Connect the target vanishing point with the two corner points respectively to obtain two corresponding auxiliary lines;

[0016] According to the preset distance, a target point is determined on each of the two auxiliary lines, and the first distance is less than the preset distance.

[0017] In a possible implementation, determining that each corner point satisfies a preset condition includes:

[0018] Convert the pixel coordinates of each corner point into the corresponding vehicle coordinates in the vehicle coordinate system;

[0019] If the longitudinal distance between each vehicle coordinate and the origin of the vehicle coordinate system is greater than a second preset value, it is determined that each corner point meets the preset condition, and the second preset value corresponds to the preset distance.

[0020] In a possible implementation, determining the target distance between the vehicle and the target object according to the two target points, the two corner points and the first preset value includes:

[0021] Based on the two target points, determine the width of the target object;

[0022] The target distance is determined according to the width, the first preset value, and the two corner points.

[0023] In a possible implementation, the target distance includes a longitudinal target distance and a lateral target distance between the vehicle and the target object;

[0024] Determine the target distance according to the width, the first preset value, and the two corner points, including:

[0025] According to the pixel coordinates of each corner point, determine the camera coordinates of each corner point in the camera coordinate system;

[0026] Determine the longitudinal target distance according to the width, the first preset value, and the camera coordinates of the two corner points;

[0027] The lateral target distance is determined according to the longitudinal target distance, the first preset value and the camera coordinates of any corner point.

[0028] In a possible implementation, determining the width of the target object according to the two target points includes:

[0029] According to the pixel coordinates of each target point, the vehicle coordinates of each target point in the vehicle coordinate system are determined;

[0030] Based on the vehicle coordinates of the two target points, the width is determined.

[0031] In a second aspect, the present invention provides a visual distance measuring device for a vehicle, the visual distance measuring device for the vehicle comprising:

[0032] An acquisition unit, configured to acquire a captured image through a visual sensor, wherein the captured image includes at least one object;

[0033] A determination unit is used to determine a bounding box and a vanishing line of a target object in a captured image, wherein the target object is any one of the at least one object, and to determine two corner points of the target object according to a lower border of the bounding box, wherein the lower border is used to indicate the ground. If each corner point meets a preset condition, two corresponding target points are determined according to the two corner points and a target vanishing point on the vanishing line, and a target distance between the vehicle and the target object is determined according to the two target points, the two corner points and a first preset value.

[0034] Among them, the first distance between the target point and the visual sensor is smaller than the second distance between the corner point and the visual sensor, one target point is located on the line connecting the target vanishing point and a corner point, and two target points are located in the area formed by the lower border and the lower edge of the captured image, and the lower edge is used to indicate the ground.

[0035] In a possible implementation manner, the determining unit is specifically configured to:

[0036] Connect the target vanishing point with the two corner points respectively to obtain two corresponding auxiliary lines;

[0037] According to the preset distance, a target point is determined on each of the two auxiliary lines, and the first distance is less than the preset distance.

[0038] In a possible implementation manner, the determining unit is specifically configured to:

[0039] Convert the pixel coordinates of each corner point into the corresponding vehicle coordinates in the vehicle coordinate system;

[0040] If the longitudinal distance between each vehicle coordinate and the origin of the vehicle coordinate system is greater than a second preset value, it is determined that each corner point meets the preset condition, and the second preset value corresponds to the preset distance.

[0041] In a possible implementation manner, the determining unit is specifically configured to:

[0042] Based on the two target points, determine the width of the target object;

[0043] The target distance is determined according to the width, the first preset value, and the two corner points.

[0044] In a possible implementation, the target distance includes a longitudinal target distance and a lateral target distance between the vehicle and the target object; the determination unit is specifically configured to:

[0045] According to the pixel coordinates of each corner point, determine the camera coordinates of each corner point in the camera coordinate system;

[0046] Determine the longitudinal target distance according to the width, the first preset value, and the camera coordinates of the two corner points;

[0047] The lateral target distance is determined according to the longitudinal target distance, the first preset value and the camera coordinates of any corner point.

[0048] In a possible implementation manner, the determining unit is specifically configured to:

[0049] According to the pixel coordinates of each target point, the vehicle coordinates of each target point in the vehicle coordinate system are determined;

[0050] Based on the vehicle coordinates of the two target points, the width is determined.

[0051] In a third aspect, the present invention provides a vehicle, the vehicle comprising: a processor and a memory. The memory is used to store computer program code, the computer program code comprising computer instructions. When the processor executes the computer instructions, the vehicle executes the visual ranging method of the vehicle as described in the first aspect and any possible implementation thereof.

[0052] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon. When the computer instructions are executed on a vehicle, the vehicle executes a visual ranging method for the vehicle as in the first aspect or any one of the possible implementations of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 One of the structural schematic diagrams of the visual ranging device for a vehicle provided by an embodiment of the present invention;

[0054] Figure 2 One of the flow charts of the visual ranging method for a vehicle provided by an embodiment of the present invention;

[0055] Figure 3 Schematic diagram of the positional relationship among a vehicle, a corner point, a vanishing line and a target vanishing point in an image captured in an embodiment of the present invention;

[0056] Figure 4 A second flow chart of a method for visual ranging of a vehicle provided by an embodiment of the present invention;

[0057] Figure 5 A third flow chart of a method for visual ranging of a vehicle provided by an embodiment of the present invention;

[0058] Figure 6 Schematic diagram of conversion between a pixel coordinate system and a vehicle coordinate system in an embodiment of the present invention;

[0059] Figure 7 A fourth flow chart of a method for visual ranging of a vehicle provided by an embodiment of the present invention;

[0060] Figure 8 A schematic diagram of determining a target distance in an embodiment of the present invention;

[0061] Fig. 9 The second structural schematic diagram of the visual ranging device for a vehicle provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, unless otherwise specified, "multiple" means two or more. In addition, the use of "based on" or "according to" means openness and inclusiveness, because the process, steps, calculations or other actions "based on" or "according to" one or more of the conditions or values ​​may be based on additional conditions or values ​​beyond the described values ​​in practice.

[0064] In order to solve the problem of low accuracy when using a visual ranging method to measure the distance between an obstacle and a vehicle, an embodiment of the present invention provides a vehicle visual ranging method, device and storage medium. When using the vehicle visual ranging method provided by an embodiment of the present invention to measure the distance between a vehicle and an obstacle, first, a captured image is acquired by a visual sensor on the vehicle, and the captured image includes at least one object, namely, an obstacle. Then, the bounding box of any object of the at least one object and the vanishing line are determined, and when the two corner points of the lower frame indicating the ground in the determined bounding box meet the preset conditions, two corresponding target points are determined according to the two corner points and the target vanishing point on the vanishing line, and the first distance between the target point and the visual sensor is less than the second distance between the corner point and the visual sensor. Finally, according to the two target points, the two corner points and the first preset value, the target distance between the vehicle and any object can be determined. Through the vanishing line and the two corner points farther away from the visual sensor, the two target points closer to the visual sensor can be determined. In this way, the object at a farther position can be projected to a position closer to the visual sensor. By measuring the distance between the two closer target points, the width or height of the object at a farther position can be accurately calculated, and then the target distance between the visual sensor and the object at a farther position can be accurately calculated, which solves the problem of low accuracy when measuring the distance between an obstacle and a vehicle using the visual ranging method.

[0065] The execution subject of the vehicle visual ranging method provided in the embodiment of the present invention is the vehicle visual ranging device. The vehicle visual ranging device can be a vehicle-mounted terminal, or a central processing unit (CPU) in the above-mentioned vehicle-mounted terminal, or a control module in the above-mentioned vehicle-mounted terminal for measuring the distance between the vehicle and the obstacle, or a client in the above-mentioned vehicle-mounted terminal for measuring the distance between the vehicle and the obstacle. The embodiment of the present invention takes the vehicle-mounted terminal executing the vehicle visual ranging method as an example to illustrate the vehicle visual ranging method provided in the embodiment of the present invention.

[0066] The vehicle visual ranging method provided by the embodiment of the present invention can be applicable to a vehicle, on which a visual sensor and an on-board terminal are installed, and the visual sensor and the on-board terminal are connected by wired communication or wireless communication.

[0067] The visual sensor is used to capture obstacles around the vehicle and generate a captured image, wherein the captured image includes at least one object, and each object corresponds to an obstacle. The visual sensor is also used to send the captured image to the vehicle-mounted terminal.

[0068] Exemplarily, the visual sensor may be a camera having a monocular camera.

[0069] Figure 1 One of the structural schematic diagrams of the visual ranging device of the vehicle is shown in FIG. Figure 1 As shown, the visual distance measuring device of the vehicle may include: a processor 11, a memory 12, a communication interface 13 and a bus 14. The processor 11, the memory 12 and the communication interface 13 may be connected via the communication bus 14.

[0070] The processor 11 is the control center of the visual ranging device of the vehicle, and can be a processor 11 or a general term for multiple processing elements. For example, the processor 11 can be a general-purpose CPU or other general-purpose processors 11. The general-purpose processor 11 can be a microprocessor 11 or any conventional processor 11.

[0071] As an embodiment, the processor 11 may include one or more CPUs, for example, Figure 1 CPU0 and CPU1 are shown.

[0072] The memory 12 may be a read-only memory 12 (ROM) or other types of static storage devices that can store static information and instructions, a random access memory 12 (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory 12 (EEPROM), a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0073] In a possible implementation, the memory 12 may exist independently of the processor 11, and the memory 12 may be connected to the processor 11 via a bus 14 to store instructions or program codes. When the processor 11 calls and executes the instructions or program codes stored in the memory 22, the vehicle visual ranging method provided in the following embodiment of the present invention can be implemented.

[0074] In another possible implementation, the memory 12 may also be integrated with the processor 11 .

[0075] The communication interface 13 is used for connecting the visual distance measuring device of the vehicle with other devices through a communication network, and the communication network may be Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc. The communication interface 13 may include a receiving unit for receiving data and a sending unit for sending data.

[0076] The bus 14 may be an Industry Standard Architecture (ISA) bus 14, a Peripheral Component Interconnect (PCI) bus 14, or an Extended Industry Standard Architecture (EISA) bus 14. The bus 14 may be divided into an address bus 14, a data bus 14, a control bus 14, etc. For ease of representation, Figure 1 Although only one thick line is used in the figure, it does not mean that there is only one bus 14 or only one type of bus 14.

[0077] It should be pointed out that Figure 1 The structure shown in the figure does not constitute a limitation on the visual distance measuring device of the vehicle, except Figure 1 In addition to the components shown, the visual distance measuring device of the vehicle may include more or fewer components than shown, or combine certain components, or arrange the components differently.

[0078] The vehicle visual ranging method provided by an embodiment of the present invention is described below with reference to the accompanying drawings.

[0079] like Figure 2 As shown, the vehicle visual ranging method provided by the embodiment of the present invention includes the following steps 201-205.

[0080] 201. The vehicle-mounted terminal obtains a captured image through a visual sensor, where the captured image includes at least one object.

[0081] Optionally, the visual sensor may be a camera with a monocular camera. The camera may be mounted on the outside of the vehicle to capture objects around the vehicle. The objects may include obstacles, street lights and trees on the roadside, and pedestrians and vehicles on the road, but are not limited thereto. The embodiments of the present invention are described by taking the vehicle as an example.

[0082] In one scenario, the camera can be installed at the position of the license plate light of the rear license plate frame of the vehicle. When the vehicle is driving, the camera can be used to photograph other vehicles behind the vehicle, generate photographed images, and send the photographed images to the vehicle terminal.

[0083] 202. The vehicle-mounted terminal determines a boundary box and a vanishing line of a target object in the captured image, where the target object is any one of the at least one object.

[0084] After the vehicle terminal obtains the captured image, the bounding box of the target object can be detected by a pre-stored detection algorithm. The shape of the bounding box can be any polygon. Exemplarily, the shape of the bounding box can be any one of a rectangle and a triangle. When the object is a vehicle, the shape of the bounding box can be a rectangle.

[0085] Straight lines that are parallel to each other in space will intersect at one point in the image, which is the vanishing point. If there are lane lines on the ground in the captured image, then extend the lane lines on the ground so that at least two lane lines will intersect at one point in the captured image, which is the vanishing point. At this time, in the pixel coordinate system, the straight line that passes through the vanishing point and is parallel to the v-axis of the pixel coordinate system is used as the vanishing line. If there are no lane lines on the ground in the captured image, the image feature matching can be used to calculate the direction of movement of the vehicle using the visual odometer method to determine the vanishing point. Similarly, in the pixel coordinate system, the straight line that passes through the vanishing point and is parallel to the v-axis of the pixel coordinate system is used as the vanishing line.

[0086] 203. The vehicle-mounted terminal determines two corner points of the target object according to the lower border of the bounding box.

[0087] After the vehicle terminal detects the bounding box of the target object through the pre-stored detection algorithm, it can determine the two corner points corresponding to the lower border of the bounding box. The coordinates of the two corner points are pixel coordinates in the pixel coordinate system. Among them, the lower border can be used to indicate the ground.

[0088] 204. If each of the corner points meets a preset condition, the vehicle-mounted terminal determines two corresponding target points according to the two corner points and the target vanishing point on the vanishing line.

[0089] The first distance between the target point and the visual sensor is less than the second distance between the corner point and the visual sensor. One target point is located on the line connecting the target vanishing point and a corner point, and two target points are located in the area formed by the lower border and the lower edge of the captured image, so that the two target points are not at the edge position in the captured image. The lower edge is used to indicate the ground.

[0090] A vanishing line may include multiple vanishing points, and since a target point is located on the line connecting the target vanishing point and a corner point, any vanishing point among the multiple vanishing points that can satisfy the requirement of making the target point located within the area formed by the lower border and the lower edge of the captured image can be the target vanishing point.

[0091] Optionally, determining that each corner point meets the preset condition may specifically include: first, the vehicle-mounted terminal converts the pixel coordinates of each corner point into corresponding vehicle coordinates in the vehicle coordinate system. Then, the vehicle-mounted terminal needs to determine the magnitude relationship between the longitudinal distance between each vehicle coordinate and the origin of the vehicle coordinate system and a second preset value, and if the longitudinal distance between each vehicle coordinate and the origin of the vehicle coordinate system is greater than the second preset value, then it is determined that each corner point meets the preset condition.

[0092] It should be understood that the vehicle coordinate system uses the center point of the two front wheels of the vehicle as the coordinate origin, the direction pointing to the rear of the vehicle as the positive direction of the X axis, the direction pointing to the right side of the vehicle as the positive direction of the Y axis, and the direction pointing to the roof as the positive direction of the Z axis. The longitudinal distance refers to the distance between the vehicle and the vehicle in front or behind.

[0093] Figure 3 FIG. 1 shows a schematic diagram of the positional relationship between the vehicle, each corner point, the vanishing line, and the target vanishing point in the captured image. Figure 3 As shown, P1 and P2 are two corner points, P3 and P4 are two target points, and corner point P1 corresponds to target point P3, corner point P2 corresponds to target point P4, L is a vanishing line, W is a target vanishing point, and M1 and M2 are two auxiliary lines.

[0094] For example, Figure 3 As shown, the process of determining two target points according to the target vanishing point and two corner points by the vehicle terminal may include: first, the vehicle terminal connects the target vanishing point W with the corner point P1 to obtain the auxiliary line M1, and connects the target vanishing point W with the corner point P2 to obtain the auxiliary line M2. Then, the vehicle terminal determines the target point P3 on the auxiliary line M1 and the target point P4 on the auxiliary line M2 according to the preset distance. Moreover, the first distance between the target point P3 and the visual sensor, or the first distance between the target point P4 and the visual sensor is less than the preset distance, and the second preset value corresponds to the preset distance. In other words, the two target points P3 and P4 are closer to the visual sensor than the two corner points P1 and P2.

[0095] It should be understood that in the vehicle coordinate system, the auxiliary lines M1 and M2 are parallel and on the same horizontal plane, so extending the auxiliary lines M1 and M2 in the pixel coordinate system will intersect at the vanishing line, and the actual lengths of the line segments P1 P2 and P3 P4 in the vehicle coordinate system are also equal. The second preset value is used to represent the distance in the vehicle coordinate system, while the preset distance is the distance in the pixel coordinate system. However, in the same coordinate system, the value of the preset distance is equal to the second preset value. Generally, the second preset value can be 20 meters.

[0096] 205. The vehicle-mounted terminal determines a target distance between the vehicle and the target object according to the two target points, the two corner points and the first preset value.

[0097] The first preset value is used to indicate the distance between the visual sensor and the origin of the corresponding camera coordinate system.

[0098] When the visual ranging method of the vehicle provided by the present invention is used to measure the distance between the vehicle and the obstacle, a captured image is obtained by a visual sensor on the vehicle, and the captured image includes at least one object. The bounding box and vanishing line of any object in the at least one object are determined, and when the two corner points of the lower frame indicating the ground in the determined bounding box are far from the visual sensor, two target points that are close to the visual sensor and located in the area composed of the lower frame and the lower edge of the captured image can be obtained according to the two corner points and the target vanishing point on the vanishing line. Finally, the target distance between the vehicle and any object in the captured image can be determined according to the two target points, the two corner points and the first preset value. As can be seen from the above, in the present invention, two target points that are close to the visual sensor are determined by the vanishing line and the two corner points that are far from the visual sensor, and when the distance between the object and the visual sensor is close, the target distance between the object and the visual sensor can be calculated very accurately. That is to say, the present invention projects an obstacle located at a farther position to a position closer to the visual sensor, thereby accurately calculating the target distance between the visual sensor and the obstacle located at a farther position, thereby solving the problem of low accuracy when using the visual ranging method to measure the distance between the obstacle and the vehicle.

[0099] Combination Figure 2 ,like Figure 4 As shown, the above step 205 may specifically include the following steps 401 and 402.

[0100] 401. The vehicle-mounted terminal determines the width of the target object according to the two target points;

[0101] After the vehicle terminal determines the two target points P3 and P4, the actual width of the target object, that is, the length of the line segment P1 P2, can be determined based on the coordinates of the two target points P3 and P4 using the Euclidean distance calculation formula.

[0102] 402. The vehicle-mounted terminal determines the target distance according to the width, the first preset value, and the two corner points.

[0103] The vehicle terminal can determine the target distance between the visual sensor and the target object, that is, the target distance between the vehicle and the target object, by using the principle of similar triangles based on the width, the first preset value, and the coordinates of the two corner points P1 and P2.

[0104] Combination Figure 4 ,like Figure 5 As shown, the above step 401 may specifically include the following steps 501 and 502.

[0105] 501. The vehicle-mounted terminal determines the vehicle coordinates of each target point in the vehicle coordinate system according to the pixel coordinates of each target point.

[0106] Figure 6 FIG. 2 shows a schematic diagram of the conversion between the pixel coordinate system and the vehicle coordinate system. Figure 6 As shown in the figure, vO1u represents the pixel coordinate system, O1 represents the coordinate origin of the pixel coordinate system, and the horizontal coordinate u and vertical coordinate v of the pixel coordinate system are the column number and row number of the image array respectively. O2 is the projection center of the camera. XO3Y represents the vehicle coordinate system, O3 represents the coordinate origin of the vehicle coordinate system, that is, the center point of the two front wheels of the vehicle, and X and Y represent the horizontal coordinate and vertical coordinate of the vehicle coordinate system respectively.

[0107] Among them, the distance between the projection center O2 of the camera and the coordinate origin O3 of the vehicle coordinate system represents the height H of the camera. The camera plane where O2 is located is parallel to the camera imaging plane where the coordinate origin O1 of the pixel coordinate system is located, that is, the camera imaging plane where the coordinate origin O1 of the pixel coordinate system is located is the plane where the imaging plane of the target object is obtained through the camera lens. The distance between the coordinate origin O1 of the pixel coordinate system and the projection center O2 of the camera is the focal length f of the camera, and the straight line between the two is the camera optical axis, and the camera optical axis is perpendicular to the camera plane where the projection center O2 of the camera is located and the camera imaging plane where the coordinate origin O1 of the pixel coordinate system is located. The coordinate origin O3 of the vehicle coordinate system can be determined by the projection position of the projection center O2 of the camera on the ground, and by calibrating the position of the intersection point M of the camera optical axis with the Y axis in the vehicle coordinate system, the vertical distance O3M between point M and the camera imaging plane can be obtained. Point P is the projection point of the target point P3 or the target point P4 on the Y axis in the vehicle coordinate system. Point Q is the projection point of the target point P3 or the target point P4 in the vehicle coordinate system.

[0108] In combination with the above description, taking the target point P3 as an example, the specific process of determining the vehicle coordinates of the target point P3 in the vehicle coordinate system is introduced below.

[0109] Exemplarily, according to the following formula (1), the angle between the camera optical axis and the Y axis of the vehicle coordinate system can be determined.

[0110]

[0111] Among them, α represents the angle between the camera optical axis and the Y axis of the vehicle coordinate system; H represents the vertical distance between the camera and the Y axis of the vehicle coordinate system, that is, the height of the camera; O3M represents the vertical distance between the intersection M of the camera optical axis with the Y axis in the vehicle coordinate system and the camera plane.

[0112] Using the pinhole imaging principle, the angle between the camera optical axis and the imaging light of the target point P3 in the camera can be determined according to the following formula (2).

[0113]

[0114] Among them, γ represents the angle between the camera optical axis and the imaging light of the target point P3 in the camera; O1v1 represents the length of the pixel value in the pixel coordinate system; y_pixel represents the length of the unit pixel in the v-axis direction of the pixel coordinate system; f represents the focal length of the camera.

[0115] According to the following formulas (3) to (5), the Y-axis coordinate of the target point P3 in the vehicle coordinate system can be determined.

[0116] β=α-γ (3)

[0117]

[0118] y3=O3P (5)

[0119] Wherein, β represents the angle between the imaging light of the target point P3 in the camera and the Y-axis in the vehicle coordinate system; y3 represents the Y-axis coordinate of the target point P3 in the vehicle coordinate system.

[0120] Using the Pythagorean theorem, the length of line segment O2v1 can be determined according to the following formula (6).

[0121]

[0122] Using trigonometric functions, the length of line segment O2P can be determined according to the following formula (7).

[0123]

[0124] Depend on Figure 6It can be seen that △v1P3O2 is similar to △O2QP. According to the following formulas (8) to (10), the X-axis coordinate of the target point P3 in the vehicle coordinate system can be determined.

[0125]

[0126]

[0127] x3=PQ (10)

[0128] Wherein, x3 represents the X-axis coordinate of the target point P3 in the vehicle coordinate system.

[0129] In summary, by using the above formulas (1) to (10), the principle of similar triangles, the Pythagorean theorem, trigonometric functions, and the principle of pinhole imaging, the vehicle coordinates of the target point P3 in the pixel coordinate system and the target point P4 in the vehicle coordinate system can be determined respectively.

[0130] 502. The vehicle-mounted terminal determines the width according to the vehicle coordinates of the two target points.

[0131] By using the above formulas (1) to (10), the vehicle coordinates (x3, y3) of the target point P3 in the vehicle coordinate system and the vehicle coordinates (x4, y4) of the target point P4 in the vehicle coordinate system can be determined. Then, according to the following formula (11), the distance between the target point P3 and the target point P4, that is, the actual width of the target object, can be determined.

[0132]

[0133] Among them, W actual Indicates the width of the target object.

[0134] Optionally, the target distance may include a longitudinal target distance and a lateral target distance between the vehicle and the target object.

[0135] Combination Figure 5 ,like Figure 7 As shown, the above step 402 may specifically include the following steps 701 to 703.

[0136] 701. The vehicle-mounted terminal determines the camera coordinates of each corner point in the camera coordinate system according to the pixel coordinates of each corner point.

[0137] Optionally, according to the pixel coordinates of each corner point and the intrinsic parameter matrix M of the camera, the pixel coordinates of each corner point can be converted into camera coordinates in the camera coordinate system.

[0138] Exemplarily, taking corner point P1 as an example, the camera coordinates of corner point P1 in the camera coordinate system can be determined according to the following formula (12).

[0139]

[0140] Among them, f x 、f y 、c x 、c y is the intrinsic parameter coefficient of the camera; u and v are the pixel coordinates of the corner point P1 in the pixel coordinate system, x c and c and are the camera coordinates of the corner point P1 in the camera coordinate system.

[0141] Optionally, since there is a deviation angle between the camera coordinate system and the world coordinate system, the camera coordinate system needs to be rotated by a rotation matrix R to eliminate the error effect caused by the deviation angle between the two coordinate systems.

[0142] Exemplarily, according to the following formula (13), the camera coordinates of the corner point P1 after adjustment in the camera coordinate system can be determined.

[0143]

[0144] Among them, x w ,y w and z w is the camera coordinate of the corner point P1 after adjustment in the camera coordinate system, and R is the rotation matrix.

[0145] For example, assuming that the deflection angles between the three coordinate axes of the camera coordinate system and the world coordinate system are α1, β1, and γ1, respectively, then, according to the following formula (14), the rotation matrix R can be determined by the rotation matrix corresponding to each coordinate axis.

[0146]

[0147] in, is the rotation matrix corresponding to the X axis; is the rotation matrix corresponding to the Y axis; is the rotation matrix corresponding to the Z axis.

[0148] For example, the rotation matrix corresponding to the X-axis is The rotation matrix corresponding to the Y axis And the rotation matrix corresponding to the Z axis It can be expressed as:

[0149]

[0150]

[0151]

[0152] 702. The vehicle-mounted terminal determines the longitudinal target distance according to the width, the first preset value, and the camera coordinates of the two corner points;

[0153] Figure 8 A schematic diagram of determining the target distance is shown, such as Figure 8 As shown, in the camera coordinate system, the width of the target object, the first preset value Z1, and the X-axis coordinate x of the corner point P1 in the camera coordinate system are known. w1 , and the X-axis coordinate x of the corner point P2 in the camera coordinate system w2 .

[0154] Exemplarily, the longitudinal target distance can be determined by using the principle of similar triangles according to the following formula (15).

[0155]

[0156] The first preset value Z1 is 1; d1 is the longitudinal target distance.

[0157] 703. The vehicle-mounted terminal determines the lateral target distance according to the longitudinal target distance, the first preset value and the camera coordinates of any corner point.

[0158] For example, taking the case where the target object is located at the rear right side of the vehicle, the lateral target distance can be determined by using the principle of similar triangles according to the following formula (16).

[0159]

[0160] It should be understood that the above takes an object in a captured image as an example to introduce the target distance between an object and a vehicle. The distance measurement process between each other object and the vehicle is the same and will not be repeated here. Figure 8 The area represented by Q in is the area consisting of the lower border of the target object's bounding box and the lower edge of the captured image.

[0161] The above mainly introduces the solution provided by the embodiment of the present invention from the perspective of the device. It is understandable that in order to realize the above functions, the device includes a hardware structure and / or software module corresponding to each function. Those skilled in the art should easily realize that, in combination with the algorithm steps of each example described in the embodiment disclosed in this article, the present invention can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0162] Fig. 9A possible schematic diagram of the composition of the vehicle visual distance measuring device 900 involved in the above embodiment is shown. Fig. 9 As shown, the visual distance measurement device 900 of the vehicle may include: an acquisition unit 901 and a determination unit 902 .

[0163] The acquisition unit 901 is used to acquire a captured image through a visual sensor, and the captured image includes at least one object. The determination unit 902 is used to determine the bounding box and vanishing line of the target object in the captured image, the target object is any object of the at least one object, and determine the two corner points of the target object according to the lower frame of the bounding box, the lower frame is used to indicate the ground, if each corner point meets the preset conditions, then according to the two corner points and the target vanishing point on the vanishing line, two corresponding target points are determined, and according to the two target points, the two corner points and the first preset value, the target distance between the vehicle and the target object is determined. The first distance between the target point and the visual sensor is less than the second distance between the corner point and the visual sensor, one target point is located on the line connecting the target vanishing point and one corner point, and two target points are located in the area formed by the lower frame and the lower edge of the captured image, and the lower edge is used to indicate the ground.

[0164] Optionally, the determination unit 902 is specifically configured to: connect the target vanishing point with the two corner points respectively to obtain two corresponding auxiliary lines, and determine a target point on each of the two auxiliary lines according to a preset distance, wherein the first distance is less than the preset distance.

[0165] Optionally, the determination unit 902 is specifically used to: convert the pixel coordinates of each corner point into corresponding vehicle coordinates in the vehicle coordinate system; if the longitudinal distance between each vehicle coordinate and the origin of the vehicle coordinate system is greater than a second preset value, it is determined that each corner point meets the preset conditions, and the second preset value corresponds to the preset distance.

[0166] Optionally, the determination unit 902 is specifically configured to: determine a width of the target object according to the two target points, and determine a target distance according to the width, a first preset value, and two corner points.

[0167] Optionally, the determination unit 902 is specifically used to: the above-mentioned target distance includes the longitudinal target distance and the lateral target distance between the vehicle and the target object; the determination unit is specifically used to: determine the camera coordinates of each corner point in the camera coordinate system according to the pixel coordinates of each corner point, and determine the longitudinal target distance according to the width, the first preset value, and the camera coordinates of the two corner points, and determine the lateral target distance according to the longitudinal target distance, the first preset value and the camera coordinates of any one of the corner points.

[0168] Optionally, the determination unit 902 is specifically configured to: determine the vehicle coordinates of each target point in the vehicle coordinate system according to the pixel coordinates of each target point, and determine the width according to the vehicle coordinates of the two target points.

[0169] Of course, the vehicle visual distance measurement device 900 provided in the embodiment of the present invention includes but is not limited to the above modules.

[0170] In actual implementation, the acquisition unit 901 and the determination unit 902 may be composed of Figure 1 The processor 11 shown calls the program code in the memory 12 to implement. The specific execution process can be referred to Figure 2 , Figure 4 , Figure 5 and Figure 7 The description of the visual ranging method of the vehicle shown will not be repeated here.

[0171] Another embodiment of the present invention further provides a vehicle, the vehicle comprising: a processor and a memory. The memory is used to store computer program codes, and the computer program codes include computer instructions. When the processor executes the computer instructions, the vehicle executes the vehicle visual distance measurement method shown in the above method embodiment.

[0172] Another embodiment of the present invention further provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on a visual ranging device of a vehicle, the visual ranging device of the vehicle executes each step executed by the visual ranging device of the vehicle in the method flow shown in the above method embodiment.

[0173] Another embodiment of the present invention further provides a chip system, which is applied to a visual ranging device 900 of a vehicle. The chip system includes one or more interface circuits and one or more processors 11. The interface circuit and the processor 11 are interconnected through a line. The interface circuit is used to receive a signal from a memory 12 of the visual ranging device 900 of the vehicle and send the signal to the processor 11, wherein the signal includes a computer instruction stored in the memory 12. When the processor 11 executes the computer instruction, the visual ranging device of the vehicle executes each step executed by the visual ranging device of the vehicle in the method flow shown in the above method embodiment. Alternatively, when the computer instruction is executed on the vehicle, the vehicle executes each step in the method flow shown in the above method embodiment.

[0174] In another embodiment of the present invention, a computer program product is also provided. The computer program product includes instructions. When the instructions are executed on the visual ranging device 900 of the vehicle, the visual ranging device 900 of the vehicle executes each step executed by the visual ranging device 900 of the vehicle in the method flow shown in the above method embodiment.

[0175] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using a software program, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer execution instruction is loaded and executed on the computer, the process or function according to the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains one or more servers that can be integrated with the medium. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state disk (SSD)).

[0176] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for visual ranging of a vehicle, characterized in that: include: Acquire a captured image by a visual sensor, wherein the captured image includes at least one object; Determine a bounding box and a vanishing line of a target object in the captured image, wherein the target object is any one of the at least one object; Determine two corner points of the target object according to a lower border of the bounding box, wherein the lower border is used to indicate the ground; If each of the corner points meets a preset condition, two corresponding target points are determined according to the two corner points and the target vanishing point on the vanishing line; The first distance between the target point and the visual sensor is smaller than the second distance between the corner point and the visual sensor, one of the target points is located on a line connecting the target vanishing point and one of the corner points, and two of the target points are located in an area formed by the lower frame and the lower edge of the captured image, and the lower edge is used to indicate the ground; Determining a target distance between the vehicle and the target object according to the two target points, the two corner points and a first preset value; Determining two corresponding target points according to the two corner points and the target vanishing point on the vanishing line includes: Connect the target vanishing point with the two corner points respectively to obtain two corresponding auxiliary lines; According to a preset distance, one target point is determined on each of the two auxiliary lines, and the first distance is smaller than the preset distance.

2. The method for visual distance measurement of a vehicle according to claim 1, characterized in that: Determining that each of the corner points meets a preset condition includes: Convert the pixel coordinates of each corner point into corresponding vehicle coordinates in the vehicle coordinate system; If the longitudinal distance between each of the vehicle coordinates and the origin of the vehicle coordinate system is greater than a second preset value, it is determined that each of the corner points meets the preset condition, and the second preset value corresponds to the preset distance.

3. The method for visual distance measurement of a vehicle according to claim 1, characterized in that: The step of determining a target distance between the vehicle and the target object according to the two target points, the two corner points and the first preset value comprises: Determining the width of the target object according to the two target points; The target distance is determined according to the width, the first preset value, and the two corner points.

4. The method for visual distance measurement of a vehicle according to claim 3, characterized in that: The target distance includes a longitudinal target distance and a lateral target distance between the vehicle and the target object; The determining the target distance according to the width, the first preset value, and the two corner points includes: Determine the camera coordinates of each corner point in a camera coordinate system according to the pixel coordinates of each corner point; Determine the longitudinal target distance according to the width, the first preset value, and the camera coordinates of the two corner points; The lateral target distance is determined according to the longitudinal target distance, the first preset value and the camera coordinates of any one of the corner points.

5. The method for visual distance measurement of a vehicle according to claim 3, characterized in that: Determining the width of the target object according to the two target points includes: Determining the vehicle coordinates of each target point in the vehicle coordinate system according to the pixel coordinates of each target point; The width is determined according to the vehicle coordinates of the two target points.

6. A visual distance measuring device for a vehicle, characterized in that: include: An acquisition unit, configured to acquire a captured image through a visual sensor, wherein the captured image includes at least one object; a determination unit, configured to determine a bounding box and a vanishing line of a target object in the captured image, wherein the target object is any one of the at least one object, and to determine two corner points of the target object according to a lower border of the bounding box, wherein the lower border is used to indicate a ground surface, and if each of the corner points satisfies a preset condition, to determine two corresponding target points according to the two corner points and a target vanishing point on the vanishing line, and to determine a target distance between the vehicle and the target object according to the two target points, the two corner points and a first preset value; The first distance between the target point and the visual sensor is smaller than the second distance between the corner point and the visual sensor, one of the target points is located on a line connecting the target vanishing point and one of the corner points, and two of the target points are located in an area formed by the lower frame and the lower edge of the captured image, and the lower edge is used to indicate the ground; The determining unit is specifically configured to: Connect the target vanishing point with the two corner points respectively to obtain two corresponding auxiliary lines; According to a preset distance, one target point is determined on each of the two auxiliary lines, and the first distance is smaller than the preset distance.

7. A vehicle, characterized in that: The vehicle comprises: a processor and a memory; the memory is used to store computer program code, and the computer program code comprises computer instructions; when the processor executes the computer instructions, the vehicle executes the visual ranging method for the vehicle as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The method comprises computer instructions, which, when executed on a vehicle, enable the vehicle to execute the vehicle visual distance measurement method according to any one of claims 1 to 5.

Images