A calibration method, device, electronic device and storage medium
By adjusting the homography matrix and using the position information of lane lines and pixel coordinates for matrix correction, the problem of inaccurate homography matrix in smart cars is solved and the distance measurement accuracy is improved.
Patent Information
- Application Number
- CN202011189288.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-10-30
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2040-10-30
AI Technical Summary
During the autonomous driving of smart cars, the homography matrix is inaccurate due to vehicle mechanical vibration and uneven road surface, which affects the accuracy of the distance measurement results.
By acquiring the road surface images of multiple lane lines collected by the image acquisition device and their position points under the world coordinate system, adjusting the initial homography matrix to determine the current homography matrix, and using the position information of the lane lines and pixel coordinates to make the matrix Correct to maintain the relative position of the image acquisition device and the vehicle.
It alleviates the inaccuracy of homography matrix caused by mechanical vibration and uneven road surface during vehicle driving, and improves the accuracy of distance measurement results.
Smart Images

Figure CN112419423B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer vision technology, and in particular to a calibration method, device, electronic device and storage medium. Background Art
[0002] With the rapid development of artificial intelligence technology, the combination of traditional industries and information technology has brought convenience to people's lives. For example, the combination of the automotive industry and information technology can produce smart cars that can drive automatically. In the process of smart cars driving automatically, ranging is a very important link. Among the ranging sensors used in smart car assisted driving, visual sensors can obtain richer road structure and environmental information and are relatively cheap.
[0003] In visual ranging, monocular visual ranging technology is widely used because it has the advantages of low cost, simple system installation, good stability, etc. compared with multi-eye visual ranging technology. In monocular visual ranging, a homography matrix is needed. Based on the pixel coordinates of the photographed target in the image coordinate system and the homography matrix, the world coordinates of the target in the world coordinate system can be obtained, and the distance information between the target and the preset position point can be obtained based on the world coordinates. Therefore, the accuracy of the homography matrix directly affects the accuracy of the ranging result.
[0004] The homography matrix is obtained through pre-calibration. During the calibration, a reference object is placed manually, and then a reference object picture is obtained based on the vehicle-mounted camera. The homography matrix of the vehicle-mounted camera is determined by the pixels corresponding to the reference object in the reference object picture and the coordinates of the reference object in the world coordinate system. This calibration method has low calibration efficiency, and because the vehicle will cause mechanical vibration during driving, the position of the vehicle-mounted camera will change. The homography matrix previously obtained by calibrating the vehicle-mounted camera will no longer be accurate when determining the distance of the target object. Summary of the invention
[0005] The embodiments of the present disclosure at least provide a calibration solution.
[0006] In a first aspect, an embodiment of the present disclosure provides a calibration method, including:
[0007] Acquire a road surface image including a plurality of lane lines acquired by an image acquisition device, and world coordinates of a plurality of position points on each lane line in a world coordinate system;
[0008] Based on the position information of each lane line in the road surface image, the world coordinates of the multiple position points and the initial homography matrix of the image acquisition device, the current homography matrix of the image acquisition device is determined.
[0009] After obtaining the road surface image containing multiple lane lines and the world coordinates of multiple position points on each lane line in the world coordinate system captured by the image acquisition device, the initial homography matrix can be adjusted through the position information of each lane line in the road surface image and the world coordinates of the multiple position points to determine the current homography matrix of the image acquisition device; illustratively, the image acquisition device can be set on a moving vehicle, so that as the vehicle moves, the road surface image is always the road surface image acquired by the image acquisition device while maintaining the current relative position relationship with the vehicle. In this way, when the road surface image contains multiple lane lines, the initial homography matrix of the image acquisition device can be continuously corrected, thereby alleviating the problem of the homography matrix being no longer accurate due to mechanical vibration during vehicle driving or uneven road surface on which the vehicle is driving.
[0010] In a possible implementation, the world coordinates of multiple position points on each lane line are obtained in the following manner:
[0011] Determine the world coordinates of the intersection of the target line segment in the road surface image and each lane line in the world coordinate system;
[0012] Based on the world coordinates of each intersection point, the world coordinates of multiple position points on each lane line in the world coordinate system are obtained.
[0013] In a possible implementation manner, determining the world coordinates of the intersection of the target line segment in the road surface image and each lane line in the world coordinate system includes:
[0014] Selecting a plurality of initial position points in the world coordinate system, wherein the distance between each of the initial position points and the origin of the world coordinate system is less than a preset distance threshold;
[0015] Based on the initial homography matrix, the target line segment formed by connecting the initial pixel coordinates corresponding to the multiple initial position points in the road surface image and the world coordinates of the intersection of the target line segment and each lane line in the world coordinate system are determined.
[0016] The disclosed embodiment selects an initial position point that is relatively close to the origin in the world coordinate system, so that the error of the physical distance corresponding to the initial pixel coordinates of the initial position point in the image coordinate system obtained based on the initial homography matrix is relatively small, and thus the error of the world coordinates corresponding to the intersection point obtained based on the initial pixel coordinates in the world coordinate system is also relatively small.
[0017] In a possible implementation, determining, based on the initial homography matrix, the target line segment formed by connecting the initial pixel coordinates corresponding to the multiple initial position points in the road surface image, and the world coordinates of the intersection of the target line segment and each lane line in the world coordinate system includes:
[0018] Based on the initial homography matrix and the initial position coordinates of each of the initial position points in the world coordinate system, determining the initial pixel coordinates corresponding to each of the initial position points in the road surface image;
[0019] Performing straight line fitting on the initial pixel coordinates corresponding to the multiple initial position points to obtain the target line segment;
[0020] Obtaining pixel coordinates of intersection points of the target line segment and each lane line in the road surface image;
[0021] Based on the intersection pixel coordinates of the intersection and the initial homography matrix, the world coordinates of the intersection in the world coordinate system are determined.
[0022] In a possible implementation, acquiring the world coordinates of a plurality of position points on each lane line in a world coordinate system based on the world coordinates of each intersection point includes:
[0023] Based on the world coordinates of each of the intersections and a preset position interval, the world coordinates of a plurality of position points located on the same lane line as each of the intersections in the world coordinate system are obtained.
[0024] In a possible implementation, determining the current homography matrix of the image acquisition device based on the position information of each lane line in the road surface image, the world coordinates of the multiple position points, and the initial homography matrix of the image acquisition device includes:
[0025] Determining initial pixel coordinates of the plurality of position points in the road surface image based on the world coordinates of the plurality of position points and the initial homography matrix;
[0026] Based on the position information of each lane line in the road surface image, correct the initial pixel coordinates corresponding to multiple position points on the lane line to obtain corrected pixel coordinates of the multiple position points on the lane line in the road surface image;
[0027] Based on the corrected pixel coordinates and the world coordinates of multiple position points on each lane line, a current homography matrix of the image acquisition device is obtained.
[0028] In the disclosed embodiments, it is proposed that the initial pixel coordinates of multiple position points in the road surface image can be determined based on the world coordinates of the multiple position points and the initial homography matrix. When the initial homography matrix is no longer accurate, the initial pixel coordinates can be quickly corrected based on the initial pixel coordinates of the multiple position points and the position information of the lane lines in the road surface image, so as to quickly determine the current homography matrix of the image acquisition device based on the corrected pixel coordinates.
[0029] In a possible implementation, based on the position information of each lane line in the road surface image, correcting the initial pixel coordinates corresponding to multiple position points on the lane line to obtain the corrected pixel coordinates of the multiple position points on the lane line in the road surface image includes:
[0030] Based on the position information of each lane line in the road surface image and the initial pixel coordinates corresponding to the multiple position points on the lane line, determine the deviation angle between the fitting straight line of the multiple initial pixel coordinates and the fitting straight line corresponding to the lane line in the road surface image;
[0031] Based on the determined deviation angle, the initial pixel coordinates corresponding to the multiple position points on the lane line are corrected to obtain the corrected pixel coordinates of the multiple position points on the lane line in the road surface image.
[0032] In the disclosed embodiment, when the initial homography matrix is no longer accurate, the initial pixel coordinates of the multiple position points in the image coordinate system determined based on the initial homography matrix will deviate from the corresponding lane line. By determining the deviation angle between the initial pixel coordinates of the multiple position points on each lane line in the image coordinate system and the lane line, the initial pixel coordinates of the multiple position points on the lane line can be corrected based on the deviation angle. In this way, the corrected pixel coordinates of the multiple position points on the lane line in the image coordinate system can be determined more accurately.
[0033] In a possible implementation, the determining, based on the position information of each lane line in the road surface image and the initial pixel coordinates corresponding to the multiple position points on the lane line, a deviation angle between a fitting straight line of the multiple initial pixel coordinates and a fitting straight line corresponding to the lane line in the road surface image includes:
[0034] Based on the position information of each lane line in the road surface image, generating a first lane line equation in an image coordinate system corresponding to each lane line in the road surface image;
[0035] Performing straight line fitting on the initial pixel coordinates corresponding to a plurality of position points on each lane line to generate a second lane line equation corresponding to each lane line;
[0036] Based on the first lane line equation and the second lane line equation corresponding to each lane line, a deviation angle of a straight line represented by the second lane line equation relative to the straight line represented by the first lane line equation is determined.
[0037] In a possible implementation manner, after determining the current homography matrix of the image acquisition device, the method further includes:
[0038] Acquire a target image obtained after the image acquisition component photographs the target object;
[0039] Based on the target image, determining the pixel coordinates of the target object in the target image;
[0040] Based on the pixel coordinates and the current homography matrix, the world coordinates of the target object in the world coordinate system are determined.
[0041] In a possible implementation manner, after determining the world coordinates of the target object in the world coordinate system, the method further includes:
[0042] Based on the world coordinates of the target object in the world coordinate system and the coordinates of the preset position point in the world coordinate system, the distance between the target object and the preset position point is determined.
[0043] After obtaining the current homography matrix of the image acquisition device, the embodiment of the present disclosure can use the current homography matrix to accurately determine the world coordinates of the target object in the world coordinate system, and then determine the distance to the target object.
[0044] In a second aspect, an embodiment of the present disclosure provides a calibration device, including:
[0045] An acquisition module, used to acquire a road surface image including a plurality of lane lines acquired by an image acquisition device, and world coordinates of a plurality of position points on each lane line in a world coordinate system;
[0046] A determination module is used to determine the current homography matrix of the image acquisition device based on the position information of each lane line in the road surface image, the world coordinates of the multiple position points and the initial homography matrix of the image acquisition device.
[0047] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the calibration method described in the first aspect are performed.
[0048] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the calibration method described in the first aspect are executed.
[0049] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.
[0051] Figure 1 A flow chart of a calibration method provided by an embodiment of the present disclosure is shown;
[0052] Figure 2 A flow chart of a method for obtaining world coordinates of multiple position points on each lane line provided by an embodiment of the present disclosure is shown;
[0053] Figure 3 A flow chart of a method for determining a current homography matrix of an image acquisition device provided by an embodiment of the present disclosure is shown;
[0054] Figure 4a One of the flow charts for determining the current homography matrix of an image acquisition device provided by an embodiment of the present disclosure is shown;
[0055] Figure 4b A second flowchart of determining a current homography matrix of an image acquisition device provided by an embodiment of the present disclosure is shown;
[0056] Figure 4c A third flowchart of determining a current homography matrix of an image acquisition device provided by an embodiment of the present disclosure is shown;
[0057] Figure 5 A flow chart of a method for performing distance measurement based on a current homography matrix of an image acquisition device provided by an embodiment of the present disclosure is shown;
[0058] Figure 6 A schematic diagram of the structure of a calibration device provided by an embodiment of the present disclosure is shown;
[0059] Figure 7 A schematic structural diagram of an electronic device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0060] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.
[0061] In the field of autonomous driving and robotics, it is often necessary to rely on image acquisition devices for visual ranging. The principle of relying on image acquisition devices for visual ranging is to determine the pixel coordinates of the target object photographed by the image acquisition device in the image coordinate system, and then determine the world coordinates of the target object in the world coordinate system based on the homography matrix of the image acquisition device, and then determine the distance between the preset position point and the target object according to the world coordinates of the preset position point and the world coordinates of the target object. The preset position point here can be the set origin of the world coordinate system. Therefore, the accuracy of the homography matrix directly affects the accuracy of the ranging result. The homography matrix of the image acquisition device on the vehicle can be obtained by pre-calibrating the position relationship between the image acquisition device and the vehicle. After the image acquisition device on the vehicle is calibrated, the distance of the target object can be determined according to the homography matrix. However, during the driving of the vehicle, due to the mechanical vibration caused by long-term driving or when the vehicle is driving on an uneven surface, the relative position of the image acquisition device and the vehicle may change, and the initial homography matrix may no longer be accurate.
[0062] Based on the above research, the present disclosure provides a calibration method. After obtaining a road surface image containing multiple lane lines captured by an image acquisition device and the world coordinates of multiple position points on each lane line in the world coordinate system, the initial homography matrix can be adjusted through the position information of each lane line in the road surface image and the world coordinates of the multiple position points to determine the current homography matrix of the image acquisition device; illustratively, the image acquisition device can be set on a moving vehicle, so that as the vehicle moves, the road surface image is always the road surface image acquired by the image acquisition device while maintaining a current relative position relationship with the vehicle. In this way, when the road surface image contains multiple lane lines, the initial homography matrix of the image acquisition device can be continuously corrected, thereby alleviating the problem that the homography matrix is no longer accurate due to mechanical vibration during vehicle driving or uneven road surface on which the vehicle is driving.
[0063] The technical solutions in the present disclosure will be clearly and completely described below in conjunction with the drawings in the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure claimed for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.
[0064] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.
[0065] To facilitate understanding of this embodiment, a calibration method disclosed in the embodiment of the present disclosure is first introduced in detail. The execution subject of the calibration method provided in the embodiment of the present disclosure is generally a computer device with certain computing capabilities, and the computer device includes, for example: a terminal device or a server or other processing device, and the terminal device can be a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, etc. In some possible implementations, the calibration method can be implemented by a processor calling a computer-readable instruction stored in a memory.
[0066] The calibration method provided in the embodiment of the present disclosure is described below by taking the execution subject as a terminal device as an example.
[0067] See also Figure 1As shown, it is a flow chart of a calibration method provided in an embodiment of the present disclosure. The calibration method can be applied to a processor in a vehicle-mounted device. The calibration method includes steps S101 to S102, wherein:
[0068] S101, obtaining a road surface image including a plurality of lane lines captured by an image acquisition device, and world coordinates of a plurality of position points on each lane line in a world coordinate system.
[0069] Exemplarily, the image acquisition device may include a monocular camera, or a monocular camera, and may include a camera for acquiring RGB images, grayscale images or depth images. The image acquisition device may be set on a vehicle. When the vehicle is traveling on a road, the image acquisition device may capture a road surface image of the current road surface, and then send the road surface image to a processor in the on-board device. The processor detects lane lines in the road surface image. If the road surface image contains multiple lane lines, including at least two non-overlapping lane lines, the current homography matrix of the image acquisition device may be determined based on multiple position points on each lane line in the road surface image.
[0070] Here, the multiple position points on each lane line are the included position points of the lane line on the current road surface. These position points may be position points existing on the road lane line, that is, position points that actually exist in the geographical environment.
[0071] For example, the world coordinate system can be pre-established in the following manner:
[0072] A world coordinate system is established with the center point of the vehicle's front axle or the mapping point of the vehicle's center on the ground as the origin, the vehicle's forward direction as the X-axis, the direction perpendicular to the vehicle's forward direction as the Y-axis, and the direction pointing to the sky as the Z-axis. After obtaining the world coordinate system, how to obtain the world coordinates of multiple position points on each lane line in the world coordinate system will be described in detail later.
[0073] S102, determining a current homography matrix of the image acquisition device based on the position information of each lane line in the road surface image, the world coordinates of multiple position points, and the initial homography matrix of the image acquisition device.
[0074] Exemplarily, the position information of each lane line in the road surface image can be represented by the positions of the pixel points that constitute the lane line in the road surface image. For example, the pixel points that constitute the lane line can be screened out from the road surface image through image recognition technology, and the pixel coordinates of these pixel points in the road surface image can be further obtained. Then, the straight line equation of the lane line in the road surface image can be determined by the pixel coordinates corresponding to these pixel points, and the position information of the lane line in the road surface image can be represented by the straight line equation.
[0075] Exemplarily, the initial homography matrix can be a homography matrix calibrated before the vehicle travels. Here, the homography matrix calibrated before the vehicle travels can be calibrated by currently known calibration methods, such as placing a reference object on a stationary road surface, and determining the homography matrix of the image acquisition device on the stationary vehicle through the geographical location of the reference object in the world coordinate system and the pixel coordinates of the reference object in the image acquired by the image acquisition device.
[0076] As the vehicle travels, the relative positional relationship between the image acquisition device and the vehicle may change. Correspondingly, the homography matrix obtained by calibrating the image acquisition device before the vehicle travels will no longer be accurate. The disclosed embodiment can calibrate the image acquisition device on the vehicle by continuously acquiring road surface images containing multiple lane lines as the vehicle travels. Each calibration process of the image acquisition device can be called a correction process of the homography matrix obtained by the last calibration of the image acquisition device. Therefore, the initial homography matrix here can also be the homography matrix after the last correction.
[0077] Exemplarily, based on the initial homography matrix of the image acquisition device, the world coordinates of multiple position points can be transformed to obtain the corresponding positions of the multiple position points in the road surface image. If the initial homography matrix is accurate, the straight lines corresponding to the corresponding positions of the multiple position points in the road surface image should coincide with the position information of the lane lines in the road surface image. Otherwise, the initial homography matrix is no longer accurate. At this time, the initial homography matrix can be adjusted based on the corresponding positions of the multiple position points in the road surface image and the position information of the lane lines in the road surface image to obtain the current homography matrix of the image acquisition device.
[0078] The calibration method proposed in S101 to S102 above, after obtaining the road surface image containing multiple lane lines captured by the image acquisition device and the world coordinates of multiple position points on each lane line in the world coordinate system, can complete the adjustment of the initial homography matrix through the position information of each lane line in the road surface image and the world coordinates of the multiple position points, and determine the current homography matrix of the image acquisition device; illustratively, the image acquisition device can be set on a moving vehicle, so that as the vehicle moves, the road surface image is always the road surface image acquired by the image acquisition device while maintaining the current relative position relationship with the vehicle. In this way, when the road surface image contains multiple lane lines, the initial homography matrix of the image acquisition device can be continuously corrected, thereby alleviating the problem that the homography matrix is no longer accurate due to mechanical vibration during vehicle driving or uneven road surface on which the vehicle is driving.
[0079] The above-mentioned processes S101 to S102 will be analyzed below in conjunction with specific embodiments.
[0080] Regarding the above S101, if Figure 2 As shown, the world coordinates of multiple position points on each lane line can be obtained in the following manner, specifically including S201 to S202:
[0081] S201, determining the world coordinates of the intersection of the target line segment in the road surface image and each lane line in the world coordinate system;
[0082] S202, based on the world coordinates of each intersection point, obtaining the world coordinates of a plurality of position points on each lane line in the world coordinate system.
[0083] A large number of test results show that the closer the position point is to the origin in the world coordinate system, the pixel coordinates obtained after projecting it to the road image according to the homography matrix. As the homography matrix changes, the deviation in the world coordinate system is smaller when it is projected back to the world coordinate system according to the changed homography matrix. Therefore, the target line segment can be generated based on this, so that the distance between the endpoint of the target line segment and the origin in the world coordinate system is less than the preset distance threshold. In this way, the distance between the intersection point determined based on the target line segment in the road image and each lane line and the origin in the world coordinate system is also less than the preset distance threshold. Therefore, when the intersection point in the road image is converted to the world coordinate system based on the initial homography matrix, the deviation of the obtained intersection point in the world coordinate system will also be very small. It can be considered that the intersection point is located on the lane line in the world coordinate system. Based on this, the world coordinates of multiple position points on each lane line in the world coordinate system can be obtained based on the world coordinates of each intersection point.
[0084] Specifically, with respect to the above S201, when determining the world coordinates of the intersection of the target line segment in the road surface image and each lane line in the world coordinate system, the following S2011 to S2012 are included:
[0085] S2011, selecting multiple initial position points in the world coordinate system, wherein the distance between each initial position point and the origin of the world coordinate system is less than a preset distance threshold.
[0086] As described above, the pixel coordinates of the position point that is closer to the origin of the coordinate system in the world coordinate system are obtained by projecting it onto the road image according to the homography matrix. As the homography matrix changes, the deviation in the world coordinate system becomes smaller when it is projected back to the world coordinate system according to the changed homography matrix. Therefore, the position point whose distance from the origin of the world coordinate system is less than a preset distance threshold is selected as the initial position point. Specifically, after the world coordinate system is established, multiple initial position points can be selected according to the origin of the world coordinate system and the preset distance threshold.
[0087] For example, here we can explain that the initial pixel coordinates corresponding to the initial position point in the road surface image have a smaller deviation when projected to the world coordinate system as the posture of the image acquisition device changes:
[0088] In a road image, as the posture of the image acquisition device changes, the offset of the physical distance corresponding to the pixel point in the road image corresponding to a near position point (a position point whose distance from the origin in the world coordinate system is less than a preset distance threshold) in the world coordinate system is smaller than the offset of the physical distance corresponding to the pixel point in the road image corresponding to a far position point (a position point whose distance from the origin in the world coordinate system is greater than or equal to the preset distance threshold) in the world coordinate system, wherein the physical distance may specifically include the distance between the horizontal axis in the world coordinate system and the distance between the vertical axis in the world coordinate system, for example, the physical distance between the pixel point corresponding to the near position point and the vertical axis in the world coordinate system is 5 meters or even smaller, and the physical distance between the pixel point corresponding to the far position point and the vertical axis in the world coordinate system is 20 meters or even larger, then after the homography matrix changes, it has been proved according to multiple experiments that the offset of the physical distance of the pixel point corresponding to the near position point in the world coordinate system is smaller than the physical offset of the pixel point corresponding to the far position point in the world coordinate system.
[0089] Taking a specific test result as an example, the pixel coordinates corresponding to a position point M 5m away from the ordinate axis in the world coordinate system in the road image are (400, 500). When the posture of the image acquisition device changes, the pixel coordinates corresponding to the position point M in the road image in the world coordinate system may become (400, 510). At this time, if the pixel coordinates (400, 500) determined by the image acquisition device before the posture change is still used as the pixel coordinates of the position point M, although the pixel coordinates (400, 510) of the position point M after the homography matrix changes and the pixel coordinates (400, 500) of the position point M before the homography matrix changes differ by 10 pixels in the ordinate direction, the corresponding physical distance has a deviation of only 50cm. Therefore, the pixel coordinates of the position point in the road image that is closer to the origin in the world coordinate system have a smaller deviation in the corresponding physical distance in the world coordinate system as the posture of the image acquisition device changes.
[0090] S2012, based on the initial homography matrix, determine a target line segment formed by connecting initial pixel coordinates corresponding to a plurality of initial position points in the road surface image, and a world coordinate of an intersection point between the target line segment and each lane line in a world coordinate system.
[0091] Exemplarily, based on the initial homography matrix, the initial pixel coordinates corresponding to multiple initial position points in the road surface image can be determined, and further, the target line segment can be obtained by straight line fitting based on the multiple initial pixel coordinates. The intersection of the target line segment and each lane line in the road surface image can be detected by image recognition technology, and can also be determined by the straight line equations corresponding to the target line segment and the lane line respectively.
[0092] Exemplarily, after obtaining the multiple initial position points, the initial pixel coordinates corresponding to the multiple initial position points in the road surface image may be obtained according to the initial homography matrix.
[0093] Then, based on the initial pixel coordinates, the target line segment is connected. After obtaining the target line segment, the pixel coordinates of the intersection between the target line segment and each lane line in the image coordinate system corresponding to the road surface image can be further determined. The pixel coordinates of the intersection can be used to represent the position information of the intersection in the road surface image. Since the selected initial position point is a position point that is close to the origin of the world coordinate system in the world coordinate system, the world coordinates of the intersection obtained through the initial homography matrix in the world coordinate system can also be approximately used to represent the intersection of the line segment formed by the initial position point on the road surface and the lane line, that is, the world coordinates of the intersection between the target line segment and each lane line in the road surface image in the world coordinate system can be approximately considered to be located on the lane line in the world coordinate system.
[0094] The disclosed embodiment selects an initial position point that is relatively close to the origin in the world coordinate system, so that the error of the physical distance corresponding to the initial pixel coordinates of the initial position point in the image coordinate system obtained based on the initial homography matrix is relatively small, and thus the error of the world coordinates corresponding to the intersection point obtained based on the initial pixel coordinates in the world coordinate system is also relatively small.
[0095] Specifically, with respect to the above S2012, when determining, based on the initial homography matrix, a target line segment formed by connecting the initial pixel coordinates corresponding to the plurality of initial position points in the road surface image, and the world coordinates of the intersection of the target line segment and each lane line in the world coordinate system, the following S20121 to S20124 are included:
[0096] S20121, based on the initial homography matrix and the initial position coordinates of each initial position point in the world coordinate system, determine the initial pixel coordinates corresponding to each initial position point in the road surface image.
[0097] Specifically, here we can use the world coordinate matrix formed by the world coordinates of multiple initial position points in the world coordinate system as a known quantity, use the initial homography matrix of the image acquisition device as a known quantity, and then substitute it into the conversion equation between pixel coordinates and world coordinates to obtain the initial pixel coordinates corresponding to the multiple initial position points.
[0098] S20122, performing straight line fitting on the initial pixel coordinates corresponding to the multiple initial position points to obtain a target line segment.
[0099] Exemplarily, the multiple initial position points include two initial position points, and the target line segment is a line segment formed by connecting initial pixel coordinates corresponding to the two initial position points.
[0100] S20123, obtaining the pixel coordinates of the intersection points between the target line segment and each lane line in the road surface image.
[0101] Exemplarily, when determining the pixel coordinates of the intersection, the pixel coordinates of the intersection in the image coordinate system corresponding to the road surface image can be obtained based on the straight line equation corresponding to the target line segment in the road surface image and the straight line equation corresponding to each lane line in the road surface image. Alternatively, the road surface image can be directly input into a pre-established pixel coordinate recognition model to determine the intersection pixel coordinates corresponding to each intersection.
[0102] Here, the pixel coordinate determination model may first perform image recognition based on the road surface image to determine the position of the intersection in the road surface image, and then determine the intersection pixel coordinates of these intersections in the image coordinate system based on the determined position of each intersection in the road surface image.
[0103] S20124, determining the world coordinates of the intersection point in the world coordinate system based on the intersection pixel coordinates of the intersection point and the initial homography matrix.
[0104] Specifically, here we can construct a pixel coordinate matrix based on the pixel coordinates of the intersection points in the image coordinate system as a known quantity, take the initial homography matrix of the image acquisition device as a known quantity, and then substitute it into the conversion equation between the pixel coordinates and the world coordinates to obtain the world coordinates of the multiple intersection points in the world coordinate system.
[0105] Further, with respect to the above S202, when obtaining the world coordinates of multiple position points on each lane line in the world coordinate system based on the world coordinates of each intersection point, it may include:
[0106] Based on the world coordinates of each intersection point and a preset position interval, the world coordinates of a plurality of position points located on the same lane line as each intersection point in the world coordinate system are determined.
[0107] According to the above-mentioned world coordinates of the intersection point, it is located on the linear equation corresponding to the lane line in the world coordinate system, and the linear equation corresponding to the lane line in the world coordinate system can be determined. For example, when multiple lane lines are parallel lane lines and the vehicle is traveling between the parallel lane lines, the linear equation corresponding to the lane line in the world coordinate system may be a straight line parallel to the vehicle's driving direction. If the vehicle's driving direction is the X-axis direction of the world coordinate system, the linear equation of the lane line in the world coordinate system may be determined according to the world coordinates of the intersection point in the world coordinate system. For example, the world coordinate system of the intersection point corresponding to a certain lane line L is (a, b), then the linear equation corresponding to the lane line L in the world coordinate system may be recorded as y=b. Further, according to a preset position interval, such as an interval of 5 meters, that is, based on the world coordinates of the position points on the same lane line as the intersection point and 5 meters, 10 meters, 20 meters and 25 meters away from the world coordinates of the intersection point, the world coordinates of multiple position points on the same lane line as each intersection point may be obtained in the same manner.
[0108] After obtaining the world coordinate system of multiple position points on each lane line, the current homography matrix of the image acquisition device can be further determined. Specifically, for the above S102, when determining the current homography matrix of the image acquisition device based on the position information of each lane line in the road surface image, the world coordinates of the multiple position points and the initial homography matrix of the image acquisition device, as follows: Figure 3 As shown, the following S301 to S303 may be included:
[0109] S301, determining initial pixel coordinates of the plurality of position points in the road surface image based on the world coordinates of the plurality of position points and an initial homography matrix.
[0110] Exemplarily, the initial pixel coordinates of multiple position points on each lane line in the image coordinate system where the road surface image is located can be determined based on the world coordinates of the multiple position points and the initial homography matrix. Specifically, the world coordinate matrix formed by the world coordinates of the multiple position points in the world coordinate system can be taken as a known quantity, and the initial homography matrix of the image acquisition device can be taken as a known quantity, and then substituted into the conversion equation between pixel coordinates and world coordinates to obtain the initial pixel coordinates of the multiple position points in the image coordinate system where the road surface image is located.
[0111] S302: Based on the position information of each lane line in the road surface image, the initial pixel coordinates corresponding to the multiple position points on the lane line are corrected to obtain the corrected pixel coordinates of the multiple position points on the lane line in the road surface image.
[0112] For example, if the initial homography matrix is no longer accurate, there will be deviations between the initial pixel coordinates of the multiple position points and the position information of the lane lines in the road surface image. Based on this idea, the calibration method proposed in the embodiment of the present disclosure can correct the initial homography matrix according to the deviation.
[0113] Exemplarily, the position information of each lane line in the road surface image can be represented by a straight line equation of the lane line in the image coordinate system corresponding to the road surface image.
[0114] Exemplarily, after determining that the initial pixel coordinates of multiple position points on each lane line deviate from the lane line, the initial pixel coordinates of the multiple position points on the lane line can be corrected to obtain the corrected pixel coordinates of the multiple position points on the lane line in the road surface image.
[0115] By correcting the initial pixel coordinates, the positional relationship between the corrected pixel coordinates corresponding to the multiple position points on each lane line obtained here and the lane line in the road surface image can be consistent with the positional relationship between the world coordinates of the multiple position points on the lane line and the lane line in the world coordinate system, so that the current homography matrix of the image acquisition device can be further determined based on the corrected pixel coordinates of the multiple position points on the lane line and the world coordinates of the multiple position points.
[0116] S303, obtaining a current homography matrix of the image acquisition device based on the corrected pixel coordinates and world coordinates of multiple position points on each lane line.
[0117] Specifically, a pixel coordinate matrix can be constructed based on the corrected pixel coordinates of multiple position points on each lane line, and a world coordinate matrix can be constructed based on the world coordinates of multiple position points on each lane line in the world coordinate system. Then, the pixel coordinate matrix and the world coordinate matrix are used as known quantities, and the current homography matrix of the image acquisition device is used as an unknown quantity. They are substituted into the conversion equation of the pixel coordinates and the world coordinates to determine the current homography matrix of the image acquisition device, which will be described in detail later.
[0118] In the disclosed embodiments, it is proposed that the initial pixel coordinates of multiple position points in the road surface image can be determined based on the world coordinates of the multiple position points and the initial homography matrix. When the initial homography matrix is no longer accurate, the initial pixel coordinates can be quickly corrected based on the initial pixel coordinates of the multiple position points and the position information of the lane lines in the road surface image, so as to quickly determine the current homography matrix of the image acquisition device based on the corrected pixel coordinates.
[0119] Specifically, with respect to the above S302, when the initial pixel coordinates corresponding to the multiple position points on the lane line are corrected based on the position information of each lane line in the road surface image to obtain the corrected pixel coordinates of the multiple position points on the lane line in the road surface image, the following steps S3021 to S3022 may be included:
[0120] S3021, based on the position information of each lane line in the road surface image and the initial pixel coordinates corresponding to the multiple position points on the lane line, determine the deviation angle between the fitting straight line of the multiple initial pixel coordinates and the fitting straight line corresponding to the lane line in the road surface image.
[0121] Here, as the vehicle travels, when the relative position of the image acquisition device and the vehicle changes, the initial homography matrix of the image acquisition device may no longer be accurate, and the initial pixel coordinates of multiple position points on the lane line determined based on the initial homography matrix are no longer accurate, resulting in the fitting straight lines of these initial pixel coordinates and the fitting straight lines corresponding to the lane line in the road surface image may not coincide with each other. In this way, there is a deviation angle between the fitting straight lines corresponding to the multiple initial pixel coordinates and the fitting straight line corresponding to the lane line in the road surface image.
[0122] Specifically, the fitting straight line corresponding to the lane line in the road surface image is the perceived lane line in the road surface image, that is, lane line detection is performed on the road surface image to obtain multiple pixel points constituting the lane line, and then straight line fitting is performed based on the pixel coordinates corresponding to the multiple pixel points to obtain the fitting straight line corresponding to the lane line in the road surface image.
[0123] Specifically, when determining the deviation angle between the fitting straight line of the multiple initial pixel coordinates and the fitting straight line corresponding to the lane line in the road surface image based on the position information of each lane line in the road surface image and the determined initial pixel coordinates corresponding to the multiple position points on the lane line, the following S30211 to S30213 may be included:
[0124] S30211, generating a first lane line equation in an image coordinate system corresponding to each lane line in the road surface image based on position information of each lane line in the road surface image;
[0125] S30212, performing straight line fitting on initial pixel coordinates corresponding to a plurality of position points on each lane line to generate a second lane line equation corresponding to each lane line;
[0126] S30213: Based on the first lane line equation and the second lane line equation corresponding to each lane line, determine the deviation angle of the straight line represented by the second lane line equation relative to the straight line represented by the first lane line equation.
[0127] When determining the first lane line equation in the image coordinate system corresponding to each lane line, multiple pixel points of the lane line in the image coordinate system can be first identified, and then based on the multiple pixel points, the first lane line equation in the image coordinate system corresponding to the lane line can be determined by the least squares method.
[0128] Similarly, when performing straight line fitting on the initial pixel coordinates of multiple position points on each lane line in the image coordinate system, the straight line fitting can also be performed according to the least squares method. Specifically, the second lane line equation corresponding to the lane line can be obtained according to the following formulas (1), (2) and (3):
[0129]
[0130]
[0131]
[0132] Among them, (x i ,y i ) represents the initial pixel coordinates of the i-th position point on the same lane line; n represents the number of multiple position points on the same lane line; Represents the average value of the horizontal coordinate values in the initial pixel coordinates of multiple position points on the lane line; represents the average value of the ordinate values in the initial pixel coordinates of multiple position points on the lane line; b, b 0 and b 1 represents the unknown parameter in the equation of the second line.
[0133] Substituting the initial pixel coordinates of multiple points on the lane line into the above formulas (1) to (3), the unknown parameter b in the second straight line equation corresponding to the lane line is obtained. 0 and b 1 Then, we can get the equation of the second straight line corresponding to the lane line: y = b 1 xb 0 .
[0134] Similarly, in the same way, the second straight line equation corresponding to each lane line can be obtained.
[0135] Furthermore, after obtaining the first straight line equation and the second straight line equation corresponding to each lane line, the deviation angle of the straight line represented by the second lane line equation relative to the straight line represented by the first lane line equation can be determined based on the slope of the straight line equation.
[0136] S3022: Based on the determined deviation angle, correct the initial pixel coordinates corresponding to the multiple position points on the lane line to obtain the corrected pixel coordinates of the multiple position points on the lane line in the road surface image.
[0137] After obtaining the deviation angle between the straight line where the initial pixel coordinates of the multiple position points on each lane line are located and the straight line where the lane line is located, the initial pixel coordinates of the multiple position points can be further corrected according to the deviation angle. Specifically, the initial pixel coordinates of the multiple position points can be corrected by the following formula (4) and formula (5):
[0138]
[0139]
[0140] Among them, (x i ,y i ) represents the initial pixel coordinate of the i-th position point among multiple position points on any lane line, (x i ',y i ') represents the corrected pixel coordinates of the ith position point; n represents the number of multiple position points on the lane line; θ represents the deviation angle of the straight line represented by the second lane line equation corresponding to any lane line relative to the straight line represented by the first lane line equation; M(θ) represents the corresponding rotation matrix when the straight line represented by the first lane line equation corresponding to any lane line is rotated to the straight line represented by the second lane line equation.
[0141] In the same way, the corrected pixel coordinates of multiple position points on each lane line in the road surface image can be obtained.
[0142] In the disclosed embodiment, the initial pixel coordinates of multiple position points on each lane line in the image coordinate system are determined based on the initial homography matrix. When the initial homography matrix is no longer accurate, the initial pixel coordinates of the multiple position points in the image coordinate system determined based on the initial homography matrix will deviate from the corresponding lane line. By determining the deviation angle between the initial pixel coordinates of the multiple position points on each lane line in the image coordinate system and the lane line, the initial pixel coordinates of the multiple position points on the lane line can be corrected based on the deviation angle. In this way, the corrected pixel coordinates of the multiple position points on the lane line in the image coordinate system can be determined more accurately.
[0143] With respect to the above S303, after obtaining the corrected pixel coordinates of the multiple position points on each lane line in the road surface image, the current homography matrix of the image acquisition device can be determined based on the corrected pixel coordinates of the multiple position points on each lane line and the world coordinates. Specifically, the current homography matrix of the image acquisition device can be determined according to the following process:
[0144] The world coordinates of multiple position points on each lane line in the world coordinate system are: (X 1 ,Y 1 )~(X N ,Y N ), where N represents the total number of position points on the lane line in the road image, the recorded world coordinate matrix is A, the corrected pixel coordinate matrix is C, and the current homography matrix of the image acquisition device is B, which is specifically expressed as follows:
[0145]
[0146] Then, the world coordinate matrix A, the corrected pixel coordinate matrix C and the current homography matrix B are substituted into the transformation equation between pixel coordinates and world coordinates. The transformation equation is expressed by the following formula (6):
[0147] A = B × C (6);
[0148] Solving the transformation equation, we can obtain the current homography matrix B of the image acquisition device = (AA T )*(CA T ) -1 .
[0149] The following example takes a road image containing two lane lines as an example. Figures 4a to 4c A specific implementation method for determining the current homography matrix of an image acquisition device is given:
[0150] like Figure 4a As shown in FIG. 1 , the image of the road surface is obtained by the image acquisition device installed on the moving vehicle. The lane lines in the road surface image are identified to obtain two lane lines, namely lane line L1 and lane line L2. Then, two position points close to the origin of the world coordinate system are selected, and the initial pixel coordinates of the two position points in the image coordinate system where the road surface image is located are obtained according to the initial homography matrix, that is, Figure 4a The initial pixel coordinates of A1 and A2 in the image coordinate system are obtained; then the intersection point B1 of the target line segment formed by A1 and A2 with the lane line L1 and the intersection point C1 with the lane line L2 are determined.
[0151] The initial pixel coordinates of the intersection points B1 and C1 are transformed according to the initial homography matrix to obtain the world coordinates of the intersection points B1 and C1 in the world coordinate system. Considering that A1 and A2 are obtained by selecting position points close to the origin of the world coordinate system and the initial homography matrix, the world coordinates of the intersection points B1 and C1 in the world coordinate system here can still be considered to be located on the straight line equations corresponding to the lane lines L1 and L2 on the road surface. Then, according to the straight line equations of the lane lines L1 and L2 in the world coordinate system, the world coordinates of the intersection points B1 and C1 in the world coordinate system can continue to obtain multiple other position points on the lane line L1 on the road surface and multiple other position points on the lane line L2 on the road surface.
[0152] Then, based on the initial homography matrix, we can obtain the initial pixel coordinates of multiple position points on the lane line L1 in the image coordinate system where the road surface image is located, that is, Figure 4b The initial pixel coordinates of B2, B3, B4 and B5 in the image coordinate system are obtained, and correspondingly, the initial pixel coordinates of multiple position points on the lane line L2 in the image coordinate system where the road surface image is located are obtained, that is, Figure 4b where C2, C3, C4 and C5 are the initial pixel coordinates in the image coordinate system.
[0153] Then, straight line fitting is performed based on B1, B2, B3, B4 and B5 to obtain straight line L1', and straight line fitting is performed based on C1, C2, C3, C4 and C5 to obtain straight line L2'. Then, in the image coordinate system, the deviation angle θ1 of straight line L1' relative to straight line L1 and the deviation angle θ2 of straight line L2' relative to straight line L2 are further determined. Then, according to the above-mentioned formulas (4) and (5), the initial pixel coordinates of B1, B2, B3, B4 and B5 are corrected to obtain: Figure 4c The corrected pixel coordinates of B1', B2', B3', B4' and B5' are shown, and the initial pixel coordinates of C1, C2, C3, C4 and C5 are corrected to obtain the following: Figure 4c The corrected pixel coordinates of C1', C2', C3', C4' and C5' are shown.
[0154] Then, the world coordinates of multiple position points on lane line L1 and lane line L2 in the world coordinate system and the corrected pixel coordinates in the image coordinate system are substituted into the conversion equation between pixel coordinates and world coordinates to obtain the current homography matrix of the image acquisition device.
[0155] Furthermore, after obtaining the current homography matrix of the image acquisition device, the distance of the target object can be measured based on the current homography matrix, such as Figure 5As shown, it is a flow chart of the ranging method provided by the embodiment of the present disclosure, which specifically includes the following steps S501 to S504:
[0156] S501, obtaining a target image obtained by an image acquisition device after photographing a target object;
[0157] S502, based on the target image, determining the pixel coordinates of the target object in the target image;
[0158] S503, determining the world coordinates of the target object in the world coordinate system based on the pixel coordinates and the current homography matrix;
[0159] S504: Determine the distance between the target object and the preset position point based on the world coordinates of the target object in the world coordinate system and the coordinates of the preset position point in the world coordinate system.
[0160] Taking a vehicle as an example, the preset position point here can be the projection of the center point of the front axle of the vehicle on the ground, or the projection of the center of the vehicle body on the ground. When it is used as the origin of the world coordinate system, the coordinates of the origin in the world coordinate system are known. The preset position point can be used as the corresponding vehicle ranging point when measuring the distance between the target object and the vehicle.
[0161] The entire process of S501 to S504 refers to the process of measuring distance through the current homography matrix after obtaining the current homography matrix of the image acquisition device. Because the target object in the target object image has an area size, after obtaining the target object image, the distance measurement point of the target object must be determined based on the target object image, and then the distance between the target object and the vehicle is determined based on the world coordinates of the distance measurement point and the preset position point in the world coordinate system.
[0162] Specifically, after obtaining the target image where the target object is located, based on image recognition technology, the annotation box where the target object is located is obtained. For example, the center point of the tangent line between the annotation box and the ground can be used as the ranging point, and then the pixel coordinates of the ranging point are used as the pixel coordinates of the target object in the image coordinate system.
[0163] After obtaining the pixel coordinates of the target object in the image coordinate system, the pixel coordinates of the target object in the image coordinate system and the current homography matrix are input into the conversion equation between the pixel coordinates and the world coordinates, and the world coordinates of the target object in the world coordinate system can be obtained. Then, based on the world coordinates of the target object in the world coordinate system and the world coordinates of the preset position point, the Euclidean distance between the two is calculated to determine the distance between the target object and the vehicle.
[0164] After obtaining the current homography matrix of the image acquisition device, the embodiment of the present disclosure can use the current homography matrix to accurately determine the world coordinates of the target object in the world coordinate system, and then determine the distance to the target object.
[0165] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.
[0166] Based on the same technical concept, a calibration device corresponding to the calibration method is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned calibration method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.
[0167] Reference Figure 6 FIG. 6 is a schematic diagram of a calibration device 600 provided in an embodiment of the present disclosure, and the calibration device 600 includes:
[0168] An acquisition module 601 is used to acquire a road surface image including a plurality of lane lines acquired by an image acquisition device, and world coordinates of a plurality of position points on each lane line in a world coordinate system;
[0169] The determination module 602 is used to determine the current homography matrix of the image acquisition device based on the position information of each lane line in the road surface image, the world coordinates of multiple position points and the initial homography matrix of the image acquisition device.
[0170] In a possible implementation, the acquisition module 601 is used to acquire the world coordinates of multiple position points on each lane line in the following manner:
[0171] Determine the world coordinates of the intersection of the target line segment in the road surface image and each lane line in the world coordinate system;
[0172] Based on the world coordinates of each intersection point, the world coordinates of multiple position points on each lane line in the world coordinate system are obtained.
[0173] In a possible implementation, when the acquisition module 601 is used to determine the world coordinates of the intersection of the target line segment in the road surface image and each lane line in the world coordinate system, it includes:
[0174] Selecting multiple initial position points in the world coordinate system, where the distance between each initial position point and the origin of the world coordinate system is less than a preset distance threshold;
[0175] Based on the initial homography matrix, determine the target line segment formed by connecting the initial pixel coordinates corresponding to the multiple initial position points in the road surface image, and the world coordinates of the intersection of the target line segment and each lane line in the world coordinate system.
[0176] In a possible implementation, when the acquisition module 601 is used to acquire the world coordinates of multiple position points on each lane line in the world coordinate system based on the world coordinates of each intersection point, it includes:
[0177] Based on the world coordinates of each intersection point and a preset position interval, the world coordinates of multiple position points located on the same lane line as each intersection point in the world coordinate system are obtained.
[0178] In a possible implementation, the acquisition module 601, when used to determine, based on the initial homography matrix, a target line segment formed by connecting initial pixel coordinates corresponding to a plurality of initial position points in the road surface image, and a world coordinate of an intersection of the target line segment and each lane line in a world coordinate system, includes:
[0179] Based on the initial homography matrix and the initial position coordinates of each initial position point in the world coordinate system, determining the initial pixel coordinates corresponding to each initial position point in the road surface image;
[0180] Perform straight line fitting on the initial pixel coordinates corresponding to multiple initial position points to obtain the target line segment;
[0181] Obtain the pixel coordinates of the intersection points between the target line segment and each lane line in the road surface image;
[0182] Based on the intersection pixel coordinates of the intersection point and the initial homography matrix, the world coordinates of the intersection point in the world coordinate system are determined.
[0183] In a possible implementation, the determination module 602, when used to determine the current homography matrix of the image acquisition device based on the position information of each lane line in the road surface image, the world coordinates of the plurality of position points, and the initial homography matrix of the image acquisition device, includes:
[0184] Determine initial pixel coordinates of the multiple location points in the road surface image based on the world coordinates of the multiple location points and the initial homography matrix;
[0185] Based on the position information of each lane line in the road surface image, the initial pixel coordinates corresponding to the multiple position points on the lane line are corrected to obtain the corrected pixel coordinates of the multiple position points on the lane line in the road surface image;
[0186] Based on the corrected pixel coordinates of multiple position points on each lane line and the world coordinates, the current homography matrix of the image acquisition device is obtained.
[0187] In a possible implementation, the determination module 602, when used to correct the initial pixel coordinates corresponding to multiple position points on the lane line based on the position information of each lane line in the road surface image to obtain the corrected pixel coordinates of the multiple position points on the lane line in the road surface image, includes:
[0188] Based on the position information of each lane line in the road surface image and the determined initial pixel coordinates corresponding to the multiple position points on the lane line, determine the deviation angle between the fitting straight line of the multiple initial pixel coordinates and the fitting straight line corresponding to the lane line in the road surface image;
[0189] Based on the determined deviation angle, the initial pixel coordinates corresponding to the multiple position points on the lane line are corrected to obtain the corrected pixel coordinates of the multiple position points on the lane line in the road surface image.
[0190] In a possible implementation, the determination module 602 is used to determine the deviation angle between the fitting straight line of the multiple initial pixel coordinates and the fitting straight line corresponding to the lane line in the road surface image based on the position information of each lane line in the road surface image and the determined initial pixel coordinates corresponding to the multiple position points on the lane line, including:
[0191] Based on the position information of each lane line in the road surface image, a first lane line equation in an image coordinate system corresponding to each lane line in the road surface image is generated;
[0192] Perform straight line fitting on the initial pixel coordinates corresponding to multiple position points on each lane line to generate a second lane line equation corresponding to each lane line;
[0193] Based on the first lane line equation and the second lane line equation corresponding to each lane line, a deviation angle of the straight line represented by the second lane line equation relative to the straight line represented by the first lane line equation is determined.
[0194] In a possible implementation, after determining the current homography matrix of the image acquisition device, the determination module 602 is further configured to:
[0195] Acquire a target image obtained after the image acquisition component photographs the target object;
[0196] Based on the target image, determining the pixel coordinates of the target object in the target image;
[0197] Based on the pixel coordinates and the current homography matrix, the world coordinates of the target object in the world coordinate system are determined.
[0198] In a possible implementation manner, after determining the world coordinates of the target object in the world coordinate system, the determination module 602 is further configured to:
[0199] Based on the world coordinates of the target object in the world coordinate system and the coordinates of the preset position point in the world coordinate system, the distance between the target object and the preset position point is determined.
[0200] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.
[0201] Corresponds to Figure 1 The present disclosure also provides an electronic device 700, such as Figure 7 FIG. 7 is a schematic diagram of the structure of an electronic device 700 provided in an embodiment of the present disclosure, including:
[0202] Processor 701, memory 702, and bus 703; memory 702 is used to store execution instructions, including internal memory 7021 and external memory 7022; the internal memory 7021 here is also called internal memory, which is used to temporarily store the calculation data in the processor 701, as well as the data exchanged with the external memory 7022 such as a hard disk. The processor 701 exchanges data with the external memory 7022 through the internal memory 7021. When the electronic device 700 is running, the processor 701 and the memory 702 communicate through the bus 703, so that the processor 701 executes the following instructions: obtain a road surface image including multiple lane lines captured by an image acquisition device, and the world coordinates of multiple position points on each lane line in the world coordinate system; determine the current homography matrix of the image acquisition device based on the position information of each lane line in the road surface image, the world coordinates of multiple position points and the initial homography matrix of the image acquisition device.
[0203] The embodiment of the present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the calibration method in the above method embodiment are executed. The storage medium can be a volatile or non-volatile computer-readable storage medium.
[0204] The present disclosure also provides a computer program product that carries a program code. The program code includes instructions that can be used to execute the steps of the calibration method described in the above method embodiment. For details, please refer to the above method embodiment, which will not be repeated here.
[0205] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is implemented as a computer storage medium. In another optional embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).
[0206] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.
[0207] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0208] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0209] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0210] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.
Claims
1. A calibration method, It is characterized in that include: Acquire a road surface image including a plurality of lane lines acquired by an image acquisition device, and world coordinates of a plurality of position points on each lane line in a world coordinate system; Determine a current homography matrix of the image acquisition device based on the position information of each lane line in the road surface image, the world coordinates of the plurality of position points, and an initial homography matrix of the image acquisition device; Obtain the world coordinates of multiple points on each lane line as follows: Selecting a plurality of initial position points in the world coordinate system, wherein the distance between each of the initial position points and the origin of the world coordinate system is less than a preset distance threshold; Based on the initial homography matrix and the initial position coordinates of each of the initial position points in the world coordinate system, determining the initial pixel coordinates corresponding to each of the initial position points in the road surface image; Performing straight line fitting on the initial pixel coordinates corresponding to the multiple initial position points to obtain a target line segment; Obtaining pixel coordinates of intersection points of the target line segment and each lane line in the road surface image; Determine the world coordinates of the intersection point in the world coordinate system based on the intersection pixel coordinates of the intersection point and the initial homography matrix; Based on the world coordinates of each of the intersections and a preset position interval, the world coordinates of a plurality of position points located on the same lane line as each of the intersections in the world coordinate system are obtained.
2. The calibration method according to claim 1, It is characterized in that The determining of the current homography matrix of the image acquisition device based on the position information of each lane line in the road surface image, the world coordinates of the plurality of position points and the initial homography matrix of the image acquisition device comprises: Determining initial pixel coordinates of the plurality of position points in the road surface image based on the world coordinates of the plurality of position points and the initial homography matrix; Based on the position information of each lane line in the road surface image, correct the initial pixel coordinates corresponding to multiple position points on the lane line to obtain corrected pixel coordinates of the multiple position points on the lane line in the road surface image; Based on the corrected pixel coordinates and the world coordinates of multiple position points on each lane line, a current homography matrix of the image acquisition device is obtained.
3. The calibration method according to claim 2, It is characterized in that The method of correcting the initial pixel coordinates corresponding to a plurality of position points on the lane line based on the position information of each lane line in the road surface image to obtain the corrected pixel coordinates of the plurality of position points on the lane line in the road surface image includes: Based on the position information of each lane line in the road surface image and the initial pixel coordinates corresponding to the multiple position points on the lane line, determine the deviation angle between the fitting straight line of the multiple initial pixel coordinates and the fitting straight line corresponding to the lane line in the road surface image; Based on the determined deviation angle, the initial pixel coordinates corresponding to the multiple position points on the lane line are corrected to obtain the corrected pixel coordinates of the multiple position points on the lane line in the road surface image.
4. The calibration method according to claim 3, It is characterized in that The determining, based on the position information of each lane line in the road surface image and the initial pixel coordinates corresponding to the multiple position points on the lane line, a deviation angle between a fitting straight line of the multiple initial pixel coordinates and a fitting straight line corresponding to the lane line in the road surface image includes: Based on the position information of each lane line in the road surface image, generating a first lane line equation in an image coordinate system corresponding to each lane line in the road surface image; Performing straight line fitting on the initial pixel coordinates corresponding to a plurality of position points on each lane line to generate a second lane line equation corresponding to each lane line; Based on the first lane line equation and the second lane line equation corresponding to each lane line, a deviation angle of a straight line represented by the second lane line equation relative to the straight line represented by the first lane line equation is determined.
5. The calibration method according to any one of claims 1 to 4, It is characterized in that After determining the current homography matrix of the image acquisition device, the method further includes: Acquire a target image obtained after the image acquisition component photographs the target object; Based on the target image, determining the pixel coordinates of the target object in the target image; Based on the pixel coordinates and the current homography matrix, the world coordinates of the target object in the world coordinate system are determined.
6. The calibration method according to claim 5, It is characterized in that After determining the world coordinates of the target object in the world coordinate system, the method further includes: Based on the world coordinates of the target object in the world coordinate system and the coordinates of the preset position point in the world coordinate system, the distance between the target object and the preset position point is determined.
7. A calibration device, It is characterized in that include: An acquisition module, used to acquire a road surface image including a plurality of lane lines acquired by an image acquisition device, and world coordinates of a plurality of position points on each lane line in a world coordinate system; A determination module, configured to determine a current homography matrix of the image acquisition device based on the position information of each lane line in the road surface image, the world coordinates of the plurality of position points, and an initial homography matrix of the image acquisition device; The acquisition module is specifically used for: Selecting a plurality of initial position points in the world coordinate system, wherein the distance between each of the initial position points and the origin of the world coordinate system is less than a preset distance threshold; Based on the initial homography matrix and the initial position coordinates of each of the initial position points in the world coordinate system, determining the initial pixel coordinates corresponding to each of the initial position points in the road surface image; Performing straight line fitting on the initial pixel coordinates corresponding to the multiple initial position points to obtain a target line segment; Obtaining pixel coordinates of intersection points of the target line segment and each lane line in the road surface image; Determine the world coordinates of the intersection point in the world coordinate system based on the intersection pixel coordinates of the intersection point and the initial homography matrix; Based on the world coordinates of each of the intersections and a preset position interval, the world coordinates of a plurality of position points located on the same lane line as each of the intersections in the world coordinate system are obtained.
8. An electronic device, It is characterized in that include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the calibration method as described in any one of claims 1 to 6 are performed.
9. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the calibration method according to any one of claims 1 to 6 are executed.
