Target positioning method and device, computer readable storage medium and electronic device

By performing dislocation analysis and mapping relationship determination on lane lines and real-time adjustment of shooting height, the problem of low accuracy of traditional target positioning methods is solved, and more accurate target object positioning is achieved.

CN118429942BActive Publication Date: 2024-11-26CHENGDU TIANFU INVO TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410508840.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-04-25
Publication Date
2024-11-26
Estimated Expiration
2044-04-25

AI Technical Summary

Technical Problem

The traditional target positioning method has low accuracy in positioning target objects such as obstacles and cannot effectively adapt to changes in shooting height caused by vehicle load changes.

Method used

By performing misalignment analysis on the lane lines in the target image, the misalignment data of the lane lines are determined, and based on the mapping relationship between the misalignment data and the shooting height, the shooting height of the target shooting device is determined in real time, thereby calculating the accurate position of the target object.

Benefits of technology

提高了目标物体位置的精确度,适应车辆载重变化,减少了定位误差。

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118429942B_ABST
    Figure CN118429942B_ABST
Patent Text Reader

Abstract

The present application relates to the field of autonomous driving technology, and specifically to a target positioning method and device, as well as a computer-readable storage medium and an electronic device, which solves the problem that the traditional target positioning method has low positioning accuracy for target objects such as obstacles. The target positioning method first performs a misalignment analysis on the lane line in the target image to determine the misalignment data of the lane line, and then determines the shooting height of the target shooting device based on the misalignment data of the lane line and the mapping relationship between the misalignment data and the shooting height. Finally, the position of the target object is determined based on the shooting height of the target shooting device and the imaging parameters of the target shooting device, so that the accurate shooting height of the target shooting device can be obtained according to the mapping relationship between the misalignment data and the shooting height, and then the position of the target object is calculated using the accurate shooting height, thereby improving the accuracy of the position of the target object.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and specifically to a target positioning method and device, as well as a computer-readable storage medium and an electronic device. Background Art

[0002] In the field of autonomous driving technology, road obstacle location is a very important technology. Smart vehicles generally use on-board sensors such as cameras and radars to perceive the surrounding environment of the vehicle, and then use corresponding obstacle location algorithms to determine the location of obstacles around the vehicle, thereby controlling the vehicle to take corresponding measures to avoid obstacles. Among them, when using a camera to locate obstacles, the camera collects environmental images and identifies obstacles in the environmental images through image recognition technology.

[0003] However, traditional target positioning methods have low positioning accuracy for target objects such as obstacles. Summary of the invention

[0004] In view of this, the embodiments of the present application provide a target positioning method and device, as well as a computer-readable storage medium and an electronic device, which solve the problem that traditional target positioning methods have low positioning accuracy for target objects such as obstacles.

[0005] In a first aspect, an embodiment of the present application provides a target positioning method, including: performing a misalignment analysis on lane lines in a target image to determine misalignment data of the lane lines, wherein the target image is obtained based on multiple panoramic images, and the multiple panoramic images are images obtained by using a target shooting device to shoot the surrounding area of ​​a target vehicle, and the misalignment data is used to characterize the broken, misaligned and deflected conditions of the lane lines; based on the misalignment data of the lane lines and the mapping relationship between the misalignment data and the shooting height, determining the shooting height of the target shooting device; based on the shooting height of the target shooting device and the imaging parameters of the target shooting device, determining the position of the target object, wherein the target object is an object in the surrounding area of ​​the target vehicle.

[0006] In combination with the first aspect of the present application, in some embodiments, a lane line in a target image is subjected to a misalignment analysis to determine the misalignment data of the lane line, including: traversing the pixels corresponding to the lane line in the target image to determine the misalignment position of the lane line, wherein the misalignment position divides one lane line into two lane lines; determining a baseline lane line and a misaligned lane line in the two lane lines; determining an offset size between the misaligned lane line and the baseline lane line at the misaligned position; determining a deflection angle between the misaligned lane line and the baseline lane line; and determining the misalignment data of the lane line based on the offset size and the deflection angle.

[0007] In combination with the first aspect of the present application, in some embodiments, determining the offset size of the staggered lane line and the baseline lane line at the staggered position includes: determining the number of staggered pixels between one end of the staggered lane line close to the staggered position and one end of the baseline lane line close to the staggered position; based on the number of staggered pixels, determining the offset size at the staggered position.

[0008] In combination with the first aspect of the present application, in some embodiments, determining the deflection angle of the offset lane line and the baseline lane line includes: generating a plane rectangular coordinate system using a line parallel to the baseline lane line and a line perpendicular to the baseline lane line as coordinate axes of the plane rectangular coordinate system; calculating the slope of the offset lane line based on the plane rectangular coordinate system; and determining the deflection angle of the offset lane line and the baseline lane line based on the slope of the offset lane line.

[0009] In combination with the first aspect of the present application, in some embodiments, pixels corresponding to lane lines in a target image are traversed to determine the misaligned position of the lane lines, including: traversing pixels corresponding to lane lines in the target image, and for each current pixel traversed, determining the gradient of the current pixel; and determining the misaligned position of the lane line based on the gradient of each current pixel.

[0010] In combination with the first aspect of the present application, in some embodiments, the mapping relationship between the misalignment data and the shooting height is determined based on the following steps: determining multiple shooting heights of the shooting equipment on the calibration vehicle; for each current shooting height among the multiple shooting heights, performing a misalignment analysis on the lane line in the current calibration image, and determining the misalignment data of the lane line in the current calibration image, wherein the current calibration image is obtained based on multiple calibration panoramic images, and the multiple calibration panoramic images are images obtained by using the shooting equipment to shoot the surrounding area of ​​the calibration vehicle, and the misalignment data is used to characterize the broken, misaligned and deflected conditions of the lane line in the current calibration image; establishing a mapping relationship between the current shooting height and the misalignment data of the lane line in the current calibration image, and obtaining a mapping relationship between the current shooting height and the misalignment data corresponding to the current shooting height; based on the mapping relationship between multiple shooting heights and the misalignment data corresponding to each of the multiple shooting heights, determine the mapping relationship between the misalignment data and the shooting height.

[0011] In combination with the first aspect of the present application, in some embodiments, the target shooting device is a surround-view camera, and the multiple panoramic images are images obtained by shooting the surrounding area of ​​the target vehicle using the surround-view camera when the target vehicle's driving speed is less than a speed threshold.

[0012] In the second aspect, an embodiment of the present application provides a target positioning device, including: a misalignment data determination module, configured to perform misalignment analysis on lane lines in a target image, and determine misalignment data of the lane lines, wherein the target image is obtained based on multiple panoramic images, and the multiple panoramic images are images obtained by using a target shooting device to shoot the surrounding area of ​​a target vehicle, and the misalignment data is used to characterize the broken, misaligned and deflected conditions of the lane lines; a shooting height determination module, configured to determine the shooting height of the target shooting device based on the misalignment data of the lane lines and the mapping relationship between the misalignment data and the shooting height; a target object positioning module, configured to determine the position of the target object based on the shooting height of the target shooting device and the imaging parameters of the target shooting device, wherein the target object is an object in the surrounding area of ​​the target vehicle.

[0013] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, which stores instructions. When the instructions are executed by a processor of an electronic device, the electronic device can execute the target positioning method mentioned in the first aspect above.

[0014] In a fourth aspect, an embodiment of the present application provides an electronic device, the electronic device comprising: a processor; a memory for storing computer executable instructions; a processor, for executing computer executable instructions to implement the target positioning method mentioned in the first aspect above.

[0015] A target positioning method provided in an embodiment of the present application first performs a misalignment analysis on lane lines in a target image to determine misalignment data of the lane lines, then determines the shooting height of a target shooting device based on the misalignment data of the lane lines and a mapping relationship between the misalignment data and the shooting height, and finally determines the position of the target object based on the shooting height of the target shooting device and the imaging parameters of the target shooting device, so that the accurate shooting height of the target shooting device can be obtained based on the mapping relationship between the misalignment data and the shooting height, and then the position of the target object is calculated using the accurate shooting height, thereby improving the accuracy of the position of the target object. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 The figure is a schematic diagram of an application scenario of a target positioning method provided in an embodiment of the present application.

[0017] Figure 2 The figure is a flow chart of a target positioning method provided in one embodiment of the present application.

[0018] Figure 3 Shown is a schematic diagram of a target image provided by an embodiment of the present application.

[0019] Figure 4 Shown is a schematic diagram of a target image provided by another embodiment of the present application.

[0020] Figure 5 Shown is a schematic diagram of a target image provided by another embodiment of the present application.

[0021] Figure 6 The figure shows a schematic diagram of determining the mapping relationship between misalignment data and shooting height provided by an embodiment of the present application.

[0022] Figure 7 The figure is a schematic diagram of the mapping relationship between the misalignment data and the shooting height provided in one embodiment of the present application.

[0023] Figure 8 FIG. 1 is a schematic diagram of a method for calculating the position of a target object provided in one embodiment of the present application.

[0024] Fig. 9 FIG. 1 is a schematic diagram showing the relationship between the shooting height and the position of the target object provided in one embodiment of the present application.

[0025] Fig.10 The figure is a flow chart of a target positioning method provided in one embodiment of the present application.

[0026] Fig.11 Shown Figure 5 A magnified view of the target image at the misaligned position is shown.

[0027] Fig.12 Shown is a schematic structural diagram of a target positioning device provided in one embodiment of the present application.

[0028] Fig.13 Shown is a schematic structural diagram of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0030] Overview

[0031] With the development of visual technology, the requirements for image recognition accuracy are getting higher and higher. Using image recognition to infer the position of the target object has gradually become the mainstream. In automatic parking scenarios and low-speed automatic driving scenarios, the type and position of the target object are usually identified, and then the vehicle is controlled to take corresponding measures to achieve functions such as obstacle avoidance and obstacle stopping. The quality of obstacle avoidance and obstacle stopping functions depends to a certain extent on the positioning accuracy of the target object.

[0032] The traditional target positioning method uses image recognition combined with a fixed shooting height to estimate the position of the target object. However, in actual applications, the number of passengers and the weight of the passengers vary, so the load of the vehicle varies. The shooting height of the shooting equipment varies with the vehicle load. The heavier the vehicle load, the lower the height of the vehicle body suspension, and the shooting height of the camera fixed on the vehicle body will also be lower. In other words, the fixed shooting height is not necessarily the real shooting height. Therefore, the traditional target positioning method uses a fixed shooting height to estimate the position of the target object, which has errors, resulting in low accuracy in estimating the position of the target object using a fixed shooting height.

[0033] In view of the problem that the traditional target positioning method has low accuracy in locating obstacles, the embodiment of the present application provides a target positioning method, which first performs a misalignment analysis on the lane lines in the target image to determine the misalignment data of the lane lines, and then determines the shooting height of the target shooting device based on the misalignment data of the lane lines and the mapping relationship between the misalignment data and the shooting height, and finally determines the position of the target object based on the shooting height of the target shooting device and the imaging parameters of the target shooting device. In other words, the present application can determine the shooting height of the target shooting device in real time by performing a misalignment analysis on the lane lines in the target image, and then determines the shooting height of the target shooting device based on the misalignment data of the lane lines and the mapping relationship between the misalignment data and the shooting height, so that the position of the target object can be calculated based on the actual shooting height, thereby improving the accuracy of the position of the target object.

[0034] Example scenarios

[0035] Figure 1 The figure is a schematic diagram of an application scenario of a target positioning method provided in an embodiment of the present application. Figure 1 The scene shown includes a computing device 110, an image acquisition device 120 that is communicatively connected to the computing device 110, a control device 130, and a target vehicle 140. Specifically, the image acquisition device 120 captures a plurality of panoramic images and sends the plurality of panoramic images to the computing device 110. The computing device 110 is used to perform misalignment analysis on lane lines in the target image and determine misalignment data of the lane lines, wherein the target image is obtained based on the plurality of panoramic images, and the plurality of panoramic images are images obtained by using a target shooting device to capture the surrounding area of ​​the target vehicle, and the misalignment data is used to characterize the fracture, misalignment and deflection of the lane lines; based on the misalignment data of the lane lines and the mapping relationship between the misalignment data and the shooting height, the shooting height of the target shooting device is determined; based on the shooting height of the target shooting device and the imaging parameters of the target shooting device, the position of the target object is determined, wherein the target object is an object in the surrounding area of ​​the target vehicle, and the position of the target object is sent to the control device 130, and then the control device 130 controls the target vehicle 140 to avoid obstacles.

[0036] The computing device 110 may be a vehicle-mounted computing device, such as a computer, a tablet, a mobile phone, or other device with computing functions. The computing device 110 may be installed on the target vehicle, or may be a cloud, a server, or other device. The image acquisition device 120 may be a camera, such as a surround view camera. The control device 130 may be a controller.

[0037] Exemplary Methods

[0038] Figure 2 FIG. 1 is a flow chart of a target positioning method provided by an embodiment of the present application. Figure 2 As shown, the target positioning method includes the following steps.

[0039] Step 210, performing misalignment analysis on the lane lines in the target image to determine misalignment data of the lane lines.

[0040] The target image is obtained based on multiple panoramic images. Exemplarily, multiple panoramic images can be stitched together to obtain the target image. Multiple panoramic images are images obtained by photographing the surrounding area of ​​the target vehicle using a target shooting device. The target shooting device can be installed on the roof, side, front, rear, and other locations of the target vehicle. The surrounding area of ​​the target vehicle can be an area with the target vehicle as the center and a preset distance as the radius, and the preset distance can be determined according to the shooting distance of the target shooting device. For example, the shooting distance of the target shooting device is 10 meters, that is, in the absence of obstruction, an object 10 meters away from the target vehicle can be photographed by the target shooting device, then the preset distance can be set to 10 meters.

[0041] In some embodiments, multiple panoramic images may be acquired first, and then the multiple panoramic images may be stitched together to obtain a target image, and then the lane lines in the target image may be segmented to obtain the lane lines. The lane lines in the target image may be segmented using a deep learning model. The deep learning model may be a segnet model.

[0042] In some embodiments, the target shooting device is a surround-view camera. The surround-view camera can be a four-way fisheye camera. The stitching of multiple panoramic images can be to first identify the black and white checkerboard of the panoramic image, then record the corner positions of the black and white checkerboard, and overlap the corresponding corner points of two adjacent roads to one position, thereby forming a panoramic stitching image (i.e., the target image). The multiple panoramic images are images obtained by photographing the surrounding area of ​​the target vehicle using a surround-view camera when the target vehicle's driving speed is less than a speed threshold. The situation where the target vehicle's driving speed is less than the speed threshold may be a situation where the target vehicle is driving in an automatic parking scene or a low-speed automatic driving scene. The low-speed automatic driving scene may be a scene such as a traffic jam, a queue to pay a fee, or a queue to enter a scenic spot.

[0043] The lane line is an image of a line area in a target image. In a real environment, there are lane lines around the target vehicle. Therefore, at least one of the multiple panoramic images obtained by photographing the surrounding area of ​​the target vehicle using a target photographing device has lane lines, so the target image obtained based on the multiple panoramic images has lane lines.

[0044] Misalignment analysis is to analyze the fracture, misalignment and deflection of lane lines. Misalignment data is used to characterize the fracture, misalignment and deflection of lane lines.

[0045] Figure 3 Shown is a schematic diagram of a target image provided by an embodiment of the present application. Figure 4 Shown is a schematic diagram of a target image provided by another embodiment of the present application. Figure 5 Shown is a schematic diagram of a target image provided by another embodiment of the present application.

[0046] See also Figure 3 ,After the target vehicle rolls off the assembly line, if there is no load in the target vehicle, the lane lines in the target image will not be broken, dislocated or deflected. It can be seen that, Figure 3 Lane markings are shown as straight lines. Figure 4 ,If the target vehicle is loaded, the lane lines in the target image are broken, dislocated and deflected. It can be seen that Figure 4 The lane lines shown have 4 misaligned positions C1 (in Figure 4 (circled in dashed circle in the figure). Figure 5 ,If the target vehicle is loaded, the lane lines in the target image are broken, dislocated and deflected. It can be seen that Figure 5 The lane lines shown have 4 misaligned positions C2 (in Figure 5 (circled in dashed circle).

[0047] The heavier the target vehicle is, the more serious the fracture dislocation and deflection will be. Figure 4 and Figure 5 For explanation. When the load of the target vehicle is M1, multiple panoramic images are obtained by photographing the surrounding area of ​​the target vehicle using the target photographing device, and the target image T1 is obtained based on the multiple panoramic images. When the load of the target vehicle is M2, multiple panoramic images are obtained by photographing the surrounding area of ​​the target vehicle using the target photographing device, and the target image T2 is obtained based on the multiple panoramic images. Wherein, M1<M2. Figure 4 The lane lines shown are the lane lines in the target image T1. Figure 5 The lane lines shown are the lane lines in the target image T2. It can be seen that Figure 5 The lane line shown in FIG. 1 is broken, misaligned and deflected at the misaligned position C2 relative to FIG. Figure 4The lane line shown in the figure is more seriously broken, misaligned and deflected at the misaligned position C1, that is, Figure 5 The lane lines shown are relative to Figure 4 The lane line fracture dislocation size is larger and the deflection angle is larger.

[0048] In actual applications, as the load of the target vehicle increases, the two lane lines above and below the offset position may be completely broken and no contact will exist.

[0049] Step 220 , determining the shooting height of the target shooting device based on the misalignment data of the lane line and the mapping relationship between the misalignment data and the shooting height.

[0050] In the mapping relationship between the misaligned data and the shooting height, the misaligned data and the shooting height are one-to-one corresponding. After determining the misaligned data of the lane line in the target image, the same or closest misaligned data can be searched in the mapping relationship according to the misaligned data of the lane line in the target image, and then the shooting height corresponding to the found misaligned data is used as the shooting height of the target shooting device.

[0051] The mapping relationship between the misalignment data and the shooting height may be a pre-calibrated mapping relationship. The mapping relationship between the misalignment data and the shooting height may be determined based on the following steps.

[0052] First, multiple shooting heights of the shooting equipment on the calibration vehicle are determined, and for each current shooting height among the multiple shooting heights, the lane line in the current calibration image is analyzed for misalignment, and the misalignment data of the lane line in the current calibration image is determined; a mapping relationship between the current shooting height and the misalignment data of the lane line in the current calibration image is established to obtain a mapping relationship between the current shooting height and the misalignment data corresponding to the current shooting height; based on the mapping relationship between the multiple shooting heights and the misalignment data corresponding to each of the multiple shooting heights, a mapping relationship between the misalignment data and the shooting height is determined.

[0053] The current calibration image is obtained based on multiple calibration panoramic images. The multiple calibration panoramic images are images obtained by photographing the surrounding area of ​​the calibration vehicle using a photographing device. The misalignment data is used to characterize the broken, misaligned and deflected conditions of the lane lines in the current calibration image.

[0054] Figure 6 FIG. 1 is a schematic diagram of determining the mapping relationship between misalignment data and shooting height provided by an embodiment of the present application. Figure 6 As shown, the following example illustrates the process of determining the mapping relationship between the misalignment data and the shooting height.

[0055] First, when the calibration vehicle is unloaded, measure the shooting height H1 of the camera on the calibration vehicle, and use the camera to shoot the surrounding area of ​​the calibration vehicle to obtain multiple calibration panoramic images, and obtain the calibration image based on the multiple calibration panoramic images; perform misalignment analysis on the lane lines in the calibration image to determine the misalignment data D1 of the lane lines in the calibration image; establish a mapping relationship between the shooting height H1 of the camera on the calibration vehicle and the misalignment data D1 of the lane lines in the calibration image. Specifically, the shooting height can be measured by laser ranging, ruler ranging, etc.

[0056] Second, the load of the calibration vehicle is changed, for example, a heavy object is placed in the calibration vehicle. In this embodiment, a load of 40 kg is placed first. When the calibration vehicle has a load of 40 kg, the above calibration process is repeated to establish a mapping relationship between the shooting height H2 of the camera on the calibration vehicle and the lane line misalignment data D2 in the calibration image.

[0057] Third, continue to increase the load of the calibration vehicle. In this embodiment, the load can be increased by 5 kg each time until the load is increased to 300 kg, completing the process of determining the mapping relationship. After each increase in load, repeat the above calibration process to establish a mapping relationship between the shooting height H3 of the camera on the calibration vehicle and the misalignment data D3 of the lane line in the calibration image, a mapping relationship between the shooting height H4 of the camera on the calibration vehicle and the misalignment data D4 of the lane line in the calibration image, ..., the shooting height H of the camera on the calibration vehicle is H4. n and the lane line misalignment data D in the calibration image n Specifically, in this embodiment, when the load increases to 300 kg, the load of the calibration vehicle needs to be changed 521 times, and the last time the shooting height H of the shooting device on the calibration vehicle is obtained. 522 and the lane line misalignment data D in the calibration image 522 The mapping relationship between them.

[0058] Figure 7 FIG. 1 is a schematic diagram showing the mapping relationship between the misalignment data and the shooting height provided by an embodiment of the present application. Figure 7 , it can be seen that different misaligned data correspond to different shooting heights. In practical applications, the actual misaligned data may be between two adjacent erroneous data in the mapping relationship. At this time, the actual erroneous data can be equated with the misaligned data with a smaller difference, which can minimize the error.

[0059] Step 230 : determining the position of the target object based on the shooting height of the target shooting device and the imaging parameters of the target shooting device.

[0060] Specifically, the target object is an object in the surrounding area of ​​the target vehicle. Based on the shooting height of the target shooting device and the imaging parameters of the target shooting device, the position of the target object can be determined by the computing device automatically calculating the position of the target object according to a preset algorithm. Specifically, the position of the target object has a mapping relationship with the imaging pixel position of the target image. The imaging pixel position of the target image is determined by the shooting height of the target shooting device and the imaging parameters of the target shooting device. Therefore, the position of the target object can be obtained according to the shooting height of the target shooting device and the imaging parameters of the target shooting device.

[0061] For example, Figure 8 FIG. 1 is a schematic diagram of a method for calculating the position of a target object provided by an embodiment of the present application. Figure 8 As shown, after the shooting height of the target shooting device is determined, the imaging plane and the imaging point of the target object can be determined in combination with the imaging parameters of the target shooting device. The distance between the imaging point of the target object and the shooting point of the target shooting device has a mapping relationship with the distance between the shooting point of the target shooting device and the target object. Therefore, the position of the target object can be obtained according to the mapping relationship between the distance between the imaging point of the target object and the shooting point of the target shooting device, the distance between the imaging point of the target object and the shooting point of the target shooting device, and the distance between the shooting point of the target shooting device and the target object.

[0062] In practical applications, if the shooting height changes, the imaging plane will also change, and the imaging point of the target object on the imaging plane will also change accordingly. At this time, based on the mapping relationship between the distance between the imaging point of the target object and the shooting point of the target shooting device and the distance between the shooting point of the target shooting device and the target object, the position of the target object will naturally change. For example, Fig. 9 FIG. 1 is a schematic diagram showing the relationship between the shooting height and the position of the target object provided by an embodiment of the present application. Fig. 9 As shown, shooting height 1 corresponds to imaging plane 1, imaging plane 1 corresponds to imaging point 1 of the target object, and the target object d position 1 is calculated through the imaging point 1 of the target object and shooting height 1. Similarly, shooting height 2 corresponds to imaging plane 2, imaging plane 2 corresponds to imaging point 2 of the target object, and the target object position 2 is calculated through the imaging point 2 of the target object and shooting height 2. It can be seen that the position of the target object finally calculated is different for different shooting heights. Therefore, the present application obtains the accurate shooting height of the target shooting device according to the misaligned data of the lane lines in the target image and the mapping relationship between the misaligned data and the shooting height, and then uses the accurate shooting height to calculate the position of the target object, thereby improving the accuracy of the position of the target object. In addition, the target image is obtained in real time, so the shooting height is also calculated in real time, further improving the accuracy of the shooting height.

[0063] Fig.10 FIG. 1 is a flow chart of a target positioning method provided by an embodiment of the present application. Fig.10 As shown, performing misalignment analysis on lane lines in a target image and determining misalignment data of lane lines includes the following steps.

[0064] Step 1010, traverse the pixels corresponding to the lane lines in the target image to determine the misaligned positions in the lane lines.

[0065] Specifically, the offset position divides one lane line into two lane lines. Figure 4 and Figure 5 As shown, the misaligned positions C1 and C2 both divide one lane line into two lane lines. That is, at the misaligned position, the lane line is broken, misaligned, and deflected.

[0066] In some embodiments, pixels corresponding to lane lines in a target image are traversed to determine misaligned positions in lane lines. This can be performed as follows: pixels corresponding to lane lines in a target image are traversed, and for each current pixel traversed, a gradient of the current pixel is determined; based on the gradient of each current pixel, the misaligned position in the lane line is determined.

[0067] Specifically, if the gradient of the current pixel is large, it may indicate that there is a grayscale mutation in the pixel value of the current pixel, that is, there is a misalignment of the lane line.

[0068] Step 1020, determining the baseline lane line and the offset lane line among the two lane lines.

[0069] Specifically, the lane line parallel to the vertical direction may be determined as the reference lane line, and the other lane line of the two lane lines may be determined as the offset lane line. Figure 4 and Figure 5 As shown in the figure, the lane line parallel to the vertical direction is the reference lane line. Figure 4 The lane line between the two offset positions C1 on the left is the reference lane line.

[0070] Step 1030, determining the offset size between the misaligned lane line and the reference lane line at the misaligned position.

[0071] The offset size may be a size on the image, for example, the offset size may be expressed as the number of pixels.

[0072] In some embodiments, determining the offset size of the staggered lane line and the baseline lane line at the staggered position can be performed by: determining the number of staggered pixels between one end of the staggered lane line close to the staggered position and one end of the baseline lane line close to the staggered position; based on the number of staggered pixels, determining the offset size at the staggered position.

[0073] Specifically, the number of misaligned pixels between one end of the misaligned lane line close to the misaligned position and one end of the baseline lane line close to the misaligned position can be: in the horizontal direction (the direction perpendicular to the baseline lane line), the number of misaligned pixels between one end of the misaligned lane line close to the misaligned position and one end of the baseline lane line close to the misaligned position.

[0074] Step 1040, determining the deflection angle between the offset lane line and the reference lane line.

[0075] The deflection angle between the offset lane line and the reference lane line may be determined by taking the reference lane line as a reference and the clockwise direction as a positive direction of the deflection angle, and calculating the deflection angle of the offset lane line relative to the reference lane line.

[0076] In some embodiments, determining the deflection angle between the offset lane line and the baseline lane line can be performed as follows: using a line parallel to the baseline lane line and a line perpendicular to the baseline lane line as coordinate axes of the plane rectangular coordinate system, generating a plane rectangular coordinate system; based on the plane rectangular coordinate system, calculating the slope of the offset lane line; based on the slope of the offset lane line, determining the deflection angle between the offset lane line and the baseline lane line.

[0077] Step 1050, determining lane line misalignment data based on the offset size and the deflection angle.

[0078] The lane line misalignment data may include an offset size between the misaligned lane line and the reference lane line at the misaligned position, and a deflection angle between the misaligned lane line and the reference lane line.

[0079] Fig.11 Shown Figure 5 The target image shown is a magnified image at the misaligned position. Fig.11 As shown in , a plane rectangular coordinate system is generated with the line parallel to the baseline lane line as the y-axis and the line perpendicular to the baseline lane line as the x-axis. The pixels at both ends of the misaligned lane line are taken as two points A and B, and the slope of the straight line connecting the pixel points A and B in the plane rectangular coordinate system is calculated. Then, the angle between the misaligned lane line and the baseline lane line is calculated based on the slope. This angle is the deflection angle. It can be seen that the misaligned lane line rotates counterclockwise relative to the baseline lane line, so the angle can be a negative angle. In addition, as Fig.11 As shown, it can be seen that the number of misaligned pixels is 4, so the offset size can be 4. In practical applications, the number of misaligned pixels can be calculated by rounding off.

[0080] Combination of the above Figures 1 to 11 , describes the method embodiment of the present application in detail, and the following is combined with Fig.12, describes the device embodiment of the present application in detail. It should be understood that the description of the method embodiment corresponds to the description of the device embodiment, so the parts not described in detail can refer to the previous method embodiment.

[0081] Exemplary Devices

[0082] Fig.12 FIG. 1 is a schematic diagram of the structure of a target positioning device provided by an embodiment of the present application. Fig.12 As shown, the target positioning device 1200 of the embodiment of the present application includes: a misalignment data determination module 1210, a shooting height determination module 1220 and a target object positioning module 1230.

[0083] Specifically, the misalignment data determination module 1210 is configured to perform misalignment analysis on the lane lines in the target image to determine the misalignment data of the lane lines, wherein the target image is obtained based on multiple panoramic images, and the multiple panoramic images are images obtained by using a target shooting device to shoot the surrounding area of ​​the target vehicle, and the misalignment data is used to characterize the broken misalignment and deflection of the lane lines. The shooting height determination module 1220 is configured to determine the shooting height of the target shooting device based on the misalignment data of the lane lines and the mapping relationship between the misalignment data and the shooting height. The target object positioning module 1230 is configured to determine the position of the target object based on the shooting height of the target shooting device and the imaging parameters of the target shooting device, wherein the target object is an object in the surrounding area of ​​the target vehicle.

[0084] In some embodiments, the misaligned data determination module 1210 is further configured to traverse the pixels corresponding to the lane line in the target image to determine the misaligned position in the lane line, wherein the misaligned position divides one lane line into two lane lines; determine the baseline lane line and the misaligned lane line in the two lane lines; determine the offset size between the misaligned lane line and the baseline lane line at the misaligned position; determine the deflection angle between the misaligned lane line and the baseline lane line; and determine the misaligned data of the lane line based on the offset size and the deflection angle.

[0085] In some embodiments, the misaligned data determination module 1210 is further configured to determine the number of misaligned pixels between one end of the misaligned lane line close to the misaligned position and the end of the reference lane line close to the misaligned position; and determine the offset size at the misaligned position based on the number of misaligned pixels.

[0086] In some embodiments, the misaligned data determination module 1210 is further configured to generate a plane rectangular coordinate system using a line parallel to the baseline lane line and a line perpendicular to the baseline lane line as coordinate axes of the plane rectangular coordinate system; calculate the slope of the misaligned lane line based on the plane rectangular coordinate system; and determine the deflection angle between the misaligned lane line and the baseline lane line based on the slope of the misaligned lane line.

[0087] In some embodiments, the misaligned data determination module 1210 is further configured to traverse the pixels corresponding to the lane line in the target image, and determine the gradient of each current pixel traversed; based on the gradient of each current pixel, determine the misaligned position in the lane line.

[0088] In some embodiments, the target positioning device 1200 further includes: a mapping relationship determination module 1240 .

[0089] Specifically, the mapping relationship determination module 1240 is configured to determine multiple shooting heights of the shooting equipment on the calibration vehicle; for each current shooting height among the multiple shooting heights, perform a misalignment analysis on the lane line in the current calibration image, and determine the misalignment data of the lane line in the current calibration image, wherein the current calibration image is obtained based on multiple calibration panoramic images, and the multiple calibration panoramic images are images obtained by using the shooting equipment to shoot the surrounding area of ​​the calibration vehicle, and the misalignment data is used to characterize the broken, misaligned and deflected conditions of the lane line in the current calibration image; establish a mapping relationship between the current shooting height and the misalignment data of the lane line in the current calibration image, and obtain a mapping relationship between the current shooting height and the misalignment data corresponding to the current shooting height; based on the mapping relationship between multiple shooting heights and the misalignment data corresponding to each of the multiple shooting heights, determine the mapping relationship between the misalignment data and the shooting height.

[0090] Exemplary Electronic Devices

[0091] Fig.13 FIG. 1 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Fig.13 As shown, the electronic device 1300 includes: one or more processors 1301 and a memory 1302; and computer program instructions stored in the memory 1302, and when the computer program instructions are executed by the processor 1301, the processor 1301 executes a target positioning method as in any of the above embodiments.

[0092] The processor 1301 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0093] The memory 1302 may include one or more computer program products, and the computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1301 may execute the program instructions to implement the steps in the target positioning method of each embodiment of the present application described above and / or other desired functions.

[0094] In one example, the electronic device 1300 may further include: an input device 1303 and an output device 1304, which are connected via a bus system and / or other forms of connection mechanisms ( Fig.13 ) interconnected.

[0095] In addition, the input device 1303 may also include, for example, a keyboard, a mouse, a microphone, etc.

[0096] The output device 1304 can output various information to the outside, and can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto.

[0097] Of course, to simplify, Fig.13 Only some of the components related to the present application in the electronic device 1300 are shown, and components such as a bus, an input device / output interface, etc. are omitted. In addition, according to specific application conditions, the electronic device 1300 may also include any other appropriate components.

[0098] Exemplary computer-readable storage media

[0099] In addition to the above methods and devices, the embodiments of the present application may also be a computer program product, including computer program instructions, which, when executed by a processor, enable the processor to execute the steps in the target positioning method of any of the above embodiments.

[0100] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0101] In addition, an embodiment of the present application may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the target positioning method according to various embodiments of the present application described in the above "Exemplary Method" section of this specification.

[0102] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0103] The basic principles of the present application are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present application. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, not for limitation, and the above details do not limit the present application to being implemented by adopting the above specific details.

[0104] The block diagrams of the devices, apparatuses, equipment, and systems involved in this application are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0105] It should also be noted that in the apparatus, device and method of the present application, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0106] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0107] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

[0108] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A target positioning method, characterized in that: include: Performing a misalignment analysis on the lane lines in the target image to determine misalignment data of the lane lines, wherein the target image is obtained based on a plurality of panoramic images, the plurality of panoramic images being images obtained by photographing the surrounding area of ​​the target vehicle using a target photographing device, and the misalignment data being used to characterize the fracture, misalignment and deflection of the lane lines; Determining the shooting height of the target shooting device based on the misalignment data of the lane line and the mapping relationship between the misalignment data and the shooting height; Based on the shooting height of the target shooting device and the imaging parameters of the target shooting device, the position of the target object is determined, wherein the target object is an object in the surrounding area of ​​the target vehicle.

2. The target positioning method according to claim 1, characterized in that: The performing misalignment analysis on the lane lines in the target image to determine misalignment data of the lane lines includes: Traversing pixels corresponding to the lane line in the target image to determine a misaligned position in the lane line, wherein the misaligned position divides one lane line into two lane lines; Determine a reference lane line and an offset lane line among the two lane lines; Determine an offset size between the misaligned lane line and the reference lane line at the misaligned position; Determining a deflection angle between the offset lane line and the reference lane line; Based on the offset size and the deflection angle, misalignment data of the lane line is determined.

3. The target positioning method according to claim 2, characterized in that: The determining of the offset size between the staggered lane line and the reference lane line at the staggered position includes: Determine the number of misaligned pixels between an end of the misaligned lane line close to the misaligned position and an end of the reference lane line close to the misaligned position; Based on the number of misaligned pixels, the offset size at the misaligned position is determined.

4. The target positioning method according to claim 2, characterized in that: The determining of the deflection angle between the misaligned lane line and the reference lane line comprises: Using a line parallel to the reference lane line and a line perpendicular to the reference lane line as coordinate axes of a plane rectangular coordinate system, respectively, to generate the plane rectangular coordinate system; Calculating the slope of the misaligned lane line based on the plane rectangular coordinate system; Based on the slope of the offset lane line, a deflection angle between the offset lane line and the reference lane line is determined.

5. The target positioning method according to claim 2, characterized in that: The traversing pixels corresponding to the lane line in the target image to determine the misaligned position of the lane line includes: Traversing pixels corresponding to lane lines in the target image, and determining a gradient of each current pixel traversed; Based on the gradient of each current pixel, a misaligned position in the lane line is determined.

6. The target positioning method according to any one of claims 1 to 5, characterized in that: The mapping relationship between the misalignment data and the shooting height is determined based on the following steps: Determine a plurality of shooting heights of a shooting device on a calibration vehicle; For each current shooting height among the multiple shooting heights, performing a misalignment analysis on the lane line in the current calibration image, and determining misalignment data of the lane line in the current calibration image, wherein the current calibration image is obtained based on a plurality of calibration panoramic images, and the plurality of calibration panoramic images are images obtained by using the shooting device to shoot the surrounding area of ​​the calibration vehicle, and the misalignment data is used to characterize the fracture, misalignment and deflection of the lane line in the current calibration image; Establishing a mapping relationship between the current shooting height and the misaligned data of the lane line in the current calibration image, and obtaining a mapping relationship between the current shooting height and the misaligned data corresponding to the current shooting height; Based on the mapping relationship between the plurality of shooting heights and the misalignment data respectively corresponding to the plurality of shooting heights, the mapping relationship between the misalignment data and the shooting heights is determined.

7. The target positioning method according to any one of claims 1 to 5, characterized in that: The target shooting device is a surround-view camera, and the multiple panoramic images are images obtained by shooting the surrounding area of ​​the target vehicle using the surround-view camera when the driving speed of the target vehicle is less than a speed threshold.

8. A target positioning device, characterized in that: include: a misalignment data determination module, configured to perform misalignment analysis on lane lines in a target image to determine misalignment data of the lane lines, wherein the target image is obtained based on a plurality of panoramic images, the plurality of panoramic images being images obtained by photographing the surrounding area of ​​a target vehicle using a target photographing device, and the misalignment data being used to characterize a fracture, misalignment and deflection condition of the lane lines; a shooting height determination module, configured to determine the shooting height of the target shooting device based on the misalignment data of the lane line and a mapping relationship between the misalignment data and the shooting height; The target object positioning module is configured to determine the position of the target object based on the shooting height of the target shooting device and the imaging parameters of the target shooting device, wherein the target object is an object in the surrounding area of ​​the target vehicle.

9. A computer-readable storage medium, characterized in that: The storage medium stores instructions, and when the instructions are executed by a processor of an electronic device, the electronic device is enabled to execute the target positioning method described in any one of claims 1 to 7.

10. An electronic device, characterized in that: The electronic device comprises: processor; memory for storing computer executable instructions; The processor is used to execute the computer executable instructions to implement the target positioning method described in any one of claims 1 to 7 above.