Positioning mapping establishment method and device for fisheye camera, medium and product

Abstract

The embodiment of the invention relates to the technical field of image processing, and discloses a fisheye camera positioning mapping establishment method and device, a medium and a product. The method comprises the following steps: calculating an internal parameter and a distortion coefficient of the fisheye camera through a calibration board; determining external parameters of the fisheye camera according to the world coordinates and the pixel coordinates of the at least six dotting positions; world coordinate interpolation is carried out based on known point locations of the vector map, interpolation points and world coordinates of the interpolation points are obtained, and an interpolation point set is formed; projecting the interpolation point set to a pixel coordinate system to obtain a pixel coordinate of each point in the interpolation point set; and establishing a mapping table of the fisheye camera according to the world coordinate and the pixel coordinate of each point in the interpolation point set. By adopting the scheme, the world coordinate corresponding to the position of each pixel point can be obtained only through matching of tables, the calculation amount is reduced, meanwhile, the actual position of the target is recognized in a mapping mode, and the position recognition accuracy can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, medium and product for establishing positioning mapping of a fisheye camera. Background Art

[0002] In recent years, with the rapid development of science and technology and the gradual improvement of image processing technology, the ability to quickly and conveniently process acquired images has become a crucial technology.

[0003] In the vehicle-road collaboration scenario, various sensors such as cameras, millimeter-wave radars, and lidars can be installed in the road infrastructure to collect road traffic information in real time, providing drivers with a safer driving environment and more efficient traffic flow management. This technology also supports the development of autonomous driving technology, and helps achieve a higher level of autonomous driving by enhancing the reliability and safety of single-vehicle intelligence. Therefore, it is critical to provide autonomous driving vehicles with information such as the location, heading, and speed of other traffic participants so that autonomous driving vehicles can make more reliable decisions.

[0004] However, due to the obvious distortion of fisheye cameras, some technical solutions first remove the distortion of fisheye cameras and use perspective changes on pictures with less distortion to establish the correspondence between pixel coordinates and world coordinates. However, this method not only requires complex calculations and extremely high requirements for hardware parsing capabilities, but also may introduce calculation errors in the process of removing distortion, affecting positioning accuracy. Therefore, how to simply and accurately establish the mapping relationship between the pixel coordinates of fisheye cameras and world coordinates is a technical problem in this field. Summary of the invention

[0005] One purpose of the present application is to provide a method, device, medium and product for establishing the positioning mapping of a fisheye camera, at least to solve the problem that the existing mapping relationship has high computational complexity and poor positioning accuracy. The present application calculates the internal parameters and distortion coefficients of the fisheye camera, and determines the external parameters of the fisheye camera according to the world coordinates and pixel coordinates of at least six dot positions within the field of view of the fisheye camera. On this basis, the world coordinates are interpolated between the known points of the vector map to obtain an interpolation point set, and then the interpolation point set is projected to the pixel coordinate system to obtain the pixel coordinates of each point in the interpolation point set, and then the mapping table of the fisheye camera is established according to the world coordinates and pixel coordinates of each point in the interpolation point set. By adopting this scheme, the external parameters of the fisheye camera can be determined by the dot position, and interpolation is performed based on the known points of the vector map to obtain a certain amount of interpolation point set to construct a mapping table, so that there is no need for complex calculations in the subsequent use process. The world coordinates corresponding to the position of each pixel point can be obtained by simply matching the table, which reduces the amount of calculation. At the same time, the actual position of the target is identified by mapping, which can improve the accuracy of position recognition.

[0006] To achieve the above objectives, some embodiments of the present application provide the following aspects:

[0007] In a first aspect, some embodiments of the present application further provide a method for establishing positioning mapping of a fisheye camera, the method comprising:

[0008] Calculate the internal parameters and distortion coefficients of the fisheye camera through the calibration plate;

[0009] Determine the external parameters of the fisheye camera according to the world coordinates and pixel coordinates of at least six marking positions within the field of view of the fisheye camera;

[0010] Obtaining a vector map where the field of view of the fisheye camera is located, performing world coordinate interpolation based on points with known coordinates in the vector map, obtaining interpolation points and world coordinates of the interpolation points, and forming an interpolation point set;

[0011] According to the internal parameters, distortion coefficients and external parameters of the fisheye camera, the interpolation point set is projected to a pixel coordinate system to obtain the pixel coordinates of each point in the interpolation point set;

[0012] A mapping table of the fisheye camera is established according to the world coordinates and the pixel coordinates of each point in the interpolation point set.

[0013] In a second aspect, some embodiments of the present application further provide an electronic device, comprising: one or more processors; and a memory storing computer program instructions, wherein the computer program instructions, when executed, cause the processor to perform the steps of the method described above.

[0014] In a third aspect, some embodiments of the present application further provide a computer-readable medium having computer program instructions stored thereon, wherein the computer program instructions can be executed by a processor to implement the method as described above.

[0015] In a fourth aspect, some embodiments of the present application further provide a computer program product, comprising a computer program / instruction, which implements the steps of the method described above when executed by a processor.

[0016] Compared with the related art, in the solution provided by the embodiment of the present application, the external parameters of the fisheye camera are determined by calculating the internal parameters and distortion coefficients of the fisheye camera, and according to the world coordinates and pixel coordinates of at least six dot positions within the field of view of the fisheye camera, on this basis, the world coordinates are interpolated between the known points of the vector map to obtain an interpolation point set, and then the interpolation point set is projected to the pixel coordinate system to obtain the pixel coordinates of each point in the interpolation point set, and then the mapping table of the fisheye camera is established according to the world coordinates and pixel coordinates of each point in the interpolation point set. By adopting this solution, the external parameters of the fisheye camera can be determined by the dot position, and interpolation is performed based on the known points of the vector map to obtain a certain amount of interpolation point set to construct a mapping table, so that in the subsequent use process, there is no need for complex calculations, and the world coordinates corresponding to the position of each pixel point can be obtained by simply matching the table, which reduces the amount of calculation. At the same time, the actual position of the target can be identified by mapping, which can improve the accuracy of position recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0018] Figure 1 An exemplary flow chart of a method for establishing positioning mapping of a fisheye camera provided according to some embodiments of the present application;

[0019] Figure 2 A schematic diagram of determining internal parameters and distortion parameters of a fisheye camera using a calibration plate provided in some embodiments of the present application;

[0020] Figure 3 A schematic diagram of RTK point location provided according to some embodiments of the present application;

[0021] Figure 4 An exemplary flow chart of a method for establishing positioning mapping of a fisheye camera provided according to some embodiments of the present application;

[0022] Figure 5 A schematic diagram of a vector map provided according to some embodiments of the present application;

[0023] Figure 6 A schematic diagram of an interpolation process provided according to some embodiments of the present application;

[0024] Figure 7 An exemplary structural diagram of the electronic device is disclosed. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] First embodiment

[0027] The first embodiment of the present application relates to a method for establishing a positioning mapping of a fisheye camera. Figure 1 As shown, the method may include the following steps:

[0028] Step S101, calculating the internal parameters and distortion coefficients of the fisheye camera through a calibration plate;

[0029] Among them, a fisheye camera is a camera with an extreme wide-angle lens. Fisheye cameras have an ultra-wide-angle view and can provide a very wide field of view, usually reaching or even exceeding a 180-degree viewing angle. This enables it to capture a large amount of scene information in a single photo, making it very suitable for shooting grand landscapes, indoor panoramas, and special scenes that require a wide viewing angle. Fisheye cameras have strong distortion, and the images of fisheye cameras have obvious distortion, straight lines become curves, and the shapes of objects are distorted. This distortion can be cleverly used in some cases to create unique artistic effects. Fisheye cameras have unique visual effects, and due to their special optical properties, photos taken with fisheye cameras often have a strong visual impact. Objects in the picture may be stretched, deformed, or appear round or spherical.

[0030] The checkerboard calibration board is a widely used tool in machine vision, photogrammetry and other fields. It is composed of alternating black and white square grids to form a regular checkerboard pattern. This sharp contrast makes it easy to identify and detect in the image. The size of the grid is usually uniform, and different size specifications can be selected according to different application requirements. The side length of each grid of the checkerboard calibration board and the overall size ratio are precisely designed and manufactured. This is crucial for accurate calibration. For example, a common checkerboard calibration board may have different grid numbers such as 8×8 and 10×10, and the side length of each grid can range from a few millimeters to tens of millimeters. In a machine vision system, the camera's internal parameters (such as focal length, principal point position, distortion coefficient, etc.) and external parameters (camera position and posture) need to be accurately calibrated. The checkerboard calibration board can provide known geometric features. By taking multiple images of the calibration board at different angles, the camera parameters are calculated using algorithms. This is crucial to ensure the measurement precision and accuracy of the machine vision system. For example, in industrial automation production, it is necessary to accurately measure the size, position, and posture of objects, and camera calibration can ensure the reliability of these measurement results.

[0031] Fisheye cameras have unique internal parameters and distortion coefficients due to their special lens design. Their internal parameters can include focal length, principal point position, and pixel size. The focal length of a fisheye camera is usually much shorter than that of an ordinary camera, which enables it to obtain an ultra-wide-angle field of view. The specific value of the focal length depends on the design and specifications of the fisheye lens, and is generally between a few millimeters and more than ten millimeters. The shorter focal length allows the fisheye camera to include more scenes in the picture when shooting, but it also causes more obvious image distortion. The principal point is the intersection of the camera's optical axis and the imaging plane. For fisheye cameras, it is more complicated to determine the position of the principal point. Due to the distortion of the fisheye lens, the principal point may not be at the center of the imaging plane, but may be offset. Determining the position of the principal point is very important for accurate camera calibration and image correction. The pixel size of a fisheye camera is usually similar to that of an ordinary camera, but due to the special optical properties of the fisheye lens, the distribution of pixels in the image may be uneven. In the edge area of ​​the image, the pixels may be stretched or compressed, resulting in a change in pixel size. Understanding the change pattern of pixel size is very helpful for image processing and analysis.

[0032] Distortion coefficients can include radial distortion, tangential distortion, and distortion correction parameters. The radial distortion of fisheye cameras is very obvious, which is caused by the curvature of the lens and the refraction of light. Radial distortion turns straight lines in the image into curves, and the farther away from the center, the more severe the distortion. In addition to radial distortion, fisheye cameras also have tangential distortion, which is caused by factors such as lens installation errors and manufacturing accuracy. Tangential distortion causes objects in the image to tilt and shift, especially at the edge of the image. In order to obtain accurate image information, the distortion of the fisheye camera needs to be corrected. Distortion correction can be achieved through software algorithms, such as using the distortion coefficients obtained by camera calibration to correct the image. Distortion can also be reduced through hardware design, such as using special lens structures or optical elements.

[0033] In short, the internal parameters and distortion coefficients of the fisheye camera have an important impact on its imaging quality and application effect. This solution can determine the internal parameters and distortion parameters of the fisheye camera through the calibration board, which can be beneficial for subsequent correction and optimization to obtain better image effects.

[0034] Figure 2 This is a schematic diagram of determining the internal parameters and distortion parameters of a fisheye camera according to the calibration plate provided in some embodiments of the present application. The internal parameters and distortion coefficients of a fisheye camera are obtained by converting the camera coordinate system to the pixel coordinate system. A chessboard is placed at different positions in the field of view of the fisheye camera, and the pixel position information of the intersection of the chessboard is found using the findChessboardCorners function of opencv. The world coordinates of the chessboard can be obtained according to the specifications of the chessboard (the size and number of rows and columns of the chessboard). Assuming that the Z value is 0, the calibrate function in the fisheye module in opencv is called according to the pixel coordinates of the intersection of the chessboard and the corresponding world coordinates to calculate the internal parameter K and distortion coefficient D of the fisheye camera.

[0035] Step S102, determining the external parameters of the fisheye camera according to the world coordinates and pixel coordinates of at least six dot positions within the field of view of the fisheye camera;

[0036] Then calculate the external parameters of the fisheye camera. The external parameters are the parameters that convert the world coordinate system to the camera coordinate system. Specifically, according to the RTK point information and the pixel position information corresponding to the points in the fisheye camera field of view, call the solvePnP function in opencv to solve the external parameters of the fisheye camera. Figure 3 The schematic diagram of the RTK dot positions provided according to some embodiments of the present application is within the field of view of the fisheye camera. Due to the large distortion of the fisheye and the calculation deviation, the external parameters are calculated using software. The number of dot positions can be six to eight, or more.

[0037] Step S103, obtaining a vector map where the field of view of the fisheye camera is located, performing world coordinate interpolation based on the points with known coordinates in the vector map, obtaining interpolation points and world coordinates of the interpolation points, and forming an interpolation point set;

[0038] A vector map is a map that stores geographic information in the form of vector data. A vector map is composed of geometric figures such as points, lines, and surfaces, which can accurately represent the location, shape, and size of geographic entities. Compared with raster maps, vector maps will not appear pixelated during zooming in or out, and always maintain clear lines and accurate shapes. Vector maps only store geometric information and attribute information of geographic entities, rather than a large amount of pixel data like raster maps. Therefore, the amount of data in vector maps is relatively small, which is convenient for storage, transmission, and processing. Points are the most basic geometric figures in vector maps, indicating the location of geographic entities. For example, landmark buildings, road intersections, etc. in a city can be represented by points. Lines are composed of a series of continuous points, representing the shape and outline of geographic entities. For example, roads, rivers, railways, etc. can be represented by lines.

[0039] Among them, when interpolating world coordinates between points with known coordinates in a vector map, specifically, linear interpolation can be used. Linear interpolation is a simple interpolation method that is suitable for situations where data changes are relatively slow. For two known points, linear interpolation between them can obtain the world coordinates of any position. Polynomial interpolation can also be used. Polynomial interpolation can fit higher-order curves and is suitable for situations where data changes are relatively complex. For multiple known points, a polynomial interpolation function can be used to approximate the world coordinate changes between them. For example, using quadratic polynomial interpolation, a quadratic polynomial function can be determined by three known points, and then interpolation calculations are performed at any position. During the interpolation process, nodes can be evenly distributed. If the known points are evenly distributed in space, interpolation nodes with equal spacing can be selected. For example, for 6 known points, the interval between them can be equally divided into parts, and then interpolation is performed at the middle position of each part. Nodes can also be distributed unevenly as needed. According to the characteristics and needs of the data, unevenly distributed interpolation nodes can be selected. For example, if the data changes more drastically in some areas, more interpolation nodes can be added in these areas to improve the accuracy of interpolation. The selection of non-uniformly distributed nodes needs to be analyzed and adjusted according to the specific situation, and the optimal node distribution can be determined through experiments and optimization.

[0040] In this solution, a series of interpolation points and their world coordinates can be obtained through the interpolation method, thereby forming an interpolation point set.

[0041] Step S104, projecting the interpolation point set to a pixel coordinate system according to the internal parameters, distortion coefficients, and external parameters of the fisheye camera to obtain the pixel coordinates of each point in the interpolation point set;

[0042] After the interpolation is completed, the interpolated points can be projected into the field of view of the fisheye camera using the internal parameters, distortion coefficients and external parameters of the fisheye camera, and based on this, most of the pixels have corresponding world coordinate points.

[0043] Step S105 , establishing a mapping table of the fisheye camera according to the world coordinates and pixel coordinates of each point in the interpolation point set.

[0044] After obtaining the correspondence between each point, a mapping table of the fisheye camera can be established. After obtaining this mapping table, if the target in the fisheye camera can be located, for example, the center position of a vehicle can be located, the coordinates of the vehicle in the world coordinate system can be obtained to determine the actual position of the target on the road. At the same time, after obtaining the target detection frame of the vehicle, the actual position of each boundary of the vehicle on the road can be obtained.

[0045] In the solution provided by the embodiment of the present application, the internal parameters and distortion coefficients of the fisheye camera are calculated, and the external parameters of the fisheye camera are determined according to the world coordinates and pixel coordinates of at least six dot positions within the field of view of the fisheye camera. On this basis, the world coordinates are interpolated between the known points of the vector map to obtain an interpolation point set, and then the interpolation point set is projected to the pixel coordinate system to obtain the pixel coordinates of each point in the interpolation point set, and then the mapping table of the fisheye camera is established according to the world coordinates and pixel coordinates of each point in the interpolation point set. By adopting this solution, the external parameters of the fisheye camera can be determined by the dot positions, and interpolation is performed based on the known points of the vector map to obtain a certain amount of interpolation point set to construct a mapping table, so that in the subsequent use process, there is no need for complex calculations, and the world coordinates corresponding to the position of each pixel point can be obtained by simply matching the table, which reduces the amount of calculation. At the same time, the actual position of the target can be identified by mapping, which can improve the accuracy of position recognition.

[0046] Second embodiment

[0047] The second embodiment of the present application relates to a method for establishing positioning mapping of a fisheye camera. The second implementation is an improvement based on the first embodiment, and the specific improvement is that the method also includes: obtaining a vector map of the field of view of the fisheye camera; obtaining the coordinate information of the lane line from the vector map; correspondingly, obtaining the vector map of the field of view of the fisheye camera, performing world coordinate interpolation based on the points with known coordinates in the vector map, obtaining interpolation points and the world coordinates of the interpolation points, and forming an interpolation point set, including: performing world coordinate interpolation based on the coordinate information of the lane line, obtaining interpolation points and the world coordinates of the interpolation points, and forming an interpolation point set. Figure 4 As shown, the method may include the following steps:

[0048] Step S401, calculating the internal parameters and distortion coefficients of the fisheye camera through a calibration plate;

[0049] Step S402, determining the external parameters of the fisheye camera according to the world coordinates and pixel coordinates of at least six dot positions within the field of view of the fisheye camera;

[0050] Step S403, performing world coordinate interpolation based on the coordinate information of the lane line to obtain interpolation points and world coordinates of the interpolation points to form an interpolation point set;

[0051] Figure 5 Schematic diagram of a vector map provided according to some embodiments of the present application. The vector map contains information such as the coordinates of lane lines, such as Figure 5 As shown, the coordinates of the left and right boundary lines of the lane can be obtained through the vector map.

[0052] Among them, multiple points can be read from the left and right boundary lines of the lane, and the coordinate information of each point can be read. The coordinate information is the coordinate in the world coordinate system.

[0053] The lane boundary lines are interpolated using the linear interpolation method to obtain dense lane line interpolation points and the world coordinates of the interpolation points.

[0054] First, the lane boundary line is interpolated along the direction of the lane using linear interpolation to make the coordinates of the lane boundary line denser.

[0055] Through such a setting, the present solution can provide more dense interpolation points on the boundary line for subsequent interpolation points of the lane area, thereby obtaining more accurate and dense interpolation points.

[0056] Step S404, projecting the interpolation point set to a pixel coordinate system according to the internal parameters, distortion coefficients, and external parameters of the fisheye camera to obtain the pixel coordinates of each point in the interpolation point set;

[0057] Step S405 , establishing a mapping table of the fisheye camera according to the world coordinates and pixel coordinates of each point in the interpolation point set.

[0058] In this embodiment, optionally, after the lane boundary lines are interpolated using a linear interpolation method to obtain dense lane line interpolation points and the world coordinates of the interpolation points, the method further includes:

[0059] The left boundary line and the right boundary line formed by the interpolated lane boundary line are interpolated within the lane area to obtain the interpolation points within the lane area and the world coordinates of the interpolation points.

[0060] Figure 6 FIG. 1 is a schematic diagram of an interpolation process provided according to some embodiments of the present application. Figure 6 As shown, the left and right lane boundary lines after interpolation are interpolated to obtain the ground point coordinates in the lane area (the lane coordinates in the map are the lane coordinates on the actual ground), traverse the left lane boundary line after interpolation, find the nearest boundary line point in the right lane boundary line, interpolate between the two points, and obtain the interpolation point in the lane area and the world coordinates of the interpolation point.

[0061] Through such a setting, in addition to obtaining the interpolation points of the lane boundary line, the present solution can also obtain the interpolation points within the lane area, so that the density of the interpolation points is higher, which can improve the precision of the established mapping table and improve the accuracy of subsequent positioning of the target in the image.

[0062] Optionally, interpolating the left boundary line and the right boundary line formed by the interpolated lane boundary line within the lane area to obtain the interpolation points within the lane area and the world coordinates of the interpolation points includes:

[0063] The interpolation points on the left boundary line are traversed, and the nearest corresponding interpolation point is found among the interpolation points on the right boundary line. Interpolation is performed between the interpolation point and its corresponding interpolation point according to the first resolution to obtain the interpolation point in the lane area and the world coordinates of the interpolation point.

[0064] Interpolation is performed at a fixed resolution, and the interpolation point and these two points are added to the interpolation point set for output.

[0065] The fixed resolution may be to insert an interpolation point every 10 centimeters, so that more dense interpolation points can be obtained in the lane area and other areas.

[0066] The linear interpolation formula is as follows:

[0067] diffX=x A -x B

[0068] diffY=yA -y B

[0069] diffZ=z A -z B

[0070]

[0071] num=floor(dis / gridSize)

[0072] d x =diffX / num

[0073] d y =diffY / num

[0074] d z =diffZ / num

[0075] x i =x A +i*d x

[0076] y i =y A +i*d y

[0077] z i =z A +i*d z

[0078] in:

[0079] A(x A ,y A ,z A ) is the point on the left lane, B(x B ,y B ,z B ) is the nearest point of the right lane found;

[0080] diffX, diffY, and diffZ are the position differences between point A and point B on the three axes respectively;

[0081] dis is the distance between point A and point B;

[0082] gridSize is the resolution of the interpolation;

[0083] num is the number of interpolation points;

[0084] d x ,d y ,d z Respectively represent the stride on the three axes;

[0085] (xi ,y i ,z i ) represents the coordinates of the i-th interpolation point.

[0086] By setting the resolution, this solution can determine a clear interval distance during the interpolation process, complete the interpolation more accurately, and improve the density and uniformity of the points after interpolation.

[0087] On the basis of the above solution, optionally, if it is identified that there is no interpolation point in the target area, interpolation points that are aligned with each other are searched around the target area;

[0088] Interpolation is performed between the mutually aligned interpolation points at a second resolution; wherein the second resolution is smaller than the first resolution.

[0089] Most pixels have corresponding world coordinate points. Due to the barrel distortion directly below the fisheye camera, the field of view is stretched and the ground points cannot completely cover the pixels. There are no pixels covering the ground points in the area of ​​interest. The nearest pixels covered by the ground points are found on the left and right. The ground points are supplemented by interpolating at a smaller resolution based on the world coordinate points corresponding to the pixels.

[0090] Through such a setting, this solution can perform more intensive interpolation based on the actual situation of the barrel distortion of the fisheye camera to ensure that subsequent positioning or other applicable requirements are met.

[0091] In the above solutions, optionally, the mapping table of the fisheye camera is established in the form of a dictionary.

[0092] Specifically, in order to facilitate algorithm query, the corresponding relationship is stored in the form of a dictionary. The definition formula of the keyword is as follows, and the value is the world coordinate corresponding to the pixel. In this way, the corresponding relationship between the pixel and the world coordinate is established.

[0093] indice=x+imageW*y

[0094] in:

[0095] x is the horizontal coordinate of the pixel coordinate;

[0096] y is the ordinate of the pixel coordinate;

[0097] imageW is the width of the image.

[0098] This solution can establish a more accurate positioning correspondence when there are fewer points on site, and the effective positioning distance is greater than the method of removing distortion.

[0099] In addition, some embodiments of the present application also provide an electronic device. The electronic device may be a digital computer in various forms, such as a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, etc. The electronic device may also be a mobile device in various forms, such as a personal digital processing, a cellular phone, a smart phone, a wearable device, and other similar computing devices.

[0100] The electronic device includes: one or more processors; and a memory storing computer program instructions, wherein when the computer program instructions are executed, the processor executes the steps of the method provided in any one or more of the above embodiments. Figure 7 An exemplary structural diagram of the electronic device is disclosed. Figure 7 As shown, the electronic device includes: one or more processors 701, memory 702, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Among them, the components shown in this article, their connections and relationships, and their functions are only used as examples, and are not intended to limit the implementation of the present application described and / or required herein.

[0101] The electronic device may further include: an input device 703 and an output device 704. The processor 701, the memory 702, the input device 703 and the output device 704 may be connected via a bus or other means. Figure 7 The example of connecting through bus is taken in the following.

[0102] The input device 703 can receive input digital or character information, and generate key signal input related to the user settings and function control of the electronic device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick and other input devices. The output device 704 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display and a plasma display. In some embodiments, the display device may be a touch screen.

[0103] To provide interaction with a user, the electronic device may be a computer. The computer has: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball), through which the user can provide input to the computer. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0104] In the embodiments of the present application, a computer program / instruction is stored on a computer-readable medium, and when the computer program / instruction is executed by a processor, the steps of the method provided by any one or more of the above embodiments are implemented. The computer-readable medium may be included in the electronic device described in the above embodiments; or it may exist independently without being assembled into the device. The above computer-readable medium carries one or more computer-readable instructions.

[0105] The memory 702 can be used as a non-transient computer-readable storage medium, which can be used to store non-transient software programs, non-transient computer executable programs and modules. The processor 701 executes various functional applications and data processing of the server by running the non-transient software programs, instructions and modules stored in the memory 702, so as to implement the program instructions / modules corresponding to the method provided by any one or more of the above embodiments in the embodiments of the present application.

[0106] The memory 702 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 702 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage devices. In some embodiments, the memory 702 may optionally include a memory remotely arranged relative to the processor 701, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0107] It should be noted that the computer-readable medium described in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device.

[0108] Computer readable media include permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, modules of programs or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0109] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0110] In the above-described embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. For example, it can be implemented by using an application specific integrated circuit (ASIC), a general-purpose computer or any other similar hardware device. In certain embodiments, the software program of the present application can be executed by a processor to implement the above steps or functions. Similarly, the software program of the present application (including relevant data structures) can be stored in a computer-readable recording medium, for example, a RAM memory, a magnetic or optical drive or a floppy disk and similar devices. In addition, some steps or functions of the present application can be implemented by hardware, for example, as a circuit that cooperates with a processor to perform each step or function.

[0111] The computer program product provided in the embodiment of the present application includes one or more computer programs / instructions, and when the computer program / instructions are executed by the processor, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from a computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0112] The flow chart or block diagram in the accompanying drawings shows the possible architecture, function and operation of the equipment, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated system for hardware that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0113] The scope of the present application is defined by the appended claims rather than the above description, and it is intended that all changes that fall within the meaning and scope of the equivalent elements of the claims are included in the present application. Any figure mark in the claims should not be regarded as limiting the claims involved. In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the device claim may also be implemented by one unit or device through software or hardware. The words "first", "second", etc. are only used to distinguish the description, and do not indicate any particular order, nor can they be understood as indicating or implying relative importance.

[0114] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily mention changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims, and the above embodiments should be regarded as exemplary and non-restrictive.

Claims

1. A method for establishing positioning mapping of a fisheye camera, characterized in that: Calculate the internal parameters and distortion coefficients of the fisheye camera through the calibration plate; Determine the external parameters of the fisheye camera according to the world coordinates and pixel coordinates of at least six marking positions within the field of view of the fisheye camera; Obtaining a vector map where the field of view of the fisheye camera is located, performing world coordinate interpolation based on points with known coordinates in the vector map, obtaining interpolation points and world coordinates of the interpolation points, and forming an interpolation point set; According to the internal parameters, distortion coefficients and external parameters of the fisheye camera, the interpolation point set is projected to a pixel coordinate system to obtain the pixel coordinates of each point in the interpolation point set; A mapping table of the fisheye camera is established according to the world coordinates and the pixel coordinates of each point in the interpolation point set.

2. The method according to claim 1, characterized in that Obtaining a vector map where the field of view of the fisheye camera is located, performing world coordinate interpolation based on the points with known coordinates in the vector map, obtaining interpolation points and world coordinates of the interpolation points, and forming an interpolation point set, including: World coordinate interpolation is performed based on the coordinate information of the lane line to obtain interpolation points and the world coordinates of the interpolation points to form an interpolation point set.

3. The method according to claim 2, characterized in that Perform world coordinate interpolation based on the coordinate information of the lane line to obtain the interpolation point and the world coordinate of the interpolation point, including: The lane boundary lines are interpolated using the linear interpolation method to obtain dense lane line interpolation points and the world coordinates of the interpolation points.

4. The method according to claim 3, characterized in that After the lane boundary lines are interpolated using a linear interpolation method to obtain dense lane line interpolation points and the world coordinates of the interpolation points, the method further includes: The left boundary line and the right boundary line formed by the interpolated lane boundary line are interpolated within the lane area to obtain the interpolation points within the lane area and the world coordinates of the interpolation points.

5. The method according to claim 4, characterized in that Interpolate the left boundary line and the right boundary line formed by the interpolated lane boundary line within the lane area, and obtain the interpolation points within the lane area and the world coordinates of the interpolation points, including: The interpolation points on the left boundary line are traversed, and the nearest corresponding interpolation point is found among the interpolation points on the right boundary line. Interpolation is performed between the interpolation point and its corresponding interpolation point according to the first resolution to obtain the interpolation point in the lane area and the world coordinates of the interpolation point.

6. The method according to claim 5, characterized in that The method further comprises: If it is identified that there is no interpolation point in the target area, then interpolation points that are aligned with each other are searched around the target area; Interpolation is performed between the mutually aligned interpolation points at a second resolution; wherein the second resolution is smaller than the first resolution.

7. The method according to claim 1, characterized in that The mapping table of the fisheye camera is established in the form of a dictionary.

8. An electronic device, characterized in that: The electronic device comprises: one or more processors; and A memory storing computer program instructions, which, when executed, cause the processor to perform the steps of the method as claimed in any one of claims 1 to 7.

9. A computer readable medium having a computer program / instructions stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images