A vehicle panoramic image calibration method, device, equipment and storage medium

Abstract

The application discloses a vehicle panoramic image calibration method, device and equipment and a storage medium. The method comprises the following steps: if it is detected that the original checkerboard calibration field exists a calibration obstacle, the number of calibration points of the original checkerboard calibration field is determined, and the density of the checkerboard calibration field is increased; the preset surround-view camera on the vehicle is used to collect images of the target checkerboard calibration field after the density of the checkerboard calibration field is increased, and a target image is determined; based on the number of calibration points, feature point searching is performed on the target image, and it is determined whether the corresponding number of target feature points is detected; if not, the predicted feature points are obtained based on the quadratic programming method according to the newly-added feature points of the target checkerboard calibration field; the calibration of the target checkerboard calibration field is realized according to the predicted feature points and the detected target feature points, and the vehicle panoramic image calibration is performed based on the calibrated target checkerboard calibration field. The efficiency of the vehicle panoramic image calibration can be effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicles, in particular to a vehicle panoramic image calibration method, device, equipment and storage medium. BACKGROUND

[0002] With the development of vehicle automation technology, more and more vehicles are equipped with panoramic image function. When the vehicle encounters an extremely narrow road section, starting the panoramic image function will assist the driver to safely pass through. When the vehicle manufacturer equips the panoramic image function, the panoramic image function needs to be calibrated, the purpose of which is to make the four look-around cameras better integrated. In the process of panoramic image calibration, there are often problems of unrecognized key feature points of panoramic image calibration due to light or ground site wear and tear, thereby causing workers to repeatedly calibrate and affecting the calibration efficiency.

[0003] Therefore, how to improve the calibration efficiency in an automated manner without changing the external environment when detecting calibration obstacles is a problem to be solved at present. SUMMARY

[0004] The present application provides a vehicle panoramic image calibration method, device, equipment and storage medium, which can effectively improve the efficiency of vehicle panoramic image calibration.

[0005] According to an aspect of the present application, a vehicle panoramic image calibration method is provided, comprising:

[0006] If it is detected that the original checkerboard calibration field exists a calibration obstacle, the number of calibration points of the original checkerboard calibration field is determined, and the density of the checkerboard calibration field is increased;

[0007] The look-around camera preset on the vehicle is used to collect images of the target checkerboard calibration field after increasing the density of the checkerboard, and determine the target image;

[0008] Based on the number of calibration points, feature point search is performed on the target image to determine whether the corresponding number of target feature points is detected, if not, the predicted feature points are obtained based on the quadratic programming method according to the newly added feature points of the target checkerboard calibration field;

[0009] According to the predicted feature points and the detected target feature points, the calibration of the target checkerboard calibration field is realized, and the vehicle panoramic image calibration is performed based on the calibrated target checkerboard calibration field.

[0010] According to another aspect of the present application, a vehicle panoramic image calibration device is provided, comprising:

[0011] The density increasing module is configured to determine the number of calibration points of the original grid calibration field and increase the grid density of the grid calibration field if it is detected that the original grid calibration field has the calibration obstacle condition.

[0012] The image acquisition module is configured to acquire images of the target grid calibration field after the grid density is increased by using the preset surround-view camera on the vehicle, and determine target images.

[0013] The prediction module is configured to search for feature points on the target images based on the number of calibration points, and determine whether a corresponding number of target feature points are detected, and if not, obtain predicted feature points based on the secondary planning method according to the newly added feature points of the target grid calibration field.

[0014] The calibration module is configured to calibrate the target grid calibration field according to the predicted feature points and the detected target feature points, and calibrate the panoramic image of the vehicle based on the calibrated target grid calibration field.

[0015] According to another aspect of the present application, an electronic device is provided, which comprises:

[0016] at least one processor; and

[0017] a memory connected to the at least one processor in communication; wherein

[0018] the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the vehicle panoramic image calibration method according to any one of the embodiments of the present application.

[0019] According to another aspect of the present application, a computer readable storage medium is provided, which stores computer instructions for enabling a processor to implement the vehicle panoramic image calibration method according to any one of the embodiments of the present application when executed by the processor.

[0020] The technical scheme of the embodiment of the present application, if the original chessboard calibration field is detected to exist a calibration obstacle, the number of calibration points of the original chessboard calibration field is determined, and the chessboard density of the chessboard calibration field is increased; the preset surround-view camera on the vehicle is used to collect images of the target chessboard calibration field after the chessboard density is increased, and a target image is determined; based on the number of calibration points, feature point searching is performed on the target image, and it is determined whether the corresponding number of target feature points is detected, if not, the predicted feature points are obtained based on the quadratic programming method according to the new feature points of the target chessboard calibration field; the calibration of the target chessboard calibration field is realized according to the predicted feature points and the detected target feature points, and the vehicle panoramic image calibration is performed based on the calibrated target chessboard calibration field. By using the quadratic programming method, the feature points that cannot be recognized are virtually constructed by using the feature points that have been recognized, which can effectively improve the calibration efficiency and facilitate the realization of the subsequent vehicle panoramic image function.

[0021] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0023] Figure 1 is a flowchart of a vehicle panoramic image calibration method provided by the first embodiment of the present application;

[0024] Figure 2A is a flowchart of a vehicle panoramic image calibration method provided by the second embodiment of the present application;

[0025] Figure 2B is a schematic diagram of increasing the calibration field chessboard density provided by the second embodiment of the present application;

[0026] Figure 3 is a structural block diagram of a vehicle panoramic image calibration device provided by the third embodiment of the present application;

[0027] Figure 4 is a structural schematic diagram of an electronic device provided by the fourth embodiment of the present application. DETAILED DESCRIPTION

[0028] In the following, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application, so that those skilled in the art can better understand the technical solutions of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work should fall within the scope of protection of the present application.

[0029] It should be noted that the terms "first", "second", "target", "candidate", "alternative" and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0030] Embodiment one

[0031] Figure 1 is a flowchart of a vehicle panoramic image calibration method provided by the first embodiment of the present application; the present embodiment can be applied to the case where the vehicle needs to be calibrated for panoramic image before or after leaving the factory so that the cameras of the vehicle can be better integrated. The method can be executed by a vehicle panoramic image calibration device, which can be realized in the form of hardware and / or software, and can be configured in an electronic device, such as a vehicle. As shown in the figure, the vehicle panoramic image calibration method comprises the following steps. Figure 1

[0032] S101, if it is detected that the original checkerboard calibration field exists a calibration obstacle, the number of calibration points of the original checkerboard calibration field is determined, and the density of the checkerboard calibration field is increased.

[0033] ​The original chessboard calibration field refers to an original calibration field environment of the vehicle. The original chessboard calibration field can be a 2*2 size and density chessboard calibration field, that is, it has four black and white grid areas distributed at intervals, wherein the white grid area is two and the black grid area is two. The calibration obstacle condition can be that the light is too bright and / or the calibration field ground is worn. If the original chessboard calibration field is 2*2 size and density, the corresponding number of calibration points is five. After increasing the chessboard density of the chessboard calibration field, the chessboard calibration field can be changed to 4*4 size and density, that is, it has 16 black and white grid areas distributed at intervals, wherein the white grid area is 8 and the black grid area is 8.

[0034] Optionally, detecting that the original chessboard calibration field has a calibration obstacle condition comprises: using a preset surround-view camera on the vehicle to collect an image of the original chessboard calibration field to determine an original image; performing image analysis and processing on the original image, and determining whether the original chessboard calibration field has a light too bright and / or calibration field ground wear condition according to the processing result, and if so, determining that the original chessboard calibration field has a calibration obstacle condition.

[0035] Optionally, the original image can be analyzed for brightness to determine whether there is a region with brightness greater than a preset brightness threshold in the original image, and if so, it is determined that the original chessboard calibration field has a light too bright condition. The original image can also be subjected to target detection to determine whether there is a wear area in the chessboard, and if so, it is determined that the original chessboard calibration field has a field ground wear condition.

[0036] Optionally, increasing the chessboard density of the chessboard calibration field comprises: if the density of the original chessboard calibration field is 2*2, then according to the geometric shape features of the original chessboard calibration field, each grid area of the original chessboard calibration field is divided into four sub-areas to obtain a target chessboard calibration field with a density of 4*4, thereby increasing the chessboard density of the chessboard calibration field.

[0037] Optionally, the original chessboard calibration field can be monitored by the surround-view camera, and when it is detected that the original chessboard calibration field has a calibration obstacle condition, relevant personnel are notified to correct the chessboard density of the calibration field from 2*2 to 4*4, that is, to increase the chessboard density of the chessboard calibration field, so as to facilitate subsequent recalibration of all feature points of the original chessboard calibration field by the surround-view camera.

[0038] S102, using a preset surround-view camera on the vehicle to collect an image of the target chessboard calibration field after increasing the chessboard density, to determine a target image.

[0039] The number of the surround-view ray heads can be four, and the surround-view cameras can be pre-configured at different positions in front and back of the vehicle to collect images of the environment around the vehicle. The target image refers to an image collected by the surround-view cameras after fusion, which can represent the situation of the chessboard field on the target chessboard field after the density is increased.

[0040] Optionally, at least two surround-view cameras pre-configured on the vehicle can be used to collect images of the target chessboard field after the density of the chessboard is increased, to obtain at least two candidate images, and based on a preset image fusion algorithm, the candidate images collected by different surround-view cameras can be fused to determine the target image.

[0041] In S103, based on the number of the calibration points, feature point searching is performed on the target image to determine whether the corresponding number of target feature points is detected. If not, the predicted feature points are obtained based on the quadratic programming method according to the newly added feature points of the target chessboard field.

[0042] For example, if the number of calibration points is five, it is determined whether five feature points can be detected on the target image. If not, it is determined that there are unidentified feature points. At this time, the positions of the unidentified feature points are predicted based on the quadratic programming method according to the newly added feature points after the density of the chessboard is increased, to obtain the predicted feature points. When five feature points except the newly added feature points can be determined on the target image, it is considered that the calibration of the original chessboard field is achieved.

[0043] Optionally, the feature point searching on the target image to determine whether the corresponding number of target feature points is detected includes: starting from the area with a higher confidence, searching from inside to outside, and matching the calibration points on the original chessboard field on the target image; and based on the relationship between the number of the matched calibration points and the number of the calibration points of the original chessboard field, it is determined whether the corresponding number of target feature points is detected.

[0044] Optionally, based on a preset rule, the confidence of different black and white regions on the target image can be set from inside to outside from large to small. Further, starting from the area with a higher confidence, searching from inside to outside, and matching the corresponding calibration points on the original chessboard field on the target image are performed. If the number of the matched calibration points is the same as the number of the calibration points of the original chessboard field, it is determined that the corresponding number of target feature points is detected. At this time, the operation of constructing the predicted feature points according to the newly added feature points is not needed to be performed, and the calibration of the target chessboard field can be achieved according to the matched feature points.

[0045] Optionally, if the number of matched calibration points is different from the number of calibration points of the original grid calibration field, for example, the number of matched calibration points is less than the number of calibration points of the original grid calibration field, the predicted feature points can be determined according to the new feature points, specifically, the predicted feature points are obtained based on the quadratic programming method according to the position information and position structure relationship of the new feature points of the target grid calibration field, including: determining the objective function and constraint condition of the quadratic programming problem; based on the objective function and constraint condition of the quadratic programming method, the positions of the feature points located in the central region of the new feature points are predicted according to the position information and position structure relationship of the new feature points of the target grid calibration field, and the predicted feature points are obtained.

[0046] Optionally, the objective function and constraint condition of the quadratic programming problem are determined, including: based on the principle that the distance between the predicted feature points and each new feature point is as possible as the same, the objective function of the quadratic programming problem is determined; based on the principle that the difference between the distance of the predicted feature points and the upper row grid and the distance of the predicted feature points and the lower row grid is as possible as small, and the principle that the predicted feature points are located in the boundary region composed of the new feature points, the constraint condition of the quadratic programming problem is determined.

[0047] S104, according to the predicted feature points and the detected target feature points, the calibration of the target grid calibration field is realized, and the vehicle panoramic image calibration is carried out based on the calibrated target grid calibration field.

[0048] Wherein, the target feature point refers to the feature point matched by the feature point search on the target image. The calibrated target grid calibration field refers to the calibration field in which all the calibration point positions of the original grid calibration field are calibrated on the target image.

[0049] It should be noted that, based on the quadratic programming method, the feature points that cannot be identified are virtually constructed by using the identified feature points, which can better fuse the four surround-view cameras, solve the problem that the key feature points of the panoramic image calibration cannot be identified due to light or ground site wear, and improve the calibration efficiency.

[0050] Optionally, before the vehicle panoramic image calibration based on the calibrated target grid calibration field, it further includes: obtaining the independent intrinsic parameters of each surround-view camera of the vehicle, adjusting the camera resolution of each surround-view camera to the maximum to improve the calibration accuracy; adjusting the light brightness so that the light is not too bright or too dark to ensure the clarity of the feature points and the boundary in the target grid calibration field.

[0051] It should be noted that factors affecting calibration pass rate are mainly divided into environmental factors and intrinsic factors. Intrinsic factors mainly include: camera resolution: the higher the camera resolution within the same field of view, the higher the calibration accuracy; independent intrinsic parameters of each camera: each camera has differences, and obtaining the independent intrinsic parameters of four cameras will improve calibration accuracy. Environmental factors mainly include lighting factors and site factors. Lighting factors, i.e., excessively bright or dim light, will cause corner points and boundaries to be blurred, resulting in the inability to detect corner points or low detection accuracy. Site factors mainly revolve around two aspects: checkerboard pattern manufacturing accuracy: the higher the accuracy of the checkerboard boundary points in the calibration field, the smaller the error; cleanliness of the calibration field: damage, dirt, and other factors can also cause the inability to detect feature points or low detection accuracy. The technical solution of this invention mainly addresses the ground reflection or wear that may be caused by site factors. By increasing the checkerboard density and combining it with a quadratic programming problem to predict unidentified feature points, it effectively improves calibration efficiency and solves the problem of key feature point recognition failure in panoramic image calibration due to lighting or ground wear, which is conducive to the subsequent realization of vehicle panoramic imaging functions.

[0052] The technical solution of this invention involves determining the number of calibration points in the original checkerboard calibration field and increasing the checkerboard density if calibration obstacles are detected. A pre-installed surround-view camera on the vehicle is used to acquire images of the target checkerboard calibration field after the density increase, thus defining the target image. Based on the number of calibration points, feature point search is performed on the target image to determine if the corresponding number of target feature points are detected. If not, predicted feature points are obtained based on the newly added feature points in the target checkerboard calibration field using a quadratic programming method. The target checkerboard calibration field is then calibrated using the predicted feature points and the detected target feature points. Finally, a panoramic image calibration of the vehicle is performed based on the calibrated target checkerboard calibration field. By using a quadratic programming method to virtually construct virtual features for unidentifiable features, calibration efficiency can be effectively improved. Panoramic image calibration experiments allow for better fusion of the vehicle's cameras, facilitating the subsequent implementation of panoramic image functionality.

[0053] Example 2

[0054] Figure 2A This is a flowchart of a vehicle panoramic image calibration method provided in Embodiment 2 of the present invention; Figure 2B This is a schematic diagram of the increased grid density of the calibration field provided in Embodiment 2 of the present invention; based on the above embodiments, this embodiment provides a preferred example for calibrating all feature points of the original grid calibration field.

[0055] like Figure 2A As shown, the vehicle panoramic image calibration method may include the following process:

[0056] (1) Increase the density of the chessboard, that is, increase the density of the chessboard in the standard area. For example, the 2x2 chessboard can be increased to 4x4.

[0057] For example, see Figure 2B The left image shows the original 2x2 checkerboard calibration field, containing five feature points located at the intersections of the boundary lines, represented by solid black circles. The right image shows the 4x4 checkerboard after increasing the checkerboard density of the calibration field. In addition to the five calibration points in the original checkerboard calibration field, four new feature points are added, represented by solid white circles. When calibrating the original checkerboard calibration field in the left image, there may be situations where feature points located at the center of the calibration field cannot be found. In this case, the newly added feature points are used to predict the unidentified feature points (represented by solid gray circles in the right image) to achieve the final calibration.

[0058] (2) Search for feature points. During the feature point search process, only the feature points before increasing the chessboard density are recorded (that is, the five feature points of the original chessboard calibration field are searched first). The feature point search prioritizes searching the region with higher confidence and searches from the outside to the inside.

[0059] (3) If any feature points in the original chessboard calibration field are not identified, the calibration is deemed to have failed. In this case, it is necessary to search for the remaining feature points, i.e., to add new feature points. These remaining feature points are then used to construct virtual feature points for the unidentified feature points. In the quadratic programming process, the objective function should first be defined, i.e., the objective function should ensure that the selection of all blue points is as smooth as possible. Secondly, the cost function should be selected to minimize the difference between the cost function and the upper and lower rows of the chessboard.

[0060] (4) If all feature points of the original chessboard calibration field are identified, the calibration is directly determined to be successful, and the calibration of the original chessboard calibration field is completed.

[0061] Example 3

[0062] Figure 3 This is a structural block diagram of a vehicle panoramic image calibration device provided in Embodiment 3 of the present invention. This embodiment is applicable to situations where panoramic image calibration tests are required before or after a vehicle leaves the factory to improve the fusion of the various cameras. The vehicle panoramic image calibration device can be implemented in hardware and / or software and configured in a device with vehicle panoramic image calibration functionality, such as... Figure 3 As shown, the device specifically includes:

[0063] The density increasing module 301 is used to determine the number of calibration points in the original chessboard calibration field and increase the chessboard density of the chessboard calibration field if calibration obstacles are detected in the original chessboard calibration field.

[0064] The image acquisition module 302 is used to acquire images of the target chessboard calibration field after the chessboard density is increased by using a pre-set surround view camera on the vehicle, and to determine the target image.

[0065] The prediction module 303 is used to search for feature points on the target image based on the number of calibration points, and determine whether the corresponding number of target feature points are detected. If not, the prediction feature points are obtained based on the newly added feature points of the target chessboard calibration field and the quadratic programming method.

[0066] The calibration module 304 is used to calibrate the target chessboard calibration field based on the predicted feature points and the detected target feature points, and to perform vehicle panoramic image calibration based on the calibrated target chessboard calibration field.

[0067] The technical solution of this invention involves determining the number of calibration points in the original checkerboard calibration field and increasing the checkerboard density if calibration obstacles are detected. A pre-installed surround-view camera on the vehicle is used to acquire images of the target checkerboard calibration field after the density increase, thus defining the target image. Based on the number of calibration points, a feature point search is performed on the target image to determine if the corresponding number of target feature points are detected. If not, predicted feature points are obtained based on the newly added feature points in the target checkerboard calibration field using a quadratic programming method. The target checkerboard calibration field is then calibrated using the predicted feature points and the detected target feature points. Finally, a panoramic vehicle image calibration is performed based on the calibrated target checkerboard calibration field. By using a quadratic programming method to virtually construct virtual features for unidentifiable features using already identified features, calibration efficiency can be effectively improved, facilitating the subsequent implementation of the panoramic vehicle image function.

[0068] Furthermore, the density increasing module 301 is specifically used for:

[0069] The vehicle uses a pre-installed surround-view camera to capture images of the original chessboard grid and determine the original image.

[0070] The original image is processed by image analysis. Based on the processing results, it is determined whether there is excessive light or wear on the ground of the original checkerboard calibration field. If so, it is determined that there is a calibration obstacle in the original checkerboard calibration field.

[0071] Furthermore, the density increasing module 301 is also used for:

[0072] If the density of the original chessboard calibration field is 2×2, then based on the geometric characteristics of the original chessboard calibration field, each grid region of the original chessboard calibration field is divided into four sub-regions to obtain a target chessboard calibration field with a density of 4×4, thereby increasing the chessboard density of the chessboard calibration field.

[0073] Furthermore, the prediction module 303 is specifically used for:

[0074] Starting from the region with high confidence, the search proceeds from the inside out, matching the calibration points on the original chessboard calibration field on the target image;

[0075] Based on the relationship between the number of matched calibration points and the number of calibration points in the original chessboard calibration field, it is determined whether the corresponding number of target feature points have been detected.

[0076] Furthermore, the prediction module 303 may include:

[0077] The element is defined to determine the objective function and constraints of the quadratic programming problem.

[0078] The prediction unit is used to predict the position of feature points located in the central region of the newly added feature points based on the objective function and constraints of the quadratic programming method, according to the position information and positional structure relationship of the newly added feature points in the target chessboard calibration field, and thus obtain the predicted feature points.

[0079] Furthermore, the specific use of the unit is as follows:

[0080] Based on the principle of making the distance between the predicted feature points and each newly added feature point as equal as possible, the objective function of the quadratic programming problem is determined.

[0081] Based on the principles of minimizing the difference between the predicted feature point and the upper row of the chessboard, and between the predicted feature point and the lower row of the chessboard, and the principle that the predicted feature point lies within the boundary region formed by the newly added feature points, the constraints of the quadratic programming problem are determined.

[0082] Furthermore, the above-mentioned device is also used for:

[0083] Obtain the independent intrinsic parameters of each surround-view camera in the vehicle, and adjust the camera resolution of each surround-view camera to the maximum to improve calibration accuracy;

[0084] Adjust the light intensity to ensure that the light is neither too bright nor too dim, so as to guarantee the clarity of feature points and boundaries in the target chessboard marking field.

[0085] Example 4

[0086] Figure 4 This is a schematic diagram of the structure of the electronic device provided in Embodiment 4 of the present invention. Figure 4A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0087] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0088] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0089] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as vehicle panoramic image calibration methods.

[0090] In some embodiments, the vehicle panoramic image calibration method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the vehicle panoramic image calibration method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the vehicle panoramic image calibration method by any other suitable means (e.g., by means of firmware).

[0091] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0092] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0093] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0094] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0095] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0096] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0097] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0098] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for calibrating a vehicle panoramic image, characterized in that, include: If a calibration obstacle is detected in the original chessboard calibration field, the number of calibration points in the original chessboard calibration field is determined, and the chessboard density of the chessboard calibration field is increased. The method of increasing the grid density of the grid calibration field includes: if the density of the original grid calibration field is 2×2, then according to the geometric shape characteristics of the original grid calibration field, each grid region of the original grid calibration field is divided into four sub-regions to obtain a target grid calibration field with a density of 4×4, thereby increasing the grid density of the grid calibration field. Using a pre-installed surround-view camera on the vehicle, images of the target chessboard calibration field after the chessboard grid density is increased are acquired to determine the target image; Based on the number of calibration points, feature point search is performed on the target image to determine whether the corresponding number of target feature points are detected. If not, based on the newly added feature points in the target chessboard calibration field, the predicted feature points are obtained using a quadratic programming method. The newly added feature points are other feature points besides the calibration points in the original chessboard calibration field after increasing the chessboard density of the chessboard calibration field. Based on the predicted feature points and the detected target feature points, the target chessboard calibration field is calibrated, and the vehicle panoramic image is calibrated based on the calibrated target chessboard calibration field. The step of obtaining predicted feature points based on the newly added feature points determined by the target chessboard grid marking, using a quadratic programming method, includes: Based on the principle of making the distance between the predicted feature points and each newly added feature point the same, the objective function of the quadratic programming problem is determined. Based on the principles of minimizing the difference between the distances of the predicted feature points to the upper and lower grid squares, and the principle that the predicted feature points are located within the boundary region formed by the newly added feature points, the constraints of the quadratic programming problem are determined. Based on the objective function and constraints of the quadratic programming problem, the position of the feature point located in the central region of the newly added feature point is predicted according to the position information and positional structure relationship of the newly added feature point in the target chessboard calibration field, thus obtaining the predicted feature point.

2. The method according to claim 1, characterized in that, An obstacle was detected in the original chessboard calibration field, including: The vehicle uses a pre-installed surround-view camera to capture images of the original chessboard grid and determine the original image. The original image is processed by image analysis. Based on the processing results, it is determined whether there is excessive light or wear on the ground of the original checkerboard calibration field. If so, it is determined that there is a calibration obstacle in the original checkerboard calibration field.

3. The method according to claim 1, characterized in that, Feature point search is performed on the target image to determine whether the corresponding number of target feature points have been detected, including: Starting from the region with high confidence, the search proceeds from the inside out, matching the calibration points on the original chessboard calibration field on the target image; Based on the relationship between the number of matched calibration points and the number of calibration points in the original chessboard calibration field, it is determined whether the corresponding number of target feature points have been detected.

4. The method according to claim 1, characterized in that, Before performing vehicle panoramic image calibration based on the calibrated target chessboard calibration field, the following steps are also included: Obtain the independent intrinsic parameters of each surround-view camera in the vehicle, and adjust the camera resolution of each surround-view camera to the maximum to improve calibration accuracy; Adjust the light intensity to ensure that the light is neither too bright nor too dim, so as to guarantee the clarity of feature points and boundaries in the target chessboard marking field.

5. A vehicle panoramic image calibration device, characterized in that, include: The density increase module is used to determine the number of calibration points in the original chessboard calibration field and increase the chessboard density of the chessboard calibration field if calibration obstacles are detected in the original chessboard calibration field. The density increasing module is specifically used to divide each grid region of the original chessboard calibration field into four sub-regions according to the geometric shape characteristics of the original chessboard calibration field if the density of the original chessboard calibration field is 2×2, thereby obtaining a target chessboard calibration field with a density of 4×4, thus increasing the chessboard density of the chessboard calibration field. The image acquisition module is used to acquire images of the target chessboard calibration field after the chessboard density is increased by using the pre-set surround view camera on the vehicle, and to determine the target image. The prediction module is used to search for feature points on the target image based on the number of calibration points to determine whether the corresponding number of target feature points are detected. If not, the predicted feature points are obtained based on the newly added feature points in the target chessboard calibration field using a quadratic programming method. The newly added feature points are other feature points besides the calibration points in the original chessboard calibration field after increasing the chessboard density of the chessboard calibration field. The calibration module is used to calibrate the target chessboard calibration field based on the predicted feature points and the detected target feature points, and to perform vehicle panoramic image calibration based on the calibrated target chessboard calibration field. The prediction module includes a determination unit and a prediction unit; The determining unit is used for: Based on the principle of making the distance between the predicted feature points and each newly added feature point the same, the objective function of the quadratic programming problem is determined. Based on the principles of minimizing the difference between the distances of the predicted feature points to the upper and lower grid squares, and the principle that the predicted feature points are located within the boundary region formed by the newly added feature points, the constraints of the quadratic programming problem are determined. The prediction unit is used to predict the position of the feature point located in the central region of the newly added feature point based on the objective function and constraints of the quadratic programming problem, and according to the position information and positional structure relationship of the newly added feature point in the target chessboard calibration field, so as to obtain the predicted feature point.

6. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the vehicle panoramic image calibration method according to any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the vehicle panoramic image calibration method according to any one of claims 1-4.

Images