Engineering vehicle panoramic image calibration method and device and engineering vehicle

Abstract

This invention relates to the field of engineering vehicle technology, and provides a method, apparatus, and engineering vehicle for panoramic image calibration of engineering vehicles. The method, used on the vehicle side, includes: acquiring a list of calibration parameters for existing vehicle models stored on the vehicle side; determining first calibration parameters based on the list of calibration parameters; generating a first panoramic image of the vehicle to be calibrated based on the first calibration parameters; determining second calibration parameters based on a calibration tool pre-set outside the vehicle to be calibrated and the first panoramic image, wherein the second calibration parameters are external parameters of a camera on the vehicle to be calibrated; and generating a second panoramic image of the vehicle to be calibrated based on the second calibration parameters. This invention enables direct calibration of the vehicle to be calibrated on the vehicle side and generates a panoramic image adapted to the vehicle, eliminating the need for third-party calibration and saving significant manpower.

Description

Technical Field

[0001] This invention relates to the field of engineering vehicle technology, and in particular to a method, device, and engineering vehicle for panoramic image calibration. Background Technology

[0002] With the rapid development of image and computer vision technologies, panoramic imaging technology is widely used in the automotive field. For some large vehicles, such as construction vehicles (excavators, pile drivers, etc.), the driver's blind spot is larger than that of ordinary passenger cars, and the risk of collisions and scrapes is higher. Therefore, panoramic imaging is particularly important for large vehicles.

[0003] Because large vehicles, especially engineering vehicles, have many models and significant size variations, the camera's coordinates in the vehicle's coordinate system change considerably after installation on a new vehicle model. Even using a panoramic image generation program matched to the camera, it's impossible to generate a panoramic image adapted to the new vehicle model. Therefore, panoramic image calibration is required before each new vehicle model is launched, generating a panoramic image adapted to that model on the driver's cab display. Furthermore, during engineering operations, collisions can deform the camera bracket, causing camera displacement, necessitating recalibration of the panoramic image. Otherwise, the generated panoramic image will suffer from misalignment and distortion due to improper stitching. Currently, most engineering vehicle manufacturers purchase third-party cameras and their corresponding panoramic image calibration and stitching algorithms. Third-party calibration is required for new model launches and when camera displacement necessitates recalibration, making it impossible to use existing panoramic image generation programs on the vehicle itself to calibrate and generate panoramic images for the vehicle to be calibrated (e.g., a new model). Summary of the Invention

[0004] This invention provides a method, apparatus, and engineering vehicle for calibrating panoramic images of engineering vehicles, thereby solving the problem in the prior art that it is impossible to calibrate the vehicle to be calibrated and generate panoramic images at the vehicle end.

[0005] This invention provides a method for calibrating panoramic images of engineering vehicles, for use on the vehicle side, comprising:

[0006] Retrieve the list of calibration parameters for existing vehicle models stored on the vehicle side;

[0007] The first calibration parameter is determined based on the calibration parameter list;

[0008] A first panoramic image of the vehicle to be calibrated is generated based on the first calibration parameters.

[0009] The second calibration parameters are determined based on the calibration tool pre-set outside the vehicle to be calibrated and the first panoramic image. The second calibration parameters are the external parameters of the camera on the vehicle to be calibrated.

[0010] A second panoramic image of the vehicle to be calibrated is generated based on the second calibration parameters.

[0011] According to the present invention, a panoramic image calibration method for engineering vehicles is provided, wherein the first calibration parameter is the calibration parameter of an existing vehicle model that is closest to the external dimensions of the vehicle to be calibrated.

[0012] According to the present invention, a method for calibrating panoramic images of engineering vehicles, wherein generating a first panoramic image of the vehicle to be calibrated based on the first calibration parameters includes:

[0013] Call the dynamic library related to video reading in the SDK integrated on the vehicle to obtain the video frames captured in real time by the camera on the vehicle to be calibrated;

[0014] The system calls the dynamic library related to format conversion in the SDK integrated on the vehicle side to perform format conversion on the video frames;

[0015] The system calls the dynamic library related to video stitching in the SDK integrated on the vehicle side, and uses the first calibration parameters to stitch the format-converted video frames into the first panoramic image.

[0016] According to the present invention, a method for calibrating panoramic images of engineering vehicles, after generating a first panoramic image of the vehicle to be calibrated based on the first calibration parameters, further includes: calling a dynamic library related to the output video in the SDK integrated on the vehicle side, and storing the first panoramic image in the video buffer on the vehicle side.

[0017] According to the present invention, a method for calibrating panoramic images of engineering vehicles, wherein determining second calibration parameters based on a calibration tool pre-installed outside the vehicle to be calibrated and the first panoramic image includes:

[0018] Call the dynamic library related to video reading in the SDK integrated on the vehicle to obtain the video data of the calibration tool captured in real time by the camera on the vehicle to be calibrated;

[0019] The coordinate position of the camera on the vehicle to be calibrated in the vehicle coordinate system is determined based on the video data from the calibration tool.

[0020] A temporary panoramic image is generated based on video data captured by the camera under the same shooting conditions as the first panoramic image and the coordinate position.

[0021] Based on the difference in stitching seams in the overlapping area of ​​the temporary panoramic image and the first panoramic image, the coordinate position is adjusted, and the adjusted coordinate position is determined as the second calibration parameter.

[0022] According to the present invention, a method for calibrating panoramic images of engineering vehicles, after generating a second panoramic image of the vehicle to be calibrated based on the second calibration parameters, further includes:

[0023] The second calibration parameters and the corresponding vehicle information of the vehicle to be calibrated are sent to the remote vehicle management system for storage.

[0024] According to the present invention, a method for calibrating panoramic images of engineering vehicles, after determining the second calibration parameters based on a calibration tool pre-set outside the vehicle to be calibrated and the first panoramic image, further includes: determining the position of the three-color line at the rear of the vehicle to be calibrated in the rear video frame based on the second calibration parameters and the rear video frame captured by the camera on the vehicle to be calibrated.

[0025] The present invention also provides a panoramic image calibration device for engineering vehicles, for use on the vehicle side, comprising:

[0026] The calibration parameter list acquisition module is used to acquire the calibration parameter list of existing vehicle models stored on the vehicle side;

[0027] The first calibration parameter determination module is used to determine the first calibration parameter based on the calibration parameter list;

[0028] The first panoramic image generation module is used to generate a first panoramic image of the vehicle to be calibrated based on the first calibration parameters.

[0029] The second calibration parameter determination module is used to determine the second calibration parameters based on the calibration tool pre-set outside the vehicle to be calibrated and the first panoramic image. The second calibration parameters are the external parameters of the camera on the vehicle to be calibrated.

[0030] The second panoramic image generation module is used to generate a second panoramic image of the vehicle to be calibrated based on the second calibration parameters.

[0031] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the panoramic image calibration method for engineering vehicles described above.

[0032] The present invention also provides an engineering vehicle including the above-described electronic equipment.

[0033] The present invention provides a method, apparatus, and engineering vehicle for calibrating panoramic images of engineering vehicles. It determines first calibration parameters using a list of existing calibration parameters for different vehicle models, generates a first panoramic image of the vehicle to be calibrated based on these parameters, and determines second calibration parameters (external parameters of the camera on the vehicle) based on a pre-installed calibration tool on the vehicle's exterior and the first panoramic image. The second calibration parameters are then used to generate a second panoramic image of the vehicle. This method enables direct calibration of the vehicle at the vehicle's exterior and generates a second panoramic image adapted to it, eliminating the need for third-party calibration and saving significant manpower. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0035] Figure 1 This is a flowchart illustrating the panoramic image calibration method for engineering vehicles provided by the present invention.

[0036] Figure 2 This is a schematic diagram of the structure of the panoramic image calibration device for engineering vehicles provided by the present invention;

[0037] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0039] The panoramic image calibration method for engineering vehicles according to embodiments of the present invention is used on the vehicle side, such as... Figure 1 As shown, the method includes:

[0040] Step S110: Obtain the calibration parameter list of existing vehicle models stored on the vehicle side. Specifically, the vehicle side has already stored the calibration parameters of previously calibrated existing vehicle models and integrates a panoramic image generation program. The calibration parameter list is stored in the vehicle side's memory, for example, in the same memory as the panoramic image generation program, to facilitate the panoramic image generation program's reading of the calibration parameters. When the user needs to calibrate the vehicle to be calibrated, they can obtain the above calibration parameter list through the drop-down selection list of the panoramic image generation program's human-machine interface. The calibration parameter list displays each existing vehicle model and its corresponding calibration parameters. Here, calibration parameters can be understood as the external parameters of at least four cameras on the front, rear, left, and right of the vehicle (for engineering vehicles with counterweights, cameras are also set at the rear end of the counterweight), that is, the three-dimensional coordinates in the vehicle coordinate system. Calibration is the process of determining the accurate three-dimensional coordinates of at least four cameras in the vehicle coordinate system.

[0041] Step S120: Determine the first calibration parameter based on the calibration parameter list. The calibration parameter list contains calibration parameters for existing vehicle models. If the vehicle to be calibrated is an existing vehicle model, the calibration parameter corresponding to that existing vehicle model can be directly selected to generate a panoramic image. If the vehicle to be calibrated is a new vehicle model, a calibration parameter from the calibration parameter list can be selected as the first calibration parameter. Since there are significant differences in body size between new and existing vehicle models, this first calibration parameter is not an accurate calibration parameter for the new vehicle model. Therefore, the first panoramic image generated based on the first calibration parameter can only be used as a reference for the panoramic image of the new vehicle model.

[0042] Step S130: Generate a first panoramic image of the vehicle to be calibrated based on the first calibration parameters. Specifically, a panoramic image generation program integrated into the vehicle can be used to generate the first panoramic image based on the first calibration parameters and real-time video frames captured by at least four cameras on the vehicle.

[0043] Step S140: Determine the second calibration parameters based on the calibration tool pre-set outside the vehicle to be calibrated and the first panoramic image. The second calibration parameters are the external parameters of the camera on the vehicle to be calibrated.

[0044] Step S150: Generate a second panoramic image of the vehicle to be calibrated based on the second calibration parameters. Specifically, a panoramic image generation program integrated into the vehicle can be used to generate the second panoramic image based on the second calibration parameters and real-time video frames captured by at least four cameras on the vehicle.

[0045] The vehicle panoramic image calibration method provided in this embodiment determines first calibration parameters through a list of existing vehicle model calibration parameters on the vehicle side. Based on the first calibration parameters, a first panoramic image of the vehicle to be calibrated is generated. Second calibration parameters are determined based on a calibration tool pre-set outside the vehicle and the first panoramic image. These second calibration parameters are the external parameters of the camera on the vehicle. A second panoramic image of the vehicle to be calibrated is then generated based on these second calibration parameters. This method enables direct calibration of the vehicle on the vehicle side and generates a second panoramic image adapted to the vehicle, eliminating the need for third-party calibration and saving significant manpower.

[0046] In some embodiments, in step S120, the first calibration parameter is the calibration parameter of an existing vehicle model whose external dimensions are closest to those of the vehicle to be calibrated. Specifically, the calibration parameter of an existing vehicle model whose external dimensions are closest to those of the vehicle to be calibrated, selected by the user from the calibration parameter list, is used as the first calibration parameter. Since the first calibration parameter is the calibration parameter of an existing vehicle model whose external dimensions are closest to those of the vehicle to be calibrated, the second calibration parameter can be quickly determined based on the first calibration parameter. One way to determine the closest external dimensions is by calculating the vehicle volume using the length, width, and height dimensions; the vehicle with the closest volume is considered to have the closest external dimensions.

[0047] In some embodiments, the panoramic image generation program is integrated into the vehicle via a software development kit (SDK), and step S130 includes:

[0048] Call the dynamic library related to reading video in the SDK integrated on the vehicle side to obtain the video frames captured in real time by the camera on the vehicle to be calibrated, for example: capturing a video stream in NV12 format with a frame rate of 30fps.

[0049] The system calls the dynamic library related to format conversion in the SDK integrated on the vehicle side to perform format conversion on the video frames, for example, converting the video frames obtained above into RGBA video rendering format.

[0050] The system calls the dynamic library related to video stitching in the SDK integrated on the vehicle side, and uses the first calibration parameters to stitch the format-converted video frames into the first panoramic image.

[0051] Furthermore, after generating the first panoramic image of the vehicle to be calibrated based on the first calibration parameters, the method further includes: calling the dynamic library related to the output video in the SDK integrated on the vehicle side, storing the first panoramic image in the video buffer on the vehicle side, preferably, storing it in a dedicated video buffer on the vehicle side. The dedicated video buffer has a faster read and write speed and can quickly read the first panoramic image, thereby quickly retrieving the real-time video frames of the video buffer and presenting the image of the first panoramic image according to the preset interface layout plan.

[0052] In some embodiments, step S140 includes:

[0053] The system calls the dynamic library related to video reading in the SDK integrated on the vehicle to obtain video data of the calibration tool captured in real time by the camera on the vehicle to be calibrated. The calibration tool can be a calibration cloth with a grid pattern. The calibration cloth can be placed arbitrarily around the vehicle to be calibrated at a distance of 1 to 2 meters (this range ensures that the camera on all models of the vehicle to be calibrated can capture the calibration tool, e.g., 1.5 meters).

[0054] The coordinates of the camera on the vehicle to be calibrated in the vehicle coordinate system are determined based on the video data from the calibration tool. Since the video data from the calibration tool itself has reference data, such as the fixed side length of the grid on the calibration cloth, the coordinates of the camera in the vehicle coordinate system can be obtained by using the camera's intrinsic parameters (mainly focal length) and the ratio of the size of multiple grids in the video data to their actual size.

[0055] A temporary panoramic image is generated based on video data captured by a camera under the same shooting conditions as the first panoramic image and the coordinate position. "Same shooting conditions" means the shooting conditions are the same as those of the video frames captured by the camera used to generate the first panoramic image. In this embodiment, it mainly refers to the same shooting scene, such as keeping the engineering vehicle stationary and removing the calibration tool to obtain the same shooting scene as the first panoramic image.

[0056] Based on the difference in stitching seams in the overlapping area of ​​the temporary panoramic image and the first panoramic image, the coordinate positions are adjusted so that the temporarily panoramic image generated after the coordinate position adjustment is consistent with the image in the overlapping area of ​​the first panoramic image. The adjusted coordinate positions are then determined as the second calibration parameter. After successful calibration, the second calibration parameter is saved. Although the first panoramic image of the vehicle to be calibrated, generated using the first calibration parameter, cannot display a suitable range around the vehicle, the stitching of video frames in the first panoramic image is accurate. Due to positional errors in the calibration tool (e.g., uneven ground causing the calibration cloth to be bent or uneven), the stitching of the temporary panoramic image may be inaccurate, even with significant errors, resulting in noticeable misalignment at the stitching seams. The coordinate position can be adjusted by the difference in the stitching seam between the overlapping area of ​​the first panoramic image and the temporary panoramic image (using the calibration parameters of an existing model that are closer to the external dimensions of the vehicle to be calibrated as the first calibration parameter; the larger the overlapping area between the first panoramic image and the temporary panoramic image, the faster the subsequent adjustment). For example, by changing the z-coordinate of the coordinate position, a temporary panoramic image is generated again. Through image recognition, when the overlapping area images of the first panoramic image and the regenerated temporary panoramic image are consistent, that is, when the stitching seam of the regenerated temporary panoramic image is aligned and the stitching seam of the regenerated temporary panoramic image is consistent with the image at the stitching seam of the first panoramic image, the adjustment is complete.

[0057] In some embodiments, after step S150, the method further includes: after successful calibration, sending the second calibration parameters and the corresponding vehicle information (e.g., vehicle model) of the vehicle to be calibrated to a remote vehicle management system for storage. This allows developers to actively retrieve new model parameters from the remote vehicle management system based on the vehicle information, name the new model, add the new model name and the second calibration parameters to the calibration parameter list to update the calibration parameter list, and then release it according to the prescribed process. When the same type of vehicle is used for the first time subsequently, the updated calibration parameter list can be obtained from the remote vehicle management system, allowing the selection of the second calibration parameters to directly generate the second panoramic image without further calibration.

[0058] In some embodiments, after step S150, the method further includes: determining the position of the three-color lines at the rear of the vehicle in the rear video frames based on the second calibration parameters and the rear video frames captured by the camera on the vehicle to be calibrated (e.g., a rear-facing camera located at the rear end of the vehicle). The three-color lines can be laid out such that the red line is 1 meter from the rear of the vehicle, the yellow line is 2 meters from the rear of the vehicle, and the green line is 3 meters from the rear of the vehicle. Because the second calibration parameters corresponding to the vehicle to be calibrated are used, the obtained three-color lines at the rear of the vehicle are more accurate.

[0059] The panoramic image calibration device for engineering vehicles provided by the present invention is described below. The panoramic image calibration device for engineering vehicles described below and the panoramic image calibration method for engineering vehicles described above can be referred to in correspondence.

[0060] The panoramic image calibration device for engineering vehicles of the present invention is used on the vehicle end, such as... Figure 2 As shown, the device includes:

[0061] The calibration parameter list acquisition module 210 is used to acquire the calibration parameter list of existing vehicle models stored on the vehicle side.

[0062] The first calibration parameter determination module 220 is used to determine the first calibration parameter based on the calibration parameter list.

[0063] The first panoramic image generation module 230 is used to generate a first panoramic image of the vehicle to be calibrated based on the first calibration parameters.

[0064] The second calibration parameter determination module 240 is used to determine a second calibration parameter based on a calibration tool pre-set outside the vehicle to be calibrated and the first panoramic image. The second calibration parameter is the external parameter of the camera on the vehicle to be calibrated.

[0065] The second panoramic image generation module 250 is used to generate a second panoramic image of the vehicle to be calibrated based on the second calibration parameters.

[0066] The panoramic image calibration device for engineering vehicles provided by this invention determines first calibration parameters based on an existing calibration parameter list for the vehicle model. A first panoramic image of the vehicle to be calibrated is generated based on these first calibration parameters. Second calibration parameters are determined based on a calibration tool pre-installed outside the vehicle and the first panoramic image. These second calibration parameters are the external parameters of the camera on the vehicle. A second panoramic image of the vehicle to be calibrated is then generated based on these second calibration parameters. This achieves direct calibration of the vehicle at the vehicle end and generates a second panoramic image adapted to the vehicle, eliminating the need for third-party calibration and saving significant manpower.

[0067] Optionally, the first calibration parameter is the calibration parameter of an existing model that is closest to the external dimensions of the vehicle to be calibrated.

[0068] Optionally, the first panoramic image generation module 230 includes:

[0069] The video frame acquisition module is used to call the dynamic library related to reading video in the SDK integrated on the vehicle side to acquire video frames captured in real time by the camera on the vehicle to be calibrated.

[0070] The format conversion module is used to call the dynamic libraries related to format conversion in the SDK integrated on the vehicle side to convert the format of the video frames.

[0071] The video stitching module is used to call the dynamic library related to video stitching in the SDK integrated on the vehicle side, and use the first calibration parameters to stitch the format-converted video frames into the first panoramic image.

[0072] Optionally, the first panoramic image generation module 230 further includes a panoramic image output module, which calls the dynamic library related to the output video in the SDK integrated on the vehicle side to store the first panoramic image in the video buffer on the vehicle side.

[0073] Optionally, the second calibration parameter determination module 240 includes:

[0074] The calibration video frame acquisition module is used to call the dynamic library related to reading video in the SDK integrated on the vehicle side to obtain the video data of the calibration tool captured in real time by the camera on the vehicle to be calibrated.

[0075] The coordinate calibration module is used to determine the coordinate position of the camera on the vehicle to be calibrated in the vehicle coordinate system based on the video data of the calibration tool.

[0076] The temporary panoramic image generation module is used to generate a temporary panoramic image based on video data captured by the camera under the same shooting conditions as the first panoramic image and the coordinate position.

[0077] The coordinate adjustment module is used to adjust the coordinate position based on the difference in stitching seams in the overlapping area of ​​the temporary panoramic image and the first panoramic image, and to determine the adjusted coordinate position as the second calibration parameter.

[0078] Optionally, the engineering vehicle panoramic image calibration device of the present invention further includes: a second calibration parameter storage module, used to send the second calibration parameters and the corresponding vehicle information of the vehicle to be calibrated to a remote vehicle management system for storage.

[0079] Optionally, the engineering vehicle panoramic image calibration device of the present invention further includes: a three-color line determination module, used to determine the position of the three-color line at the rear of the vehicle to be calibrated in the rear video frame based on the second calibration parameters and the rear video frame captured by the camera on the vehicle to be calibrated.

[0080] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communication interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 can call logical instructions in the memory 330 to execute a panoramic image calibration method for engineering vehicles, the method including:

[0081] Retrieve the list of calibration parameters for existing vehicle models stored on the vehicle side.

[0082] The first calibration parameter is determined based on the calibration parameter list.

[0083] A first panoramic image of the vehicle to be calibrated is generated based on the first calibration parameters.

[0084] The second calibration parameters are determined based on the calibration tool pre-set on the outside of the vehicle to be calibrated and the first panoramic image. The second calibration parameters are the external parameters of the camera on the vehicle to be calibrated.

[0085] A second panoramic image of the vehicle to be calibrated is generated based on the second calibration parameters.

[0086] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0087] The present invention also provides an engineering vehicle that includes the aforementioned electronic equipment. For example, the electronic equipment may be a vehicle ECU (Electronic Control Unit) integrated with a program for executing the aforementioned panoramic image calibration method for engineering vehicles. The engineering vehicle may be a crane, excavator, pile driver, or other construction machinery, or a vehicle such as an aerial work platform truck, fire truck, or cement mixer truck. Because this engineering vehicle possesses the electronic equipment for executing the aforementioned panoramic image calibration method for engineering vehicles, it also exhibits the corresponding technical effects.

[0088] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is able to execute the engineering vehicle panoramic image calibration method provided by the above methods, the method comprising:

[0089] Retrieve the list of calibration parameters for existing vehicle models stored on the vehicle side.

[0090] The first calibration parameter is determined based on the calibration parameter list.

[0091] A first panoramic image of the vehicle to be calibrated is generated based on the first calibration parameters.

[0092] The second calibration parameters are determined based on the calibration tool pre-set on the outside of the vehicle to be calibrated and the first panoramic image. The second calibration parameters are the external parameters of the camera on the vehicle to be calibrated.

[0093] A second panoramic image of the vehicle to be calibrated is generated based on the second calibration parameters.

[0094] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the panoramic image calibration method for engineering vehicles provided by the methods described above, the method comprising:

[0095] Retrieve the list of calibration parameters for existing vehicle models stored on the vehicle side.

[0096] The first calibration parameter is determined based on the calibration parameter list.

[0097] A first panoramic image of the vehicle to be calibrated is generated based on the first calibration parameters.

[0098] The second calibration parameters are determined based on the calibration tool pre-set on the outside of the vehicle to be calibrated and the first panoramic image. The second calibration parameters are the external parameters of the camera on the vehicle to be calibrated.

[0099] A second panoramic image of the vehicle to be calibrated is generated based on the second calibration parameters.

[0100] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0101] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for calibrating panoramic images of engineering vehicles, characterized in that, For use on the vehicle side, including: Retrieve the list of calibration parameters for existing vehicle models stored on the vehicle side; The first calibration parameter is determined based on the calibration parameter list; A first panoramic image of the vehicle to be calibrated is generated based on the first calibration parameters. The second calibration parameters are determined based on the calibration tool pre-set outside the vehicle to be calibrated and the first panoramic image. The second calibration parameters are the external parameters of the camera on the vehicle to be calibrated. A second panoramic image of the vehicle to be calibrated is generated based on the second calibration parameters. The step of determining the second calibration parameters based on a calibration tool pre-set outside the vehicle to be calibrated and the first panoramic image includes: Call the dynamic library related to reading video in the SDK integrated on the vehicle to obtain the video data of the calibration tool captured in real time by the camera on the vehicle to be calibrated; The coordinate position of the camera on the vehicle to be calibrated in the vehicle coordinate system is determined based on the video data from the calibration tool. A temporary panoramic image is generated based on video data captured by the camera under the same shooting conditions as the first panoramic image and the coordinate position. Based on the difference in stitching seams in the overlapping area of ​​the temporary panoramic image and the first panoramic image, the coordinate position is adjusted, and the adjusted coordinate position is determined as the second calibration parameter.

2. The method for calibrating panoramic images of engineering vehicles according to claim 1, characterized in that, The first calibration parameter is the calibration parameter of an existing model that is closest to the external dimensions of the vehicle to be calibrated.

3. The method for calibrating panoramic images of engineering vehicles according to claim 1, characterized in that, The process of generating a first panoramic image of the vehicle to be calibrated based on the first calibration parameters includes: Call the dynamic library related to video reading in the SDK integrated on the vehicle to obtain the video frames captured in real time by the camera on the vehicle to be calibrated; The system calls the dynamic library related to format conversion in the SDK integrated on the vehicle side to perform format conversion on the video frames; The system calls the dynamic library related to video stitching in the SDK integrated on the vehicle side, and uses the first calibration parameters to stitch the format-converted video frames into the first panoramic image.

4. The method for calibrating panoramic images of engineering vehicles according to claim 3, characterized in that, After generating the first panoramic image of the vehicle to be calibrated based on the first calibration parameters, the method further includes: calling the dynamic library related to the output video in the SDK integrated on the vehicle side, and storing the first panoramic image in the video buffer on the vehicle side.

5. The method for calibrating panoramic images of engineering vehicles according to any one of claims 1 to 4, characterized in that, After generating the second panoramic image of the vehicle to be calibrated based on the second calibration parameters, the method further includes: The second calibration parameters and the corresponding vehicle information of the vehicle to be calibrated are sent to the remote vehicle management system for storage.

6. The method for calibrating panoramic images of engineering vehicles according to any one of claims 1 to 4, characterized in that, After determining the second calibration parameters based on the calibration tool pre-set outside the vehicle to be calibrated and the first panoramic image, the method further includes: determining the position of the three-color line at the rear of the vehicle to be calibrated in the rear video frame based on the second calibration parameters and the rear video frame captured by the camera on the vehicle to be calibrated.

7. A panoramic image calibration device for engineering vehicles, characterized in that, For use on the vehicle side, including: The calibration parameter list acquisition module is used to acquire the calibration parameter list of existing vehicle models stored on the vehicle side; The first calibration parameter determination module is used to determine the first calibration parameter based on the calibration parameter list; The first panoramic image generation module is used to generate a first panoramic image of the vehicle to be calibrated based on the first calibration parameters. The second calibration parameter determination module is used to determine the second calibration parameters based on the calibration tool pre-set outside the vehicle to be calibrated and the first panoramic image. The second calibration parameters are the external parameters of the camera on the vehicle to be calibrated. The second panoramic image generation module is used to generate a second panoramic image of the vehicle to be calibrated based on the second calibration parameters. The second calibration parameter determination module includes: The calibration video frame acquisition module is used to call the dynamic library related to reading video in the SDK integrated on the vehicle side to obtain the video data of the calibration tool captured in real time by the camera on the vehicle to be calibrated; The coordinate calibration module is used to determine the coordinate position of the camera on the vehicle to be calibrated in the vehicle coordinate system based on the video data of the calibration tool. A temporary panoramic image generation module is used to generate a temporary panoramic image based on video data captured by a camera under the same shooting conditions as the first panoramic image and the coordinate position. The coordinate adjustment module is used to adjust the coordinate position based on the difference in stitching seams in the overlapping area of ​​the temporary panoramic image and the first panoramic image, and to determine the adjusted coordinate position as the second calibration parameter.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the engineering vehicle panoramic image calibration method as described in any one of claims 1 to 6.

9. An engineering vehicle, characterized in that, Including the electronic device as described in claim 8.

Images