Image stitching method, device and server for a laser camera and a dome camera

Abstract

This application is applicable to the field of image processing technology, and provides an image stitching method, device and server for a laser camera and a spherical camera. The method includes obtaining an image to be processed, where the image to be processed is an image obtained by the laser camera and the spherical camera at the same position, and the relative position between the laser camera and the spherical camera is fixed; stitching the image to be processed according to the geometric parameters of the laser camera and the spherical camera to obtain a panoramic image. It can be seen that this application can stitch RGB images to obtain an RGB panoramic image, and at the same time stitch depth images to obtain a panoramic depth image, solving the technical problem that the panoramic image obtained in the prior art contains fewer information dimensions, achieving the effects of increasing the information dimension of image stitching and improving the accuracy of image stitching.

Description

Technical Field

[0001] This application belongs to the technical field of image processing, and particularly relates to an image stitching method, device, and server for a laser camera and a spherical camera. Background Art

[0002] Generally, a panoramic image is obtained by stitching multiple groups of images taken by a panoramic camera (such as a spherical camera) with a 360-degree view of a real scene. However, the panoramic image obtained contains fewer information dimensions. Summary of the Invention

[0003] Embodiments of this application provide an image stitching method, device, and server for a laser camera and a spherical camera, which can solve the technical problem in the prior art that the panoramic image obtained contains fewer information dimensions.

[0004] In a first aspect, embodiments of this application provide an image stitching method for a laser camera and a spherical camera, including:

[0005] Obtain an image to be processed, where the image to be processed is an image taken by a laser camera and a spherical camera at the same position, and the relative position between the laser camera and the spherical camera is fixed;

[0006] Stitch the image to be processed according to the geometric parameters of the laser camera and the spherical camera to obtain a panoramic image.

[0007] In a possible implementation manner of the first aspect, the image to be processed includes an RGB image; the geometric parameters include the internal parameters of the spherical camera and the local external parameters between the spherical camera and the laser camera; the panoramic image includes an RGB panoramic image;

[0008] Stitching the image to be processed according to the geometric parameters of the laser camera and the spherical camera to obtain a panoramic image includes:

[0009] Stitch the RGB panoramic image according to the internal parameters of the spherical camera and the local external parameters between the spherical camera and the laser camera to obtain a panoramic RGB image.

[0010] In a possible implementation manner of the first aspect, before stitching the RGB panoramic image according to the internal parameters of the spherical camera and the local external parameters between the spherical camera and the laser camera to obtain a panoramic RGB image, it includes:

[0011] Obtain the internal parameters of the spherical camera;

[0012] Obtain the local external parameters between the spherical camera and the laser camera.

[0013] In a possible implementation of the first aspect, the image to be processed includes a depth image; the geometric parameters include the local extrinsic parameters between the spherical panoramic camera and the laser camera, and the global extrinsic parameters between the spherical panoramic camera and the laser camera; the panoramic image includes a depth panoramic image;

[0014] Stitching the image to be processed according to the geometric parameters of the laser camera and the spherical panoramic camera to obtain a panoramic image, including:

[0015] Rotating the first coordinate system where the depth points of the depth image are located to the second coordinate system where the RGB panoramic image is located according to the global extrinsic parameters;

[0016] Based on the depth points in the second coordinate system, adjusting the point cloud corresponding to the depth points according to the local extrinsic parameters so that the point cloud is aligned with the pixel points in the RGB panoramic image to obtain a panoramic depth image.

[0017] In a possible implementation of the first aspect, before stitching the image to be processed according to the geometric parameters of the laser camera and the spherical panoramic camera to obtain a panoramic image, it further includes:

[0018] Obtaining the global extrinsic parameters between the spherical panoramic camera and the laser camera.

[0019] In a second aspect, an embodiment of the present application provides an image stitching device for a laser camera and a spherical panoramic camera, including:

[0020] An acquisition module, configured to acquire an image to be processed corresponding to a calibration object;

[0021] An extraction module, configured to extract feature points of the image to be processed;

[0022] A calibration module, configured to calibrate the internal parameters of the spherical panoramic camera and the extrinsic parameters between the laser camera and the spherical panoramic camera according to the feature points, where the relative positions between the laser camera and the spherical panoramic camera are fixed.

[0023] In a possible implementation of the second aspect, the image to be processed includes an RGB image; the geometric parameters include the internal parameters of the spherical panoramic camera and the local extrinsic parameters between the spherical panoramic camera and the laser camera; the panoramic image includes an RGB panoramic image;

[0024] The stitching module includes:

[0025] A stitching sub-module, configured to stitch the RGB panoramic image according to the internal parameters of the spherical panoramic camera and the local extrinsic parameters between the spherical panoramic camera and the laser camera to obtain a panoramic RGB image.

[0026] In a possible implementation manner of the second aspect, the image to be processed includes a depth image; the geometric parameters include the local external parameters between the spherical panoramic camera and the laser camera, and the global external parameters between the spherical panoramic camera and the laser camera; the panoramic image includes a depth panoramic image.

[0027] The stitching module includes:

[0028] A rotation sub-module, configured to rotate the first coordinate system where the depth points of the depth image are located to the second coordinate system where the RGB panoramic image is located according to the global external parameters;

[0029] An adjustment sub-module, configured to adjust the point cloud corresponding to the depth points according to the local external parameters based on the depth points in the second coordinate system, so that the point cloud is aligned with the pixel points in the RGB panoramic image, and a panoramic depth image is obtained.

[0030] In a third aspect, an embodiment of the present application provides a server including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method described in the first aspect above is implemented.

[0031] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the method described in the first aspect above is implemented.

[0032] The beneficial effects of the embodiments of the present application compared with the prior art are as follows:

[0033] In the embodiments of the present application, an RGB panoramic image can be obtained by stitching RGB images, and at the same time, a depth image can be stitched to obtain a panoramic depth image, which solves the technical problem that the information dimension of the panoramic image obtained in the prior art is relatively small, and achieves the effects of increasing the information dimension of image stitching and improving the accuracy of image stitching. Description of the Drawings

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0035] Figure 1 It is a schematic flowchart of an image stitching method for a laser camera and a spherical panoramic camera provided by an embodiment of the present application;

[0036] Figure 2It is a structural block diagram of an image stitching device for a laser camera and a dome camera provided by an embodiment of the present application;

[0037] Figure 3 It is a schematic structural diagram of a server provided by an embodiment of the present application. Detailed implementation manners

[0038] In the following description, for the purpose of illustration rather than limitation, specific details such as specific system architectures, technologies, etc. are presented to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0039] It should be understood that when used in the specification and appended claims of the present application, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0040] It should also be understood that the term "and / or" as used in the specification and appended claims of the present application refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0041] As used in the specification and appended claims of the present application, the term "if" can be interpreted as "when", "once", "in response to determining", or "in response to detecting" according to the context. Similarly, the phrase "if determined" or "if [the described condition or event] is detected" can be interpreted as meaning "once determined", "in response to determining", "once [the described condition or event] is detected", or "in response to detecting [the described condition or event]" according to the context.

[0042] In addition, in the description of the specification and appended claims of the present application, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0043] Reference to "one embodiment" or "some embodiments" etc. described in the specification of this application means that a specific feature, structure or characteristic described in connection with that embodiment is included in one or more embodiments of this application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments" etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "comprising", "including", "having" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0044] The automatic calibration method for a laser camera and a spherical camera provided by an embodiment of this application can be applied to a server, which is preferably a cloud server. The server is connected to a 3D camera, where the 3D camera includes a laser camera and a spherical camera, and the relative positions between the laser camera and the spherical camera are fixed. The laser camera includes a first laser camera (i.e., UpLidar laser camera), a second laser camera (i.e., MidLidar laser camera), and a third laser camera (i.e., DownLidar laser camera) with fixed relative positions. Exemplarily, the UpLidar laser camera, MidLidar laser camera, and DownLidar laser camera are arranged at intervals in the upper, middle, and lower order on the same vertical line. Additionally, the 3D camera rotates 6 times at a rotation angle of 60 degrees at the same point to finally obtain a 360-degree panoramic image.

[0045] The technical solution provided by the embodiment of this application will be introduced below through specific embodiments.

[0046] See Figure 1 , which is a schematic flowchart of the image stitching method for a laser camera and a spherical camera provided by an embodiment of this application. As an example but not a limitation, this method can be applied to a server, and this method may include the following steps:

[0047] Step S101, obtain the image to be processed.

[0048] Among them, the image to be processed is an image obtained by the laser camera and the spherical camera at the same point, the relative positions between the laser camera and the spherical camera are fixed, and the image to be processed includes an RGB image and a depth image.

[0049] Step S102, stitch the image to be processed according to the geometric parameters of the laser camera and the spherical camera to obtain a panoramic image.

[0050] Among them, the panoramic image includes an RGB panoramic image and a depth panoramic image, and the geometric parameters include the internal parameters of the spherical camera, the global external parameters between the spherical camera and the lidar camera, and the local external parameters between the spherical camera and the lidar camera. It should be noted that: the internal parameters refer to the parameters related to the characteristics of the camera itself, such as the focal length and pixel size of the camera; the external parameters refer to the parameters in the world coordinate system, such as the position and rotation direction of the camera. The local external parameters between the spherical camera and the lidar camera refer to the external parameters corresponding to the co-viewing shooting of the spherical camera and the lidar camera during the rotation process.

[0051] Exemplarily, according to the geometric parameters of the lidar camera and the spherical camera, the image to be processed is stitched to obtain a panoramic image, including:

[0052] Step S201, according to the internal parameters of the spherical camera and the global external parameters between the spherical camera and the lidar camera, stitch the RGB panoramic image to obtain a panoramic RGB image.

[0053] In a specific application, the spherical camera and the lidar camera rotate 6 times at the same point to capture 6 RGB images. The image coordinate system of each RGB image is determined according to the internal parameters, and then the 6 RGB images are adjusted to the same reference coordinate system according to the local external parameters. The middle 60 degrees of each RGB image are intercepted and stitched into a 360-degree panoramic image.

[0054] Exemplarily, according to the geometric parameters of the lidar camera and the spherical camera, the image to be processed is stitched to obtain a panoramic image, including:

[0055] Step 301, rotate the first coordinate system where the depth points of the depth image are located to the second coordinate system where the RGB panoramic image is located according to the global external parameters;

[0056] Step 302, based on the depth points in the second coordinate system, adjust the point cloud corresponding to the depth points according to the local external parameters so that the point cloud is aligned with the pixel points in the RGB panoramic image to obtain a panoramic depth image.

[0057] It can be understood that in the embodiments of the present application, the RGB images can be stitched to obtain an RGB panoramic image, and at the same time, the depth images can be stitched to obtain a panoramic depth image, which solves the technical problem that the information dimension of the panoramic image obtained in the prior art is less, and achieves the effects of increasing the information dimension of the image stitching and improving the accuracy of the image stitching.

[0058] In a possible implementation manner, before stitching the RGB panoramic image according to the internal parameters of the spherical camera and the local external parameters between the spherical camera and the lidar camera to obtain a panoramic RGB image, it includes:

[0059] Step 401, obtain the internal parameters of the spherical camera.

[0060] Step 402: Obtain the local extrinsic parameters between the spherical camera and the laser camera.

[0061] In a possible implementation, before stitching the to-be-processed images according to the geometric parameters of the laser camera and the spherical camera to obtain a panoramic image, it further includes:

[0062] Step 501: Obtain the global extrinsic parameters between the spherical camera and the laser camera.

[0063] The following specifically describes how to obtain the intrinsic parameters of the spherical camera, the local extrinsic parameters between the spherical camera and the laser camera, and the global extrinsic parameters between the spherical camera and the laser camera.

[0064] In a possible implementation, obtaining the intrinsic parameters of the spherical camera includes:

[0065] Step 601: Obtain the first calibration image corresponding to the first calibration object;

[0066] Step 602: Extract the first feature points of the first calibration image;

[0067] Step 603: Calibrate the intrinsic parameters of the spherical camera according to the first feature points.

[0068] Wherein, the first calibration image is the first calibration RGB image corresponding to the first calibration object obtained by the spherical camera shooting at a preset distance from the first calibration object. The first calibration object is a plurality of checkerboards arranged vertically. The first feature points are 2D-2D matching points. A 2D-2D matching point refers to 2D corner points on multiple different images corresponding to the same real point in the world coordinate system. Then, the 2D corner points on multiple different images are called 2D-2D matching points.

[0069] In specific applications, extracting the first feature points of the first calibration RGB image includes: extracting the first 2D corner points of the first calibration RGB image, and determining the 2D-2D matching points corresponding to the spherical camera according to the first 2D corner points. The principle of extracting the first 2D corner points of the first calibration RGB image is as follows: Use the local average adaptive thresholding method to equalize the image histogram in the photo, then perform binarization and image dilation to separate the connections of each black block quadrilateral; then perform quadrilateral detection, identify the adjacent quadrilaterals of each quadrilateral, and record the number of adjacent quadrilaterals; classify all quadrilaterals according to the principle of whether they have four adjacent quadrilaterals. The quadrilateral with four adjacent quadrilaterals is the required checkerboard square; judge whether the number of required checkerboard squares after classification is the same as the known number of corner points. If not, perform cyclic detection until the number of checkerboard squares tested is the same as the known number of corner points. The serial numbers of each required checkerboard square can be sorted according to the proximity relationship, and then the midpoint of the line connecting the two opposite points of the two quadrilaterals on the diagonal is taken as the corner point.

[0070] In specific applications, calibrating the internal parameters of the spherical camera based on the first feature points includes:

[0071] Step 701, determining a first cost function according to the 2D-2D matching points;

[0072] Step 702, calibrating the internal parameters of the spherical camera according to the first cost function.

[0073] The first cost function is:

[0074]

[0075] Where is the abscissa of the projection of the i-th point in the current frame on the reference frame; is the ordinate of the projection of the i-th point in the current frame on the reference frame; is the abscissa of the projection of the reference frame on the current frame; is the ordinate of the projection of the reference frame on the current frame.

[0076] In a possible implementation, obtaining the global extrinsic parameters between the spherical camera and the laser camera includes:

[0077] Step 801, obtaining the second calibrated RGB image, the first intensity image, and the first depth image corresponding to the second calibration object;

[0078] Step 802, extracting the 2D-3D matching points of the second calibrated RGB image, the first intensity image, and the first depth image;

[0079] Step 803, calibrating the global extrinsic parameters between the spherical camera and the laser camera according to the 2D-3D matching points.

[0080] Among them, the second calibrated RGB image, the first intensity image, and the first depth image are the images corresponding to the second calibration object obtained by the first laser camera, the second laser camera, and the third laser camera respectively co-viewing and shooting with the spherical camera at a preset position (such as 2.5 meters) from the second calibration object. It should be noted that co-viewing shooting means having an overlapping area and a checkerboard pattern. The 2D-3D matching points refer to the 2D corner points on multiple different images and the 3D corner points converted from the 2D corner points on other images corresponding to the same real point in the world coordinate system, and the 2D corner points on multiple different images and the 3D corner points converted from the 2D corner points on other images are called 2D-3D matching points.

[0081] In specific applications, the global extrinsic parameters between the spherical camera and the laser camera include: determining a second cost function according to the 2D-3D matching points, and calibrating the global extrinsic parameters between the spherical camera and the laser camera according to the second cost function.

[0082] The second cost function is:

[0083]

[0084] Among them, x refers to the abscissa corresponding to the 2D pixel point, y refers to the ordinate corresponding to the 2D pixel point, and p 3d_up_reproject refers to the 2D pixel point where the first laser camera is back-projected onto the spherical camera, and p up_cam refers to the 2D pixel point of the second calibrated RGB image; p 3d_min_reproject refers to the 2D point where the second laser camera is back-projected onto the spherical camera, and p min_cam refers to the 2D point of the second calibrated RGB image; p 3d_down_reproject refers to the 2D pixel point where the third laser camera is back-projected onto the spherical camera, and p down_cam refers to the 2D pixel point of the second calibrated RGB image. M2 refers to the number of corresponding first 2D-3D matching points between the first laser camera and the spherical camera and between the third laser camera and the spherical camera. M3 refers to the number of corresponding first 2D-3D matching points between the second laser camera and the spherical camera.

[0085] In a possible implementation, obtaining the local extrinsic parameters between the spherical camera and the laser camera includes:

[0086] Step 901: Obtain the third calibrated RGB image, the second intensity image, and the second depth image corresponding to the second calibration object;

[0087] Step 902: Extract the 3D-3D matching points of the third calibrated RGB image, the second intensity image, and the second depth image;

[0088] Step 903: Calibrate the local extrinsic parameters between the spherical camera and the laser camera according to the 3D-3D matching points.

[0089] The third cost function is:

[0090]

[0091] Among them, p 3d_up_mid_reproject x up refers to the abscissa of the 3D-3D matching point corresponding to the first laser camera and the second laser camera back-projected onto the second intensity map corresponding to the first laser camera, p 3d_up_mid_reproject x mid refers to the abscissa of the 3D-3D matching point corresponding to the first laser camera and the second laser camera back-projected onto the second intensity map corresponding to the second laser camera, p 3d_up_mid_reproject y up refers to the ordinate of the 3D-3D matching point corresponding to the first laser camera and the second laser camera back-projected onto the second intensity map corresponding to the first laser camera, p 3d_up_mid_reproject ymid refers to the ordinate of the second intensity map corresponding to the second lidar camera obtained by back-projecting the corresponding 3D-3D matching points between the first lidar camera and the second lidar camera, p 3d_mid_down_reproject x mid refers to the abscissa of the second intensity map corresponding to the second lidar camera obtained by back-projecting the corresponding 3D-3D matching points between the second lidar camera and the third lidar camera, p 3d_up_mid_reproject x mid refers to the abscissa of the second intensity map corresponding to the second lidar camera obtained by back-projecting the corresponding 3D-3D matching points between the second lidar camera and the third lidar camera, p 3d_mid_down_reproject y mid refers to the ordinate of the second intensity map corresponding to the second lidar camera obtained by back-projecting the corresponding 3D-3D matching points between the second lidar camera and the third lidar camera, p 3d_up_mid_reproject y mid refers to the ordinate of the second intensity map corresponding to the second lidar camera obtained by back-projecting the corresponding 3D-3D matching points between the first lidar camera and the second lidar camera.

[0092] In the embodiments of the present application, RGB images can be stitched to obtain an RGB panoramic image, and at the same time, depth images can be stitched to obtain a panoramic depth image, which solves the technical problem that the information dimension of the panoramic image obtained in the prior art is less, and achieves the effects of increasing the information dimension of image stitching and improving the accuracy of image stitching.

[0093] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0094] Corresponding to the method described in the above embodiments, Figure 2 The structural block diagram of the image stitching device of the lidar camera and the dome camera provided by the embodiments of the present application is shown. For the sake of convenience of description, only the parts related to the embodiments of the present application are shown.

[0095] Refer to Figure 2 , the device includes:

[0096] An acquisition module 21, configured to acquire an image to be processed, where the image to be processed is an image obtained by the lidar camera and the dome camera shooting at the same position, and the relative position between the lidar camera and the dome camera is fixed;

[0097] A stitching module 22, configured to stitch the image to be processed according to the geometric parameters of the lidar camera and the dome camera to obtain a panoramic image.

[0098] In a possible implementation manner, the image to be processed includes an RGB image; the geometric parameters include the internal parameters of the spherical camera and the local external parameters between the spherical camera and the laser camera; the panoramic image includes an RGB panoramic image.

[0099] The stitching module includes:

[0100] A stitching sub-module, configured to stitch the RGB panoramic image according to the internal parameters of the spherical camera and the local external parameters between the spherical camera and the laser camera to obtain a panoramic RGB image.

[0101] In a possible implementation manner, the image to be processed includes a depth image; the geometric parameters include the local external parameters between the spherical camera and the laser camera and the global external parameters between the spherical camera and the laser camera; the panoramic image includes a depth panoramic image.

[0102] The stitching module includes:

[0103] A rotation sub-module, configured to rotate the first coordinate system where the depth points of the depth image are located to the second coordinate system where the RGB panoramic image is located according to the global external parameters;

[0104] An adjustment sub-module, configured to adjust the point cloud corresponding to the depth points according to the local external parameters based on the depth points in the second coordinate system so that the point cloud is aligned with the pixel points in the RGB panoramic image to obtain a panoramic depth image.

[0105] It should be noted that for the information interaction, execution process, etc. between the above-mentioned device / units, since they are based on the same concept as the method embodiments of the present application, their specific functions and the technical effects brought about can be specifically referred to in the method embodiment part, and will not be elaborated here.

[0106] Figure 3 This is a schematic structural diagram of the server provided by the embodiment of the present application. As Figure 3 shown, the server 3 of this embodiment includes: at least one processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the at least one processor 30. When the processor 30 executes the computer program 32, the steps in any of the above method embodiments are implemented.

[0107] The server 3 may be a computing device such as a cloud server. The server may include, but is not limited to, a processor 30 and a memory 31. Those skilled in the art can understand that Figure 3 this is only an example of the server 3 and does not constitute a limitation on the server 3. It may include more or fewer components than shown in the figure, or combine some components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0108] The so-called processor 30 may be a central processing unit (CPU), and the processor 30 may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0109] In some embodiments, the memory 31 may be an internal storage unit of the server 3, such as the hard disk or memory of the server 3. In other embodiments, the memory 31 may also be an external storage device of the server 3, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the server 3. Further, the memory 31 may also include both the internal storage unit of the server 3 and the external storage device. The memory 31 is used to store an operating system, application programs, a boot loader, data, and other programs, such as the program code of the computer program, etc. The memory 31 may also be used to temporarily store data that has been output or is to be output.

[0110] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above division of each functional unit and module is used as an example. In practical applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiments can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of this application. The specific working processes of the units and modules in the above system can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated here.

[0111] The embodiments of the present application further provide a computer-readable storage medium storing a computer program, which when executed by a processor can implement the steps in the above-mentioned method embodiments.

[0112] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above-mentioned method embodiments of the present application, a computer program can be used to instruct the relevant hardware to complete. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate forms, etc. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the server, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disc, etc. In some jurisdictions, according to legislation and patent practice, the computer-readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0113] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0114] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or by a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0115] In the embodiments provided in the present application, it should be understood that the disclosed device / network device and method can be implemented in other ways. For example, the device / network device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical, mechanical or other form.

[0116] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0117] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. An image stitching method for a laser camera and a dome camera, characterized in that, Including: Obtain an image to be processed, where the image to be processed is an image obtained by a laser camera and a spherical camera at the same position, and the relative position between the laser camera and the spherical camera is fixed; Stitch the image to be processed according to the geometric parameters of the laser camera and the spherical camera to obtain a panoramic image; The image to be processed includes an RGB image; the geometric parameters include the internal parameters of the spherical camera and the local external parameters between the spherical camera and the laser camera; the panoramic image includes an RGB panoramic image; Stitching the image to be processed according to the geometric parameters of the laser camera and the spherical camera to obtain a panoramic image includes: Stitch the RGB panoramic image according to the internal parameters of the spherical camera and the local external parameters between the spherical camera and the laser camera to obtain a panoramic RGB image; Or, The image to be processed includes a depth image; the geometric parameters include the local external parameters between the spherical camera and the laser camera and the global external parameters between the spherical camera and the laser camera; the panoramic image includes a depth panoramic image; Stitching the image to be processed according to the geometric parameters of the laser camera and the spherical camera to obtain a panoramic image includes: Rotate the first coordinate system where the depth points of the depth image are located to the second coordinate system where the RGB panoramic image is located according to the global external parameters; Based on the depth points in the second coordinate system, adjust the point cloud corresponding to the depth points according to the local external parameters so that the point cloud is aligned with the pixel points in the RGB panoramic image to obtain a panoramic depth image.

2. The image stitching method for a laser camera and a dome camera according to claim 1, characterized in that, Before stitching the RGB panoramic image according to the internal parameters of the spherical camera and the local external parameters between the spherical camera and the laser camera to obtain a panoramic RGB image, it includes: Obtain the internal parameters of the spherical camera; Obtain the local external parameters between the spherical camera and the laser camera.

3. The image stitching method for a laser camera and a dome camera according to claim 1, characterized in that, Before stitching the image to be processed according to the geometric parameters of the laser camera and the spherical camera to obtain a panoramic image, it further includes: Obtain the global external parameters between the spherical camera and the laser camera.

4. An image stitching device for a laser camera and a dome camera, characterized in that, Including: An acquisition module for acquiring an image to be processed, where the image to be processed is an image obtained by a laser camera and a spherical camera at the same position, and the relative position between the laser camera and the spherical camera is fixed; A stitching module for stitching the image to be processed according to the geometric parameters of the laser camera and the spherical camera to obtain a panoramic image; The image to be processed includes an RGB image; the geometric parameters include the internal parameters of the spherical camera and the local external parameters between the spherical camera and the laser camera; the panoramic image includes an RGB panoramic image; The stitching module includes: A stitching sub-module for stitching the RGB panoramic image according to the internal parameters of the spherical camera and the local external parameters between the spherical camera and the laser camera to obtain a panoramic RGB image; Or, The image to be processed includes a depth image; the geometric parameters include the local external parameters between the spherical camera and the laser camera and the global external parameters between the spherical camera and the laser camera; the panoramic image includes a depth panoramic image; The splicing module includes: A rotation sub-module for rotating a first coordinate system where depth points of the depth image are located to a second coordinate system where the RGB panoramic image is located according to the global external parameters; An adjustment sub-module for adjusting a point cloud corresponding to the depth points based on the depth points in the second coordinate system according to the local external parameters, so that the point cloud is aligned with pixel points in the RGB panoramic image to obtain a panoramic depth image.

5. A server, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, the method described in any one of claims 1 to 3 is implemented.

6. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, the method described in any one of claims 1 to 3 is implemented.

Images