Method and device for generating three-dimensional panoramic image, and electronic device

Abstract

The embodiment of the present invention provides a method and device for generating a three-dimensional panoramic image, and an electronic device, the method comprising: obtaining at least four images taken by at least four depth cameras at the same time and depth data of each pixel in the images, wherein the scenes within the field of view of adjacent depth cameras overlap; generating a first point cloud map and a second point cloud map based on the depth data of each pixel in the first image and the second image, wherein the first point cloud map and the second point cloud map have an overlapping area, and the coordinate origins are both at the first position; overlapping the first point cloud map and the second point cloud map according to the first overlapping area to obtain a first spliced ​​point cloud map, wherein the first overlapping area is the overlapping area of ​​the first point cloud map and the second point cloud map. The three-dimensional panoramic image can guide the user to make accurate judgments about the situation around the vehicle and improve the safety during driving.

Description

Technical Field

[0001] The present invention relates to the field of vehicle-mounted technology, and in particular to a method and device for generating a three-dimensional panoramic image, and electronic equipment. Background Art

[0002] With the increasing number of vehicles on the road, driving safety has received more and more attention. At present, most vehicles are equipped with driving recorders, reversing cameras and other on-board equipment to improve driving safety.

[0003] Since there are many blind spots for drivers during driving, the dashcam and reversing images alone are not enough to cover all blind spots. Therefore, the in-vehicle 360° panoramic view came into being. It collects videos in real time through 4-8 wide-angle fisheye cameras installed in the vehicle, and uses algorithms to perform video synthesis processing to form a bird's-eye view of the vehicle's surroundings. The driver can view the real-time image information of the 360-degree panoramic view around the car through the bird's-eye view, which gives the driver a wider field of vision and safer driving.

[0004] However, as users' requirements for driving safety continue to increase, the current in-vehicle 360° panoramic view has gradually failed to meet users' requirements. Summary of the invention

[0005] The embodiments of the present invention provide a method and device for generating a three-dimensional panoramic image, and an electronic device to solve the problem in the prior art that the vehicle-mounted 360° panoramic view cannot meet the increasing requirements of users for driving safety.

[0006] In a first aspect, an embodiment of the present invention provides a method for generating a three-dimensional panoramic image, the method comprising:

[0007] Acquire at least four images taken by at least four depth cameras at the same time and depth data of each pixel in the images, wherein the scenes within the field of view of adjacent depth cameras overlap;

[0008] Generate a first point cloud map and a second point cloud map based on the depth data of each pixel point in the first image and the second image, wherein the first point cloud map and the second point cloud map have an overlapping area and the coordinate origins are both at the first position;

[0009] Overlapping the first point cloud image and the second point cloud image according to a first overlapping area to obtain a first spliced ​​point cloud image, wherein the first overlapping area is an overlapping area between the first point cloud image and the second point cloud image;

[0010] A panoramic point cloud image is obtained based on the first spliced ​​point cloud image and the depth data of each pixel point in the remaining images; wherein the remaining images are images other than the first image and the second image in the at least four images;

[0011] A three-dimensional panoramic image is generated based on the panoramic point cloud map and the color data of each pixel in the at least four images.

[0012] Optionally, generating the first point cloud image and the second point cloud image based on the depth data of each pixel point in the first image and the second image includes:

[0013] Based on the depth data of each pixel in the first image, each pixel in the first image is mapped to a world coordinate system with the first position as a coordinate origin to obtain a first point cloud image;

[0014] Based on the depth data of each pixel in the second image and the first joint calibration data, mapping each pixel in the second image to a world coordinate system with the first position as the coordinate origin to obtain a second point cloud image;

[0015] The first joint calibration data is data obtained by performing joint calibration on the depth camera that captures the first image and the second image.

[0016] Optionally, obtaining a panoramic point cloud map based on the first spliced ​​point cloud map and the depth data of each pixel in the remaining image includes:

[0017] Generate a third point cloud image based on the depth data of each pixel in the third image and the second joint calibration data; wherein the second joint calibration data is data obtained by joint calibration of the depth camera that shoots the first image and the third image; the third point cloud image and the first spliced ​​point cloud image have an overlapping area, and the coordinate origin is at the first position;

[0018] Overlapping the first stitched point cloud image and the third point cloud image according to a second overlapping area to obtain a second stitched point cloud image, wherein the second overlapping area is an overlapping area between the third point cloud image and the first stitched point cloud image;

[0019] Based on the depth data of each pixel point in the fourth image, generating a fourth point cloud map with a coordinate origin at a target position of the vehicle, wherein the fourth point cloud map and the second spliced ​​point cloud map have an overlapping area;

[0020] The coordinate origin of the second stitched point cloud map is switched to the target position, and the second stitched point cloud map after the coordinate origin is switched is overlapped with the fourth point cloud map according to a third overlapping area to obtain a panoramic point cloud map, wherein the third overlapping area is an overlapping area of ​​the fourth point cloud map and the second stitched point cloud map.

[0021] Optionally, obtaining a panoramic point cloud map based on the first spliced ​​point cloud map and the depth data of each pixel in the remaining image includes:

[0022] Generate a third point cloud map and a fourth point cloud map based on the depth data of each pixel point in the third image and the fourth image; wherein the third point cloud map and the fourth point cloud map have an overlapping area, and the coordinate origins are both at the second position;

[0023] Overlapping the third point cloud image and the fourth point cloud image according to a second overlapping area to obtain a second spliced ​​point cloud image, wherein the second overlapping area is an overlapping area between the third point cloud image and the fourth point cloud image;

[0024] The coordinate origins of the first stitching point cloud map and the second stitching point cloud map are switched to the target position of the vehicle, and overlapped according to the third overlapping area to obtain a panoramic point cloud map, wherein the third overlapping area is the overlapping area of ​​the first stitching point cloud map and the second stitching point cloud map.

[0025] Optionally, the target position is the position of the driver's head of the vehicle.

[0026] In a second aspect, an embodiment of the present invention further provides a device for generating a three-dimensional panoramic image, the device comprising:

[0027] An acquisition module, used to acquire at least four images taken by the at least four depth cameras at the same time and depth data of each pixel in the image, wherein the scenes within the field of view of adjacent depth cameras overlap;

[0028] A point cloud module, configured to generate a first point cloud map and a second point cloud map based on depth data of each pixel point in the first image and the second image, wherein the first point cloud map and the second point cloud map have an overlapping area and the coordinate origins are both at a first position;

[0029] A first splicing module, configured to overlap the first point cloud image and the second point cloud image according to a first overlapping area to obtain a first spliced ​​point cloud image, wherein the first overlapping area is an overlapping area between the first point cloud image and the second point cloud image;

[0030] A second stitching module, configured to obtain a panoramic point cloud image based on the first stitching point cloud image and the depth data of each pixel point in the remaining images; wherein the remaining images are images other than the first image and the second image in the at least four images;

[0031] The panoramic module is used to generate a three-dimensional panoramic image based on the panoramic point cloud map and the color data of each pixel in the at least four images.

[0032] Optionally, the point cloud module includes:

[0033] A first point cloud unit, configured to map each pixel in the first image to a world coordinate system with the first position as a coordinate origin to obtain a first point cloud image based on depth data of each pixel in the first image;

[0034] A second point cloud unit is used to map each pixel in the second image to a world coordinate system with the first position as a coordinate origin to obtain a second point cloud image based on the depth data of each pixel in the second image and the first joint calibration data;

[0035] The first joint calibration data is data obtained by performing joint calibration on the depth camera that captures the first image and the second image.

[0036] Optionally, the second splicing module includes:

[0037] a third point cloud unit, configured to generate a third point cloud map based on the depth data of each pixel in the third image and the second joint calibration data; wherein the second joint calibration data is data obtained by performing joint calibration on the depth camera that shoots the first image and the third image; the third point cloud map and the first spliced ​​point cloud map have an overlapping area, and the coordinate origin is at the first position;

[0038] The first stitching unit is used to overlap the first stitching point cloud image and the third point cloud image according to a second overlapping area to obtain a second stitching point cloud image, wherein the second overlapping area is an overlapping area of ​​the third point cloud image and the first stitching point cloud image.

[0039] a fourth point cloud unit, configured to generate a fourth point cloud image with a coordinate origin at a target position of the vehicle based on depth data of each pixel point in the fourth image, wherein the fourth point cloud image and the second spliced ​​point cloud image have an overlapping area;

[0040] The second stitching unit is used to switch the coordinate origin of the second stitching point cloud image to the target position, and overlap the second stitching point cloud image after the coordinate origin is switched with the fourth point cloud image according to a third overlapping area to obtain a panoramic point cloud image, wherein the third overlapping area is an overlapping area of ​​the fourth point cloud image and the second stitching point cloud image.

[0041] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the method described in the first aspect above when executing the computer program.

[0042] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps in the method described in the first aspect above are implemented.

[0043] In an embodiment of the present invention, images and depth data taken by at least four depth cameras at the same time can be obtained. Since the scenes within the field of view of adjacent depth cameras overlap, scene information around the installation position of the depth cameras can be obtained. Then, based on the depth data of each pixel in the first image and the second image, a first point cloud map and a second point cloud map are generated. Among them, the first point cloud map and the second point cloud map have an overlapping area, and the coordinate origins are both in the first position, that is, the two point cloud maps have the same coordinate origin in the world coordinate system. Then, the first point cloud map and the second point cloud map can be overlapped directly according to the first overlapping area to obtain a first spliced ​​point cloud map, wherein the first overlapping area is the overlapping area of ​​the first point cloud map and the second point cloud map. Since the two point cloud maps have the same coordinate origin, they can be directly spliced ​​to obtain a first spliced ​​point cloud map. Then, based on the first stitching point cloud map and the depth data of each pixel in the remaining images, a panoramic point cloud map is obtained, and the color data is added to the panoramic point cloud map to obtain a three-dimensional panoramic image. In addition to containing color data, the three-dimensional panoramic image displays the scene around the vehicle to the user in a three-dimensional form, thereby guiding the user to make accurate judgments about the situation around the vehicle, improving safety during driving, and thus meeting the user's increasing requirements for driving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0045] Figure 1 A flowchart of the steps of a method for generating a three-dimensional panoramic image provided by an embodiment of the present invention;

[0046] Figure 2 is a schematic diagram of the layout of a depth camera of a vehicle according to an embodiment of the present invention;

[0047] Figure 3 A schematic diagram showing the target position of a vehicle provided in an embodiment of the present invention;

[0048] Figure 4 A practical application flow chart of the method for generating a three-dimensional panoramic image provided by an embodiment of the present invention;

[0049] Figure 5 A structural block diagram of a device for generating a three-dimensional panoramic image provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] It should be understood that the references to "one embodiment" or "an embodiment" throughout the specification mean that the specific features, structures, or characteristics associated with the embodiment are included in at least one embodiment of the present invention. Therefore, the references to "in one embodiment" or "in an embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0052] In various embodiments of the present invention, it should be understood that the size of the serial numbers of the following processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0053] See also Figure 1 , an embodiment of the present invention provides a method for generating a three-dimensional panoramic image, which may include:

[0054] Step 101: Obtain at least four images taken by at least four depth cameras at the same time and depth data of each pixel in the image.

[0055] It should be noted that in order to ensure the timing synchronization between different images taken by different depth cameras, here, images and depth data will be obtained for each moment, that is, for each moment, at least four images taken by at least four depth cameras at that moment and the depth data of each pixel in the image are obtained. It can be understood that when a three-dimensional panoramic image within a certain time period needs to be obtained, the data processing process for the images and depth data obtained at different moments is the same, and a three-dimensional panoramic image at each moment within the time period will be obtained, and finally a three-dimensional panoramic image within the time period will be generated. Here, only one moment is taken as an example to illustrate the process of obtaining a three-dimensional panoramic image at that moment. Preferably, when performing timing synchronization, high-precision data timing synchronization can be achieved through a precision time synchronization protocol (Precision Time Synchronization Protocol, referred to as PTP) or a pulse per second (Pulse Per Second, referred to as PPS), but it is not limited to this.

[0056] For the depth data of each pixel in the image, we can first generate the internal and external parameter matrix of the depth camera through calibration, obtain the left and right images in real time, and generate disparity data (Disparity) through the stereo matching algorithm. Finally, through the principle of triangulation, the depth data = B (baseline) × F (equivalent focal length) / Disparity (disparity data) can be obtained to obtain the depth data of each pixel.

[0057] Preferably, the depth camera includes a binocular camera, each binocular camera will capture two images, a left image and a right image. The present invention aligns the left image and the right image to a plane in advance, and uses epipolar constraints to perform pixel-by-pixel feature matching of the left and right images to generate a full-resolution depth point cloud.

[0058] It is understandable that at least four depth cameras can be installed on the same object, and the scenes within the field of view of adjacent depth cameras overlap, so as to collect scene information around the object. Specifically, when the method for generating a three-dimensional panoramic image provided in an embodiment of the present invention is applied to a vehicle, at least four depth cameras are installed around the vehicle, and the scenes within the field of view of adjacent depth cameras overlap. It is understandable that the depth camera is a camera that can collect color data and depth data, that is, the depth camera can not only obtain an image containing scene information within its field of view, but also obtain the distance between each point in the image and the depth camera. Here, at least one depth camera is installed on each of the front, rear, left and right sides of the vehicle, wherein the front side of the vehicle refers to the front side of the vehicle, the rear side refers to the rear side of the vehicle, the left side refers to the side close to the driver, and the right side refers to the side close to the co-pilot. The number of installed depth cameras is related to the field of view of each depth camera. For example, when the field of view of each depth camera is about 180°, a depth camera can be installed on each side, but it is not limited to this. Figure 2 As shown, a first depth camera 22 is installed on the front side of the vehicle 21, a second depth camera 23 is installed on the left side, a third depth camera 24 is installed on the right side, and a fourth depth camera 25 is installed on the rear side. Accordingly, the scenes within the field of view of the first depth camera 22 and the second depth camera 23 overlap, the scenes within the field of view of the first depth camera 22 and the third depth camera 24 overlap, the scenes within the field of view of the fourth depth camera 25 and the second depth camera 23 overlap, and the scenes within the field of view of the fourth depth camera 25 and the third depth camera 24 overlap.

[0059] Step 102: Generate a first point cloud image and a second point cloud image based on the depth data of each pixel in the first image and the second image.

[0060] It should be noted that the first point cloud map and the second point cloud map have an overlapping area, and the coordinate origins are both at the first position. Since the first point cloud map is generated using the first image, and the second point cloud map is generated using the second image, the first image and the second image also have an overlapping area. Specifically, the first image and the second image are two images with overlapping areas among at least four images, that is, images captured by two adjacent depth cameras with overlapping scenes within the field of view. Preferably, the first position is the position of the depth camera that captured the first image or the second image in the world coordinate system, that is, one of the two adjacent depth cameras with overlapping scenes within the aforementioned field of view. Continuing with Figure 2 For example, if the first image and the second image are images captured by the first depth camera 22 and the second depth camera 23 respectively, the first position may be the position of the first depth camera 22 or the second depth camera 23 in the world coordinate system.

[0061] It can be understood that the point cloud map is an image generated by point cloud data, and each point in the point cloud map corresponds to a pixel point and has three-dimensional coordinates. Among them, each image taken by the depth camera uses itself as a reference or coordinate origin. In order to unify the coordinate origin, one of the first image and the second image keeps its own coordinate origin unchanged and is mapped to the world coordinate system to generate a point cloud map. The other image is additionally processed in the process of mapping to the world coordinate system so that its coordinate origin is the same as the coordinate origin of the other point cloud map. In other words, the coordinate origin of one of the two images in the camera coordinate system is consistent with the coordinate origin of the point cloud map generated based on the image in the world coordinate system. Only the process of generating the point cloud map based on the other image is compensated so that the two point cloud maps have the same coordinate origin in the world coordinate system.

[0062] Step 103: Overlap the first point cloud image and the second point cloud image according to the first overlapping area to obtain a first spliced ​​point cloud image.

[0063] It should be noted that the first point cloud image and the second point cloud image have the same coordinate origin, and the overlapping area of ​​the two is the first overlapping area. Therefore, the first spliced ​​point cloud image obtained by overlapping the first overlapping area can be understood as a point cloud image generated based on different pictures observed at the same position and the same viewing angle. Figure 2 For example, if the first image is an image taken by the first depth camera 22, and the second image is an image taken by the second depth camera 23, then the first overlapping area is the area corresponding to the scene within the first field of view overlapping area 26 of the two depth cameras. Since the first overlapping area has the same coordinate origin and the same scene information, the two point cloud images can be overlapped according to the first overlapping area, and the first spliced ​​point cloud image after the overlap is equivalent to a point cloud image generated by an image and depth data obtained by a depth camera with a field of view of 270°. Specifically, it is assumed that the first point cloud image includes: a first overlapping area corresponding to the first scene information and an area other than the first overlapping area corresponding to the second scene information; the second point cloud image includes: a first overlapping area corresponding to the first scene information and an area other than the first overlapping area corresponding to the third scene information; the first spliced ​​point cloud image includes: an area other than the first overlapping area corresponding to the second scene information, a first overlapping area corresponding to the first scene information, and an area other than the first overlapping area corresponding to the third scene information.

[0064] Step 104: Obtain a panoramic point cloud image based on the first spliced ​​point cloud image and the depth data of each pixel in the remaining images.

[0065] It should be noted that the panoramic point cloud image is a 360° point cloud image around the vehicle. The remaining images are images other than the first image and the second image in the at least four images. Another spliced ​​point cloud image can be generated using the remaining images, and then the two spliced ​​point cloud images are overlapped or spliced ​​again to obtain a global point cloud image, but is not limited thereto.

[0066] Step 105: Generate a three-dimensional panoramic image based on the panoramic point cloud image and the color data of each pixel in at least four images.

[0067] It should be noted that each pixel in the image has its own color data, and the pixel corresponds to the depth data one by one. After the point cloud is generated, each point in the point cloud corresponds to a pixel. Therefore, when generating a three-dimensional panoramic image, for each point in the panoramic point cloud, the color data of the pixel corresponding to the point is added to the point.

[0068] In an embodiment of the present invention, images and depth data taken by at least four depth cameras at the same time can be obtained. Since the scenes within the field of view of adjacent depth cameras overlap, scene information around the installation position of the depth cameras can be obtained. Then, based on the depth data of each pixel in the first image and the second image, a first point cloud map and a second point cloud map are generated. Among them, the first point cloud map and the second point cloud map have an overlapping area, and the coordinate origins are both in the first position, that is, the two point cloud maps have the same coordinate origin in the world coordinate system. Then, the first point cloud map and the second point cloud map can be overlapped directly according to the first overlapping area to obtain a first spliced ​​point cloud map, wherein the first overlapping area is the overlapping area of ​​the first point cloud map and the second point cloud map. Since the two point cloud maps have the same coordinate origin, they can be directly spliced ​​to obtain the first spliced ​​point cloud map. Then, based on the first stitching point cloud map and the depth data of each pixel in the remaining images, a panoramic point cloud map is obtained, and the color data is added to the panoramic point cloud map to obtain a three-dimensional panoramic image. In addition to containing color data, the three-dimensional panoramic image displays the scene around the vehicle to the user in a three-dimensional form, thereby guiding the user to make accurate judgments about the situation around the vehicle, improving safety during driving, and thus meeting the user's increasing requirements for driving safety.

[0069] Optionally, generating a first point cloud image and a second point cloud image based on the depth data of each pixel in the first image and the second image includes:

[0070] Based on the depth data of each pixel in the first image, each pixel in the first image is mapped to a world coordinate system with the first position as the coordinate origin to obtain a first point cloud image.

[0071] Based on the depth data of each pixel in the second image and the first joint calibration data, each pixel in the second image is mapped to a world coordinate system with the first position as the coordinate origin to obtain a second point cloud image;

[0072] It should be noted that the first position can be determined as the position of the depth camera that shoots the first image in the world coordinate system. When unifying the coordinate origin, the coordinate origin is unified as the position of the depth camera that shoots the first image in the world coordinate system. Since the first image is an image that is shot using the first position as a reference or coordinate origin, it is only necessary to use the calibration data of the depth camera that shoots the first image and the depth data of each pixel in the first image to obtain the first point cloud map with the first position as the coordinate origin. Among them, the calibration data is the position, pitch angle, degree of freedom, etc. of the camera obtained in the process of calibrating the depth camera.

[0073] Since the second image does not use the position of the depth camera that took the first image in the world coordinate system as a reference or coordinate origin, it is necessary to pre-calibrate the depth cameras that took the first image and the second image together to obtain first joint calibration data, and then use the first joint calibration data and the depth data of each pixel in the second image to obtain a second point cloud map with the position of the depth camera that took the first image in the world coordinate system as the coordinate origin.

[0074] In an embodiment of the present invention, by jointly calibrating the depth camera that takes the first image and the second image, first joint calibration data can be obtained, and then using the first joint calibration data and the depth data of each pixel in the second image, a second point cloud map with the depth camera that takes the first image as the coordinate origin can be obtained.

[0075] Optionally, a panoramic point cloud image is obtained based on the first spliced ​​point cloud image and the depth data of each pixel point in the remaining image, including:

[0076] A third point cloud map is generated based on the depth data of each pixel in the third image and the second joint calibration data; wherein the second joint calibration data is data obtained by jointly calibrating the depth camera that takes the first image and the third image, and the third point cloud map and the first spliced ​​point cloud map have an overlapping area, and the coordinate origin is at the first position.

[0077] It should be noted that the at least four images also include a third image having an overlapping area with the first image, and a fourth image having an overlapping area with both the second image and the third image. Here, the first position is determined as the position of the depth camera that took the first image in the world coordinate system. The third image is an image of the at least four images other than the second image that has an overlapping area with the first image. Figure 2For example, if the first image is an image captured by the first depth camera 22, and the second image is an image captured by the second depth camera 23, then the third image is an image captured by the first depth camera 24. It can be understood that the process of generating a third point cloud map with a coordinate origin at the first position based on the depth data of each pixel in the third image and the second joint calibration data is similar to the process of mapping each pixel in the second image to a world coordinate system with the first position as the coordinate origin to obtain a second point cloud map based on the depth data of each pixel in the second image and the first joint calibration data in the aforementioned embodiment of the invention, and will not be repeated here.

[0078] The first spliced ​​point cloud image and the third point cloud image are overlapped according to the second overlapping area to obtain a second spliced ​​point cloud image, wherein the second overlapping area is an overlapping area of ​​the third point cloud image and the first spliced ​​point cloud image.

[0079] In this step, the coordinate origins of the first stitched point cloud image and the third point cloud image are both the positions of the depth camera that captured the first image in the world coordinate system, that is, the two have a unified coordinate origin. And the first point cloud image portion in the first stitched point cloud image has an overlapping area with the third point cloud image, so the first stitched point cloud image and the third point cloud image can be directly overlapped according to the overlapping area to obtain a second stitched point cloud image, but it is not limited to this. The first point cloud image and the third point cloud image can also be overlapped according to the overlapping area to obtain another stitched point cloud image, and the stitched point cloud image is stitched with the first stitched point cloud image to obtain a second stitched point cloud image.

[0080] Continue with Figure 2 For example, assume that the first image is an image captured by the first depth camera 22, the second image is an image captured by the second depth camera 23, and the third image is an image captured by the third depth camera 24. Correspondingly, the first point cloud map is a point cloud map corresponding to the first image, the second point cloud map is a point cloud map corresponding to the second image, and the third point cloud map is a point cloud map corresponding to the third image. The coordinate origins of the first point cloud map, the second point cloud map, and the third point cloud map are all the positions of the first depth camera 22 in the world coordinate system. While obtaining a first spliced ​​point cloud map based on the first point cloud map and the second point cloud map, the first point cloud map is copied, and another spliced ​​point cloud map is obtained based on the re-copied first point cloud map and the third point cloud map. The two spliced ​​point cloud maps are spliced ​​or overlapped again to generate a second spliced ​​point cloud map. It can be understood that the two stitched point cloud images used to generate the second stitched point cloud image respectively contain all the contents of the first point cloud image and the second point cloud image, and all the contents of the first point cloud image and the third point cloud image, and both contain the contents of the first point cloud image. Therefore, there is an overlapping area between the two stitched point cloud images, and the second stitched image can be obtained by overlapping them according to the overlapping area, or the two stitched point cloud images can be divided from the same position and a part of each can be taken for stitching to obtain the second stitched image.

[0081] Based on the depth data of each pixel point in the fourth image, a fourth point cloud image is generated with a coordinate origin at the target position of the vehicle, wherein the fourth point cloud image and the second spliced ​​point cloud image have an overlapping area.

[0082] In this step, when the depth cameras are all installed on the vehicle, the target position can be any position of the vehicle. Before generating the fourth point cloud map, the depth camera that takes the fourth image will be calibrated using the target position to obtain calibration data. When generating the fourth point cloud map, the calibration data will be used for processing. The fourth image is an image of the at least four images, except the first image, that has overlapping areas with the second image and the third image.

[0083] The coordinate origin of the second stitched point cloud map is switched to the target position, and the second stitched point cloud map after the coordinate origin is switched is overlapped with the fourth point cloud map according to the third overlapping area to obtain a panoramic point cloud map, wherein the third overlapping area is the overlapping area of ​​the fourth point cloud map and the second stitched point cloud map.

[0084] It should be noted that, when the target position is not the position of the depth camera that captured the first image in the world coordinate system, the second stitched point cloud image and the fourth point cloud image are unified with the coordinate origin, and then overlapped based on the overlapping area of ​​the two to obtain a global point cloud image. Figure 2 For example, the second stitched point cloud image includes all the contents of the second point cloud image and the third point cloud image, and both the second point cloud image and the third point cloud image have overlapping areas with the fourth point cloud image, that is, the overlapping area is the area corresponding to the scene in the second field of view overlapping area 27 and the third field of view overlapping area 28 of the two depth cameras. Therefore, the second stitched point cloud image and the fourth point cloud image have overlapping areas, and they can be overlapped according to the overlapping areas.

[0085] In an embodiment of the present invention, the data of the third point cloud map and the fourth point cloud map are sequentially spliced ​​into the first spliced ​​point cloud map, so that while obtaining the panoramic point cloud map, the accuracy of the point cloud data can be improved, and a better overlapping effect can be achieved in the overlapping area, thereby improving the quality of the final image.

[0086] Optionally, a panoramic point cloud image is obtained based on the first spliced ​​point cloud image and the depth data of each pixel point in the remaining image, including:

[0087] A third point cloud map and a fourth point cloud map are generated based on the depth data of each pixel point in the third image and the fourth image; wherein the third point cloud map and the fourth point cloud map have an overlapping area, and the coordinate origins are both at the second position.

[0088] It should be noted that the at least four images also include a third image having an overlapping area with the first image, and a fourth image having an overlapping area with both the second image and the third image. That is, the third image is an image of at least four images other than the second image that has an overlapping area with the first image; the fourth image is an image of at least four images other than the first image that has an overlapping area with both the second image and the third image. The second position is the position of the depth camera that takes the third image or the fourth image in the world coordinate system. The process of generating the third point cloud map and the fourth point cloud map based on the depth data of each pixel in the third image and the fourth image is similar to the process of generating the first point cloud map and the second point cloud map based on the depth data of each pixel in the first image and the second image in the above-mentioned embodiment of the invention, and will not be repeated here.

[0089] The third point cloud image and the fourth point cloud image are overlapped according to the second overlapping area to obtain a second spliced ​​point cloud image, wherein the second overlapping area is an overlapping area of ​​the third point cloud image and the fourth point cloud image.

[0090] In this step, the third point cloud image and the fourth point cloud image have the same coordinate origin, so the third spliced ​​point cloud image obtained by overlapping according to the second overlapping area can be understood as a point cloud image generated based on different pictures observed at the same position and the same viewing angle. The specific process is similar to the above step 103 and will not be repeated here.

[0091] The coordinate origins of the first stitched point cloud image and the second stitched point cloud image are switched to the target position of the vehicle, and they are overlapped according to the third overlapping area to obtain a panoramic point cloud image, wherein the third overlapping area is the overlapping area of ​​the first stitched point cloud image and the second stitched point cloud image.

[0092] It should be noted that when the coordinate origins of the first stitched point cloud map and the second stitched point cloud map are different, the coordinate origins need to be unified. When the depth cameras are installed on the vehicle, the coordinate origins of the two stitched point cloud maps are switched to the same position of the vehicle, that is, the target position. The resulting panoramic point cloud map can be understood as a point cloud map generated based on different pictures observed at the same position and the same viewing angle. Since the first image and the third image have overlapping areas, and the second image and the fourth image have overlapping areas, the first stitched point cloud map and the third stitched point cloud map also have overlapping areas, that is, the third overlapping area. Continuing with Figure 2For example, assuming that the first image is an image captured by the first depth camera 22, the second image is an image captured by the second depth camera 23, the third image is an image captured by the third depth camera 24, and the fourth image is an image captured by the fourth depth camera 25. Correspondingly, the first point cloud image is a point cloud image corresponding to the first image, the second point cloud image is a point cloud image corresponding to the second image, the third point cloud image is a point cloud image corresponding to the third image, and the fourth point cloud image is a point cloud image corresponding to the fourth image. After the coordinate origin of the first spliced ​​point cloud image and the third spliced ​​point cloud image is switched to the same position of the vehicle, there is an overlapping area, that is, the overlapping area is the area corresponding to the scene in the second field of view overlap area 27 and the fourth field of view overlap area 29 of the two depth cameras. Therefore, there is an overlapping area between the first spliced ​​point cloud image and the third spliced ​​point cloud image, that is, the third overlapping area, and they can be overlapped according to the third overlapping area.

[0093] In an embodiment of the present invention, after the third point cloud map and the fourth point cloud map are overlapped according to the second overlapping area to generate a third stitched point cloud map, the first stitched point cloud map and the third stitched point cloud map are overlapped again according to the third overlapping area, so that while obtaining the panoramic point cloud map, the accuracy of the point cloud data can be improved, and a better overlapping effect can be achieved in the overlapping area, thereby improving the quality of the final image.

[0094] Optionally, the target position is the position of the driver's head of the vehicle.

[0095] It should be noted that the position of the driver's head can be understood as the position of the head of the person sitting in the driving position, such as Figure 3 As shown, position A is the position of the driver's head. Here, a three-dimensional coordinate inside the vehicle can be determined according to the design of the interior of the vehicle as the position of the driver's head. Preferably, different designs of the interior of the vehicle correspond to different three-dimensional coordinates.

[0096] In the embodiment of the present invention, the target position is determined as the position of the driver's head of the vehicle, so that the three-dimensional panoramic image can be presented from the driver's perspective, thereby improving the driver's user experience.

[0097] The following is a specific example to illustrate the method for generating a three-dimensional panoramic image provided by an embodiment of the present invention. Figure 4As shown. Among them, depth camera No. 1 is set on the left side of the vehicle, depth camera No. 2 is set on the front of the vehicle, depth camera No. 3 is set on the right side of the vehicle, and depth camera No. 4 is set at the rear of the vehicle. The horizontal field of view of the original imaging of each depth camera is greater than 180°, the resolution is greater than 1920X 1080, and the maximum ranging accuracy of each group of cameras reaches the centimeter level, but is not limited to this. Here, the original image is used for higher resolution calibration, so that the field of view of the calibrated large image is close to the field of view of the original imaging, and the target is 180°. The front and rear depth cameras can be used as driving recorders and reversing images. The depth camera here is an RGBD camera, but it is not limited to this. A combination of multiple sets of monocular cameras and laser radars can also be used as depth cameras. Preferably, the four groups of cameras are all full-vehicle grade, with the characteristics of miniaturization, high reliability, high durability, and high adaptability. The calibration, matching, and rendering calculations of the RGBD camera are all performed in a high-performance programmable chip.

[0098] Specifically, the data collected by depth camera No. 1 and the data collected by depth camera No. 2 are spliced ​​in images and three-dimensional coordinates to obtain a spliced ​​image with depth camera No. 2 as the reference (coordinate origin) (equivalent to the first point cloud spliced ​​image in the above-mentioned invention embodiment). The data collected by depth camera No. 2 and the data collected by depth camera No. 3 are spliced ​​in images and three-dimensional coordinates to obtain a spliced ​​image with depth camera No. 2 as the reference (equivalent to the second point cloud spliced ​​image in the above-mentioned invention embodiment). The two spliced ​​point cloud images are spliced ​​into a front spliced ​​image of point A with depth camera No. 2 as the coordinate origin, which includes the front scene of the vehicle, the left scene, and the right scene. The coordinate origin of the front spliced ​​image of point A is switched to point A.

[0099] Depth camera No. 4 acquires a point cloud image of the rear of point A (equivalent to the fourth point cloud image in the above-mentioned invention embodiment), and switches the coordinate origin of the point cloud image of the rear of point A to point A. Then, the front stitching image of point A and the rear point cloud image of point A, both of which take point A as the coordinate origin, are spliced ​​and fused to obtain a 360° image with point A as the coordinate origin and spliced ​​fusion data of three-dimensional coordinates, which is equivalent to the three-dimensional panoramic image in the above-mentioned invention embodiment. Finally, it is output to the display module, and the display model displays the three-dimensional panoramic image.

[0100] In the embodiment of the present invention, the use of multiple RGBD cameras can achieve real-time high-precision, high-frame rate, and high-resolution scene reconstruction. The use of multiple RGBD cameras for high-precision joint calibration can ensure that when multiple depth cameras are spliced, the error at the splicing point is as low as centimeter level. At the same time, it can ensure that the hardware system can still maintain the original accuracy effect after the equipment experiences vibration, structural aging, high and low temperature shock during normal use.

[0101] The above describes the method for generating a 3D panoramic image provided by an embodiment of the present invention. The following describes the device for generating a 3D panoramic image provided by an embodiment of the present invention in conjunction with the accompanying drawings.

[0102] See also Figure 5 The embodiment of the present invention further provides a device for generating a three-dimensional panoramic image, the device comprising:

[0103] An acquisition module 51 is used to acquire at least four images taken by at least four depth cameras at the same time and depth data of each pixel in the image, wherein the scenes within the field of view of adjacent depth cameras overlap;

[0104] A point cloud module 52, for generating a first point cloud map and a second point cloud map based on the depth data of each pixel point in the first image and the second image, wherein the first point cloud map and the second point cloud map have an overlapping area and the coordinate origins are both at the first position;

[0105] A first stitching module 53 is used to overlap the first point cloud image and the second point cloud image according to a first overlapping area to obtain a first stitched point cloud image, wherein the first overlapping area is an overlapping area of ​​the first point cloud image and the second point cloud image;

[0106] A second stitching module 54 is used to obtain a panoramic point cloud image based on the first stitching point cloud image and the depth data of each pixel point in the remaining images; wherein the remaining images are images other than the first image and the second image in the at least four images;

[0107] The panoramic module 55 is used to generate a three-dimensional panoramic image based on the panoramic point cloud image and the color data of each pixel point in at least four images.

[0108] Optionally, the point cloud module 52 includes:

[0109] A first point cloud unit, configured to map each pixel in the first image to a world coordinate system with the first position as a coordinate origin to obtain a first point cloud image based on depth data of each pixel in the first image;

[0110] A second point cloud unit is used to map each pixel in the second image to a world coordinate system with the first position as a coordinate origin to obtain a second point cloud image based on the depth data of each pixel in the second image and the first joint calibration data;

[0111] The first joint calibration data is data obtained by performing joint calibration on a depth camera that captures the first image and the second image.

[0112] Optionally, the second splicing module 54 includes:

[0113] A third point cloud unit is used to generate a third point cloud map based on the depth data of each pixel in the third image and the second joint calibration data; wherein the second joint calibration data is data obtained by jointly calibrating the depth camera that shoots the first image and the third image; the third point cloud map and the first spliced ​​point cloud map have an overlapping area, and the coordinate origin is at the first position;

[0114] A first stitching unit is used to overlap the first stitching point cloud image and the third point cloud image according to a second overlapping area to obtain a second stitching point cloud image, wherein the second overlapping area is an overlapping area of ​​the third point cloud image and the first stitching point cloud image;

[0115] a fourth point cloud unit, for generating a fourth point cloud image with a coordinate origin at a target position of the vehicle based on depth data of each pixel point in the fourth image, wherein the fourth point cloud image and the second spliced ​​point cloud image have an overlapping area;

[0116] The second stitching unit is used to switch the coordinate origin of the second stitching point cloud map to the target position, and overlap the second stitching point cloud map after the coordinate origin is switched with the fourth point cloud map according to the third overlapping area to obtain a panoramic point cloud map, wherein the third overlapping area is the overlapping area of ​​the fourth point cloud map and the second stitching point cloud map.

[0117] Optionally, the second splicing module 54 includes:

[0118] A third point cloud unit is used to generate a third point cloud map and a fourth point cloud map based on the depth data of each pixel point in the third image and the fourth image; wherein the third point cloud map and the fourth point cloud map have an overlapping area, and the coordinate origins are both at the second position;

[0119] A first splicing unit is used to overlap the third point cloud image and the fourth point cloud image according to the second overlapping area to obtain a second spliced ​​point cloud image, wherein the second overlapping area is an overlapping area of ​​the third point cloud image and the fourth point cloud image;

[0120] The panoramic unit is used to switch the coordinate origins of the first stitching point cloud map and the second stitching point cloud map to the target position of the vehicle, and overlap them according to the third overlapping area to obtain a panoramic point cloud map, wherein the third overlapping area is the overlapping area of ​​the first stitching point cloud map and the second stitching point cloud map.

[0121] Optionally, the target position is the position of the driver's head of the vehicle.

[0122] The three-dimensional panoramic image generation device provided by the embodiment of the present invention can achieve Figures 1 to 4 To avoid repetition, the various processes of the method for generating a three-dimensional panoramic image in the method embodiment are not described here.

[0123] In an embodiment of the present invention, images and depth data taken by at least four depth cameras at the same time can be obtained. Since the scenes within the field of view of adjacent depth cameras overlap, scene information around the installation position of the depth cameras can be obtained. Then, based on the depth data of each pixel in the first image and the second image, a first point cloud map and a second point cloud map are generated. Among them, the first point cloud map and the second point cloud map have an overlapping area, and the coordinate origins are both in the first position, that is, the two point cloud maps have the same coordinate origin in the world coordinate system. Then, the first point cloud map and the second point cloud map can be overlapped directly according to the first overlapping area to obtain a first spliced ​​point cloud map, wherein the first overlapping area is the overlapping area of ​​the first point cloud map and the second point cloud map. Since the two point cloud maps have the same coordinate origin, they can be directly spliced ​​to obtain a first spliced ​​point cloud map. Then, based on the first stitching point cloud map and the depth data of each pixel in the remaining images, a panoramic point cloud map is obtained, and the color data is added to the panoramic point cloud map to obtain a three-dimensional panoramic image. In addition to containing color data, the three-dimensional panoramic image displays the scene around the vehicle to the user in a three-dimensional form, thereby guiding the user to make accurate judgments about the situation around the vehicle, improving safety during driving, and thus meeting the user's increasing requirements for driving safety.

[0124] On the other hand, an embodiment of the present application further provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for generating a three-dimensional panoramic image provided in the above-mentioned embodiments of the application is implemented.

[0125] On the other hand, an embodiment of the present application further provides a readable storage medium. When the instructions in the readable storage medium are executed by a processor of an electronic device, the electronic device can execute the method for generating a three-dimensional panoramic image provided in the above application embodiments.

[0126] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0127] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0128] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0129] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for generating a three-dimensional panoramic image, characterized in that: The method comprises: Acquire images taken by at least four depth cameras at the same time and depth data of each pixel in the images, wherein the scenes within the field of view of adjacent depth cameras overlap; Generate a first point cloud map and a second point cloud map based on the depth data of each pixel point in the first image and the second image, wherein the first point cloud map and the second point cloud map have an overlapping area and the coordinate origins are both at the first position; Overlapping the first point cloud image and the second point cloud image according to a first overlapping area to obtain a first spliced ​​point cloud image, wherein the first overlapping area is an overlapping area between the first point cloud image and the second point cloud image; Based on the first spliced ​​point cloud image and the depth data of each pixel point in the remaining image, a panoramic point cloud image is obtained, including: Generate a third point cloud image based on the depth data of each pixel in the third image and the second joint calibration data; wherein the second joint calibration data is data obtained by joint calibration of the depth camera that shoots the first image and the third image; the third point cloud image and the first spliced ​​point cloud image have an overlapping area, and the coordinate origin is at the first position; Overlapping the first stitched point cloud image and the third point cloud image according to a second overlapping area to obtain a second stitched point cloud image, wherein the second overlapping area is an overlapping area between the third point cloud image and the first stitched point cloud image; Based on the depth data of each pixel point in the fourth image, generating a fourth point cloud image with a coordinate origin at the target position of the vehicle, wherein the fourth point cloud image and the second spliced ​​point cloud image have an overlapping area; Switching the coordinate origin of the second stitched point cloud image to the target position, and overlapping the second stitched point cloud image after the coordinate origin is switched with the fourth point cloud image according to a third overlapping area to obtain a panoramic point cloud image, wherein the third overlapping area is an overlapping area of ​​the fourth point cloud image and the second stitched point cloud image; The remaining images are images other than the first image and the second image among the at least four images; A three-dimensional panoramic image is generated based on the panoramic point cloud map and the color data of each pixel point in the at least four images.

2. The method according to claim 1, characterized in that The generating the first point cloud image and the second point cloud image based on the depth data of each pixel point in the first image and the second image includes: Based on the depth data of each pixel in the first image, each pixel in the first image is mapped to a world coordinate system with the first position as a coordinate origin to obtain the first point cloud image; Based on the depth data of each pixel in the second image and the first joint calibration data, mapping each pixel in the second image to a world coordinate system with the first position as the coordinate origin to obtain the second point cloud image; The first joint calibration data is data obtained by performing joint calibration on the depth camera that captures the first image and the second image.

3. The method according to claim 1, characterized in that The step of obtaining a panoramic point cloud map based on the first spliced ​​point cloud map and the depth data of each pixel point in the remaining image includes: Generate a third point cloud map and a fourth point cloud map based on the depth data of each pixel point in the third image and the fourth image; wherein the third point cloud map and the fourth point cloud map have an overlapping area, and the coordinate origins are both at the second position; Overlapping the third point cloud image and the fourth point cloud image according to a second overlapping area to obtain a second spliced ​​point cloud image, wherein the second overlapping area is an overlapping area between the third point cloud image and the fourth point cloud image; The coordinate origins of the first stitching point cloud map and the second stitching point cloud map are switched to the target position of the vehicle, and overlapped according to the third overlapping area to obtain a panoramic point cloud map, wherein the third overlapping area is the overlapping area of ​​the first stitching point cloud map and the second stitching point cloud map.

4. The method according to claim 1 or 3, characterized in that: The target position is the position of the driver's head of the vehicle.

5. A device for generating a three-dimensional panoramic image, characterized in that: The device comprises: An acquisition module, used to acquire at least four images taken by at least four depth cameras at the same time and depth data of each pixel in the image, wherein the scenes within the field of view of adjacent depth cameras overlap; A point cloud module, configured to generate a first point cloud map and a second point cloud map based on depth data of each pixel point in the first image and the second image, wherein the first point cloud map and the second point cloud map have an overlapping area and the coordinate origins are both at a first position; A first stitching module is used to overlap the first point cloud image and the second point cloud image according to a first overlapping area to obtain a first stitched point cloud image, wherein the first overlapping area is an overlapping area of ​​the first point cloud image and the second point cloud image; a second stitching module is used to obtain a panoramic point cloud image based on the first stitched point cloud image and the depth data of each pixel point in the remaining image, including: Generate a third point cloud image based on the depth data of each pixel in the third image and the second joint calibration data; wherein the second joint calibration data is data obtained by joint calibration of the depth camera that shoots the first image and the third image; the third point cloud image and the first spliced ​​point cloud image have an overlapping area, and the coordinate origin is at the first position; Overlapping the first stitched point cloud image and the third point cloud image according to a second overlapping area to obtain a second stitched point cloud image, wherein the second overlapping area is an overlapping area between the third point cloud image and the first stitched point cloud image; Based on the depth data of each pixel point in the fourth image, generating a fourth point cloud image with a coordinate origin at the target position of the vehicle, wherein the fourth point cloud image and the second spliced ​​point cloud image have an overlapping area; Switching the coordinate origin of the second stitched point cloud image to the target position, and overlapping the second stitched point cloud image after the coordinate origin is switched with the fourth point cloud image according to a third overlapping area to obtain a panoramic point cloud image, wherein the third overlapping area is an overlapping area of ​​the fourth point cloud image and the second stitched point cloud image; The remaining images are images other than the first image and the second image among the at least four images; The panoramic module is used to generate a three-dimensional panoramic image based on the panoramic point cloud map and the color data of each pixel in the at least four images.

6. The device according to claim 5, characterized in that The point cloud module comprises: A first point cloud unit, configured to map each pixel in the first image to a world coordinate system with the first position as a coordinate origin to obtain a first point cloud image based on depth data of each pixel in the first image; A second point cloud unit is used to map each pixel in the second image to a world coordinate system with the first position as a coordinate origin to obtain a second point cloud image based on the depth data of each pixel in the second image and the first joint calibration data; The first joint calibration data is data obtained by performing joint calibration on the depth camera that captures the first image and the second image.

7. The device according to claim 5, characterized in that The second splicing module comprises: a third point cloud unit, configured to generate a third point cloud map based on the depth data of each pixel in the third image and the second joint calibration data; wherein the second joint calibration data is data obtained by performing joint calibration on the depth camera that shoots the first image and the third image; the third point cloud map and the first spliced ​​point cloud map have an overlapping area, and the coordinate origin is at the first position; A first stitching unit is used to overlap the first stitching point cloud image and the third point cloud image according to a second overlapping area to obtain a second stitching point cloud image, wherein the second overlapping area is an overlapping area of ​​the third point cloud image and the first stitching point cloud image; a fourth point cloud unit, configured to generate a fourth point cloud image with a coordinate origin at a target position of the vehicle based on depth data of each pixel point in the fourth image, wherein the fourth point cloud image and the second spliced ​​point cloud image have an overlapping area; The second stitching unit is used to switch the coordinate origin of the second stitching point cloud image to the target position, and overlap the second stitching point cloud image after the coordinate origin is switched with the fourth point cloud image according to a third overlapping area to obtain a panoramic point cloud image, wherein the third overlapping area is an overlapping area of ​​the fourth point cloud image and the second stitching point cloud image.

8. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method for generating a three-dimensional panoramic image as claimed in any one of claims 1 to 4 when executing the program.

9. A readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method for generating a three-dimensional panoramic image as described in any one of claims 1 to 4.

Images