Panoramic scanning method and related device

Abstract

The invention discloses a panoramic scanning method and a related device, and relates to the technical field of panoramic scanning. The method is applied to a control unit of a panoramic scanning system, three-dimensional point cloud information, collected by all first laser radars, of a to-be-scanned object is fused, three-dimensional image data of the to-be-scanned object is obtained, the first laser radars are fixedly installed at preset positions of the to-be-scanned object, and the first laser radars are fixedly installed at the preset positions of the to-be-scanned object. And the scanning range of all the first laser radars covers the to-be-scanned object. Under the condition that the three-dimensional image data meets a preset supplementary collection condition, according to the three-dimensional image data, determining a to-be-supplementary-collected area of the to-be-scanned object, and controlling a second laser radar to collect three-dimensional point cloud information of the to-be-supplementary-collected area; and fusing the three-dimensional point cloud information and the three-dimensional image data of the to-be-complemented area to obtain a panoramic three-dimensional image of the to-be-scanned object. On the basis, efficient cooperation of wide-area scanning and fixed-point accurate scanning of the to-be-scanned object can be achieved, the panoramic scanning efficiency is improved, and the integrity and accuracy of a scanning result are guaranteed.

Description

Technical Field

[0001] This application relates to the field of panoramic scanning technology, and in particular to a panoramic scanning method and related apparatus. Background Technology

[0002] Currently, common cargo hold scanning techniques typically employ laser ranging and 3D imaging to construct a 3D model of the cargo hold's interior, thereby understanding the cargo layout. For example, fully automated ship unloaders, operating in fully automated mode, use a lidar mounted on the underside of the main beam to scan the cargo hold of the ship to be unloaded, identifying the shape and location of the cargo inside. However, due to the limited effective measurement range of lidar, it is necessary to control the lidar's forward and backward movement to enhance the scanning range and accuracy, resulting in a relatively long cycle for a complete cargo hold scan.

[0003] Therefore, there is an urgent need for a panoramic scanning method to improve the efficiency and completeness of panoramic scanning. Summary of the Invention

[0004] In view of the above problems, this application provides a panoramic scanning method and related apparatus to improve the efficiency and completeness of panoramic scanning. The specific solution is as follows:

[0005] The first aspect of this application provides a panoramic scanning method applied to a control unit of a panoramic scanning system, the panoramic scanning system further comprising: at least one first lidar and at least one movable second lidar; the first lidar is fixedly installed at a preset position of the object to be scanned, and the scanning range of all the first lidars covers the object to be scanned;

[0006] The panoramic scanning method includes:

[0007] Acquire the three-dimensional point cloud information of the object to be scanned, collected by the first lidar;

[0008] The three-dimensional point cloud information of the object to be scanned collected by the first lidar is fused to obtain the three-dimensional image data of the object to be scanned.

[0009] When the three-dimensional image data meets the preset supplementary acquisition conditions, the supplementary acquisition area of ​​the object to be scanned is determined based on the three-dimensional image data, and the second lidar is controlled to acquire the three-dimensional point cloud information of the supplementary acquisition area.

[0010] The three-dimensional point cloud information of the area to be scanned and the three-dimensional image data are fused to obtain a panoramic three-dimensional image of the object to be scanned.

[0011] In one possible implementation, fusing the 3D point cloud information of the object to be scanned acquired by all the first lidar sensors to obtain 3D image data of the object to be scanned includes:

[0012] Based on the scanning range of each first lidar, at least one lidar group is identified, wherein the lidar group includes at least two first lidars whose scanning range overlap rate reaches a preset threshold.

[0013] Determine whether the difference rate between the three-dimensional point cloud information of the object to be scanned collected by each first lidar in the lidar group and the three-dimensional point cloud information corresponding to the target scanning range reaches a preset difference threshold, wherein the target scanning range is the intersection of the scanning ranges of all the first lidars in the lidar group;

[0014] When the difference rate reaches the preset difference threshold, the second lidar is controlled to move in order to obtain the three-dimensional point cloud information of the target scanning range collected by the second lidar.

[0015] Based on the three-dimensional point cloud information of the target scanning range, the three-dimensional point cloud information of the object to be scanned collected by each of the first lidars in the lidar group is calibrated.

[0016] The three-dimensional point cloud information of the object to be scanned corresponding to each of the first lidars is fused to obtain the three-dimensional image data of the object to be scanned.

[0017] In one possible implementation, the process of determining whether the three-dimensional image data meets the preset acquisition conditions includes:

[0018] Based on the pre-acquired geometric reference features of the object to be scanned, determine whether the deviation rate between the geometric features of the object to be scanned and the geometric reference features in the three-dimensional image data is greater than a preset supplementary sampling threshold.

[0019] If the deviation rate is greater than the preset supplementary sampling threshold, then the three-dimensional image data satisfies the preset supplementary sampling condition;

[0020] If the deviation rate is not greater than the preset supplementary acquisition threshold, then the three-dimensional image data does not meet the preset supplementary acquisition condition.

[0021] In one possible implementation, determining the area to be scanned based on the three-dimensional image data, and controlling the second lidar to acquire the three-dimensional point cloud information of the area to be scanned includes:

[0022] Determine the current first position of the second lidar;

[0023] Based on the first position and the scanning range of the second lidar, the motion trajectory of the second lidar from the first position to the target scanning position is determined, wherein the target scanning position is the position in which the scanning range of the second lidar includes the area to be sampled.

[0024] Based on the motion trajectory, the second lidar is controlled to acquire three-dimensional point cloud information of the area to be acquired.

[0025] In one possible implementation, the process of determining the location of the area to be replenished includes:

[0026] Based on the aforementioned three-dimensional image data, a three-dimensional coordinate system is established;

[0027] The location of the region in the three-dimensional image data that meets the preset re-sampling conditions in the three-dimensional coordinate system is determined. In one possible implementation, the object to be scanned is a ship's cabin;

[0028] The first lidar is fixedly installed at the bottom of the main beam of the grab unloader, and the second lidar is installed at the bottom of the mobile driver's cab of the grab unloader.

[0029] A second aspect of this application provides a panoramic scanning system, comprising: at least one first lidar, at least one second lidar, and a control unit;

[0030] The first lidar is installed at a preset position on the object to be scanned, and the scanning range of all the first lidars covers the object to be scanned;

[0031] The second lidar is a mobile lidar;

[0032] The control unit is used to implement the panoramic scanning method of the first aspect or any implementation thereof.

[0033] In one possible implementation, it further includes: a mobile device that responds to the movement control of the control unit to move the second lidar mounted on the mobile device.

[0034] A third aspect of this application provides a computer program product including computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the panoramic scanning method described in the first aspect or any implementation thereof.

[0035] A fourth aspect of this application provides an electronic device, including at least one processor and a memory connected to the processor, wherein:

[0036] The memory is used to store computer programs;

[0037] The processor is used to execute the computer program so that the electronic device can implement the panoramic scanning method of the first aspect or any implementation thereof.

[0038] The panoramic scanning method and related apparatus provided in this application utilize at least one first lidar fixed at a preset position on the object to be scanned to scan the object. Since the scanning range of all the first lidars can basically cover the object to be scanned, the three-dimensional point cloud information acquired by all the first lidars can construct the three-dimensional image data of the object to be scanned. Compared with the moving scanning method, the panoramic scanning efficiency is significantly improved. Furthermore, when it is detected that the three-dimensional image data of the object to be scanned meets the preset supplementary acquisition conditions, a moving second lidar is controlled to accurately scan the area to be supplemented. The three-dimensional point cloud information of the area to be supplemented is fused with the three-dimensional image data, thereby ensuring the integrity and accuracy of the scanning results.

[0039] Furthermore, supplementary sampling conditions can be set based on the dynamic changes of the object to be scanned. When the three-dimensional image of the object to be scanned meets the supplementary sampling conditions, it is determined that the object to be scanned has undergone dynamic changes, and the second lidar is controlled to perform supplementary sampling, thereby enhancing the response capability of this application to dynamic changes of the object to be scanned and realizing efficient coordination between wide-area scanning and fixed-point precise scanning of the object to be scanned. Attached Figure Description

[0040] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0041] Figure 1 A system architecture diagram for implementing the panoramic scanning method provided in this application;

[0042] Figure 2 A flowchart illustrating a panoramic scanning method provided in an embodiment of this application;

[0043] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0044] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0045] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0046] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0047] This application can be applied to the field of panoramic scanning. The following section takes panoramic scanning using LiDAR as an example to introduce several application scenarios that have been implemented in products.

[0048] First, we introduce one possible application scenario for this application. In situations such as cargo hold loading / unloading planning and real-time monitoring during transportation, it is necessary to obtain the complete spatial distribution and stacking status of goods within the cargo hold to meet requirements such as logistics optimization and security checks. A common approach is to use LiDAR to scan the interior of the cargo hold. For example, multiple LiDARs can be installed at preset intervals inside the cargo hold, or the cargo hold can be scanned by moving LiDARs. However, when LiDARs are fixed, there is a problem of blind spots in the panoramic scan due to cargo obstruction; while moving LiDARs significantly increases the time consumption of the panoramic scan, resulting in low scanning efficiency.

[0049] To address the aforementioned technical problems, this application provides a panoramic scanning system, comprising: at least one first lidar, at least one second lidar, and a control unit; the first lidars are installed at a preset position on the object to be scanned, and the scanning range of all the first lidars covers the object to be scanned; the second lidars are movable lidars; and the control unit is used to implement the panoramic scanning method described below.

[0050] This panoramic scanning system integrates the advantages of fixed and mobile LiDAR. By installing at least one first LiDAR at a preset location on the object to be scanned, and ensuring that the scanning range of all first LiDARs fully covers the object, stable basic scanning of the target area is achieved. Simultaneously, at least one second LiDAR is introduced. This second LiDAR is a mobile LiDAR that can be moved in response to the control unit's movement, allowing for flexible adjustment of its position and angle to compensate for the scanning blind spots of the fixed LiDAR or enhance the scanning accuracy of specific areas. The LiDAR types of the first and second LiDARs can be selected according to scanning requirements, such as multi-line LiDAR or single-line LiDAR. The second LiDAR is a mobile LiDAR that can be moved in response to the control unit's movement.

[0051] Optionally, a slide rail is set around the object to be scanned, and a second lidar is mounted on the slide rail. The second lidar, in response to the control unit, moves along the slide rail to perform supplementary scanning. In one possible implementation, the panoramic scanning system may further include a moving device, which, in response to movement control from the control unit, moves the second lidar mounted on the moving device.

[0052] The mobile device can be an intelligent robot, a track-mounted vehicle, a drone, etc. Under the control of the control unit, the mobile device can adjust its trajectory according to the instructed path, thereby causing the second lidar mounted on it to flexibly change its position and orientation. This allows the second lidar to perform omnidirectional detection of the object to be scanned from different heights and angles. Based on this, the second lidar can overcome the limitations of a fixed position and penetrate into every corner of the object to be scanned, effectively filling the scanning blind spots of the first lidar and improving the completeness of the panoramic data.

[0053] In one possible implementation, the object to be scanned is a ship's cabin. For a panoramic scanning system capable of performing a panoramic scan of the cabin, please refer to [link to relevant documentation]. Figure 1 , Figure 1 The diagram illustrates a system architecture for the panoramic scanning method provided in this application, in which first lidars are fixedly installed at the bottom of the main beam of the grab unloader, and second lidars are installed at the bottom of the mobile driver's cab of the grab unloader.

[0054] The grab unloader consists of front and rear gantries, front and rear beams, grab buckets and a hopper receiving system, a mobile operator's cab, and other components. It is a type of machinery used for unloading bulk cargo from ships. Its core function is to grab bulk materials, such as coal and ore, from the ship's hold using grab buckets and transfer the materials to the dock's conveyor system for continuous unloading. During this process, panoramic scanning provides the unloader with key data such as the ship's hold structure and the distribution of cargo within the hold, enabling it to automatically plan the optimal operating path and dynamically adjust its grabbing strategy, significantly improving unloading efficiency.

[0055] Reference Figure 1 In this example, two sets of multi-line lidars, A1 and A2, are fixedly installed at a preset interval on the bottom of the main beam. A second lidar, B, is installed at the bottom of the mobile operator's cab, and the movement of the second lidar is controlled by the mobile operator's cab. Optionally, based on the spatial coordinate analysis of the cargo hold to be unloaded and the unloader, the layout of multiple lidars can be adaptively and scientifically planned to ensure that the panoramic scanning system can meet the maximum berthing capacity of the terminal. For example, the lidar layout of the panoramic scanning system of the terminal can be designed to meet the all-round scanning coverage requirements of the largest berthing cargo hold—a 100,000-ton class ship hold.

[0056] Reference Figure 2 , Figure 2 A flowchart illustrating a panoramic scanning method provided in this application embodiment is shown below. Figure 2 As shown in the embodiment of this application, a panoramic scanning method may include steps S110 to S150, which are described in detail below.

[0057] Step S110: Obtain the 3D point cloud information of all objects to be scanned collected by the first lidar.

[0058] Each primary lidar unit collects 3D point cloud information within its respective scanning range and transmits this information to the control unit. Optionally, in the scenario of the grab unloader unloading cargo from the ship's hold, each primary lidar unit can periodically scan the hold. It is understood that during the unloading process, the grab's grasping action causes continuous changes in the cargo accumulation pattern within the hold, such as collapse, the formation of slopes, or voids. Periodic scanning generates the latest 3D image data, helping the unloading system dynamically adjust its grasping path and avoid the grab going empty or performing repetitive operations.

[0059] Step S120: The three-dimensional point cloud information of the object to be scanned collected by all the first lidars is fused to obtain the three-dimensional image data of the object to be scanned.

[0060] Optionally, feature matching, deep learning, and point cloud mapping can be used to fuse the 3D point cloud information of the object to be scanned from all the first LiDAR sensors. Since the scanning range of all the first LiDAR sensors can basically cover the object to be scanned, the fused 3D image data of the object can basically restore the basic 3D structure of the object. Therefore, the basic 3D structure of the object to be scanned can be obtained from a single acquisition by all the first LiDAR sensors, which greatly saves the laser scanning time and improves the efficiency of panoramic scanning.

[0061] In one possible implementation, the process of implementing step S120 may include: based on the scanning range of each first lidar, identifying at least one lidar group, wherein the lidar group contains at least two first lidars whose scanning range overlap rate reaches a preset threshold; determining whether the difference rate between the 3D point cloud information of the object to be scanned collected by each first lidar in the lidar group and the 3D point cloud information corresponding to the target scanning range reaches a preset difference threshold, wherein the target scanning range is the intersection of the scanning ranges of all first lidars in the lidar group; if the difference rate reaches the preset difference threshold, controlling the second lidar to move to acquire the 3D point cloud information of the target scanning range collected by the second lidar; calibrating the 3D point cloud information of the object to be scanned collected by each first lidar in the lidar group based on the 3D point cloud information of the target scanning range; and fusing the 3D point cloud information of the object to be scanned corresponding to each first lidar to obtain the 3D image data of the object to be scanned.

[0062] Verifying the overlapping region of the 3D point clouds acquired by two first-level lidars with overlapping scanning ranges can significantly improve the accuracy and reliability of the 3D point cloud information. It is understandable that individual lidars may have discrepancies in their 3D point cloud representations of the same region due to hardware deviations (such as ranging errors) or installation errors (such as angular tilt). For example, if lidar A1 has a systematic offset in the Z-axis direction due to installation tilt, while lidar A2 is correctly installed, fusing the 3D point clouds acquired by lidars A1 and A2 will result in a mismatch in the overlapping region's 3D point cloud, leading to errors in the 3D modeling of the overlapping area.

[0063] In this embodiment, the three-dimensional point cloud information collected by the second lidar is used as a third-party verification source to verify the three-dimensional point cloud information of the first lidar with overlapping areas, ensuring the consistency and reliability of the three-dimensional point cloud information used for fusion, thereby improving the accuracy of the panoramic scanning results of the object to be scanned.

[0064] Specifically, firstly, the overlap rate between the scanning ranges of each of the first lidars installed in the panoramic scanning system is calculated, and all the first lidars are divided into lidar groups based on the overlap rate. The overlap rate between the scanning ranges of any two lidars in a group reaches a preset threshold.

[0065] Furthermore, using point cloud feature matching and deep learning, the difference rate between the 3D point cloud information collected by the first LiDAR in the group and the 3D point cloud information corresponding to the overlapping area (i.e., the target scanning range) is calculated. If the difference rate between the 3D point cloud information reaches a preset difference threshold, it proves that the difference is sufficient to affect the accuracy of the panoramic scanning results. Therefore, a movable second LiDAR is used to rescan the target scanning range and collect its 3D point cloud information. Further, the 3D point cloud information of the target scanning range collected by the second LiDAR is used as a verification standard to calibrate the 3D point cloud information of each first LiDAR in the LiDAR group. Based on this, the 3D point cloud information of the object to be scanned corresponding to each first LiDAR is fused to obtain the 3D image data of the object to be scanned.

[0066] Step S130: Determine whether the three-dimensional image data meets the preset supplementary acquisition conditions. If yes, proceed to step S140; otherwise, use the three-dimensional image data to determine the panoramic three-dimensional image of the object to be scanned.

[0067] Understandably, while the 3D point cloud information acquired by the first lidar can cover the entire object to be scanned, limitations such as the lidar's blind spot, the attenuation of point cloud accuracy at long distances, and dynamic interference lead to problems in the 3D image data obtained by fusing the 3D point cloud information acquired by the first lidar, including missing local information or excessive deviations. Figure 1 Taking the panoramic scan of the ship's hold as an example, in the actual dynamic operation scenario of the grab unloader, factors such as water level fluctuations and cargo collapse or movement of unloading equipment obscuring part of the area during the unloading process can cause the three-dimensional image data of the ship's hold to have local data loss or geometric deviations exceeding the threshold.

[0068] Based on this, the embodiments of this application can set preset re-sampling conditions for verifying whether the three-dimensional image data is distorted, according to the above-mentioned problems. For example, the preset re-sampling conditions are set as follows: re-sampling is triggered when the area of ​​continuously missing point clouds in the three-dimensional image data exceeds a set value; the key geometric features of the object to be scanned are not completely captured; the deviation between the three-dimensional image data and historical data / preset standard data exceeds the allowable range, etc.

[0069] In one possible implementation, the process of determining whether the three-dimensional image data meets the preset supplementary acquisition conditions includes: based on the pre-acquired geometric reference features of the object to be scanned, determining whether the deviation rate between the geometric features of the object to be scanned and the geometric reference features in the three-dimensional image data is greater than a preset supplementary acquisition threshold; if the deviation rate is greater than the preset supplementary acquisition threshold, the three-dimensional image data meets the preset supplementary acquisition conditions; if the deviation rate is not greater than the preset supplementary acquisition threshold, the three-dimensional image data does not meet the preset supplementary acquisition conditions.

[0070] The core of determining whether 3D image data meets the preset re-acquisition conditions lies in quantifying the deviation between the geometric features in the acquired 3D image data and the preset standard geometric features (geometric reference features), and triggering the re-acquisition mechanism when the deviation reaches a threshold.

[0071] Specifically, firstly, inspectors can use CAD models, historical scanning data, or design drawings of the object to be scanned to extract key geometric features such as size, shape, and position as geometric reference features. Further, geometric features of the object to be scanned are extracted from the 3D image data obtained from the fused point cloud data. The deviation rate between the geometric features and the corresponding geometric reference features is calculated, such as distance error and feature point matching rate. When the deviation rate exceeds a preset threshold, such as size error > 2% or shape deviation > 10%, the 3D image data is deemed to meet preset re-sampling conditions, and a re-sampling mechanism needs to be triggered to correct the deviations in the 3D image data.

[0072] Taking a panoramic scan of a ship's hold containing coal as an example, the control unit pre-stores a CAD model of the empty hold as its geometric reference feature and defines the reference plane features for the placement of the coal pile within the hold. During operation, the real-time point cloud of the hold and the point cloud of the coal pile, obtained from the first lidar scan, are fused to obtain three-dimensional image data of the hold. Geometric features of the hold and surface features of the coal pile are extracted from this data and compared with the geometric reference features and reference plane features, respectively. Specifically, based on the geometric features and geometric reference features of the hold, the deviations in size and shape of each region of the hold are calculated. Based on the surface features and reference plane features of each region of the coal pile, the point cloud density and deviations in the undulating shape of each region of the coal pile are calculated. If a deviation in size or shape of a certain region of the hold is detected to be greater than 10%, it is determined that this region needs to be re-scanned. Conversely, if the comparison of each region does not meet the preset re-scanning conditions, it proves that the three-dimensional image data is relatively accurate and can be used as the result of the panoramic scan of the hold.

[0073] Optionally, the preset supplementary sampling conditions can also be understood as the dynamic change conditions of the object to be scanned. Taking the scanning of the cargo hold as an example, during the unloading process, the cargo in the cargo hold will undergo dynamic changes. The first lidar scans at regular intervals, and the deviation between the three-dimensional image data obtained by fusing the three-dimensional point cloud information of the current scan and the three-dimensional image data obtained by fusing the previous scan is used as the preset supplementary sampling conditions. In this way, the change in cargo status between the current time and the previous scan time can be determined. In order to improve the accuracy of scanning this change in cargo status, the supplementary sampling mechanism is triggered to supplement the actual cargo hold area corresponding to the area in the three-dimensional image where the status has changed. In this way, the timely response and scanning accuracy of the panoramic scan to the dynamic changes of the object to be scanned are improved.

[0074] Step S140: Based on the three-dimensional image data, determine the area to be scanned of the object to be scanned, and control the second lidar to collect the three-dimensional point cloud information of the area to be scanned.

[0075] Step S150: The three-dimensional point cloud information and three-dimensional image data of the area to be scanned are fused to obtain a panoramic three-dimensional image of the object to be scanned.

[0076] It is understandable that step S130 can determine the data in the three-dimensional image data that meets the preset supplementary acquisition conditions, determine the area corresponding to the data in the actual three-dimensional space where the object to be scanned is located, determine the area as the supplementary acquisition area, and control the second lidar to scan the supplementary acquisition area, thereby realizing supplementary acquisition.

[0077] In one possible implementation, the process of determining the location of the area to be supplemented includes: establishing a three-dimensional coordinate system based on the three-dimensional image data; and determining the location of the area in the three-dimensional image data that meets the preset supplementation conditions within the three-dimensional coordinate system. It can be understood that the three-dimensional image data can construct a three-dimensional model of the object to be scanned, a three-dimensional coordinate system is established based on this model, and the location of the model area that meets the preset supplementation conditions within the three-dimensional coordinate system is determined, thereby determining the location of the area to be supplemented.

[0078] Furthermore, the control unit controls the second lidar to acquire 3D point cloud information of the area to be scanned. Optionally, based on the 3D image data, determining the area to be scanned of the object to be scanned and controlling the second lidar to acquire the 3D point cloud information of the area to be scanned includes: determining the current first position of the second lidar; determining the motion trajectory of the second lidar from the first position to the target scanning position based on the first position and the scanning range of the second lidar, wherein the target scanning position is the position where the scanning range of the second lidar includes the area to be scanned; and controlling the second lidar to acquire the 3D point cloud information of the area to be scanned based on the motion trajectory.

[0079] In this embodiment, the location of the area to be supplemented is accurately calculated. Based on this location and the current location of the second lidar, the control unit determines the movement trajectory of the second lidar and controls its movement accordingly, enabling it to perform 3D point cloud data supplementation on the area to be supplemented at the target scanning location. It is understood that if the scanning range of the second lidar at its current location includes the area to be supplemented, then the movement trajectory can be empty, meaning that the supplementation can be completed without moving the second lidar.

[0080] First, the control unit constructs a three-dimensional coordinate system based on the positional relationship between the panoramic scanning system and the object to be scanned. Based on this three-dimensional coordinate system, the current position coordinates of the second lidar are determined. It is understandable that, to maintain consistency between the position coordinates of the second lidar and the area to be scanned, the three-dimensional coordinate system constructed here is the same as the three-dimensional coordinate system established based on the three-dimensional image data.

[0081] Furthermore, based on the scanning range of the second lidar and the location coordinates of the area to be scanned, the optimal position for the second lidar to scan the area is determined. Then, the movement trajectory of the second lidar from its current position to this optimal position is planned. Based on this movement trajectory, the movement of the second lidar is controlled. Figure 1 In the example shown, the control unit controls the movement of the mobile driver's cab (mobile device) according to the movement trajectory, so as to drive the movement of the second lidar B.

[0082] By fusing the three-dimensional point cloud information of the area to be scanned obtained above with the three-dimensional image data, a panoramic image of the object to be scanned can be obtained.

[0083] In summary, the panoramic scanning method and related apparatus provided in this application utilize at least one first lidar fixed at a preset position on the object to be scanned to scan the object. Since the scanning range of all the first lidars can basically cover the object to be scanned, the three-dimensional point cloud information acquired by all the first lidars can construct the three-dimensional image data of the object to be scanned. Compared with the moving scanning method, the panoramic scanning efficiency is significantly improved. Furthermore, when it is detected that the three-dimensional image data of the object to be scanned meets the preset supplementary acquisition conditions, the second lidar is controlled to move to accurately scan the area to be supplemented. The three-dimensional point cloud information of the area to be supplemented is fused with the three-dimensional image data, thereby ensuring the integrity and accuracy of the scanning results.

[0084] Furthermore, supplementary sampling conditions can be set based on the dynamic changes of the object to be scanned. When the three-dimensional image of the object to be scanned meets the supplementary sampling conditions, it is determined that the object to be scanned has undergone dynamic changes, and the second lidar is controlled to perform supplementary sampling, thereby enhancing the response capability of this application to dynamic changes of the object to be scanned and realizing efficient coordination between wide-area scanning and fixed-point precise scanning of the object to be scanned.

[0085] This application also provides an electronic device in its embodiments. (See reference...) Figure 3 The diagram illustrates a structural schematic suitable for implementing the electronic device in the embodiments of this application. The electronic device in the embodiments of this application may include, but is not limited to, fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 3The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0086] like Figure 3 As shown, the electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. When the electronic device is powered on, the RAM 603 also stores various programs and data required for the operation of the electronic device. The processing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0087] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 608 including, for example, memory cards, hard drives, etc.; and communication devices 609. Communication device 609 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 3 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0088] This application also provides a computer program product including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the panoramic scanning methods provided in this application.

[0089] This application also provides a computer-readable storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the panoramic scanning methods provided in this application.

[0090] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0091] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0092] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0093] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A panoramic scanning method, characterized in that, A control unit for a panoramic scanning system, the panoramic scanning system further comprising: at least one first lidar and at least one movable second lidar; the first lidar is fixedly installed at a preset position of the object to be scanned, and the scanning range of all the first lidars covers the object to be scanned; The panoramic scanning method includes: Acquire the three-dimensional point cloud information of the object to be scanned, collected by the first lidar; The three-dimensional point cloud information of the object to be scanned collected by the first lidar is fused to obtain the three-dimensional image data of the object to be scanned. When the three-dimensional image data meets the preset supplementary acquisition conditions, the supplementary acquisition area of ​​the object to be scanned is determined based on the three-dimensional image data, and the second lidar is controlled to acquire the three-dimensional point cloud information of the supplementary acquisition area. The three-dimensional point cloud information of the area to be scanned and the three-dimensional image data are fused to obtain a panoramic three-dimensional image of the object to be scanned.

2. The panoramic scanning method according to claim 1, characterized in that, The process of fusing the 3D point cloud information of the object to be scanned acquired by all the first lidar sensors to obtain 3D image data of the object to be scanned includes: Based on the scanning range of each first lidar, at least one lidar group is identified, wherein the lidar group includes at least two first lidars whose scanning range overlap rate reaches a preset threshold. Determine whether the difference rate between the three-dimensional point cloud information of the object to be scanned collected by each first lidar in the lidar group and the three-dimensional point cloud information corresponding to the target scanning range reaches a preset difference threshold, wherein the target scanning range is the intersection of the scanning ranges of all the first lidars in the lidar group; When the difference rate reaches the preset difference threshold, the second lidar is controlled to move in order to obtain the three-dimensional point cloud information of the target scanning range collected by the second lidar. Based on the three-dimensional point cloud information of the target scanning range, the three-dimensional point cloud information of the object to be scanned collected by each of the first lidars in the lidar group is calibrated. The three-dimensional point cloud information of the object to be scanned corresponding to each of the first lidars is fused to obtain the three-dimensional image data of the object to be scanned.

3. The panoramic scanning method according to claim 1, characterized in that, The process of determining whether the three-dimensional image data meets the preset acquisition conditions includes: Based on the pre-acquired geometric reference features of the object to be scanned, determine whether the deviation rate between the geometric features of the object to be scanned and the geometric reference features in the three-dimensional image data is greater than a preset supplementary sampling threshold. If the deviation rate is greater than the preset supplementary sampling threshold, then the three-dimensional image data satisfies the preset supplementary sampling condition; If the deviation rate is not greater than the preset supplementary acquisition threshold, then the three-dimensional image data does not meet the preset supplementary acquisition condition.

4. The panoramic scanning method according to claim 1, characterized in that, The step of determining the area to be scanned of the object to be scanned based on the three-dimensional image data, and controlling the second lidar to acquire the three-dimensional point cloud information of the area to be scanned, includes: Determine the current first position of the second lidar; Based on the first position and the scanning range of the second lidar, the motion trajectory of the second lidar from the first position to the target scanning position is determined, wherein the target scanning position is the position in which the scanning range of the second lidar includes the area to be sampled. Based on the motion trajectory, the second lidar is controlled to acquire three-dimensional point cloud information of the area to be acquired.

5. The panoramic scanning method according to claim 1, characterized in that, The process of determining the location of the area to be supplemented includes: Based on the aforementioned three-dimensional image data, a three-dimensional coordinate system is established; Determine the location of the region in the three-dimensional image data that meets the preset re-sampling conditions in the three-dimensional coordinate system.

6. The panoramic scanning method according to any one of claims 1-5, characterized in that, The object to be scanned is the ship's cabin; The first lidar is fixedly installed at the bottom of the main beam of the grab unloader, and the second lidar is installed at the bottom of the mobile driver's cab of the grab unloader.

7. A panoramic scanning system, characterized in that, include: At least one first lidar, at least one second lidar, and a control unit; The first lidar is installed at a preset position on the object to be scanned, and the scanning range of all the first lidars covers the object to be scanned; The second lidar is a mobile lidar; The control unit is used to implement the panoramic scanning method as described in any one of claims 1-6.

8. The panoramic scanning system according to claim 7, characterized in that, Also includes: A mobile device that responds to the movement control of the control unit to move the second lidar mounted on the mobile device.

9. A computer program product, characterized in that, Includes computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the panoramic scanning method as described in any one of claims 1 to 6.

10. An electronic device, characterized in that, It includes at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program to enable the electronic device to implement the panoramic scanning method as described in any one of claims 1 to 6.

Images