iToF camera system and path planning method for suppressing image expansion

Abstract

The present invention relates to the field of depth perception technology, and in particular to an iToF camera system and path planning method for suppressing image expansion, wherein the iToF camera system is composed of a main control unit, a laser emitting module, a projection and replication module, and a sensor receiving module. The laser emitting module includes multiple vertical cavity surface emitting lasers, the projection and replication module includes a replication optical element, and the sensor receiving module includes an iToF chip. The iToF camera system under the aforementioned framework is used to determine a depth map of a target area, so that the emitted pulsed laser is replicated into multiple beams and projected onto the target area, thereby reducing the degree of "expansion" of the depth image generated by highly reflective objects and improving the accuracy of the generated depth map, thereby improving the accuracy of the path from the AGV's robotic arm to a storage carrier in the target area determined based on the depth map, so that the AGV's robotic arm can accurately grasp the storage carrier in the target area.

Description

Technical Field

[0001] The present invention relates to the field of depth perception technology, and in particular to an iToF camera system and a path planning method for suppressing image expansion. Background Art

[0002] Automated Guided Vehicles (AGVs) can move according to instructions in warehouses, distribution centers, and other scenarios to perform tasks such as handling and charging. Typically, AGVs are equipped with perception systems, such as cameras and radar systems. These systems use indirect time-of-flight (iToF) or direct time-of-flight (dToF) technology to generate a depth map of the AGV's environment. This depth map, combined with relevant image processing techniques, can then identify and locate storage media such as pallets and boxes used to carry goods. Path planning can also be performed, allowing the AGV's robotic arm to follow the planned path to approach and transport the storage media.

[0003] However, when there are highly reflective objects around the storage medium, for example, when the storage medium is placed on a shelf, and the shelves on different levels are connected by support columns. The support columns are usually made of iron, steel, aluminum alloy, etc., all of which have smooth surfaces and are highly reflective objects. In this case, when the AGV generates a depth map of the environment through the perception system, the highly reflective objects will cause the laser emitted by the perception system to reflect multiple times. Multiple reflections will lead to errors in the phase calculation of the echo signal corresponding to the emitted laser, thereby reducing the accuracy of the depth measurement. Highly reflective objects can also cause the laser emitted by the perception system to reflect at a mirror, causing most of the light to be reflected in other directions, resulting in the received echo signal being too weak, reducing the accuracy of the depth measurement. Furthermore, when the highly reflective objects are made of different materials and the reflectivity corresponding to the different materials varies significantly, the depth values ​​of the pixels of objects that should be at the same depth in the generated depth map will be inconsistent, resulting in depth value holes or errors. In general, the impact caused by the aforementioned highly reflective objects is manifested as causing the generated depth map to appear "bloated." Therefore, the key issue is how to eliminate the depth measurement errors caused by highly reflective objects, improve the accuracy of the depth map generated by the perception system, and thus improve the accuracy of the planned path so that the AGV robotic arm can accurately transport storage carriers.

[0004] Based on this, the present application specification provides an iToF camera system and a path planning method for suppressing image expansion. Summary of the Invention

[0005] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides an iToF camera system and a path planning method for suppressing image expansion. When an AGV robotic arm grabs storage carriers on a shelf, it can solve the technical problem that the depth map of the area including the shelf generated by the iToF camera is inaccurate due to the presence of highly reflective objects on the shelf, resulting in inaccurate path planning of the AGV robotic arm and inability to accurately grab the storage carrier.

[0006] In order to achieve the above objectives, the main technical solutions adopted by the present invention include:

[0007] In a first aspect, an embodiment of the present invention provides an iToF camera system for suppressing image expansion. The iToF camera system is installed on an AGV and includes: a main control unit, a laser emitting module, a projection replication module, and a sensor receiving module; the laser emitting module includes multiple vertical cavity surface emitting lasers; the projection replication module includes a replication optical element; and the sensor receiving module includes an iToF chip;

[0008] The main control unit is used to drive the laser emitting module and the sensor receiving module;

[0009] The main control unit is used to drive the laser emitting module and the sensor receiving module;

[0010] The laser emission module is configured to emit pulsed lasers through the plurality of vertical cavity surface emitting lasers in response to a drive of the main control unit;

[0011] The projection replication module is configured to split the pulsed laser emitted by each vertical cavity surface emitting laser through the replication optical element to obtain a first light beam array, so as to project the first light beam array to a target area;

[0012] The sensing receiving module is used to respond to the drive of the main control unit to receive the echo signal of the first light beam array reflected by the target area through the iToF chip; based on the echo signal of the first light beam array, generate a depth map of the target area, so that the AGV determines the path from the AGV's robotic arm to the storage carrier in the target area based on the depth map.

[0013] Optionally, the projection replication module further includes a collimating lens; the projection replication module is specifically configured to collimate the pulsed laser emitted by each vertical cavity surface emitting laser through the collimating lens, and split each collimated pulsed laser through the replication optical element to obtain a first light beam array;

[0014] The sensing receiving module further includes a narrowband filter, and the echo signals of each first light beam array received by the iToF chip are signals filtered by the narrowband filter.

[0015] Optionally, the plurality of vertical cavity surface emitting lasers are arranged in a specified manner;

[0016] The main control unit is specifically used to drive a specified number or a specified position of vertical cavity surface emitting lasers to emit pulsed lasers.

[0017] Optionally, the plurality of vertical cavity surface emitting lasers are arranged in a specified manner;

[0018] The main control unit is specifically used to drive the vertical cavity surface emitting lasers at designated positions to emit pulsed lasers in sequence according to a preset timing, wherein the designated positions are one position or multiple positions;

[0019] The projection replication module is specifically configured to, at each moment in the preset time sequence, split the pulsed laser emitted by the vertical cavity surface emitting laser at the designated position controlled by the main control unit at that moment through the replication optical element to obtain a second beam array, so as to project the second beam array onto a sub-region in the target region;

[0020] The sensing receiving module is specifically used to receive, for each moment in the preset time sequence, the echo signal reflected by the sub-area projected by the second light beam array corresponding to the moment through the iToF chip, and generate a sub-depth map corresponding to the moment; based on the generated sub-depth map corresponding to each moment, obtain the depth map of the target area.

[0021] Optionally, there are multiple replica optical elements, and each replica optical element splits the pulse laser beam in a different number of beams;

[0022] The system also includes a component switching module;

[0023] The main control unit is specifically configured to determine a target replica optical element suitable for a target area among a plurality of replica optical elements, and send a replica optical element switching instruction to the element switching module;

[0024] The element switching module is specifically configured to switch the replica optical element in the projection replication module to the target replica optical element in response to the replica optical element switching instruction;

[0025] The projection replication module is specifically configured to split the pulse laser emitted by each vertical cavity surface emitting laser through the target replication optical element to obtain a first light beam array.

[0026] Optionally, the main control unit is specifically configured to determine a target replication optical element based on an area of ​​the target area, wherein the area of ​​the target area is positively correlated with the number of beam splitting of the target replication optical element;

[0027] Alternatively, the target replicating optical element is determined based on the number of obstacles in the target area, and the number of obstacles is positively correlated with the beam splitting number of the target replicating optical element.

[0028] Optionally, the element switching module includes an electric motor.

[0029] Optionally, the replica optical element is a diffractive optical element, a microlens array or a spatial light modulator.

[0030] Optionally, the sensor receiving module is specifically configured to send the depth map of the target area to the main control unit;

[0031] The main control unit is specifically configured to determine a path from the robotic arm of the AGV to the storage carrier in the target area according to the received depth map of the target area.

[0032] In a second aspect, an embodiment of the present invention provides a path planning method, the path planning method being based on the iToF camera system for suppressing image expansion described in any one of the first aspects; the method being applied to an AGV; the method comprising:

[0033] Generate a depth map of the target area using the iToF camera system;

[0034] locating a position of the storage carrier in the target area according to the depth map;

[0035] Based on the position of the storage carrier, a path is determined by the robotic arm of the AGV to the storage carrier in the target area.

[0036] The beneficial effects of the present invention are as follows: the iToF camera system is composed of a main control unit, a laser emitting module, a projection and replication module, and a sensor receiving module. The laser emitting module includes multiple vertical cavity surface emitting lasers, the projection and replication module includes a replication optical element, and the sensor receiving module includes an iToF chip. The depth map of the target area is determined by the iToF camera system under the aforementioned framework, so that the emitted pulsed laser is replicated into multiple beams and projected to the target area, reducing the degree of "expansion" of the depth image generated by highly reflective objects, and improving the accuracy of the generated depth map, thereby improving the accuracy of the path from the AGV's robotic arm to the storage carrier in the target area determined by the AGV based on the depth map, so that the AGV's robotic arm can accurately grasp the storage carrier in the target area. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A depth map of a target area provided in this specification where there are no highly reflective objects;

[0038] Figure 2 A depth map of a target area with highly reflective objects provided in this specification;

[0039] Figure 3 A schematic diagram of an iToF camera system provided in this specification;

[0040] Figure 4 A schematic diagram of a photocopying module provided in this specification splitting a pulsed laser beam;

[0041] Figure 5 This is a schematic diagram of the structure of an AGV provided in this manual;

[0042] Figure 6 This is a schematic diagram of the structure of an AGV provided in this manual;

[0043] Figure 7 A sub-depth map corresponding to a moment in the preset time sequence provided in this specification;

[0044] Figure 8 The depth map obtained by stitching the sub-depth maps corresponding to each moment in the preset time sequence provided in this specification;

[0045] Figure 9 A schematic diagram of an iToF camera system provided in this specification;

[0046] Figure 10 A schematic diagram of a component switching module provided in this specification;

[0047] Figure 11 This is a flowchart of a path planning method provided in this specification. DETAILED DESCRIPTION

[0048] In order to better explain the present invention and facilitate understanding, the present invention is described in detail below through specific implementation methods in conjunction with the accompanying drawings.

[0049] Automated Guided Vehicles (AGVs) are equipped with a perception system, either a radar system or a camera system, to provide the AGV with environmental information. Generally, when an AGV performs a task, the perception system and the decision-making system work together. The perception system first provides environmental information, and then the decision-making system makes decisions. For example, when an AGV performs a transport task, the perception system first generates a depth map of the target area containing the storage carrier. The decision-making system then uses this depth map to plan a path, allowing the AGV's robotic arm to successfully contact and grasp the storage carrier for transport. Therefore, the accuracy of the depth map generated by the perception system has a crucial impact on the accuracy of the decisions made by the decision-making system.

[0050] In actual applications, highly reflective objects will affect the accuracy of the generated depth map, causing the generated depth map to appear dilated. This dilation refers to the object appearing magnified during imaging. In other words, this dilation means that the area of ​​the object presented in the depth map is larger than the area the object should actually appear in the depth map. Specifically, highly reflective objects will not only cause themselves to appear significantly dilated, but also, due to signal interference, will cause ranging errors for other objects around them and even objects within the entire field of view, causing other objects around them and even objects within the entire field of view to appear dilated. It can be seen that reducing the impact of highly reflective objects on the accuracy of the generated depth map is an important issue.

[0051] like Figure 1 as well as Figure 2 As shown, Figure 1 This is a depth map of a target area without highly reflective objects provided in this application specification. Figure 2 This is a depth map of a target area with a highly reflective object provided in this application specification. Figure 1 There are no reflective columns (highly reflective objects) in the target area shown. It can be seen that the pixels corresponding to the pallet in the depth map are relatively neat, the shape of the pallet is complete and accurate, and the pixels corresponding to the wall in the depth map are also relatively neat and the depth values ​​are relatively uniform. Figure 2 middle, Figure 2 A reflective column is placed in the target area shown. It can be seen that the reflective column in the depth map has expanded significantly and affects the pallet and the wall. The pixels of the pallet in the depth map are mixed with the pixels of the expanded part of the reflective column in the depth map, resulting in an inaccurate shape of the pallet and uneven depth values ​​of the corresponding pixels of the wall in the depth map.

[0052] Currently, to suppress this expansion, pulsed laser light is typically processed through lenses designed to achieve a transmittance of nearly 99.9% on each surface. Furthermore, the lens barrel and spacers are matted to achieve an absorption rate of nearly 99.9%, minimizing the expansion range. However, this lens design is very expensive.

[0053] Based on this, the present application provides an iToF camera system composed of a main control unit, a laser emitting module, a projection and replication module, and a sensor receiving module, which can at least be used to suppress image expansion. The laser emitting module includes multiple vertical cavity surface emitting lasers, the projection and replication module includes a replication optical element, and the sensor receiving module includes an iToF chip. The iToF camera system of this architecture determines the depth map of the target area, which can reduce the "expansion" of the depth image generated by highly reflective objects and improve the accuracy of the generated depth map, thereby improving the accuracy of the path from the AGV's robotic arm to the storage carrier in the target area determined by the AGV based on the depth map, so that the AGV's robotic arm can accurately grasp the storage carrier in the target area. At the same time, the production cost is low and it has strong practicality.

[0054] iToF stands for indirect time-of-flight (iToF) technology. In one or more embodiments of this specification, an iToF camera system refers to a camera system that uses indirect time-of-flight technology to measure distance. An iToF chip refers to a chip that uses indirect time-of-flight technology to measure distance. In other words, an iToF chip is a chip that converts the phase difference of the echo signal of a received light beam into depth information.

[0055] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. Instead, these embodiments are provided to enable a clearer and more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0056] like Figure 3 As shown, Figure 3This is a schematic diagram of an iToF camera system provided in this specification. It can be seen that the iToF camera system includes: a main control unit, a laser emitting module, a projection replication module, and a sensor receiving module. The laser emitting module includes multiple vertical-cavity surface-emitting lasers (VCSELs), the projection replication module includes replication optical elements, and the sensor receiving module includes an iToF chip. In this specification, the iToF camera system can be installed on an AGV and is used to provide the AGV with at least a depth map of the target area.

[0057] In one or more embodiments of this specification, the iToF camera system includes multiple VCSELs in the laser emission module that are 940nm VCSEL arrays, arranged in a 2×2 layout, with each VCSEL having a peak power of 8W. The DOE in the projection replication module can be made of fused quartz, and the beam after splitting the pulsed laser emitted by the VCSEL is a 16×16 array with a diffraction efficiency greater than 85%. The iToF chip in the sensor receiving module has a pixel size of 640×480, a single pixel size of 5μm×5μm, and a narrowband filter with a central wavelength of 940nm and a bandwidth of 22nm.

[0058] In this specification, the main control unit is connected to the laser emitting module and is connected to the sensor receiving module. The main control unit in the iToF camera system can drive the laser emitting module and drive the sensor receiving module.

[0059] Thus, the laser emitting module can respond to the drive of the main control unit to emit pulsed lasers through multiple vertical cavity surface emitting lasers. At the same time, the sensor receiving module can respond to the drive of the main control unit to receive the echo signals reflected by the pulsed lasers emitted by multiple vertical cavity surface emitting lasers through the iToF chip.

[0060] It should be noted that the main control unit drives the laser emitting module and the sensor receiving module at the same time, that is, while driving the laser emitting module, it also drives the sensor receiving module, or the difference between the time when the main control unit drives the laser emitting module and the time when it drives the sensor receiving module is within a preset range, thereby ensuring the accuracy of the ranging.

[0061] It should be noted that this manual does not impose any specific restrictions on how the main control unit drives the laser emission module and the sensor receiving module. The currently existing synchronous control technology, pulse coding technology, multi-channel parallel processing technology and other related electrical technologies and computer technologies can be adopted.

[0062] In this specification, the projection replication module can be pre-installed in the direction of the pulsed lasers emitted by the multiple vertical cavity surface emitting lasers in the laser emission module through optical design and mechanical adjustment techniques. After the laser emission module emits the pulsed lasers, the projection replication module can use the replication optical element to split the pulsed lasers emitted by each VCSEL to obtain a first beam array, which can then be projected onto the target area.

[0063] like Figure 4 The figure shows a schematic diagram of a projection replication module provided in this specification for splitting pulsed laser beams. The VCSELs in the laser emission module are arranged in a 2×2 pattern, and the emitted pulsed lasers are injected into the projection replication module. The replication parameters of the replication optical element in the projection replication module are M×N, meaning that the replication optical element can replicate the pulsed laser emitted by each VCSEL into M×N beams, which can be projected onto the target area. The area indicated by reference numeral 1 is the replicated light spot of one VCSEL.

[0064] In this specification, the target area includes the storage carrier and the highly reflective object.

[0065] It should be noted that in one or more embodiments of the present specification, the target area can be predetermined by other optical imaging devices or infrared sensors installed in the AGV and sent to the iToF camera system so that the iToF camera system generates a depth map of the target area.

[0066] In one or more embodiments of the present specification, the projection replication module further includes a collimating lens. The projection replication module can first collimate the pulsed laser light emitted by each vertical cavity surface emitting laser through the collimating lens, and then split each collimated pulsed laser light through the replication optical element to produce a first beam array. The collimating lens is used to collimate the pulsed laser light emitted by the laser emission module to reduce the divergence angle of the pulsed laser light and improve the overall performance and reliability of the iToF camera system.

[0067] In one or more embodiments of the present specification, the replica optical element is a diffractive optical element (DOE), a microlens array, or a spatial light modulator (SLM).

[0068] In one or more embodiments of the present specification, the replicated optical element may also be a metalens, a grating, a nonlinear crystal, etc., and the present specification does not impose any specific limitation.

[0069] Although this specification does not specifically limit the specific replication optical elements used in the projection replication module, since DOE can achieve more efficient multi-beam replication and achieve wide-range light field coverage, and the energy distribution of the replicated beam can be precisely designed with higher uniformity, it can reduce hot spots and light intensity attenuation problems. In addition, by changing the DOE period, duty cycle, and etching depth, the number of replicated beams, spacing, and angles can be flexibly adjusted to meet different application requirements. Therefore, in practical applications, DOE can be used as the replication optical element in the iToF camera system described in this specification.

[0070] In this specification, the pulsed laser emitted by each vertical cavity surface laser emitter is replicated into multiple beams by replicating optical elements, so that the expansion effect of the highly reflective object on the pulsed laser covering the target area is shared by each replicated laser beam, thereby reducing the overall expansion effect of the highly reflective object on the emitted pulsed laser.

[0071] Furthermore, the sensor receiving module can respond to the main control unit's drive and receive the echo signal of the first beam array reflected by the target area through the iToF chip. Based on the echo signal, it can generate a depth map of the target area, so that the AGV can determine the path from the AGV's robotic arm to the storage carrier in the target area based on the depth map.

[0072] It should be noted that the sensor receiving module uses iToF technology to calculate the phase difference based on the echo signal received by the iToF chip, thereby generating a depth map of the target area. Generating depth maps based on echo signals using iToF technology is a relatively mature technology, and the specific process will not be repeated here.

[0073] In one or more embodiments of the present specification, the sensor receiving module further includes a narrowband filter. When the sensor receiving module receives the echo signals of the first light beam array reflected from the target area through the iToF chip in response to the drive of the main control unit, the echo signals of each first light beam array received by the iToF chip are signals filtered by the narrowband filter. The narrowband filter functions to allow the passage of echo signals with the same wavelength as the emitted pulsed laser, thereby allowing the echo signals with the same wavelength as the emitted pulsed laser to be collected by the iToF chip, thereby filtering out ambient light, improving the signal-to-noise ratio of the signal, and improving the accuracy of the generated depth map.

[0074] Usually, AGV is equipped with a decision-making system that is independent of the iToF camera system, such as Figure 5As shown in the figure, a schematic diagram of the structure of an AGV is provided in the present application. When the main control unit determines the path from the AGV's robotic arm to the storage carrier in the target area based on the depth map, the iToF camera system can send the depth map of the target area obtained by the sensor receiving module to the decision system. The decision system can segment the storage carrier and locate the storage carrier based on the depth map, thereby planning the path from the AGV's robotic arm to the storage carrier in combination with the initial position of the AGV's robotic arm and the position of the storage carrier.

[0075] In one or more embodiments of this specification, the execution of the decision, that is, the planning of the path, can be completed by the iToF camera system, that is, the main control unit also has the function of path planning, such as Figure 6 The figure shows a schematic diagram of the structure of an AGV provided in the specification of this application. The sensor receiving module can send a depth map of the target area to the main control unit, and the main control unit can determine the path from the AGV's robotic arm to the storage carrier in the target area based on the depth map.

[0076] Specifically, the main control unit can first generate a point cloud map of the target area based on the depth map, and use target detection technology to locate the position of the storage carrier in the point cloud map, so as to determine the path from the AGV's robotic arm to the storage carrier according to the position of the storage carrier.

[0077] It should be known that when locating the position of the storage carrier based on the depth map, it is not necessary to convert it into a point cloud image, but to directly segment and locate the storage carrier based on the depth map. Of course, other instance segmentation algorithms, edge detection algorithms, etc. can also be used to identify the storage carrier in the depth map or point cloud image and locate its position. This specification does not impose any specific restrictions.

[0078] It should be noted that how the master control unit performs path planning, that is, how to determine the path from the AGV arm to the storage carrier, is a relatively mature technology and is not specifically limited in this specification. For example, the optimal path from the AGV arm to the storage carrier can be planned by using the initial position of the AGV arm and the position of the storage carrier, combined with position calculation, graph search, kinematic solution, and collision avoidance technologies.

[0079] In the aforementioned method, the iToF camera system performs both environmental perception and decision execution, which can improve the integration and automation level of AGV and improve the efficiency of AGV in executing tasks.

[0080] based on Figure 3The iToF camera system shown in the figure is composed of a main control unit, a laser emitting module, a projection replication module, and a sensor receiving module. The laser emitting module includes multiple vertical cavity surface emitting lasers, the projection replication module includes replication optical elements, and the sensor receiving module includes an iToF chip. The iToF camera system with this architecture determines the depth map of the target area, which can reduce the "expansion" degree of the depth image generated by highly reflective objects and improve the accuracy of the generated depth map, thereby improving the accuracy of the path from the AGV's robotic arm to the storage carrier in the target area determined by the AGV based on the depth map, so that the AGV's robotic arm can accurately grasp the storage carrier in the target area. In addition, Figure 3 The iToF camera system shown has low manufacturing cost and strong practicality.

[0081] To reduce the degree of "expansion" in depth images generated by highly reflective objects, the iToF camera system replicates the pulsed laser light emitted by each vertical cavity surface laser emitter through a replicating optical element to obtain a first beam array. This can split a single high-intensity pulsed laser beam into multiple low-intensity beams within the first beam array, reducing the energy density of the beams. Due to the lower energy of the beams, the echo signal reflected by the first array is less affected by the expansion of highly reflective objects than the echo signal reflected by a single high-intensity pulsed laser beam, thereby reducing the possibility of overexposure or expansion effects. Furthermore, based on the foregoing, it can be seen that when the materials of objects in the target area vary, especially when the materials of highly reflective objects vary, multiple beams can more evenly cover the target area than a single beam. This improves the adaptability of the emitted pulsed laser light to surfaces of objects with different materials and reflective properties, thereby enhancing the stability and reliability of the iToF camera system.

[0082] In one or more embodiments of the present specification, the plurality of vertical cavity surface emitting lasers in the laser emission module are arranged in a specified manner. The main control unit can drive the vertical cavity surface emitting lasers in a specified number or at specified positions to emit pulsed lasers.

[0083] In one or more embodiments of the present specification, a plurality of vertical cavity surface emitting lasers in the laser emission module are arranged in a specified manner. In order to further reduce the expansion effect of highly reflective objects, in one or more embodiments of the present specification, the VCSEL can also be activated in rotation, and the VCSEL at the specified position can be activated in rotation. That is, the main control unit can also drive the vertical cavity surface emitting laser at the specified position to emit pulsed laser in sequence according to the preset timing, and the specified position is one position or multiple positions. For example, assuming that the VCSELs are arranged in a 2×2 square array, their positions are represented by top, bottom, left, and right, respectively, and assuming that the preset timing is the first moment, the second moment, and the third moment, then at the first moment, the VCSEL at the top position can be activated, at the second moment, the VCSEL at the bottom and right position can be activated, and at the third moment, the VCSEL at the left position can be activated.

[0084] Therefore, at each moment in the preset time sequence, the projection and replication module can use the replication optical element to split the pulsed laser emitted by the vertical cavity surface emitting laser at the specified position controlled by the main control unit at that moment to obtain a second beam array, and then project the second beam array to the sub-region within the target area. Then, at each moment in the preset time sequence, the sensor receiving module can use the iToF chip to receive the echo signal reflected from the sub-region projected by the second beam array at that moment, and generate a sub-depth map corresponding to that moment, thereby obtaining a depth map of the target area based on the generated sub-depth map corresponding to each moment.

[0085] It should be noted that, since the VCSEL is activated according to the preset timing, the projected area of ​​the pulsed laser emitted by the VCSEL at each moment is only a part of the entire target area, that is, a sub-area. The sub-areas projected by the pulsed laser emitted by the VCSEL at each moment in the entire preset timing together constitute the entire target area. Therefore, after obtaining the echo signal reflected by the sub-target area at each moment in the preset timing, a sub-depth map of the sub-area can be obtained based on the echo signal reflected by the sub-area corresponding to each moment, and the sub-depth maps corresponding to each moment together constitute the depth map of the entire target area. In other words, the sub-depth maps of the sub-areas corresponding to each moment in the preset timing can be spliced ​​to obtain the depth map of the entire target area. Figure 7 As shown, Figure 7 A sub-depth map corresponding to a moment in the preset time sequence provided in this application specification, such as Figure 8 As shown, Figure 8 It is a depth map obtained by stitching together the sub-depth maps corresponding to each moment in the preset time sequence provided in this specification.

[0086] Continuing with the previous example, assuming the VCSELs are arranged in a 2×2 square array, with their positions represented by top, bottom, left, and right, and assuming the time series is the first, second, and third moments, then at the first moment, the top VCSEL is activated, at the second moment, the bottom and right VCSELs are activated, and at the third moment, the left VCSEL is activated. The target area is then divided into three sub-areas. At the first moment, the top VCSEL is replicated to illuminate the first sub-area, thereby obtaining a first sub-depth map for the first sub-area. At the second moment, the pulsed laser light emitted by the bottom and right VCSELs is replicated to illuminate the second sub-area, thereby obtaining a second sub-depth map for the second sub-area. At the third moment, the pulsed laser light emitted by the left VCSEL is replicated to illuminate the third sub-area, thereby obtaining a third sub-depth map for the third sub-area. By concatenating the first, second, and third sub-depth maps, a depth map of the target area can be obtained.

[0087] The aforementioned method of activating the VCSEL at a specified position to emit pulsed laser according to a preset timing can further reduce the expansion effect of highly reflective objects, thereby further improving the accuracy of the generated depth map.

[0088] In one or more embodiments of the present specification, the plurality of vertical cavity surface emitting lasers are arranged in a specified manner, which may be a square array, a linear array, a ring array, etc., and may be pre-designed based on actual needs.

[0089] Furthermore, in practical applications, when fine resolution is required, the projection replication module should tend to generate a larger number of beams. When scanning a large area, it should tend to generate a smaller number of sub-beams distributed over a larger area. This improves scanning efficiency or resolution in different scenarios, meeting the needs of different scenarios.

[0090] Based on the above, it can be known that different situations require different beam splitting quantities of the replication optical element in the projection replication module. Therefore, in one or more embodiments of this specification, the replication optical element may be multiple, and each replication optical element may split the pulse laser beam in a different number of beams. Figure 9 As shown, Figure 9 This is a schematic diagram of an iToF camera system provided in the specification of this application. It can be seen that there are multiple replicating optical elements in the projection replication module in the iToF camera system, and the iToF camera system also includes an element switching module.

[0091] In this specification, the main control unit is communicatively connected to the element switching module. The main control unit can determine a target replica optical element suitable for the target area from among the multiple replica optical elements and send a replica optical element switching instruction to the element switching module. The element switching module can then, in response to the replica optical element switching instruction, switch the replica optical element in the projection replication module to the target replica optical element. The projection replication module can then split the pulsed laser light emitted by each vertical cavity surface emitting laser through the target replica optical element to produce a first beam array.

[0092] In one or more embodiments of the present specification, the main control unit may determine the target replicating optical element based on the required resolution, wherein the resolution is positively correlated with the number of beam splitting of the target replicating optical element.

[0093] In one or more embodiments of the present specification, the main control unit may determine the target replica optical element based on a required scanning rate, wherein the scanning rate is negatively correlated with the number of beam splitting of the target replica optical element.

[0094] It should be noted that the required resolution and the required scanning rate are both determined in advance based on actual business and actual scenarios.

[0095] Furthermore, in practical applications, when the target area contains a large number of obstacles, the total number of pulsed laser beams emitted should be increased to ensure reliability. When the target area contains a small number of obstacles, the total number of pulsed laser beams emitted can be reduced to conserve resources. Furthermore, when the target area is large, meaning that the required field of view of the projection replication module is large, the number of beams can be reduced to increase the coverage area of ​​each beam to improve scanning speed and efficiency.

[0096] Therefore, in one or more embodiments of the present specification, the main control unit may determine the target replication optical element based on the area of ​​the target region, wherein the area of ​​the target region is positively correlated with the number of beam splitting of the target replication optical element.

[0097] In one or more embodiments of the present specification, the main control unit may determine the target replication optical element based on the number of obstacles in the target area, wherein the number of obstacles is positively correlated with the number of beam splitting of the target replication optical element.

[0098] In general, when determining a target replicating optical element suitable for a target area, the main control unit may determine the target replicating optical element suitable for the target area from among a plurality of replicating optical elements based on specified conditions. The specified conditions may include the aforementioned requirements regarding resolution, scanning rate, the area of ​​the target area (i.e., scanning area), and the number of obstacles in the target area.

[0099] In one or more embodiments of the present specification, when the main control unit determines that the area of ​​the target region is greater than a preset area threshold, the diffractive optical element with the smallest beam splitting parameter among the multiple replica optical elements is selected as the target diffractive optical element. When the main control unit determines that the required resolution is greater than a preset resolution threshold, the diffractive optical element with the largest beam splitting parameter is selected as the target diffractive optical element.

[0100] Based on different conditions, different replication optical elements are selected to adjust the number of light beams replicated by the projection replication module in different environments, realize dynamic changes in scanning rate (frame rate) and resolution, improve the applicability of the iToF camera system in different environments, and improve the reliability and intelligence level of the iToF camera system.

[0101] In one or more embodiments of this specification, Figure 10 As shown, Figure 10 This is a schematic diagram of an element switching module provided in this specification. It can be seen that the element switching module may include an electric motor. In the aforementioned step, when the element switching module switches the replicating optical element in the projection replication module to the target replicating optical element in response to the replicating optical element switching instruction, the element switching module may respond to the replicating optical element switching instruction by switching the replicating optical element in the projection replication module to the target replicating optical element via the electric motor.

[0102] like Figure 11 As shown, Figure 11 This is a flow chart of a path planning method provided in this specification. The method is based on Figure 3 The method of the iToF camera system shown in FIG. 1 includes the following steps:

[0103] S1000: Generate a depth map of the target area through the iToF camera system.

[0104] S1002: Locate the position of the storage carrier in the target area according to the depth map.

[0105] S1004: Based on the position of the storage carrier, determine a path for the robotic arm of the AGV to reach the storage carrier.

[0106] It should be noted that the execution subject of the path planning method is an AGV, that is, an AGV equipped with an iToF camera system for suppressing image expansion.

[0107] The specific implementation process of generating the depth map of the target area through the iToF camera system is the same as above. Figure 3 The process of generating a depth map by the iToF system shown is consistent, and the specific process will not be repeated here.

[0108] As previously described, when locating the storage carrier in the target area based on the depth map, the depth map can be first converted into a site cloud map, and then the storage carrier in the target area can be located using object detection technology, edge detection technology, or instance segmentation technology. A more detailed process is not repeated here.

[0109] As mentioned above, when determining the path for the AGV's robotic arm to reach the storage carrier based on the position of the storage carrier, the process can be executed by the AGV's decision-making system. Of course, the process can still be executed by the main control unit in the iToF camera system, and combined with dynamics, graph search and other technologies to perform path planning. The specific process will not be described in detail here.

[0110] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0111] In the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," "connect," "fixed," etc. should be understood broadly. For example, they may refer to fixed connection, detachable connection, or integration; mechanical connection or electrical connection; direct connection or indirect connection through an intermediate medium; and internal communication between two components or interaction between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0112] In the present invention, unless otherwise expressly specified or limited, when a first feature is "above" or "below" a second feature, it may mean that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediate medium. Furthermore, when a first feature is "above," "above," or "above" a second feature, it may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is at a higher level than the second feature. When a first feature is "below," "below," or "below" a second feature, it may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is at a lower level than the second feature.

[0113] In the description of this specification, the terms "one embodiment", "some embodiments", "embodiments", "examples", "specific examples" or "some examples" refer to the specific features, structures, materials or characteristics described in conjunction with the embodiment or example and included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and features of different embodiments or examples, unless they are mutually inconsistent.

[0114] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may alter, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. An iToF camera system for suppressing image expansion, characterized in that: The iToF camera system is installed on the AGV, and the iToF camera system includes: a main control unit, a laser emission module, a projection replication module and a sensor receiving module; the laser emission module includes multiple vertical cavity surface emitting lasers; the projection replication module includes a replication optical element; the sensor receiving module includes an iToF chip; The main control unit is used to drive the laser emitting module and the sensor receiving module; The laser emission module is configured to emit pulsed lasers through the plurality of vertical cavity surface emitting lasers in response to a drive of the main control unit; The projection replication module is configured to split the pulsed laser emitted by each vertical cavity surface emitting laser through the replication optical element to obtain a first light beam array, so as to project the first light beam array to a target area; The sensor receiving module is configured to receive, in response to actuation by the main control unit, an echo signal of the first light beam array reflected from the target area through the iToF chip; and generate a depth map of the target area based on the echo signal of the first light beam array, so that the AGV determines a path from the AGV's robotic arm to the storage carrier in the target area based on the depth map; There are multiple replica optical elements, and each replica optical element splits the pulse laser beam in a different number of beams; The system also includes a component switching module; The main control unit is specifically configured to determine a target replica optical element suitable for a target area among a plurality of replica optical elements, and send a replica optical element switching instruction to the element switching module; The element switching module is specifically configured to switch the replica optical element in the projection replication module to the target replica optical element in response to the replica optical element switching instruction; The projection replication module is specifically configured to split the pulse laser emitted by each vertical cavity surface emitting laser through the target replication optical element to obtain a first light beam array.

2. The iToF camera system according to claim 1, wherein: The projection replication module further includes a collimating lens; the projection replication module is specifically configured to collimate the pulsed laser emitted by each vertical cavity surface emitting laser through the collimating lens, and to split the collimated pulsed lasers through the replication optical element to obtain a first beam array; The sensing receiving module further includes a narrowband filter, and the echo signals of each first light beam array received by the iToF chip are signals filtered by the narrowband filter.

3. The iToF camera system according to claim 1, wherein: The plurality of vertical cavity surface emitting lasers are arranged in a specified manner; The main control unit is specifically used to drive a specified number or a specified position of vertical cavity surface emitting lasers to emit pulsed lasers.

4. The iToF camera system according to claim 1, wherein: The plurality of vertical cavity surface emitting lasers are arranged in a specified manner; The main control unit is specifically used to drive the vertical cavity surface emitting lasers at designated positions to emit pulsed lasers in sequence according to a preset timing, wherein the designated positions are one position or multiple positions; The projection replication module is specifically configured to, at each moment in the preset time sequence, split the pulsed laser emitted by the vertical cavity surface emitting laser at the designated position controlled by the main control unit at that moment through the replication optical element to obtain a second beam array, so as to project the second beam array onto a sub-region in the target region; The sensing receiving module is specifically used to receive, for each moment in the preset time sequence, the echo signal reflected by the sub-area projected by the second light beam array corresponding to the moment through the iToF chip, and generate a sub-depth map corresponding to the moment; based on the generated sub-depth map corresponding to each moment, obtain the depth map of the target area.

5. The iToF camera system according to claim 1, wherein: The main control unit is specifically configured to determine a target replica optical element based on an area of ​​the target area, wherein the area of ​​the target area is positively correlated with the number of beam splitting of the target replica optical element; Alternatively, the target replicating optical element is determined based on the number of obstacles in the target area, and the number of obstacles is positively correlated with the beam splitting number of the target replicating optical element.

6. The iToF camera system according to claim 1, wherein: The element switching module includes an electric motor.

7. The iToF camera system according to claim 1, wherein: The replica optical element is a diffractive optical element, a microlens array or a spatial light modulator.

8. The iToF camera system according to claim 1, wherein: The sensor receiving module is specifically configured to send the depth map of the target area to the main control unit; The main control unit is specifically configured to determine a path from the robotic arm of the AGV to the storage carrier in the target area according to the received depth map of the target area.

9. A path planning method, characterized in that: The path planning method is a method for suppressing image expansion of an iToF camera system based on any one of claims 1 to 8; The method is applied to AGV; the method includes: Generate a depth map of the target area using the iToF camera system; locating a position of the storage carrier in the target area according to the depth map; Based on the position of the storage carrier, a path is determined by the robotic arm of the AGV to the storage carrier in the target area.

Images