Upper bin depth point cloud identification method and device, electronic equipment and medium

Abstract

The invention relates to the technical field of intelligent robots, and provides an upper warehouse depth point cloud identification method and device, electronic equipment and a medium. Comprising the following steps: acquiring a 3D detection frame of a box body relative to a chassis according to the pose of an upper bin in an environment and the pose of the chassis in the environment; calculating each vertex position of the 3D detection frame in the environment; 3D point clouds are obtained according to the vertex positions; and carrying out 3D point cloud target detection by adopting a target detection network, and outputting a 3D detection frame and a type of the upper cabin, namely obtaining the pose of the upper cabin relative to the chassis. According to the method, the loading pose is identified through the depth point cloud, external pasting of a two-dimensional code is not needed, and the method is beautiful and convenient; and meanwhile, the method is not influenced by external environment, such as illumination, and is wide in application range and high in recognition precision.

Description

Technical Field

[0001] The present disclosure relates to the field of intelligent robot technology, and in particular to a method, device, electronic device and medium for identifying depth point cloud of an upper warehouse. Background Art

[0002] A robot refers to an intelligent robot that can automatically plan a driving route to a target location and can automatically move along the driving route. Specifically, the robot also has the function of transporting goods. For example, the robot is provided with an upper warehouse transfer mechanism. When the robot transfers the upper warehouse, the upper warehouse transfer mechanism is controlled to be connected to the upper warehouse so that the upper warehouse moves with the robot to achieve the transfer; and when the upper warehouse is transferred to the target location, the upper warehouse transfer mechanism is controlled to separate from the upper warehouse to complete the transfer of the upper warehouse. The upper warehouse is used to accommodate the goods purchased by users at the merchandise vending cabinet. For robots in which the chassis and the upper warehouse are designed separately, it is particularly important to identify the relative position of the upper warehouse to the chassis.

[0003] The existing technology generally adopts the method of pasting a QR code on the upper warehouse or the deep learning method. The former has too great an impact on the appearance and is not beautiful; the latter's vision is always affected by light, for example, it will be completely ineffective when the lights are turned off. Summary of the invention

[0004] The present disclosure provides a method, device, electronic device and medium for upper warehouse depth point cloud recognition, including: obtaining a 3D detection frame of a box relative to a chassis according to the posture of the upper warehouse in an environment and the posture of a chassis in an environment; calculating the position of each vertex of the 3D detection frame in the environment; obtaining a 3D point cloud according to the position of each vertex; using a target detection network to perform 3D point cloud target detection, and outputting the 3D detection frame and type of the upper warehouse, that is, obtaining the posture of the upper warehouse relative to the chassis. Compared with the prior art, the present disclosure recognizes the posture of the upper warehouse through a deep point cloud, does not require an external QR code to be pasted, is beautiful and convenient; and is not affected by the external environment, such as lighting, has a wide range of applications, and has high recognition accuracy.

[0005] In order to achieve the above objectives, the present invention adopts the following technical solutions:

[0006] The first aspect of the present disclosure provides a method for identifying a depth point cloud of an upper warehouse, comprising:

[0007] According to the position and posture of the upper warehouse in the environment and the position and posture of the chassis in the environment, a 3D detection frame of the box relative to the chassis is obtained.

[0008] Calculate the position of each vertex of the 3D detection box in the environment.

[0009] A 3D point cloud is obtained according to the positions of the vertices.

[0010] The target detection network is used to perform the 3D point cloud target detection, and the 3D detection frame and type of the upper warehouse are output, that is, the position and posture of the upper warehouse relative to the chassis are obtained.

[0011] Furthermore, the upper warehouse depth point cloud recognition method, after obtaining the position and posture of the upper warehouse relative to the chassis, further includes:

[0012] Perform 3D point cloud registration on the warehouse.

[0013] Furthermore, the above-mentioned method for identifying the depth point cloud of the upper warehouse includes:

[0014] The ICP method is used for warehouse 3D point cloud registration.

[0015] Furthermore, the upper warehouse depth point cloud recognition method calculates the positions of each vertex of the 3D detection frame in the environment, including:

[0016] Calculate the position Twc of the depth camera in the environment:

[0017] Twc=Twb×Tbc.

[0018] Wherein, Twb is the position of the chassis in the environment, and Tbc is the position of the depth camera on the chassis.

[0019] Calculate the coordinates vbi of each vertex vi of the 3D detection frame in the environment in the depth camera coordinate system:

[0020] vbi=Twc-1×vi.

[0021] Wherein, vi is the coordinates of each vertex of the 3D detection frame in the environment, and Twc is the pose of the depth camera in the environment.

[0022] Furthermore, in the upper warehouse depth point cloud recognition method, the position of the upper warehouse in the environment includes:

[0023] The position and posture of the upper warehouse in the environment is obtained through a motion capture camera.

[0024] Furthermore, in the upper warehouse depth point cloud recognition method, the position and posture of the chassis in the environment includes:

[0025] The position and posture of the chassis in the environment is obtained through a motion capture camera or chassis positioning.

[0026] Furthermore, in the upper warehouse depth point cloud recognition method, the position and posture of the chassis in the environment includes:

[0027] The position and posture of the chassis in the environment are obtained by chassis positioning.

[0028] Furthermore, in the above-mentioned method for recognizing the depth point cloud of the upper warehouse, the target detection network includes:

[0029] The target detection network is point-net++ or LiDAR R-CNN or OpenPCDet.

[0030] Furthermore, in the above-mentioned method for recognizing the depth point cloud of the upper warehouse, the target detection network includes:

[0031] The target detection network is point-net++.

[0032] Furthermore, the above-mentioned method for recognizing the depth point cloud of the upper warehouse performs 3D point cloud registration, including:

[0033] The 3D point cloud of the upper warehouse in the upper warehouse coordinate system is obtained by 3D modeling.

[0034] A second aspect of the present disclosure provides a device for identifying depth point clouds in an upper warehouse, comprising:

[0035] The first acquisition unit is used to acquire a 3D detection frame of the box relative to the chassis according to the posture of the upper warehouse in the environment and the posture of the chassis in the environment.

[0036] The calculation unit is used to calculate the positions of each vertex of the 3D detection frame in the environment.

[0037] The second acquisition unit is used to acquire a 3D point cloud according to the position of each vertex.

[0038] The target detection unit is used to use the target detection network to perform the 3D point cloud target detection, and output the 3D detection frame and type of the upper warehouse, that is, to obtain the position and posture of the upper warehouse relative to the chassis.

[0039] Furthermore, the upper warehouse depth point cloud recognition device further includes:

[0040] The registration unit is used to perform 3D point cloud registration on the upper warehouse.

[0041] Furthermore, in the upper warehouse depth point cloud recognition device, the computing unit includes:

[0042] The first calculation module is used to calculate the position Twc of the depth camera in the environment:

[0043] Twc=Twb×Tbc.

[0044] Wherein, Twb is the position of the chassis in the environment, and Tbc is the position of the depth camera on the chassis.

[0045] The second calculation module is used to calculate the coordinates vbi of each vertex vi of the 3D detection frame in the environment in the depth camera coordinate system:

[0046] vbi=Twc-1×vi.

[0047] Wherein, vi is the coordinates of each vertex of the 3D detection frame in the environment, and Twc is the pose of the depth camera in the environment.

[0048] A third aspect of the present disclosure provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0049] A fourth aspect of the present disclosure provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the above method.

[0050] The present disclosure provides a method, device, electronic device and medium for upper warehouse depth point cloud recognition, including: obtaining a 3D detection frame of a box relative to a chassis according to the posture of the upper warehouse in an environment and the posture of a chassis in an environment; calculating the position of each vertex of the 3D detection frame in the environment; obtaining a 3D point cloud according to the position of each vertex; using a target detection network to perform 3D point cloud target detection, and outputting the 3D detection frame and type of the upper warehouse, that is, obtaining the posture of the upper warehouse relative to the chassis. Compared with the prior art, the present disclosure recognizes the posture of the upper warehouse through a deep point cloud, does not require an external QR code to be pasted, is beautiful and convenient; and is not affected by the external environment, such as lighting, has a wide range of applications, and has high recognition accuracy.

[0051] In addition, the present invention performs 3D point cloud registration of the upper warehouse, further improving the accuracy of the upper warehouse posture results. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0053] Figure 1 A schematic diagram of a method for identifying a depth point cloud in an upper warehouse in an embodiment of the present disclosure;

[0054] Figure 2 A schematic diagram of a 3D detection frame in an embodiment of the present disclosure;

[0055] Figure 3 It is a flowchart of another method for identifying depth point cloud in a warehouse according to an embodiment of the present disclosure;

[0056] Figure 4This is a schematic diagram of the structure of a depth point cloud recognition device for an upper warehouse in an embodiment of the present disclosure;

[0057] Figure 5 It is a schematic diagram of the structure of another upper warehouse depth point cloud recognition device in an embodiment of the present disclosure;

[0058] Figure 6 It is a schematic diagram of the structure of an upper warehouse depth point cloud recognition electronic device in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0059] The technical solutions in the embodiments of the present disclosure will be described clearly and completely below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0060] Unless otherwise defined, all technical and scientific terms used in the present disclosure have the same meanings as those commonly understood by technicians in the field of the present disclosure; the terms used in the present disclosure are only for the purpose of describing specific embodiments, and are not intended to limit the terms "including" and "having" and any variations thereof in the specification and claims of the present disclosure and the above-mentioned drawings, and are intended to cover non-exclusive inclusions.

[0061] In the description of the embodiments of the present disclosure, the technical terms "first", "second", etc. are only used to distinguish different objects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number, specific order or primary and secondary relationship of the indicated technical features. In the description of the embodiments of the present disclosure, the meaning of "multiple" is more than two, unless otherwise clearly and specifically defined.

[0062] In the description of the embodiments of the present disclosure, the term "and / or" is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0063] In the description of the embodiments of the present disclosure, the term "multiple" refers to more than two (including two). Similarly, "multiple groups" refers to more than two groups (including two groups), and "multiple pieces" refers to more than two pieces (including two pieces).

[0064] In the description of the embodiments of the present disclosure, the orientations or positional relationships indicated by technical terms such as "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", and "circumferential" are based on the orientations or positional relationships shown in the accompanying drawings and are only for the convenience of describing the embodiments of the present disclosure and simplifying the description. They do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the embodiments of the present disclosure.

[0065] In the description of the embodiments of the present disclosure, unless otherwise clearly specified and limited, technical terms such as "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, and it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the embodiments of the present disclosure can be understood according to the specific circumstances.

[0066] Example 1

[0067] The present disclosure provides a method for identifying depth point clouds in a warehouse. Figure 1 As shown, including:

[0068] 101. According to the position of the upper warehouse in the environment and the position of the chassis in the environment, a 3D detection frame of the box relative to the chassis is obtained.

[0069] Robot: refers to an intelligent robot that can automatically plan a route to the target location and move automatically along the route.

[0070] Upper warehouse: It is used to store the goods that users purchase at the merchandise vending counter.

[0071] Chassis: A robot is a combination of multiple technologies and functions. In addition to software functions, other important parts are concentrated on the robot chassis hardware, which carries multiple functions of the robot such as positioning, navigation and obstacle avoidance.

[0072] 102. Calculate the positions of each vertex of the 3D detection frame in the environment.

[0073] like Figure 2 As shown, it is a 3D detection box, whose vertices are v1, v2, v3, v4, v5, v6, v7, and v8. The coordinates of each vertex in the environment can be obtained by calculation. Please refer to other corresponding embodiments for details.

[0074] 103. Obtain a 3D point cloud according to the positions of the vertices.

[0075] Point clouds: Point clouds refer to a set of discrete sampling points obtained by a 3D scanner. They usually contain the 3D coordinates, normals, and surface appearance properties of the sampling points. Point clouds can be used to represent the outer surface of a continuous 3D model. The surface composed of point clouds is called a point set surface, also known as a point model. After triangular meshes were used as a way to express geometric models, point clouds have become a new way to express geometric models.

[0076] 104. Use a target detection network to perform the 3D point cloud target detection, and output the 3D detection frame and type of the upper warehouse, that is, obtain the position of the upper warehouse relative to the chassis.

[0077] Target detection network: In this embodiment, the target detection network can use point-net++ or LiDAR R-CNN or OpenPCDet. In this embodiment, point-net++ is selected as the target detection network. In other embodiments, the implementer may also select other target detection networks.

[0078] An embodiment of the present disclosure provides a method for deep point cloud recognition of an upper warehouse, including: obtaining a 3D detection frame of a box relative to a chassis according to the posture of the upper warehouse in an environment and the posture of a chassis in an environment; calculating the position of each vertex of the 3D detection frame in the environment; obtaining a 3D point cloud according to the position of each vertex; using a target detection network to perform 3D point cloud target detection, and outputting the 3D detection frame and type of the upper warehouse, that is, obtaining the posture of the upper warehouse relative to the chassis. Compared with the prior art, the present disclosure recognizes the posture of the upper warehouse through a deep point cloud, without the need to externally paste a QR code, and is beautiful and convenient; at the same time, it is not affected by the external environment, such as: lighting, and has a wide range of applications and high recognition accuracy.

[0079] Example 2

[0080] The present disclosure provides a method for identifying depth point clouds in a warehouse. Figure 3 As shown, including:

[0081] 201. According to the position and posture of the upper warehouse in the environment and the position and posture of the chassis in the environment, obtain a 3D detection frame of the box relative to the chassis.

[0082] Among them, the position and posture of the upper warehouse in the environment can be obtained by a motion capture camera, and the position and posture of the chassis in the environment can be obtained by a motion capture camera or chassis positioning. In this embodiment, a motion capture camera is used to obtain the position and posture of the upper warehouse in the environment; chassis positioning obtains the position and posture of the chassis in the environment. However, the protection scope of this embodiment is not limited to this. In other embodiments, the implementer can determine the acquisition method according to the actual situation or its own needs.

[0083] 202. Calculate the position of each vertex of the 3D detection frame in the environment.

[0084] 2021. Calculate the position Twc of the depth camera in the environment:

[0085] Twc=Twb×Tbc.

[0086] Wherein, Twb is the position of the chassis in the environment, and Tbc is the position of the depth camera on the chassis.

[0087] 2022. Calculate the coordinates vbi of each vertex vi of the 3D detection frame in the environment in the depth camera coordinate system:

[0088] vbi=Twc-1×vi.

[0089] Wherein, vi is the coordinates of each vertex of the 3D detection frame in the environment, and Twc is the pose of the depth camera in the environment.

[0090] 203. Acquire a 3D point cloud according to the positions of the vertices.

[0091] 204. Use a target detection network to perform the 3D point cloud target detection, and output the 3D detection frame and type of the upper warehouse, that is, obtain the position of the upper warehouse relative to the chassis.

[0092] 205. Perform 3D point cloud registration on the warehouse.

[0093] This embodiment uses the ICP method to perform upper warehouse 3D point cloud registration to improve the accuracy of the acquired upper warehouse relative to the chassis, wherein the 3D point cloud of the upper warehouse in the upper warehouse coordinate system is obtained by 3D modeling.

[0094] Among them, point cloud registration refers to inputting two point clouds (source) and (target), and outputting a transformation to make the degree of overlap as high as possible. The transformation can be rigid (rigid) or not. This article only considers rigid transformation, that is, the transformation only includes rotation and translation. Point cloud registration can be divided into two steps: coarse registration (CoarseRegistration) and fine registration (Fine Registration). Coarse registration refers to a relatively rough registration when the transformation between the two point clouds is completely unknown. The main purpose is to provide a better initial value of the transformation for fine registration; the fine registration criterion is to give an initial transformation and further optimize it to obtain a more accurate transformation. The most widely used point cloud fine registration algorithm is the iterative closest point algorithm (Iterative Closest Point, ICP) and various variants of the ICP algorithm.

[0095] The present disclosure provides a method for upper warehouse depth point cloud recognition, including: obtaining a 3D detection frame of a box relative to a chassis according to the posture of the upper warehouse in an environment and the posture of a chassis in an environment; calculating the position of each vertex of the 3D detection frame in the environment; obtaining a 3D point cloud according to the position of each vertex; using a target detection network to perform 3D point cloud target detection, and outputting the 3D detection frame and type of the upper warehouse, that is, obtaining the posture of the upper warehouse relative to the chassis. Compared with the prior art, the present disclosure recognizes the posture of the upper warehouse through a deep point cloud, does not need to externally paste a QR code, is beautiful and convenient; and is not affected by the external environment, such as: light, has a wide range of applications, and has high recognition accuracy.

[0096] In addition, the present invention performs 3D point cloud registration of the upper warehouse, further improving the accuracy of the upper warehouse posture results.

[0097] Example 3

[0098] The present disclosure provides a device for identifying depth point cloud in a warehouse. Figure 4 As shown, including:

[0099] The first acquisition unit 31 is used to acquire a 3D detection frame of the box relative to the chassis according to the posture of the upper warehouse in the environment and the posture of the chassis in the environment.

[0100] The calculation unit 32 is used to calculate the positions of each vertex of the 3D detection frame in the environment.

[0101] The second acquisition unit 33 is used to acquire a 3D point cloud according to the position of each vertex.

[0102] The target detection unit 34 is used to use a target detection network to perform the 3D point cloud target detection and output the 3D detection frame and type of the upper warehouse, that is, to obtain the position and posture of the upper warehouse relative to the chassis.

[0103] It should be noted here that the detailed description of each component of this embodiment can refer to other embodiments and will not be repeated here.

[0104] An embodiment of the present disclosure provides a depth point cloud recognition device for an upper warehouse, including: a first acquisition unit acquires a 3D detection frame of a box relative to a chassis according to the posture of the upper warehouse in an environment and the posture of a chassis in an environment; a calculation unit calculates the position of each vertex of the 3D detection frame in the environment; a second acquisition unit acquires a 3D point cloud according to the position of each vertex; a target detection unit uses a target detection network to perform 3D point cloud target detection, and outputs the 3D detection frame and type of the upper warehouse, that is, acquires the posture of the upper warehouse relative to the chassis. Compared with the prior art, the present disclosure recognizes the posture of the upper warehouse through a deep point cloud, does not need to externally paste a QR code, is beautiful and convenient; and is not affected by the external environment, such as light, has a wide range of applications, and has high recognition accuracy.

[0105] Example 4

[0106] The present disclosure provides a device for identifying depth point cloud in a warehouse. Figure 5 As shown, including:

[0107] The first acquisition unit 41 is used to acquire a 3D detection frame of the box relative to the chassis according to the posture of the upper warehouse in the environment and the posture of the chassis in the environment.

[0108] The calculation unit 42 is used to calculate the positions of each vertex of the 3D detection frame in the environment.

[0109] The first calculation module 421 is used to calculate the position Twc of the depth camera in the environment:

[0110] Twc=Twb×Tbc.

[0111] Wherein, Twb is the position of the chassis in the environment, and Tbc is the position of the depth camera on the chassis.

[0112] The second calculation module 422 is used to calculate the coordinates vbi of each vertex vi of the 3D detection frame in the environment in the depth camera coordinate system:

[0113] vbi=Twc-1×vi.

[0114] Wherein, vi is the coordinates of each vertex of the 3D detection frame in the environment, and Twc is the pose of the depth camera in the environment.

[0115] A second acquisition unit 43, configured to acquire a 3D point cloud according to each vertex position;

[0116] The target detection unit 44 is used to use a target detection network to perform the 3D point cloud target detection and output the 3D detection frame and type of the upper warehouse, that is, to obtain the position and posture of the upper warehouse relative to the chassis.

[0117] The registration unit 45 is used to perform 3D point cloud registration of the upper warehouse.

[0118] It should be noted here that the detailed description of each component of this embodiment can refer to other embodiments and will not be repeated here.

[0119] An embodiment of the present disclosure provides a depth point cloud recognition device for an upper warehouse, including: a first acquisition unit acquires a 3D detection frame of a box relative to a chassis according to the posture of the upper warehouse in an environment and the posture of a chassis in an environment; a calculation unit calculates the position of each vertex of the 3D detection frame in the environment; a second acquisition unit acquires a 3D point cloud according to the position of each vertex; a target detection unit uses a target detection network to perform 3D point cloud target detection, and outputs the 3D detection frame and type of the upper warehouse, that is, acquires the posture of the upper warehouse relative to the chassis. Compared with the prior art, the present disclosure recognizes the posture of the upper warehouse through a deep point cloud, does not need to externally paste a QR code, is beautiful and convenient; and is not affected by the external environment, such as light, has a wide range of applications, and has high recognition accuracy.

[0120] In addition, the registration unit of the embodiment of the present disclosure performs 3D point cloud registration of the upper warehouse, further improving the accuracy of the upper warehouse posture results.

[0121] Example 5

[0122] The present disclosure provides an electronic device for deep point cloud recognition in a warehouse, such as Figure 6 As shown, it includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above-mentioned upper warehouse depth point cloud recognition method are implemented.

[0123] A processing device (e.g., a central processing unit, a graphics processing unit, etc.) 51 can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 52 or a program loaded from a storage device 58 into a random access memory (RAM) 53. Various programs and data required for the operation of the electronic device are also stored in the RAM 53. The processing device 51, the ROM 52, and the RAM 53 are connected to each other via a bus 54. An input / output (I / O) interface 55 is also connected to the bus 54.

[0124] Typically, the following devices may be connected to the I / O interface 55: an input device 56 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 57 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 58 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 59. The communication device 59 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Figure 6 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 6 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0125] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 59, or installed from the storage device 58, or installed from the ROM 52. When the computer program is executed by the processing device 51, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0126] The present disclosure provides an upper warehouse depth point cloud recognition electronic device, comprising: obtaining a 3D detection frame of a box relative to a chassis according to the posture of the upper warehouse in an environment and the posture of a chassis in an environment; calculating the position of each vertex of the 3D detection frame in the environment; obtaining a 3D point cloud according to the position of each vertex; using a target detection network to perform 3D point cloud target detection, and outputting the 3D detection frame and type of the upper warehouse, that is, obtaining the posture of the upper warehouse relative to the chassis. Compared with the prior art, the present disclosure recognizes the posture of the upper warehouse through a deep point cloud, does not need to externally paste a QR code, is beautiful and convenient; and is not affected by the external environment, such as: light, has a wide range of applications, and has high recognition accuracy.

[0127] In addition, the present invention performs 3D point cloud registration of the upper warehouse, further improving the accuracy of the upper warehouse posture results.

[0128] Example 6

[0129] An embodiment of the present disclosure provides a computer-readable storage medium for upper warehouse depth point cloud recognition, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by one or more processors, the above-mentioned upper warehouse depth point cloud recognition method is implemented.

[0130] It should be noted that the computer-readable medium in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0131] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0132] The computer-readable medium may be included in the device; or it may exist independently without being installed in the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains the 3D detection frame of the box relative to the chassis according to the position of the upper warehouse in the environment and the position of the chassis in the environment; calculates the position of each vertex of the 3D detection frame in the environment; obtains the 3D point cloud according to the position of each vertex; uses the target detection network to perform 3D point cloud target detection, and outputs the 3D detection frame and type of the upper warehouse, that is, obtains the position of the upper warehouse relative to the chassis.

[0133] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0134] The present disclosure provides a deep point cloud recognition medium for an upper warehouse, including: obtaining a 3D detection frame of a box relative to a chassis according to the posture of the upper warehouse in an environment and the posture of a chassis in an environment; calculating the positions of each vertex of the 3D detection frame in the environment; obtaining a 3D point cloud according to the positions of each vertex; using a target detection network to perform 3D point cloud target detection, and outputting the 3D detection frame and type of the upper warehouse, that is, obtaining the posture of the upper warehouse relative to the chassis. Compared with the prior art, the present disclosure recognizes the posture of the upper warehouse through a deep point cloud, does not require an external QR code to be pasted, is beautiful and convenient; and is not affected by the external environment, such as light, has a wide range of applications, and has high recognition accuracy.

[0135] In addition, the present invention performs 3D point cloud registration of the upper warehouse, further improving the accuracy of the upper warehouse posture results.

[0136] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0137] The units described in some embodiments of the present disclosure may be implemented by software or hardware. The units described may also be set in a processor, for example, it may be described as: a processor includes a receiving unit, an information acquisition unit, a target determination unit, and a distribution unit. The names of these units do not constitute limitations on the units themselves in some cases.

[0138] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0139] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

[0140] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, not to limit them; although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention. In particular, as long as there is no structural conflict, the various technical features mentioned in each embodiment can be combined in any way. The present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions that fall within the scope of the claims.

Claims

1. A method for identifying depth point cloud of an upper warehouse, characterized in that: include: According to the position of the upper warehouse and the position of the chassis in the environment, a 3D detection frame of the box relative to the chassis is obtained; Calculate the position of each vertex of the 3D detection frame in the environment; Acquire a 3D point cloud according to the positions of each vertex; The target detection network is used to perform the 3D point cloud target detection, and the 3D detection frame and type of the upper warehouse are output, that is, the position and posture of the upper warehouse relative to the chassis are obtained.

2. The method for identifying the depth point cloud of an upper warehouse according to claim 1, characterized in that: After obtaining the position of the upper compartment relative to the chassis, it also includes: Perform 3D point cloud registration on the warehouse.

3. The method for identifying the depth point cloud of an upper warehouse according to claim 2, characterized in that: include: The ICP method is used for warehouse 3D point cloud registration.

4. The method for identifying the depth point cloud of an upper warehouse according to claim 1, characterized in that: Calculating the positions of each vertex of the 3D detection frame in the environment includes: Calculate the position Twc of the depth camera in the environment: Twc=Twb×Tbc. Wherein, Twb is the position of the chassis in the environment, and Tbc is the position of the depth camera on the chassis; Calculate the coordinates vbi of each vertex vi of the 3D detection frame in the environment in the depth camera coordinate system: vbi=Twc-1×vi. Wherein, vi is the coordinates of each vertex of the 3D detection frame in the environment, and Twc is the pose of the depth camera in the environment.

5. The method for identifying the depth point cloud of an upper warehouse according to claim 1, characterized in that: The position of the upper warehouse in the environment includes: The position and posture of the upper warehouse in the environment is obtained through a motion capture camera.

6. The method for identifying the depth point cloud of an upper warehouse according to claim 1, characterized in that: The position and posture of the chassis in the environment include: The position and posture of the chassis in the environment is obtained through a motion capture camera or chassis positioning.

7. The method for identifying the depth point cloud of an upper warehouse according to claim 6, characterized in that: The position and posture of the chassis in the environment include: The position and posture of the chassis in the environment are obtained by chassis positioning.

8. The method for identifying the depth point cloud of an upper warehouse according to claim 1, characterized in that: The target detection network includes: The target detection network is point-net++ or LiDARR-CNN or OpenPCDet.

9. The method for identifying the depth point cloud of an upper warehouse according to claim 8, characterized in that: The target detection network includes: The target detection network is point-net++.

10. The method for identifying the depth point cloud of an upper warehouse according to claim 2, characterized in that: Perform 3D point cloud registration, including: The 3D point cloud of the upper warehouse in the upper warehouse coordinate system is obtained by 3D modeling.

11. A device for identifying depth point cloud in an upper warehouse, characterized in that: include: A first acquisition unit is used to acquire a 3D detection frame of the box relative to the chassis according to the posture of the upper warehouse in the environment and the posture of the chassis in the environment; A calculation unit, used to calculate the position of each vertex of the 3D detection frame in the environment; A second acquisition unit, used for acquiring a 3D point cloud according to the position of each vertex; The target detection unit is used to use the target detection network to perform the 3D point cloud target detection, and output the 3D detection frame and type of the upper warehouse, that is, to obtain the position and posture of the upper warehouse relative to the chassis.

12. The upper warehouse depth point cloud recognition device according to claim 11, characterized in that: Also includes: The registration unit is used to perform 3D point cloud registration on the upper warehouse.

13. The upper warehouse depth point cloud recognition device according to claim 11, characterized in that: The computing unit comprises: The first calculation module is used to calculate the position Twc of the depth camera in the environment: Twc=Twb×Tbc. Wherein, Twb is the position of the chassis in the environment, and Tbc is the position of the depth camera on the chassis; The second calculation module is used to calculate the coordinates vbi of each vertex vi of the 3D detection frame in the environment in the depth camera coordinate system: vbi=Twc-1×vi. Wherein, vi is the coordinates of each vertex of the 3D detection frame in the environment, and Twc is the pose of the depth camera in the environment.

14. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 10 are implemented.

15. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

Images