Warehousing system, workstation, and method for detecting overhang of target articles

Abstract

Provided in the embodiments of the present disclosure are a warehousing system, a workstation, and a method for detecting the overhang of target articles. The warehousing system comprises: a reference mark; a movable carrier configured to carry articles; a handling apparatus configured to transport the movable carrier so as to transport same to a target area; and a detection mechanism configured to determine, on the basis of the reference mark, whether the articles on the movable carrier overhang the movable carrier, or whether there is a deviation in the pose of the movable carrier.

Description

Warehouse systems, workstations, and methods for detecting protruding objects

[0001] This disclosure claims priority to Chinese Patent Application No. 202520735737.1, filed on April 17, 2025; and priority to Chinese Patent Application No. 202510679629.1, filed on May 23, 2025; the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure belongs to the field of warehousing and logistics equipment technology, and in particular relates to warehousing systems, workstations and methods for detecting protruding target objects. Background Technology

[0003] In warehousing and logistics, to improve work efficiency, save labor, and enhance safety, warehousing systems can include mobile carriers and handling equipment. Mobile carriers can buffer items in batches, and handling equipment can move these mobile carriers to improve the efficiency of a single item transfer. For example, when items need to be picked, the mobile carriers containing the items to be picked can be moved to a workstation by handling equipment. The workstation's loading and unloading equipment (such as robotic arms or robotic hands) can then remove the items from the mobile carriers for picking. Summary of the Invention

[0004] The first aspect of this disclosure provides a warehousing system, including:

[0005] Reference markers;

[0006] A mobile vehicle configured to carry items;

[0007] The handling equipment is configured to handle mobile vehicles to move them to a target area.

[0008] The testing agency is configured to determine, based on reference markers, whether an item on a movable vehicle protrudes from the movable vehicle, or to determine whether there is a deviation in the pose of the movable vehicle.

[0009] In a second aspect of this disclosure, a method for detecting protruding objects is provided, applied to the warehousing system described in the first aspect of this disclosure. The method for detecting protruding objects includes:

[0010] Acquire at least one target image; wherein the at least one target image is acquired based on at least one acquisition direction, and the target image includes a target object and a reference object; detect the target object and the reference object in the at least one target image using a target detection model to obtain the detection box corresponding to the target object and the detection box corresponding to the reference object; determine the prominence information of the target object relative to the reference object based on the positional relationship between the detection box corresponding to the target object and the detection box corresponding to the reference object.

[0011] A third aspect of this disclosure provides a workstation, including:

[0012] Reference markers;

[0013] The loading and unloading equipment has a docking area on one side. The loading and unloading equipment is configured to dock with a mobile vehicle located in the docking area to transfer goods between the mobile vehicle and the loading and unloading equipment.

[0014] The inspection agency is configured to determine, based on reference markers, whether an item on a movable vehicle protrudes from the movable vehicle, or whether the movable vehicle is tilted. Attached Figure Description

[0015] The accompanying drawings, which are included to provide a further understanding of this disclosure and form part of this disclosure, illustrate exemplary embodiments of the present disclosure and are used to explain the disclosure, but do not constitute an undue limitation of the disclosure. In the drawings:

[0016] Figure 1A is a structural schematic diagram of a warehousing system provided in an embodiment of the present disclosure from a first-view perspective;

[0017] Figure 1B is a schematic diagram of a warehousing system provided in an embodiment of this disclosure;

[0018] Figure 1C is a schematic diagram of the structure of a workstation provided in an embodiment of this disclosure;

[0019] Figure 2 is a structural schematic diagram of a warehousing system provided in an embodiment of the present disclosure from a second perspective.

[0020] Figure 3 is a structural schematic diagram of a warehousing system provided in an embodiment of the present disclosure from a third-person perspective;

[0021] Figure 4 is a schematic diagram of the movement path of a mobile vehicle moving towards the docking area according to an embodiment of this disclosure;

[0022] Figure 5 is a schematic diagram of the movement path of a mobile vehicle moving towards the docking area according to an embodiment of this disclosure;

[0023] Figure 6 is a schematic diagram of the movement path of a mobile vehicle moving towards the docking area according to an embodiment of this disclosure;

[0024] Figure 7 is a schematic diagram of the movement path of a mobile vehicle moving towards the docking area according to an embodiment of this disclosure;

[0025] Figure 8 is a schematic diagram of the structure of a detection mechanism and a reference mark provided in one embodiment of this disclosure;

[0026] Figure 9 is a schematic diagram of the structure of one of the detection mechanisms and reference marks provided in an embodiment of this disclosure;

[0027] Figure 10 is a schematic diagram of an application scenario of a target object protrusion detection method provided in an embodiment of this disclosure;

[0028] Figure 11 is a schematic flowchart of a target object protrusion detection method provided in an embodiment of this disclosure;

[0029] Figure 12 is a flowchart illustrating another target protrusion detection method provided in an embodiment of this disclosure;

[0030] Figure 13 is a schematic flowchart of another target object protrusion detection method provided in an embodiment of this disclosure;

[0031] Figure 14 is a schematic flowchart of another target object protrusion detection method provided in an embodiment of this disclosure;

[0032] Figure 15A is a schematic diagram of the positional relationship between a first detection frame, a second detection frame, and a third detection frame provided in an embodiment of this disclosure;

[0033] Figure 15B is a schematic diagram showing the positional relationship between a first detection frame, a second detection frame, and a third detection frame according to another embodiment of the present disclosure.

[0034] Figure 15C is a schematic diagram of a first target image and a second target image provided in an embodiment of this disclosure;

[0035] Figure 16 is a flowchart illustrating a target detection model training method provided in an embodiment of this disclosure;

[0036] Figure 17 is a flowchart illustrating another object detection model training method provided in an embodiment of this disclosure;

[0037] Figure 18a is a schematic diagram of another detection mechanism and reference mark provided in an embodiment of this disclosure;

[0038] Figure 18b is a schematic diagram of another detection mechanism and reference mark provided in an embodiment of this disclosure;

[0039] Figure 19 is a schematic diagram of another detection mechanism and reference mark provided in an embodiment of the present disclosure;

[0040] Figure 20 is a schematic diagram of the detection mechanism and positioning mechanism provided in an embodiment of this disclosure;

[0041] Figure 21 is a schematic diagram of the detection mechanism and positioning mechanism provided in one embodiment of this disclosure;

[0042] Figure 22 is a schematic diagram of another warehousing system provided in an embodiment of the present disclosure;

[0043] Figure 23 is a schematic diagram of a target protrusion detection device provided in an embodiment of this disclosure;

[0044] Figure 24 is a schematic diagram of the structure of a target detection model training device provided in an embodiment of this disclosure;

[0045] Figure 25 is a schematic diagram of the internal structure of a robot provided in an embodiment of this disclosure;

[0046] Figure 26 is a schematic diagram of the structure of a control device provided in an embodiment of this disclosure.

[0047] Explanation of reference numerals in the attached drawings: 10-Workstation; 20-Mobile vehicle; 30-Handling equipment; 40-Dock docking equipment; 50-Running surface; 60-Limiting mechanism; 70-Target area; 80-Vehicle; 100-Warehouse system; 11-Loading and unloading equipment; 12-Connecting area; 13-Entry area; 14-Detection mechanism; 15-Reference mark; 16-Work platform; 17-Positioning mechanism; 18-Mounting frame; 19-Positioning mark; 21-Cargo location; 21a-Cargo location opening; 111-Pick-and-place mechanism; 112-Longitudinal beam; 113-Crossbeam; 114-Support frame; 115-Guiding mechanism; 116-Horizontal moving device; 117-Vertical moving device; 141-Camera; 142-LiDAR; 143-Signal transmitting component; 151-Reference line; 152-Scanning surface; 153-Signal receiving component; 191-Visual mark; 192-Positioning surface; A-Working plane; B-Dating surface; C-Target plane. Detailed Implementation

[0048] To enable those skilled in the art to better understand the technical solutions in this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this disclosure.

[0049] It should be noted that many specific details are set forth in the following description in order to provide a full understanding of this disclosure. However, this disclosure may also be implemented in other ways different from those described herein. Therefore, the scope of protection of this disclosure is not limited to the specific embodiments disclosed below.

[0050] With the rapid development of e-commerce, it plays an increasingly important role in consumers' lives. In e-commerce, the storage and handling of goods plays a crucial role. To facilitate the storage and handling of goods, they are usually stored and handled in warehousing systems.

[0051] Figure 1A is a structural schematic diagram of a warehousing system provided in an embodiment of this disclosure from a first perspective; Figure 2 is a structural schematic diagram of a warehousing system provided in an embodiment of this disclosure from a second perspective; Figure 3 is a structural schematic diagram of a warehousing system provided in an embodiment of this disclosure from a third perspective; and Figure 1B is a schematic diagram of a warehousing system provided in an embodiment of this disclosure. Referring to Figures 1A to 3, in order to improve the efficiency of handling and picking goods, the warehousing system of this disclosure includes a movable carrier 20 and a handling device 30. The movable carrier 20 is configured to carry items. For example, the movable carrier 20 can be a movable shelf, and the movable carrier 20 includes one or more storage locations 21, which are configured to carry items. For example, the movable carrier 20 includes one or more rows of storage locations 21 arranged along the length direction, and each row of storage locations 21 includes one or more layers of storage locations 21 arranged along the height direction.

[0052] It should be noted that, referring to Figure 1A, the length direction of the movable vehicle 20 can be referenced to the direction shown in x, the width direction can be referenced to the direction shown in y, and the height direction can be referenced to the direction shown in z.

[0053] The movable carrier 20 can be provided with a cargo space 21 along the width direction, that is, the movable carrier 20 is a single-row cargo space 21. The cargo space opening 21a of the single-row cargo space 21 is located on one side of the movable carrier 20 and is connected to the cargo space 21 so as to allow items to enter and exit the movable carrier 20.

[0054] The carrier 80 (which may include a movable carrier 20) may include multiple storage locations (also referred to as storage locations or placement locations). Each placement location can be configured to hold a target object (also referred to as an item), which may be a container, a cargo box, a material bin, goods, or the original packaging of goods. This disclosure does not limit the scope of the embodiments; the following embodiments use a cargo box as an example to illustrate the concept.

[0055] For example, the cargo space can be a rectangular prism-shaped storage space, and multiple cargo spaces on the vehicle 80 can be neatly arranged along the length, width, and height of the vehicle 80. The cargo box can be a specially designed accessory for the vehicle 80, a regular cargo box (also called a material box), or packaging for goods (also called the original box); this disclosure does not limit the specific type of cargo box.

[0056] The movable carrier 20 can be provided with two or more storage locations 21 along the width direction. The two or more storage locations 21 can share a storage location opening 21a. That is, a storage location opening 21a is provided on one side of the movable carrier 20. The storage location opening 21a is used for the entry and exit of items on multiple storage locations 21 along the width direction. In other words, the movable carrier 20 is a multi-depth movable carrier 20.

[0057] The movable carrier 20 can be provided with two cargo positions 21 along the width direction. Each cargo position 21 has its own cargo position opening 21a. That is, cargo position openings 21a are provided on both sides of the movable carrier 20 along the width direction. One cargo position opening 21a is connected to one of the two cargo positions 21 to allow items to enter and exit the cargo position 21. The other cargo position opening 21a is connected to the other cargo position 21 to allow items to enter and exit the cargo position 21. Thus, the movable carrier 20 is a back-to-back single-depth carrier.

[0058] In some examples, the movable vehicle 20 has cargo openings 21a on both sides, and the movable vehicle 20 is provided with two or more cargo positions 21 communicating with each cargo opening 21a, so that the movable vehicle 20 is a back-to-back multi-deep vehicle.

[0059] By setting multiple storage locations 21 on the mobile carrier 20, the storage density of the mobile carrier 20 is increased, the number of items handled in a single operation is increased, and the handling efficiency of the warehousing system is improved.

[0060] It should be noted that items may include goods, containers containing goods such as bins, original boxes containing goods, parcels containing goods, single containers, etc. There are no restrictions on the type of items here.

[0061] The vehicle 80 can be a shelf, pallet, turnover cart, cage cart, etc.

[0062] For example, the carrier 80 can be a movable shelf. The movable shelf includes at least one partition that divides the carrier 80 into at least two layers. Each partition of the carrier 80 has at least one storage location, and each storage location can accommodate at least one cargo box. It should be noted that the carrier 80 includes, but is not limited to, partitioned shelves, fixed shelves, picking shelves, and movable shelves. The carrier 80 provided in this embodiment can refer to any carrier configured to hold cargo boxes. The carrier 80 can be a movable carrier 20.

[0063] For example, the carrier 80 can be either a high-density storage rack or a low-density storage rack. When the carrier 80 is a high-density storage rack, the gaps between the cargo boxes are smaller.

[0064] For example, the carrier 80 can be a single-sided carrier, a double-sided carrier, or a four-sided carrier (i.e., the goods can be retrieved from four sides). This disclosure does not limit this; however, it uses a double-sided carrier as an example for illustrative purposes. For instance, when the carrier 80 is a double-sided shelf, partitions can be provided between multiple storage locations along the vertical direction; each side of the carrier 80 can have 30 storage locations, for a total of 60 storage locations on both sides. This disclosure does not limit the type of carrier 80 or the number of storage locations it includes; this is merely an illustrative example.

[0065] The handling equipment 30 is configured to handle the mobile carrier 20 to change the position of the mobile carrier 20 within the warehousing system, for example, to move the mobile carrier 20 to the target area 70 of the warehousing system.

[0066] The warehousing system may also include a docking device 40, which docks with a mobile carrier 20 to transfer goods between the docking device 40 and the mobile carrier 20. The target area 70 includes a connection area 12 on one side of the docking device 40. A handling device 30 can move the mobile carrier 20 from other areas of the warehousing system to the connection area 12 of the docking device 40, enabling the docking device 40 to dock with the mobile carrier 20 and thus transfer goods between the docking device 40 and the mobile carrier 20.

[0067] It is understandable that the mobile vehicle 20 docks with the docking device 40 in the docking area 12. In other words, the docking device 40 can only successfully dock with the cargo position 21 on the mobile vehicle 20 when the mobile vehicle 20 is in the docking area 12, so as to realize the transfer of goods.

[0068] The handling equipment 30 may also be called an automated handling equipment, a carrier handling equipment, or a handling robot. The handling equipment 30 is configured as a handling carrier 80.

[0069] For example, the handling equipment 30 can be a handling robot, such as an Automated Guided Vehicle (AGV). In some application scenarios, the handling robot can move under the carrier 80 and then lift the carrier 80 off the ground, thereby carrying the carrier 80 to move.

[0070] The handling equipment 30 can be an Automated Guided Vehicle (AVG), such as a lifting handling robot. The lifting handling robot can enter the bottom of the mobile carrier 20 and lift the mobile carrier 20 by raising the supporting structure of the lifting handling robot, so that the mobile carrier 20 is removed from the operating surface of the storage system (such as the ground or the raised platform). Then the lifting robot drives the mobile carrier 20 to move along the ground to dock with the docking equipment 40.

[0071] In other examples, the handling equipment 30 may also be a forklift, etc. The mobile carrier 20 can be moved by the forklift. The present disclosure does not limit the structural type of the handling equipment 30, as long as it ensures that the mobile carrier 20 can be moved smoothly to the docking area 12 of the docking equipment 40.

[0072] The handling equipment 30 can also be a conveyor line, which moves the movable carrier 20.

[0073] The warehousing system may include workstation 10, and docking equipment 40 may be loading and unloading equipment 11 in workstation 10. Loading and unloading equipment 11 may remove items from mobile carrier 20 or place items in mobile carrier 20.

[0074] The number of workstations 10 in the warehousing system 100 can be one or more. Each workstation 10 can be equipped with a corresponding vehicle docking area, which includes the target area 70 in the warehousing system. The vehicle docking area includes one or more docking positions, which are configured to dock vehicles 80.

[0075] The handling equipment 30 can move the carriers 80 from the inventory area (also known as the storage area) in the warehousing system to the docking position of the corresponding carrier docking area of ​​the workstation 10, so that the workstation 10 can perform pick-and-place operations on the carriers 80. For example, the workstation 10 can remove the boxes from the carriers 80 docked at the docking position, or place the boxes on the storage position of the carriers 80.

[0076] The warehousing system may also include a control device, which may also be called a control system. The control device may be coupled to the handling equipment 30 and the workstation 10 respectively, and is configured to control the operation of the handling equipment 30 and the workstation 10. For example, the control device may control the movement of the handling equipment 30 to handle the carrier 80, or it may control the workstation 10 to perform pick-up and drop operations on the carrier 80 parked at the docking position.

[0077] The control device can be a server or a terminal device, or a device deployed with a Warehouse Management System (WMS) and a Robot Management System (RMS). The terminal device can include at least one of a personal computer, laptop computer, smartphone, tablet computer, and portable wearable device; the server can include a standalone server or a server cluster consisting of multiple servers, and this disclosure does not limit the specific type of server.

[0078] The control device communicates with the handling equipment 30 and the workstation 10 for data communication. For example, the control device can communicate with the handling equipment 30 and the workstation 10 via a local area network (LAN), a wireless local area network (WLAN), or other networks.

[0079] Workstation 10 may also include a work platform 16 located on one side of loading and unloading equipment 11, the work platform 16 being configured for handling items by the operation object.

[0080] Referring to Figure 1A, the length direction of the loading and unloading equipment 11 can be referenced to the x-direction, the width direction can be referenced to the y-direction, and the height direction can be referenced to the z-direction. It should be noted that the loading and unloading direction of the loading and unloading equipment 11 is the y-direction.

[0081] The connecting area 12 and the working platform 16 can be located on opposite sides of the loading / unloading equipment 11 along the loading / unloading direction. For example, the connecting area 12 can be located in front of the loading / unloading equipment 11 along the loading / unloading direction, and the working platform 16 can be located behind the loading / unloading equipment 11 along the loading / unloading direction. The loading / unloading equipment 11 separates the working platform 16 from the connecting area 12, providing safety for operators or robotic arms on one side of the working platform 16. Of course, in some examples, it is not ruled out that the working platform 16 and the connecting area 12 can be located on adjacent sides or the same side of the loading / unloading equipment 11.

[0082] The loading and unloading equipment 11 is configured to transfer items between the mobile carrier 20 and the work platform 16. For example, after the handling equipment 30 moves the mobile carrier 20 to the docking area 12 of the loading and unloading equipment 11, the loading and unloading equipment 11 can remove the items to be processed on the mobile carrier 20 and transfer them to the work platform 16 for the operator to process the items on the work platform 16.

[0083] Workstation 10 may include, but is not limited to, picking workstation 10, sorting workstation 10, or packing workstation 10. Accordingly, loading and unloading equipment 11 takes items out of the mobile carrier 20 or places items in the mobile carrier 20, which can realize the picking, sorting or packing of items.

[0084] The docking device 40 can be a conveyor line in the workstation 10, and the target area 70 can be a docking area on one side of the conveyor line. The handling device 30 can move the movable carrier 20 to the docking area on one side of the conveyor line, where operators or robotic arms can transfer items between the movable carrier 20 and the conveyor line.

[0085] The warehousing system may include a storage area with fixed carriers configured to store items. A retrieval device may be provided on one side of the fixed carrier. The retrieval device can move along the storage location side of the fixed carrier to dock with any storage location of the fixed carrier, thereby retrieving items from any storage location on the fixed carrier and placing them in a movable location or storing items in any storage location of the fixed carrier.

[0086] One side of the fixed vehicle has a passageway, which can serve as a docking area for the pick-and-place device. The handling equipment 30 can move the movable vehicle 20 to the passageway and dock it with the pick-and-place device to transfer items between the movable vehicle 20 and the fixed vehicle via the pick-and-place device.

[0087] A track is provided on the fixed carrier, and the pick-and-place device can travel along the track to move along the length of the fixed carrier. The track can extend from one end of the fixed carrier along its length, so that the pick-and-place device can run to one end of the fixed carrier along its length. In this example, the area at one end of the fixed carrier along its length can be the docking area of ​​the pick-and-place device to dock with the movable carrier 20.

[0088] The docking equipment 40 is a pick-and-place device, and the end area of ​​the tunnel or fixed vehicle along the length direction can be used as the docking area for the pick-and-place device.

[0089] The pick-and-place device may include a lifting mechanism and a pick-and-place mechanism, with the pick-and-place mechanism mounted on the lifting mechanism. The lifting mechanism is suspended from a track on one side of the stationary carrier via a traveling assembly, such as drive wheels, and can travel along the track to move the pick-and-place mechanism along the length of the stationary carrier, thereby switching between different cargo positions within the stationary carrier. The pick-and-place mechanism can move up and down along the lifting mechanism, allowing it to move along the height of the stationary carrier, thereby switching between different cargo positions within the stationary carrier.

[0090] The retrieval and placement devices in the storage area can also enable cargo handling between fixed and mobile vehicles.

[0091] Additional sorting equipment can also be installed within the storage area. Temporary storage shelves can be installed on one side of the sorting equipment, while the other side can serve as a receiving area. Handling equipment can move movable carriers 20 to the receiving area and dock with the sorting equipment to transfer items between the temporary storage shelves and the movable carriers, thus completing the sorting operation. In this example, the target area is the receiving area on one side of the sorting equipment in the storage area, and the docking equipment is the sorting equipment itself.

[0092] The cargo handling equipment can be a robotic arm or similar loading and unloading equipment in a workstation, and there are no restrictions on the structure of the cargo handling equipment.

[0093] Workstation 10 or hub station is equipped with sorting equipment and temporary storage racks. The temporary storage racks are located on one side of the sorting equipment, and the other side of the sorting equipment can be a receiving area. Handling equipment can move movable vehicles 20 to the receiving area and dock with the sorting equipment to transfer items between the temporary storage racks and the movable vehicles, thus realizing the sorting operation. In this example, the target area is the receiving area on one side of the sorting equipment in the workstation or hub station, and the docking equipment is the sorting equipment. The sorting equipment in workstation 10 can be the loading and unloading equipment 11 of workstation 10, or it can be an additionally installed sorting robotic arm, etc.

[0094] Figure 1C shows a schematic diagram of the structure of a workstation 10 provided in some embodiments of this disclosure. The structure and working principle of the workstation 10 provided in the embodiments of this disclosure will be described below with reference to Figure 1C.

[0095] As shown in Figure 1C, the workstation 10 may include a support frame 114, a guide mechanism 115, and a pick-and-place mechanism 111.

[0096] A guide mechanism 115 is disposed to a support frame 114. The guide mechanism 115 is movable laterally and vertically relative to the support frame 114. A pick-and-place mechanism 111 is disposed to the guide mechanism 115 so that the pick-and-place mechanism 111 is movable laterally and / or vertically relative to the support frame 114.

[0097] The pick-and-place mechanism 111 is coupled to the control device. When the handling equipment 30 moves the carrier 80 to the parking position of the workstation 10 under the control of the control device, the pick-and-place mechanism 111 moves laterally (e.g., in the L direction) and / or vertically (e.g., in the H direction) to reach the position corresponding to the cargo location 21 in the carrier 80, thereby picking up or placing the cargo box at the cargo location 21.

[0098] The guiding mechanism 115 may include a lateral moving device 116, a moving rod (also referred to as a longitudinal beam) 112, and a vertical moving device 117. The lateral moving device 116 is disposed to the support frame 114 and coupled to a control device. The lateral moving device 116 is capable of lateral movement relative to the support frame 114. The moving rod 112 extends vertically and is disposed to the lateral moving device 116, thereby enabling lateral movement of the moving rod 112 relative to the support frame 114. The vertical moving device 117 is disposed to the moving rod 112 and coupled to the control device. The vertical moving device 117 is capable of vertical movement relative to the moving rod 112. A pick-and-place mechanism 111 is disposed to the vertical moving device 117, thereby enabling vertical movement of the pick-and-place mechanism 111 relative to the moving rod 112. Therefore, the pick-and-place mechanism 111 can move laterally and / or vertically on the support frame 114.

[0099] Workstation 10 may also be without support frame 114. When workstation 10 is without support frame 114, pick-and-place mechanism 111 can move laterally and / or vertically through guide mechanism 115.

[0100] When the workstation 10 is not equipped with the support frame 114, the workstation 10 may include at least one guide mechanism 115 and a pick-and-place mechanism 111. The pick-and-place mechanism 111 is disposed on the guide mechanism 115, and the pick-and-place mechanism 111 can move laterally and / or vertically through the guide mechanism 115.

[0101] The lateral moving device 116 is coupled to the moving rod 112 and is configured to drive the moving rod 112 to move laterally; the moving rod 112 extends in the vertical direction, and the pick-and-place mechanism 111 is disposed on the moving rod 112 so that the moving rod 112 drives the pick-and-place mechanism 111 to move laterally; the vertical moving device 117 is coupled to the pick-and-place mechanism 111 and is configured to drive the pick-and-place mechanism 111 to move vertically relative to the moving rod 112.

[0102] The lateral moving device 116 can be disposed on the moving rod 112 or on the pick-and-place mechanism 111; the vertical moving device 117 can be disposed on the moving rod 112. For example, the lateral moving device 116 and the vertical moving device 117 can be two drive motors. This embodiment does not limit the specific positions of the lateral moving device 116 and the vertical moving device 117, as long as the lateral moving device 116 can drive the moving rod 112 to move laterally, and the vertical moving device 117 can drive the pick-and-place mechanism 1113 to move vertically relative to the moving rod 112.

[0103] The guiding mechanism 115 can be a robotic arm (such as an automated robotic arm), and the pick-and-place mechanism 111 can be connected to the robotic arm. The pick-and-place mechanism 111 can move laterally and / or vertically via the robotic arm (for example, the robotic arm can move laterally and / or vertically with the pick-and-place mechanism 111). The robotic arm can be fixed to the ground or a guide rail. The robotic arm can be permanently fixed to the ground or a guide rail, or it can be temporarily fixed to the ground or a guide rail. This disclosure does not limit this aspect.

[0104] The pick-and-place mechanism 111 may include a pick-and-place component that can move along the depth direction.

[0105] When the pick-and-place mechanism 111 reaches the position corresponding to the storage location 21 through lateral and / or vertical movement, the pick-and-place component can be controlled to extend along the depth direction, i.e., the pick-and-place component can extend to the storage location 21 in the direction toward the movable carrier, to pick up or place a box from the storage location 21. For example, as shown in FIG1C, when the pick-and-place mechanism 111 reaches the position corresponding to the storage location, the pick-and-place component (not shown in FIG2) can move along the depth direction (such as the Z direction) to the storage location 15 to pick up the box placed on the storage location 15.

[0106] The picking and placing component can be a claw, a fork, or a suction cup connected to the picking and placing mechanism 111. When the picking and placing component is a claw, it can pick up and place the cargo box from the cargo position of the carrier 80 by gripping; when the picking and placing component is a fork, it can pick up and place the cargo box from the cargo position of the carrier 80 by holding; when the picking and placing component is a suction cup, it can pick up and place the cargo box from the cargo position of the carrier 80 by suction. This embodiment does not limit the specific structure of the picking and placing mechanism 111. It should be noted that this embodiment can be illustrated by using a suction cup as an example.

[0107] In actual operation, when the handling equipment 30 carries the movable carrier 20 on the operating surface 50 (e.g., the ground) of the storage system, the items may shake on the storage position 21 of the movable carrier 20 due to unevenness of the operating surface 50, vibration during the movement of the movable carrier 20, or the inertia of the movable carrier 20. In some cases, the items may even protrude from the storage position opening 21a of the movable carrier 20, which may cause the items to fall off the movable carrier 20.

[0108] When the handling equipment 30 is a conveyor line, the position of the movable carrier 20 or the items on the movable carrier 20 may shake due to unevenness of the conveyor line, causing the items to bulge out or even fall off the movable carrier 20.

[0109] During the movement of the transport equipment 30 carrying the mobile carrier 20, the position of the mobile carrier 20 on the transport equipment 30 may be deviated due to unevenness of the running surface 50. This may cause the mobile carrier 20 to fall off the transport equipment 30, or the mobile carrier 20 to be unable to carry out subsequent work smoothly after reaching the target area 70.

[0110] The deviation in the position of the movable carrier 20 on the handling equipment 30 includes the parallel movement of the movable carrier 20 or the tilting of the movable carrier 20.

[0111] When the mobile carrier 20 carrying the items arrives at the docking area 12 of the docking device 40, items protruding from the mobile carrier 20 or the mobile carrier 20 with a deviated posture may interfere with the docking device 40, thereby damaging the items or the docking device 40 or affecting the normal operation of the docking device 40. In addition, the mobile carrier 20 with a deviated posture may not be able to dock with the docking device 40 normally. Referring to Figure 1A, taking the loading and unloading equipment 11 of workstation 10 as an example, the loading and unloading equipment 11 of workstation 10 may include a crossbeam 113, a longitudinal beam 112 and a pick-and-place mechanism 111. It can be understood that the crossbeam 113 and the longitudinal beam 112 may be part of the support frame 114. The pick-and-place mechanism 111 is set on the longitudinal beam 112 and can move vertically along the longitudinal beam 112. The longitudinal beam 112 is slidably connected to the crossbeam 113, that is, the longitudinal beam 112 can move along the length direction of the crossbeam 113, thereby driving the pick-and-place mechanism 111 to move in the length direction.

[0112] In other words, the pick-and-place mechanism 111 can move within the working plane of the workstation 10 (which can be understood as the plane defined by the longitudinal beam 112 and the crossbeam 113, or, referring to Figures 2 and 3, the working plane can be the plane marked by the dashed line A, i.e. the plane of the loading and unloading equipment 11 facing the docking area 12) via the longitudinal beam 112 and the crossbeam 113, thereby docking with the movable carrier 20 parked on one side of the workstation 10. The pick-and-place mechanism 111 transfers the target items on the movable carrier 20 to the workstation 10 for picking or packing, etc. Alternatively, the pick-and-place mechanism 111 transfers the items on the work platform 16 to the movable carrier 20, and transports the movable carrier 20 to the storage area via the handling equipment 30, and can transfer the items on the movable carrier 20 to the fixed carrier for storage via the pick-and-place device in the storage area.

[0113] For example, the mobile vehicle 20 can also store items directly without having to transfer them to a fixed vehicle.

[0114] The loading and unloading equipment 11 may include a robotic arm, which drives the picking and placing mechanism 111 to move freely within the working plane A; for example, driven by the robotic arm, it moves in the height direction and the length direction, and the plane formed by the height direction and the length direction of the picking and placing mechanism 111 is the working plane A.

[0115] The robotic arm can be fixedly mounted on workstation 10; alternatively, it can be fixedly mounted on the ground. Alternatively, the robotic arm can be connected to the ground via slide rails, tracks, or other components, meaning it can be mounted on a slide rail. The connection between the robotic arm and the ground can be either detachable or non-detachable.

[0116] Referring to Figure 2, the mobile carrier 20 typically has multiple storage locations 21, each of which may store different target items. For different target items, the pick-and-place mechanism 111 moves vertically along the longitudinal beam 112 or horizontally along the transverse beam 113 via the longitudinal beam 112. That is, the pick-and-place mechanism 111 can move at least within the working plane A of the workstation 10 and reach the storage location 21 where the target item is located, thereby retrieving the target item.

[0117] Referring to Figure 3, the movable carrier 20 moves under the transport of the transport equipment 30. Generally, the movable carrier 20 can be moved along the movement path (indicated by the arrow) in Figure 3.

[0118] Figure 4 is a schematic diagram of the movement path of a movable carrier towards a docking area according to an embodiment of this disclosure. Referring to Figures 3 and 4, the handling equipment 30 can move the movable carrier 20 towards the workstation 10 (i.e., towards the loading / unloading equipment 11) in the direction shown by s1 in Figures 3 and 4. After the movable carrier 20 moves to a preset distance from the workstation 10, the handling equipment 30 can move the movable carrier 20 to the docking area 12 in the direction shown by s2. In the docking area 12, after the movable carrier 20 completes the docking operation with the loading / unloading equipment 11, it can exit the docking area 12 in the direction shown by s3.

[0119] Figure 5 is a second schematic diagram of the movement path of a mobile vehicle moving towards the docking area according to an embodiment of the present disclosure, and Figure 6 is a third schematic diagram of the movement path of a mobile vehicle moving towards the docking area according to an embodiment of the present disclosure. Referring to Figures 5 and 6, the transport device 30 can directly reach the first end of the workstation 10 from a distance along the direction shown in s2, and then move to the docking area 12 via the first end of the workstation 10.

[0120] Referring to FIG5, in this example, after the movable vehicle 20 completes the docking work with the loading and unloading equipment 11 in the docking area 12, it can exit the docking area 12 in the direction shown by s3.

[0121] Referring to Figure 6, in this example, after the movable vehicle 20 completes the docking operation with the loading and unloading equipment 11 in the docking area 12, it can exit the docking area 12 from the second end of the workstation 10 along the direction shown in s2. It can be understood that the first end and the second end of the workstation 10 are the two ends of the workstation 10 opposite each other along the length direction x.

[0122] Figure 7 is a schematic diagram of the movement path of a movable carrier to the docking area according to an embodiment of this disclosure. Referring to Figure 7, the handling equipment 30 can move to the docking area 12 of the loading and unloading equipment 11 in an "S" shaped movement path. For example, the handling equipment 30 can first move in the direction shown in s4 (e.g., parallel to the length direction of the workstation 10), then move a certain distance towards the workstation 10 in the direction shown in s5, then move a certain distance parallel to the length direction of the workstation 10 in the direction shown in s6, and then move the movable carrier 20 towards the workstation 10 in the direction shown in s3. After the movable carrier 20 has moved to a preset distance from the workstation 10, the handling equipment 30 can move the movable carrier 20 to the docking area 12 in the direction shown in s2. In the docking area 12, after the movable carrier 20 completes the docking operation with the loading and unloading equipment 11, it can exit the docking area 12 from the other end of the workstation 10 in the direction shown in s2.

[0123] This "S"-shaped path allows the handling equipment 30 to move flexibly in narrow or congested environments, avoiding collisions with workstations 10 or other equipment. Upon reaching the docking area 12, the handling equipment 30 can precisely dock with the movable carrier 20 for loading and unloading operations. This flexible movement path selection improves the overall efficiency and safety of the warehousing system.

[0124] This embodiment does not restrict the direction of the movement path of the transport equipment 30 in moving the movable carrier 20 to the docking area 12 of the workstation 10, as long as the movable carrier 20 can eventually reach the docking area 12 of the workstation 10.

[0125] As the mobile carrier 20 moves along the moving path toward the docking area 12 under the drive of the handling equipment 30, the cargo opening 21a of the mobile carrier 20 can always face the loading and unloading equipment 11 of the workstation 10. When it is necessary to change the direction of the moving path of the handling equipment 30, the chassis of the handling equipment 30 does not need to turn. It is only necessary to rotate the wheel system on the chassis to change the forward direction, thereby improving the stability of the mobile carrier 20 on the handling equipment 30.

[0126] In the docking area 12, a portion of the pick-and-place mechanism 111 (e.g., suction cup, gripping fork, or telescopic fork) can dock with the target item on the movable carrier 20 in the longitudinal direction (i.e., the pick-and-place direction, as shown in the y direction in FIG1A), thereby transporting the target item to the work platform 16 of the workstation 10; or, the pick-and-place mechanism 111 moves the target item on the work platform 16 to the movable carrier 20.

[0127] After the goods are processed, the handling equipment 30 moves the movable carrier 20 away from the receiving area 12 in the direction shown by s3 in Figures 4 to 5, or in the direction shown by s2 in Figures 6 and 7. For example, the movable carrier 20 can be moved to the waiting area in the warehousing system to wait for the next time it needs to be picked.

[0128] As shown in Figures 1A and 3, when the handling equipment 30 moves the movable carrier 20 to the receiving area 12, the pick-and-place mechanism 111 needs to move different target items from different storage locations 21 of the movable carrier 20. That is, the pick-and-place mechanism 111 needs to move within the working plane A constructed by the longitudinal beams 112 and the transverse beams 113. During the movement of the pick-and-place mechanism 111 within the working plane A, there should be no obstruction from any items; typically, a certain distance is required between the movable carrier 20 and the target items carried on the movable carrier 20 and the working plane A.

[0129] The handling equipment 30 needs to move the movable carrier 20 to a safe distance range to ensure the normal handling of the target item by the pick-and-place mechanism 111. Referring to Figure 3, the plane shown by the dashed line B represents the end face (hereinafter referred to as docking surface B) where the storage port 21a of the movable carrier 20 is located. The distance range shown between A and B in Figure 3 can be represented as the distance range that the movable carrier 20 or the target item needs to maintain with the working plane A. It can be understood that when the movable carrier 20 or the target item is outside the distance range shown by the two dashed lines in Figure 3, that is, the distance between the movable carrier 20 or the target item and the working plane A is too large, it may cause the pick-and-place mechanism 111 to have difficulty docking with the target storage port 21 of the movable carrier 20, resulting in failure to pick up the item. Usually, when the distance between the movable carrier 20 or the target item and the working plane A is too large, the handling robot can move the movable carrier 20 so that the movable carrier 20 or the target item is moved to a suitable distance from the working plane A.

[0130] The target item on the movable carrier 20 may protrude from the end face (facing the working plane A) of the movable carrier 20 due to uneven ground. This may result in the target item being too close to the working plane A, or even intersecting with the working plane A. When the pick-up and place mechanism 111 moves within the working plane A, there is a risk of interference and collision with the target item, which may cause damage to the pick-up and place mechanism 111 or the target item.

[0131] The warehousing system and workstation 10 provided in this embodiment of the present disclosure have a detection mechanism 14 and a reference mark 15 set on the side of the moving path of the movable carrier 20 to the receiving area 12. When the movable carrier 20 moves to the receiving area 12, the storage port 21a of the movable carrier 20 faces the detection area. In this way, when an item on the movable carrier 20 protrudes from the movable carrier 20 through the storage port 21a, at least a part of the item will enter the detection area of ​​the detection mechanism 14 and block part of the reference mark 15. In this way, the detection mechanism 14 can detect in real time whether at least part of the reference mark 15 is obstructed, so as to determine whether the item on the movable carrier 20 protrudes from the movable carrier 20. For example, when the movable carrier 20 moves to the docking area 12 of the docking device 40 under the transport of the transport equipment 30, it will pass by the detection mechanism 14 and the reference mark 15. When the detection mechanism 14 detects that at least part of the reference mark 15 is obstructed, it determines that the item on the movable carrier 20 protrudes from the movable carrier 20 through the loading port 21a. In this way, the item can be pushed back into the movable carrier 20 in time to avoid the item falling, or to avoid the protruding item interfering with the docking device 40 after the movable carrier 20 enters the docking area 12, causing damage to the item or the docking device 40, or causing the docking device 40 to malfunction. This can improve the docking safety and success rate between the movable carrier 20 and the docking device 40.

[0132] The structure of the warehousing system and workstation 10 provided in the embodiments of this disclosure will be described in detail below with reference to the accompanying drawings.

[0133] Figure 8 is a schematic diagram of one type of detection mechanism and reference mark provided in an embodiment of the present disclosure, and Figure 9 is a schematic diagram of another type of detection mechanism and reference mark provided in an embodiment of the present disclosure. Referring to Figures 1A to 9, the warehousing system, such as a workstation, also includes a detection mechanism 14.

[0134] The warehousing system, such as a workstation, also includes reference objects, which may include reference markers 15. In some application scenarios, the reference objects may also include vehicles and / or collision detection devices pre-installed within the warehousing system. Specifically, based on the positional relationship between the vehicle and the goods, it can be determined whether the goods protrude beyond the vehicle (protrusion can also be called a bulge, which is not limited to this embodiment). Based on the positional relationship between the collision detection devices and the goods, it can be determined whether the cargo box exceeds the collision avoidance zone, i.e., whether there is a risk of collision with the cargo box.

[0135] Reference mark 15 can be a marking (e.g., warning tape, color strip, etc.), a structural feature (e.g., a groove, protrusion, or pattern on the ground), or a physical structure (e.g., a pole, etc.). This embodiment does not limit the type of collision detection object. Reference mark 15 is positioned on one side of the movement path of the movable vehicle 20 towards the target area 70. Reference mark 15 is located within the detection area of ​​the detection mechanism 14. During the movement of the movable vehicle 20 towards the docking area 12, the loading port 21a of the movable vehicle 20 faces the detection area.

[0136] The detection mechanism 14 of this embodiment is configured to detect whether at least a portion of the reference mark 15 is obscured in order to determine whether an article on the movable vehicle 20 protrudes from the movable vehicle 20, or to determine whether the pose of the movable vehicle 20 has deviated.

[0137] It should be noted that the movement path of the mobile vehicle 20 towards the docking area 12 can be understood as the projection area of ​​the mobile vehicle 20 on the operating surface 50 (e.g., the ground) during its movement. This can be understood as follows: by setting markers (e.g., QR codes or graphics) on the ground to pre-plan the path, the transport equipment 30 identifies the markers, causing the mobile vehicle 20 to move along the pre-planned path. Referring to the shaded areas in Figures 4 to 7, this represents the movement path of the mobile vehicle (i.e., the projection area on the ground). One side of the movement path can be understood as one side of the projection area.

[0138] Referring to Figure 3, the loading port 21a of the movable carrier 20 is located on the docking surface B of the movable carrier 20. The loading port 21a of the movable carrier 20 facing the detection area of ​​the detection mechanism 14 means that the loading port 21a of the movable carrier 20 is located outside the detection area of ​​the detection mechanism 14 and is arranged side by side with the detection area in the direction of item entry and exit. In this way, when an item protrudes from the loading port 21a, the item can enter the detection area and block the reference mark 15.

[0139] In the first direction, the detection mechanism 14 and the reference mark 15 are located on the side of the docking device 40 facing the docking area 12, and the distance between them and the docking device 40 in the direction of object retrieval of the docking device 40 is greater than or equal to zero.

[0140] The first direction is parallel to the running surface 50 of the movable carrier 20 and intersects with the loading and unloading direction of the docking device 40. For example, the first direction is the length direction of the docking device 40 (e.g., loading and unloading device 11), as shown by s2 or x in Figures 4 to 7.

[0141] Referring to Figures 3 and 4, for example, when the movable carrier 20 moves along s1 to a predetermined distance from the working plane A of the docking device 40, and then moves along s2 to the docking area 12, the reference mark 15 can be set on one side of the end of the movement path shown in the s1 direction. For example, the detection mechanism 14 and the reference mark 15 can be set on the side of the working plane A of the loading and unloading device 11 facing the entry area 13 of the movable carrier 20.

[0142] The entry zone 13 refers to the area where the movable vehicle 20 is located when it moves along s1 to a preset distance from the working plane A of the docking equipment 40. The entry zone 13 and the docking zone 12 are arranged side by side along the length of the loading and unloading equipment 11.

[0143] Referring to Figures 5 and 6, for example, when the movable carrier 20 moves only along the direction shown in s2 from a distance to the receiving area 12 of the loading and unloading equipment 11, the detection mechanism 14 and the reference mark 15 can be set on the side of the movement path along the direction of s2 facing the workstation 10. For example, the detection mechanism 14 and the reference mark 15 can be set at any position along the movement path along the direction of s2. That is, the detection mechanism 14 is arranged side by side with the loading and unloading equipment 11 along the direction shown in s2. For example, it can be set on the side of the loading and unloading equipment 11 along the length direction and at a position far away from the loading and unloading equipment 11, or it can be set at a position close to the loading and unloading equipment 11, as long as it is ensured that the loading port 21 of the movable carrier 20 faces the detection area of ​​the detection mechanism 14 during the movement of the movable carrier 20 towards the receiving area 12 along the direction shown in s2.

[0144] Referring to Figure 3, in the example above, in the direction of picking up and returning items (i.e., the y-direction) of the docking device 40, the distance between the reference mark 15 and the docking device 40 (e.g., the working plane A of the loading and unloading device 11) can be zero. This ensures that when the detection mechanism 14 detects that the reference mark 15 is obstructed, it determines that the item protrudes from the movable carrier 20 and will interfere with the loading and unloading device 11. On the other hand, it also makes the spatial layout of the detection mechanism 14, the reference mark 15, and the loading and unloading device 11 more compact, thereby reducing the spatial size of the entire workstation 10 and reserving suitable space for other areas of the warehousing system to set up the movement path of other structures or handling equipment 30.

[0145] Figure 9 is a schematic diagram of one embodiment of the detection mechanism and reference mark provided in this disclosure. Referring to Figure 9, of course, in the direction of picking up and returning the object of the docking device 40 (i.e., the y-direction), the distance between the reference mark 15 and the docking device 40 (e.g., the working plane A of the loading and unloading device 11) can also be greater than zero. In this way, a suitable safe distance can be provided for the movable carrier 20. For example, when the detection mechanism 14 detects that the reference mark 15 is blocked, the object protruding from the movable carrier 20 may not interfere with the loading and unloading device 11, and the operator can be promptly reminded to push the object back into the movable carrier 20, greatly reducing the safety hazards caused by the object protruding.

[0146] Referring to FIG7, for example, when the movable vehicle 20 moves toward the docking area 12 along the moving path directions of s4, s5, s6, s1 and s2, the detection mechanism 14 and the reference mark 15 can be set on one side of the projection area (the side facing the workstation 10) when the movable vehicle 20 moves in any direction of s3, s4, s5, s1 and s2. For example, they can be set on one side of the projection area when the movable vehicle 20 moves in the s3 direction.

[0147] It can be understood that the detection area of ​​the detection mechanism 14 refers to the spatial area that can be covered by the detection signal emitted by the detection mechanism 14. For example, the detection area of ​​the detection mechanism 14 can be the spatial area between the detection surface of the detection mechanism 14 and the reference mark 15.

[0148] By orienting the loading port 21a of the movable carrier 20 toward the detection area of ​​the detection mechanism 14, the direction in which the item on the movable carrier 20 protrudes outward through the loading port 21a is toward the direction of movement toward the detection area. In this way, when the item protrudes beyond the movable carrier 20 to a certain extent, it can enter the detection area of ​​the detection mechanism 14 and block the reference mark 15. Thus, when the detection mechanism 14 detects that at least part of the reference mark 15 is blocked, it determines that the item protrudes beyond the movable carrier 20.

[0149] When the position of the movable carrier 20 on the handling device 30 deviates and the direction of the deviation is toward the detection area, at least a portion of the movable carrier 20 may obscure the reference mark 15, thereby determining that the position of the movable carrier 20 has deviated when the detection mechanism 14 detects that at least a portion of the reference mark 15 is obscured.

[0150] In this way, when an item protrudes from the movable carrier 20, it can be pushed back into the movable carrier 20 in time to prevent it from falling. For example, the item can be pushed back into the movable carrier 20 by a worker, or the anti-protrusion mechanism can be triggered to push the protruding item back into the movable carrier 20.

[0151] When the position of the movable vehicle 20 deviates, the position of the movable vehicle 20 or the working state of the handling equipment 30 can be adjusted in time so that the movable vehicle 20 enters the target area 70 in a preset position, thereby smoothly carrying out subsequent work. It can also prevent the movable vehicle 20 from falling off the handling equipment 30 due to the position deviation.

[0152] For example, when the movable carrier 20 tilts at an angle toward the loading dock 21a, the movement direction of the handling equipment 30 can be adjusted so that when the movable carrier 20 reaches the target area 70 (e.g., the docking area 12 of the docking equipment 40), the loading dock 21a is directly opposite the docking equipment 40, thereby enabling the docking equipment 40 to dock smoothly with the movable carrier 20.

[0153] For example, when the movable carrier 20 moves in the direction of the loading dock 21a, the displacement of the handling equipment 30 can be reduced, so that when the movable carrier 20 reaches the target area 70 (e.g., the docking area 12 of the docking equipment 40), the distance between the handling equipment 30 and the docking equipment 40 increases, thereby placing the loading dock 21a of the movable carrier 20 in a dockable position with the docking equipment 40, so that the docking equipment 40 and the movable carrier 20 can dock smoothly.

[0154] When the overall weight of the movable vehicle 20 and the items is within a suitable range, the position of the movable vehicle 20 can also be manually adjusted to adjust the position of the movable vehicle 20 to a preset position.

[0155] It should be noted that the preset pose of the mobile vehicle 20 refers to the pose in which the mobile vehicle 20 can work smoothly in the target area 70.

[0156] The detection mechanism 14 can be positioned above the reference mark 15, with the detection surface of the detection mechanism 14 facing downwards. This reduces the degree of contamination caused by environmental dirt such as dust on the detection surface of the detection mechanism 14 and improves the detection accuracy of the detection mechanism 14.

[0157] The detection mechanism 14 and the reference mark 15 may be located on one side of the docking device 40 along the first direction (i.e., the direction shown in s1). For example, as shown in Figures 3 to 6, the reference mark 15 and the docking device 40 are arranged along the first direction.

[0158] During the process of the movable carrier 20 moving along the first direction to the docking area 12 via the reference mark 15, the distance between the movable carrier 20 and the reference mark 15 along the retrieval and return direction of the docking device 40 is less than or equal to the distance between the movable carrier 20 and the docking device 40.

[0159] Referring to Figure 9, it can be understood that during the process of the movable carrier 20 moving along the first direction to the docking area 12 via the reference mark 15, the distance between the movable carrier 20 and the reference mark 15 (referred to as the first distance, H1) determines that when the length of the item protruding from the movable carrier 20 is greater than or equal to H1, the item can be detected by the detection mechanism 14.

[0160] The first distance can be zero, so that the item can be located in the detection area of ​​the detection mechanism 14 even when it protrudes from the movable carrier 20 by any length, and the reference mark 15 is blocked, so as to improve the detection accuracy of the detection mechanism 14. When the item protrudes from the movable carrier 20 by a small length, it can be detected by the detection mechanism 14 in time, so that the item can be pushed back to the movable carrier 20 in time.

[0161] The first distance can be greater than zero. In this way, the item can enter the detection area of ​​the detection mechanism 14 when it protrudes from the movable carrier 20 by a preset length greater than zero and blocks the reference mark 15. This way, when the item protrudes from the movable carrier 20 by a small length and will not cause the item to fall or interfere with the loading and unloading equipment 11 of the workstation 10, the detection mechanism 14 will not be able to detect the protrusion and will not need to intervene in the item. Only when the item protrudes from the movable carrier 20 by a preset length and will cause the item to fall or interfere with the loading and unloading equipment 11 of the workstation 10, can the detection mechanism 14 detect the protrusion and intervene in the item, such as pushing the item back to the movable carrier 20. This can improve the efficiency of docking between the movable carrier 20 and the loading and unloading equipment 11 and the transfer of items.

[0162] The aforementioned preset length (i.e., the first distance) depends on the distance between the movable carrier 20 and the loading / unloading equipment 11 when the movable carrier 20 is in the docking area 12, that is, the distance between the end face (dating face B) where the cargo port 21a of the movable carrier 20 is located and the working plane A of the workstation 10 (hereinafter referred to as the second distance, see H2 in Figure 9). When the second distance is large, the preset length can be appropriately increased; when the distance between the docking face B and the working plane A of the workstation 10 is small, the preset length can be appropriately decreased.

[0163] The preset length (i.e., the first distance) can be equal to the second distance. Thus, when the detection mechanism 14 detects that at least part of the reference mark 15 is blocked, the item on the movable carrier 20 protrudes beyond the preset length of the movable carrier 20. This indicates that when the movable carrier 20 reaches the docking area 12, the item on the movable carrier 20 will extend to the working plane A of the workstation 10 and collide with the picking and placing mechanism 111 of the loading and unloading equipment 11. This requires the item to be pushed back into the storage position 21 of the movable carrier 20 in a timely manner.

[0164] In order to improve the safety of the handling of the movable carrier 20 and the workstation 10, the preset length can be less than the second distance, so that when the item protrudes from the movable carrier 20 but has not yet reached the point of interfering with the working plane A, it can be detected by the detection mechanism 14 in time, thereby reducing the detection accuracy requirements of the detection mechanism 14.

[0165] In addition, before the movable carrier 20 reaches the docking area 12, the whole unit moves slightly away from the reference mark 15 on the handling equipment 30, or the handling equipment 30 causes the movable carrier 20 to deviate away from the reference mark 15 in the movement path, but the length of the item protruding from the movable carrier 20 is greater than the preset length, for example, greater than or equal to the second distance, but the detection mechanism 14 has not yet detected that the reference mark 15 is blocked, so when the movable carrier 20 reaches the docking area 12, the item will interfere with the loading and unloading equipment 11.

[0166] By setting the preset length to be less than the second distance, a certain deviation space can be reserved for the slight tilt of the movable carrier 20. When the detection mechanism 14 does not detect that the reference mark 15 is blocked, it can also be ensured that the item will not interfere with the loading and unloading equipment 11 when the length protruding from the movable carrier 20 is greater than the preset length.

[0167] Once the preset length is determined, the setting position of the reference mark 15 can be determined, so that when the item protrudes beyond the preset length from the end face of the movable carrier 20, the reference mark 15 can be blocked and thus detected by the detection mechanism 14.

[0168] Among them, the width of the movable carrier 20 when it is carried on the handling equipment 30 is a fixed parameter. After the movement path is determined, the length and width of the projection area of ​​the movable carrier 20 on the movement path are also determined. Then the edge of the projection area is a fixed parameter. Thus, when the reference mark 15 is set on one side of a certain movement path, the first distance can be understood as the distance between the edge of the projection area corresponding to that movement path and the reference mark 15. Therefore, when setting the reference mark 15, the setting position of the reference mark 15 can be determined based on the width and preset length of the movable carrier 20 when it is carried on the handling equipment 30.

[0169] For example, when reference mark 15 is set on one side of the movement path shown in s2, the distance between reference mark 15 and the projection area of ​​the movable vehicle 20 moving in the direction shown in s1 is equal to a preset length.

[0170] To ensure the docking safety between the movable carrier 20 and the docking device 40, the first distance can be less than or equal to the distance between the reference mark 15 and the working plane A. For example, if the first distance is less than the distance between the reference mark 15 and the working plane A, and the item protrudes beyond the first distance of the movable carrier 20, it may obstruct the reference mark 15 and be detected by the detection mechanism 14. However, if the distance between the reference mark 15 and the working plane A is greater than the first distance, the item can be detected in time at a safe distance from the working plane A, allowing it to be pushed back into the movable carrier 20 promptly. Additionally, a certain deviation space can be reserved for slight tilting of the movable carrier 20. Even if the detection mechanism 14 does not detect the reference mark 15 being obstructed, it can still ensure that the item will not interfere with the loading / unloading device 11 even if its length protruding beyond the movable carrier 20 exceeds the first distance.

[0171] The following example illustrates the process of detecting protrusions of items on the mobile vehicle 20, taking the example of the mobile vehicle 20 moving towards the docking area 12 along the movement paths s4, s5, s6, s1, and s2, with the detection mechanism 14 and reference mark 15 positioned on one side of the projection area when the mobile vehicle 20 moves in the s4 direction:

[0172] During the movement of the movable carrier 20 in the direction shown in s4 under the transport of the transport equipment 30, it will pass by the detection mechanism 14 and the reference mark 15, with the loading port 21a of the movable carrier 20 facing the detection area of ​​the detection mechanism 14. When the detection mechanism 14 detects that at least part of the reference mark 15 is obstructed, it determines that the item on the movable carrier 20 is protruding from the movable carrier 20 through the loading port 21a. Thus, the item can be pushed back into the movable carrier 20 in time. For example, when the detection detects that at least part of the reference mark 15 is obstructed, the transport equipment 30 can pause its movement or issue an alarm signal to remind the operator to push the item back into the movable carrier 20 in time to avoid the item falling, or to prevent the protruding item from interfering with the docking equipment 40 after the movable carrier 20 enters the docking area 12, causing damage to the item or the docking equipment 40, or causing the docking equipment 40 to malfunction. This can improve the docking safety and success rate between the movable carrier 20 and the docking equipment 40.

[0173] Understandably, as long as the testing agency 14 detects that the reference mark 15 is damaged, regardless of the extent of the damage to the reference mark 15, it can be determined that the reference mark 15 is obscured by an object protruding from the movable vehicle 20.

[0174] Multiple detection mechanisms 14 can be provided, and correspondingly, multiple reference marks 15 can be provided. For example, referring to FIG5, when the movable vehicle can move from both ends of the workstation 10 to the docking area 12 along the moving path shown in s2, two detection mechanisms 14 can be provided, and correspondingly, two reference marks 15 can be provided. For example, one detection mechanism 14 and one reference mark 15 can be provided at the first end of the workstation 10 along the length direction to detect the movable vehicle 20 entering the docking area 12 from the first end of the workstation 10 along the s2 direction, and another detection mechanism 14 and one reference mark 15 can be provided at the second end of the workstation 10 along the length direction to detect the movable vehicle 20 entering the docking area 12 from the second end of the workstation 10 along the s2 direction.

[0175] Multiple detection mechanisms 14 can also be spaced apart along the extension direction of one of the movement paths to repeatedly detect protrusions of the object along the movement path of the movable vehicle 20. For example, referring to FIG7, when the movement path of the movable vehicle 20 is an "S" shaped structure, a detection mechanism 14 and a reference mark 15 can be set on one side of the movement path in the s4 direction so that the detection mechanism 14 can detect protrusions of the object in the movable vehicle 20 when the movable vehicle 20 travels through the movement path segment shown in s4. In addition, a detection mechanism 14 and a reference mark 15 can be set on one side of the movement path in the s2 direction so that the detection mechanism 14 can detect protrusions of the object in the movable vehicle 20 when the movable vehicle 20 travels through the movement path segment shown in s2.

[0176] The inspection agency 14 can be in a continuous inspection state during the movement of the mobile carrier 20 driven by the handling equipment 30, so as to ensure that the inspection agency 14 can detect in time whether there are protruding items in each mobile carrier 20.

[0177] The detection mechanism 14 can be in a detection state when the movable carrier 20 moves to the movement path segment corresponding to the reference mark 15. For example, the reference mark 15 is located on one side of the movement path shown in s2. When the movable carrier 20 moves to the movement path segment shown in s2, the handling device 30 can send an opening signal to the detection mechanism 14. The detection mechanism 14 starts to acquire information about the reference mark 15 based on the opening signal to determine whether the reference mark 15 is obstructed.

[0178] Of course, a position detector can also be set on the movement path shown in s2. When the position detector detects the movable vehicle 20, it sends an activation signal to the detection mechanism 14. The detection mechanism 14 starts to acquire information about the reference mark 15 based on the activation signal to determine whether the reference mark 15 is obscured.

[0179] Once the movable carrier 20 has moved to one side of the testing mechanism 14, it can pause its movement and wait for a preset time. After the testing mechanism 14 has completed its testing, it can resume its movement. The preset waiting time can be determined based on the mechanism parameters of the testing mechanism 14 itself, and is not limited here.

[0180] When the other movable vehicle 20 moves to the side of the detection mechanism 14, it does not need to stop and continues to move. The detection mechanism 14 detects the obstruction of the reference mark 15 during the movement of the movable vehicle 20, so as to improve the working efficiency of the movable vehicle 20.

[0181] Considering that protruding items may be located on different layers of the same column in the movable vehicle 20, the detection process can be carried out in multiple rounds until all protruding items are pushed back and no longer obstruct the view.

[0182] For example, the inspection agency 14 performs a first inspection on a movable carrier 20. If it detects that an item is protruding from a certain column, it pushes that column of items back. Then, the inspection agency 14 continues to perform a second inspection on the movable carrier 20. If it detects that an item is protruding from the same column, it pushes that column of items back. Then, the inspection agency 14 continues to perform a third inspection on the movable carrier 20. When no protrusions are detected, the movable carrier 20 can move to the target area 70 (e.g., the connecting area 12). In this way, it can be ensured that the protrusion of items in each layer of the movable carrier 20 is detected and promptly pushed back into the movable carrier 20.

[0183] The installation height of the detection mechanism 14 and the reference mark 15 relative to the movable vehicle 20 can be set according to actual needs.

[0184] Referring to Figure 8, when the detection mechanism 14 is set, it can be higher than or level with the top of the top shelf 21 of the target movable carrier 20, and the reference mark 15 can be lower than or level with the bottom of the lowest shelf 21 in the target movable carrier 20. In this way, when the movable carrier 20 passes by one side of the detection area under the drive of the handling equipment 30, the detection mechanism 14 can detect any items protruding from the movable carrier 20 along the entire height direction of the movable carrier 20. This enables comprehensive detection of the movable carrier 20 and effectively prevents safety hazards caused by items protruding from the movable carrier 20 at any height.

[0185] It should be noted that the target mobile vehicle 20 refers to the mobile vehicle 20 located on the handling equipment 30.

[0186] The top of the top shelf 21 of the target mobile vehicle 20 refers to the top of the space in the top shelf 21 of the target mobile vehicle 20 that can accommodate items, for example, the top of the items when items are placed on the top shelf 21.

[0187] Additionally, the bottom of the lowest level cargo space 21 of the target mobile vehicle 20 refers to the bottom of the space in the lowest level cargo space 21 of the target mobile vehicle 20 that can accommodate items, for example, the bottom of the items when items are placed on the lowest level cargo space 21.

[0188] The detection unit 14 can be set at a position higher than the top shelf 21 of the target mobile vehicle 20, and the reference mark 15 can be set on the operating surface 50 of the storage system (e.g., the ground).

[0189] The detection mechanism 14 can be positioned above the top shelf 21 of the target movable vehicle 20, and the reference mark 15 can be positioned above the bottom of the bottom shelf 21 of the target movable vehicle 20, for example, in the middle area of ​​the target movable vehicle 20. In this way, the detection mechanism 14 can accurately detect the protrusion of items at higher levels of the target movable vehicle 20. For example, a support can be set on the ground, and the reference mark 15 can be placed on the support so that the reference mark 15 is located in the middle area of ​​the target movable vehicle 20. This also reduces the distance between the detection mechanism 14 and the reference mark 15, thereby improving the detection sensitivity of the detection mechanism 14 to the reference mark 15, and thus accurately detecting the protrusion of items at higher levels of the target movable vehicle 20.

[0190] In addition, the height of the inspection agency 14 can be flexibly adjusted according to actual needs to accommodate mobile vehicles 20 of different sizes and types and the items they carry, further enhancing the versatility and practicality of the warehousing system.

[0191] The length of the reference mark 15 can be greater than or equal to the length of the movable vehicle 20. In this way, when any item protrudes from the movable vehicle 20, it can block the reference mark 15 and be detected by the detection agency 14, thereby increasing the detection range.

[0192] Furthermore, when the movable carrier 20 is in continuous motion during the detection process of the detection mechanism 14, by setting the length of the reference mark 15 to be greater than or equal to the length of the movable carrier 20, it can be ensured that the detection mechanism 14 can effectively monitor the protrusion of items in the same column of the movable carrier 20 for a fixed period of time as the movable carrier 20 moves along one side of the reference mark 15, until the items in that column pass the reference mark 15. That is, the detection time for protrusion of items on the movable carrier 20 can be increased even when the movable carrier 20 is in continuous motion. This design not only improves the safety of the warehousing system but also effectively reduces the operational risks caused by protruding items, providing a more reliable guarantee for warehousing operations.

[0193] It should be noted that the above-mentioned fixed duration is the time it takes for the movable vehicle 20 to move from one end of the reference mark 15 to the other end of the reference mark 15.

[0194] In other examples, the length of reference marker 15 may also be shorter than the length of the movable vehicle 20 to save on the setup cost of reference marker 15.

[0195] In this embodiment of the present disclosure, the detection mechanism 14 can be configured in various ways, and correspondingly, the reference mark 15 can also be configured in various ways. This embodiment of the present disclosure does not limit the configuration of the detection mechanism 14 and the reference mark 15. As long as the detection mechanism 14 can detect that the reference mark 15 is blocked, the situation of the item protruding from the movable carrier 20 can be determined. Several configurations of the detection mechanism 14 are listed below.

[0196] The inspection unit 14 includes an image acquisition device. A reference marker 15 is located on the operating surface 50 of the movable vehicle 20. For example, the reference marker 15 can be affixed to the ground to simplify its installation. The image acquisition device is configured to acquire images of the reference marker 15 and determine whether the reference marker 15 is obstructed by protruding objects on the movable vehicle 20 based on the degree of image loss.

[0197] Setting the detection mechanism 14 as an image acquisition device simplifies the structural setup of the reference mark 15. For example, the reference mark 15 can be directly set on the running surface 50 of the movable carrier 20, simplifying its installation structure. Furthermore, the reference mark 15 can be of any shape, texture, or color; for instance, its shape can be a line, surface, or other structure, allowing for flexible setup. The image acquisition device also employs mature and accurate image recognition and processing algorithms, thereby improving the detection accuracy of protruding objects.

[0198] In addition, image acquisition devices, such as cameras, can operate in low light, dust, and humid environments, thereby improving the adaptability of the inspection agency 14 to the environment. In particular, the image acquisition devices are more adaptable to various complex environments within the logistics and warehousing system.

[0199] For example, image acquisition devices may include, but are not limited to, cameras, infrared thermal imagers, and multispectral cameras.

[0200] Referring to Figures 8 and 9, for example, the image acquisition device includes a camera 141, and the reference mark 15 can be a reference line 151. The physical characteristics of the reference mark 15 include the image information of the reference line 151. The camera 141 is configured to acquire the image information of the reference line 151 to determine whether the reference line 151 is obscured by a protruding object on the movable carrier 20 based on the degree of missing image information. Alternatively, it can determine the protrusion information of the target object relative to the reference object based on the positional relationship between the detection frame corresponding to the target object and the detection frame corresponding to the reference object. For example, the camera 141 can perform a graphic continuity comparison between the acquired image information of the reference line 151 and the pre-made image information of a complete reference line 151. When the comparison shows that the graphics are not continuous, it means that the reference line 151 is obscured by a protruding object on the movable carrier 20. At this time, the warehousing system can issue an alarm or execute a preset processing procedure, such as suspending logistics operations or triggering the anti-protrusion mechanism to push the protruding object back into the storage location 21 to avoid the protruding object interfering with or damaging the logistics operations.

[0201] Furthermore, by assessing the degree of missing image information, parameters such as the size and location of protruding parts of the item can be estimated, providing more detailed information support for subsequent logistics processing. This image-based detection method not only improves the accuracy and efficiency of detection but also significantly simplifies the setup and maintenance costs of the detection facility 14.

[0202] The target plane of the image acquisition device (as shown in C in Figure 8) is perpendicular to the reference mark 15. That is, the light beam emitted by the image acquisition device can be incident on the surface of the reference mark 15 in the normal direction, thereby minimizing the light loss caused by the oblique incident light onto the reference mark 15, improving the accuracy of determining the loss degree of the reference mark 15, and thus improving the detection accuracy of the camera 141 for the protrusion of the object.

[0203] The image acquisition device can emit multiple target planes C parallel to the y-direction to acquire image information of each item arranged along the y-direction on the reference mark 15. By acquiring image information at different positions along the y-direction, the size of the protruding part of the item can be determined.

[0204] Multiple reference markers 15 can be set parallel to each other along the y-direction. The number of reference markers 15 corresponds to the number and position of the target planes C. The size of the protruding part of the object can be determined by the image information of the corresponding reference markers 15 collected from different target planes C.

[0205] An image acquisition device can emit multiple target planes C parallel to each other along the y-direction.

[0206] Multiple image acquisition devices can be arranged along the y-direction, with each image acquisition device emitting a target plane.

[0207] Furthermore, the target plane of the image acquisition device is perpendicular to the reference mark 15. When the reference mark 15 is parallel to the end face of the cargo opening 21a of the movable carrier 20, the target plane of the image acquisition device is parallel to the end face of the cargo opening 21a of the movable carrier 20. This ensures that the size of the protruding portion of each layer of items detected by the image acquisition device is greater than or equal to H1.

[0208] By aligning the target plane of the image acquisition device perpendicular to the reference mark 15, and the reference mark 15 parallel to the end face of the cargo bay 21a of the movable carrier 20, problems arising from the target plane not being perpendicular to the reference mark 15 can be avoided. For example, the problem of an item protruding a small length (e.g., less than H1) being detected by the image acquisition device can be avoided; the risk of an item protruding a large length (e.g., greater than H1) not being detected by the image acquisition device can also be avoided.

[0209] The aforementioned setup improves the accuracy and reliability of item management in the warehousing system. This vertical setup ensures that the detection signal emitted from the target plane directly illuminates protruding items on the movable carrier 20, thereby accurately detecting the protrusion of the items. Simultaneously, this setup also helps improve the stability and accuracy of the detection signal, further enhancing the safety and operational efficiency of the warehousing system.

[0210] For example, the target plane of camera 141 is perpendicular to reference line 151, so that the size of the protruding portion of each layer of the object detected by camera 141 is greater than or equal to H1. This avoids the problem of an object with a small protrusion length (e.g., less than H1) being detected by camera 141, or an object with a large protrusion length (e.g., greater than H1) not being detected by camera 141. It is understood that the emission target surface of detection mechanism 14 includes the target plane in this example.

[0211] Referring to Figure 9, the reference line 151 is parallel to the working plane A of the docking device 40. That is, the distance between any position of the reference line 151 along its length and the working plane A is equal. In this way, when the detection mechanism 14 detects that the reference line 151 is blocked, the protruding length of the object blocking any position of the reference line 151 is greater than or equal to H1, ensuring that the distance between the protruding object and the working plane A is within a controllable range. This allows the reference line 151 to continuously serve as a detection reference when the movable carrier 20 moves in a direction parallel to the working plane A of the docking device 40 (e.g., the direction shown in s2), ensuring the consistency and stability of the detection.

[0212] Furthermore, the design of the reference line 151 can be flexibly adjusted to adapt to mobile carriers 20 of different sizes and shapes, as well as different logistics operation needs. For example, parameters such as the length, width, and color of the reference line 151 can be customized according to actual needs to improve the accuracy and applicability of detection. Simultaneously, the placement of the reference line 151 can be optimized according to the layout of the warehousing system and the logistics operation process to ensure the effectiveness and efficiency of detection. In this embodiment, through real-time monitoring of the reference line 151 by the camera 141, the warehousing system can promptly detect and handle protruding items on the mobile carrier 20, effectively avoiding safety hazards and losses in logistics operations.

[0213] The image acquisition device can also be equipped with lighting equipment, such as LED light strips or ring lights, to ensure clear image acquisition of reference marker 15 even in low-light conditions. The position and brightness of the lighting equipment can be adjusted according to the actual working environment to achieve the best image acquisition effect.

[0214] Reference mark 15 may be a reflective element affixed to the operating surface 50. For example, the reflective element may include, but is not limited to, reflective strips / sheets, microprism reflective plates, laser-etched reflective patterns, adaptive reflective materials, etc.

[0215] For example, reflectors can be made of glass microspheres (such as VHB reflective film) or aluminum-based reflectors, with a silver or gold coating on the surface. Adaptive reflective materials can be electrochromic layers, photosensitive coatings, etc., applied to the ground.

[0216] By setting the reference mark 15 as a reflective element, the reflectivity of the reflective element to the detection beam is increased, thereby improving the recognition accuracy of the detection mechanism 14 of the physical features of the reference mark 15, such as the image, and thus improving the accuracy of the judgment on whether the reference mark 15 is occluded.

[0217] In the warehousing system shown in Figure 1B, after the handling equipment moves the carrier 80 to the docking position of the workstation 10, the pick-and-place mechanism 111 on the workstation 10 can move laterally and / or vertically relative to the carrier 80 to reach the target storage location on the carrier 80, and pick up or place containers such as boxes and pallets from the storage location, or pick up or place goods from the storage location. During this process, if there are protruding boxes on the movable carrier 20, the pick-and-place mechanism 111 may collide with the protruding boxes during the movement, which not only reduces safety but also affects the continuation of the work.

[0218] Figure 10 illustrates an application scenario of a target object protrusion detection method. As shown in Figure 10, if a cargo box A protrudes from the carrier 80, the picking and placing mechanism 111 may collide with the cargo box A during its upward movement. Therefore, to improve the safety of the warehousing system, it is necessary to detect whether a target object, such as a cargo box or an item, is protruding. It is understood that the target object can be a carrier, a cargo box, or an item, and the embodiments of this disclosure do not specifically limit the type of target object.

[0219] Figure 11 is a schematic flowchart of a target object protrusion detection method provided in an embodiment of this disclosure. The target object protrusion detection method provided in this disclosure will be described below with reference to Figure 11. It should be noted that the target object protrusion detection method provided in this disclosure can be implemented by the control device (which can also be called a control system) in the aforementioned warehousing system, or by the controller in the workstation 10 of the aforementioned warehousing system. This disclosure does not limit the implementation in this regard.

[0220] As shown in Figure 11, the target object protrusion detection method provided in this embodiment includes S301-S303.

[0221] S301. Acquire at least one target image.

[0222] At least one target image is acquired based on at least one acquisition direction. The target image includes both the target object and a reference object. Taking a container protrusion detection scheme on a vehicle as an example, the target object is the container, and the reference object can be a shelf or other marker to detect whether the container protrudes from the shelf or other marker. Taking a vehicle protrusion detection scheme as an example, the reference object can be a marker line or marker pole to detect whether the shelf protrudes from the marker line or marker pole, etc. For example, the reference object may include reference mark 15.

[0223] The vehicle can be a mobile vehicle. Taking the detection scheme for a protruding container on a mobile vehicle as an example, the target object is the container, and the reference object can be a shelf or other signage. The detection method is to check whether at least part of the shelf or other signage is obstructed in order to determine whether the container protrudes from the shelf or other signage or to determine whether there is a deviation in the position of the mobile vehicle.

[0224] At least one target image can be obtained by capturing images of a target object and a reference object from at least one acquisition direction using an image acquisition device (also called an image acquisition apparatus, such as a camera). The acquisition direction can be configured to indicate the direction in which the image acquisition device captures the target object and the reference object. For example, the acquisition direction can be from top to bottom (i.e., vertical), from left to right, or from right to left (i.e., horizontal), or other angles; this disclosure does not limit this.

[0225] When the acquisition direction is top-down, the target image will be a top view of the target and reference objects; when the acquisition direction is left-to-right or right-to-left, the target image will be a side view of the target and reference objects. Understandably, the acquisition direction can be adjusted according to actual needs, the location of the target object, or the relative positional relationship between the target and reference objects. For example, by acquiring images of the vehicle and containers from top to bottom, it can be determined which column of containers stands out from the vehicle setup. Alternatively, by acquiring images of the vehicle and containers from left to right or right to left, it can be determined which row of containers stands out from the vehicle setup.

[0226] The target object can be configured to represent the object to be detected. For example, the target object can be a container such as a box or pallet, a carrier such as a shelf, or other objects that need to be detected, which is not limited in the embodiments of this disclosure.

[0227] When the target object is a cargo box, the target object protrusion detection method provided in this embodiment can be configured to detect whether the cargo box protrudes from the vehicle, in order to determine whether the pick-and-place device in the workstation will collide with the cargo box. When the target object is a vehicle, the target object protrusion detection method provided in this embodiment can be configured to detect whether the vehicle protrudes from a preset area, in order to determine whether the pick-and-place device in the workstation will collide with the vehicle.

[0228] Reference objects can be configured to represent objects providing spatial reference. Reference objects include reference markers 15 or other identifying features. In some applications, reference objects may include vehicles and / or pre-installed collision detection devices within the warehousing system. Specifically, the positional relationship between the vehicle and the goods can be used to determine whether the goods protrude beyond the vehicle; and the positional relationship between the collision detection devices and the goods can be used to determine whether the cargo container exceeds the collision avoidance zone, i.e., whether there is a risk of collision with the cargo container.

[0229] The collision detection device can be a marking (e.g., warning tape, color strip, etc.), a structural feature (e.g., grooves, protrusions, or patterns on the ground), or a physical structure (e.g., a pole, etc.). This disclosure does not limit the type of collision detection device. The collision detection device can be placed on the ground or on the side of the vehicle; this disclosure does not limit the location of the collision detection device.

[0230] To improve detection accuracy, the initial image acquired by the image acquisition device can be preprocessed, and the preprocessed initial image can be determined as the target image for target object protrusion detection. The preprocessing method can be image grayscale conversion, image distortion correction, or other image preprocessing methods; this disclosure does not limit the specific methods used.

[0231] The initial image acquired by the image acquisition device can be subjected to distortion correction processing, and the distortion-corrected initial image can be determined as the target image. As shown in Figure 12, the target protrusion detection method provided in this embodiment of the present disclosure further includes S401-S402.

[0232] S401. Acquire the initial image captured by the image acquisition device.

[0233] S402. Based on the intrinsic parameter matrix and distortion parameters corresponding to the image acquisition device, the initial image is subjected to distortion removal processing to obtain the target image.

[0234] The initial image can be configured to represent the raw, unprocessed image. The target image can represent the raw image after distortion correction.

[0235] At least one initial image can be acquired using an image acquisition device based on at least one acquisition direction. Then, distortion correction processing is performed on each initial image to obtain at least one target image. The image acquisition device can acquire one or more initial images based on each acquisition direction.

[0236] When the image acquisition device acquires multiple initial images based on a first acquisition direction (e.g., from top to bottom), distortion correction processing can be performed on the initial image with the highest clarity among the multiple initial images to obtain its corresponding target image.

[0237] The intrinsic parameter matrix is ​​configured to describe the internal geometric characteristics of the image acquisition device, and the distortion parameter characterizes the degree of lens distortion of the image acquisition device. The intrinsic parameter matrix and distortion parameter can be preset within the image acquisition device and / or control unit.

[0238] S401-S402 can be executed by the image acquisition device or by the control device. This disclosure does not limit the scope of the embodiments.

[0239] When the control device executes S401-S402, the image acquisition device acquires an initial image and sends it to the control device. After receiving the initial image from the image acquisition device, the control device performs distortion correction processing on the initial image based on a pre-set intrinsic parameter matrix and distortion parameters to obtain the target image. When the image acquisition device executes S401-S402, after acquiring the initial image, it performs distortion correction processing on the initial image based on a pre-set intrinsic parameter matrix and distortion parameters to obtain the target image, and then sends the target image to the control device.

[0240] The control device can send a capture command to the image acquisition device, which then captures an image and sends it to the control device. Alternatively, sensors can be installed in the warehousing system to detect whether a vehicle has been moved to the corresponding docking point at the workstation. When the vehicle is moved to the docking point, the sensor can send a trigger signal to the image acquisition device, causing the image acquisition device to capture an image, and the control device simultaneously acquires the image captured by the image acquisition device. Alternatively, the image acquisition device can periodically capture images and send them to the control device. This disclosure does not limit the method of image acquisition.

[0241] The control device can send an acquisition command to the image acquisition device, which then acquires an initial image and sends it back to the control device. Upon receiving the initial image, the control device performs distortion correction processing on the initial image based on a pre-set intrinsic parameter matrix and distortion parameters to obtain the target image.

[0242] As shown in Figure 13, S402 may include S4021-S4024.

[0243] S4021. Convert the pixel coordinates of multiple pixels to be processed in the image to be processed into multiple normalized coordinates. For example, based on the intrinsic parameter matrix, convert the pixel coordinates of multiple pixels to be processed in the image to be processed into multiple normalized coordinates.

[0244] S4022. Based on multiple normalized coordinates, distortion parameters, and preset distortion formulas, determine multiple distortion coordinates.

[0245] S4023. Map multiple distorted coordinates onto the initial image to obtain multiple target coordinates.

[0246] S4024. Assign the pixel value at the target coordinate in the initial image to multiple pixels to be processed to obtain the target image. For example, by using interpolation, assign the pixel value at the target coordinate in the initial image to multiple pixels to be processed to obtain the target image.

[0247] The image to be processed refers to a blank image with the same resolution as the initial image. Normalized coordinates can be configured to represent the coordinates of the image to be processed on the normalized plane. Distorted coordinates can be configured to represent the distorted normalized coordinates. Target coordinates can be configured to represent the pixel values ​​corresponding to the distorted coordinates in the initial image.

[0248] A blank image with the same resolution as the initial image can be created to obtain the image to be processed. This image contains multiple pixels to be processed, each with an initial pixel value of 0. Given that the pixel coordinates of the pixels to be processed in the image are (u', v'), these coordinates can first be converted to normalized coordinates (x', y') based on a pre-set intrinsic parameter matrix. Then, the distortion parameter D and the normalized coordinates (x', y') are substituted into a preset distortion formula to obtain the distorted coordinates (x', y'). undistort y undistort The distorted coordinates (x) are obtained. undistort y undistort After that, the distorted coordinates (x) undistort y undistort The distorted coordinates are mapped onto the initial image to obtain the corresponding pixel coordinates in the initial image, i.e., the target coordinates (u,v). After obtaining the target coordinates (u,v), interpolation can be used to assign the pixel value corresponding to the target coordinates (u,v) in the initial image to the pixel value corresponding to the pixel coordinates (u',v') in the image to be processed, thus obtaining the target image. For example, if the pixel value corresponding to the target coordinates (u,v) is 255, interpolation can be used to modify the pixel value corresponding to the pixel coordinates (u',v') in the image to be processed from the initial pixel value of 0 to 255.

[0249] It should be noted that the interpolation method can be bilinear interpolation or bicubic interpolation. This disclosure does not limit the specific method used.

[0250] It is understandable that distortion correction can prevent distortion of the target object and reference object in the target image, thereby improving the accuracy of subsequent detection.

[0251] S302. Detect at least one target object and reference object in a target image using a target detection model to obtain the detection box corresponding to the target object and the detection box corresponding to the reference object.

[0252] The object detection model can be configured to identify objects and reference objects in an image and to label them using bounding boxes.

[0253] At least one target image can be input into the target detection model. After receiving the at least one target image, the target detection model detects the target object and the reference object in the at least one target image, obtains the detection box corresponding to the target object and the detection box corresponding to the reference object, and outputs them.

[0254] The detection box corresponding to the target object can be configured to represent the boundary of the target object, and the detection box corresponding to the reference object can be configured to represent the boundary of the reference object.

[0255] The target detection model can output the detection boxes corresponding to the target and the reference object in the form of data (e.g., coordinate values), or in the form of images (e.g., generating detection boxes in the target image).

[0256] The detection frame can be rectangular or other shapes, and this disclosure does not limit this.

[0257] When the detection box is rectangular, the target detection model can output the detection box corresponding to the target object and the detection box corresponding to the reference object in the form of coordinates, or it can output the detection box corresponding to the target object and the detection box corresponding to the reference object in the form of coordinates combined with the length and width of the detection box.

[0258] After obtaining the detection box using the object detection model, the coordinates of the two opposite vertices of the detection box can be determined and output using the object detection model; that is, the detection box can be represented as the top-left corner coordinates (x1, y1) and the bottom-right corner coordinates (x2, y2). After obtaining the detection box using the object detection model, the coordinates of the four vertices of the detection box can also be determined and output using the object detection model; that is, the detection box can be represented as the top-left corner coordinates (x1, y1), the bottom-left corner coordinates (x1, y2), the top-right corner coordinates (x2, y1), and the bottom-right corner coordinates (x2, y2). After obtaining the detection box using the object detection model, the center coordinates, length, and width of the detection box can also be determined and output using the object detection model. That is, the detection box can be represented as (x1, y1), (x2, y1), (x2, y2 ... center y center (x, w, h). center y center ) represents the center coordinates of the detection box, w represents the width of the detection box, and h represents the length of the detection box.

[0259] Understandably, target detection models can determine the positions of target objects and reference objects, providing a data foundation for subsequent target prominence detection. Labeling target and reference objects with bounding boxes facilitates the subsequent determination of their spatial relationship.

[0260] The object detection model may include convolutional layers, activation layers, segmentation layers, and sampling layers. The sampling layers may include downsampling layers and upsampling layers. As shown in Figure 14, S302 may include S3021-S3023.

[0261] S3021. Perform convolution activation processing on at least one target image using a target detection model to obtain a first image.

[0262] The object detection model first performs convolution processing on at least one object image to obtain a convolutional image. Then, activation processing is performed on the convolutional image to obtain the first image.

[0263] A first image can be obtained by performing multiple convolutional activation processes on at least one target image using an object detection model. For example, the first image can be obtained by performing 70 convolutional activation processes on at least one target image using an object detection model. The number of times the object detection model performs convolutional activation processes on at least one target image can be set according to actual needs, and this embodiment does not limit this.

[0264] S3022. The first image is segmented using an object detection model to obtain a multi-layer feature map.

[0265] Multi-layer feature maps include features at various resolutions.

[0266] The first image can be segmented multiple times using an object detection model to obtain multi-layer feature maps. For example, the first image can be segmented 30 times using an object detection model to obtain multi-layer feature maps. The number of times the object detection model segments the first image can be set according to actual needs, and this embodiment does not limit this.

[0267] S3023. The multi-layer feature map is sampled and processed by the target detection model to obtain the detection box corresponding to the target object and the detection box corresponding to the reference object.

[0268] Sampling processing can include upsampling and downsampling.

[0269] The object detection model first downsamples the multi-layer feature map to obtain high-level semantic features (e.g., cargo box location) in the multi-layer feature map. Then, the object detection model upsamples the downsampled multi-layer feature map to restore the resolution, thus obtaining the detection box corresponding to the target object and the detection box corresponding to the reference object.

[0270] S303. Based on the positional relationship between the detection box corresponding to the target object and the detection box corresponding to the reference object, determine the prominence information of the target object relative to the reference object.

[0271] The positional relationship between the detection box corresponding to the target object and the detection box corresponding to the reference object is determined. Based on this positional relationship, the protrusion information of the target object relative to the reference object is determined. This protrusion information can be configured to represent the protrusion state and / or protrusion position of the target object relative to the reference object.

[0272] The bulging information may include at least one of the following: the target object does not protrude from the vehicle; the target object protrudes from the vehicle and there is no collision risk; the target object protrudes from the vehicle and there is a collision risk; a horizontal bulging position; and a vertical bulging position. The options of "the target object does not protrude from the vehicle," "the target object protrudes from the vehicle and there is no collision risk," and "the target object protrudes from the vehicle and there is a collision risk" are configured to indicate the bulging state of the target object relative to a reference object, while the horizontal and vertical bulging positions are configured to indicate the bulging positions of the target object relative to the reference object.

[0273] The reference objects can be set according to the actual application scenario. Different application scenarios may have the same or different types of reference objects; different application scenarios may have the same or different numbers of reference objects. This disclosure does not limit this.

[0274] At least one target image includes a first target image; the first target image is acquired based on a first direction among at least one acquisition direction; when acquiring the first target image based on the first direction, the reference objects include a vehicle and a collision detection object; the detection box corresponding to the target object includes a first detection box, and the detection box corresponding to the reference object includes a second detection box corresponding to the vehicle and a third detection box corresponding to the collision detection object. Specifically, detecting the target object and the reference object in at least one target image using a target detection model to obtain the detection boxes corresponding to the target object and the reference object may include: detecting the target object and the reference object in the first target image using a target detection model to obtain the first detection box, the second detection box, and the third detection box.

[0275] The first direction can be from top to bottom, and the first target image is a top view of the target object and the reference object. That is, the first target image can be obtained by taking pictures of the target object and the reference object from top to bottom using an image acquisition device.

[0276] Based on the positional relationship between the detection frame corresponding to the target object and the detection frame corresponding to the reference object, the protrusion information of the target object relative to the reference object is determined, including: when the first boundary of the first detection frame is outside the second detection frame and there is no overlapping area between the first detection frame and the third detection frame, the protrusion information is determined to be that the target object is protruding from the vehicle and there is no collision risk; when the first boundary of the first detection frame is outside the second detection frame and there is an overlapping area between the first detection frame and the third detection frame, the protrusion information is determined to be that the target object is protruding from the vehicle and there is a collision risk.

[0277] The first boundary of the first detection frame can be configured to represent the boundary of the first detection frame on the side closest to the collision detection object.

[0278] The positional relationship between the detection boxes corresponding to the target object and the detection boxes corresponding to the reference object can be determined by determining the size relationship between the coordinates of the detection boxes corresponding to the target object and the coordinates of the detection boxes corresponding to the reference object.

[0279] As shown in Figure 15A, in some application scenarios, the first detection frame output by the target detection model has coordinates of the top-left corner (x1, y1), the bottom-left corner (x1, y2), the top-right corner (x2, y1) and the bottom-right corner (x2, y2); the second detection frame has coordinates of the top-left corner (x3, y3), the bottom-left corner (x3, y4), the top-right corner (x4, y3) and the bottom-right corner (x4, y4); and the third detection frame has coordinates of the top-left corner (x5, y5), the bottom-left corner (x5, y6), the top-right corner (x6, y5) and the bottom-right corner (x6, y6). When y2 ≥ y4, it is determined that the first boundary of the first detection frame is located within the second detection frame (as shown by a in Figure 15A); when y2 < y4, it is determined that the first boundary of the first detection frame is located outside the second detection frame (as shown by b in Figure 15A). When y2 < y5 and y2 > y6, or y2 < y6, it is determined that the first boundary of the first detection frame is located outside the second detection frame, and there is an overlapping area with the third detection frame (as shown by c in Figure 15A).

[0280] In other embodiments, the positional relationship between the detection frame corresponding to the target object and the detection frame corresponding to the reference object may also be directly determined.

[0281] As shown in Figure 15B, in the case where the first boundary of the detection frame 71 (i.e., the first detection frame) corresponding to the cargo box A is located within the detection frame 74 (i.e., the second detection frame) corresponding to the carrier, it can be determined that the cargo box A does not protrude from the carrier and there is no collision risk. In the case where the first boundary of the detection frame 72 (i.e., the first detection frame) corresponding to the cargo box B is located outside the detection frame 74 corresponding to the carrier, and there is no overlapping area between the detection frame 72 corresponding to the cargo box B and the detection frame 75 (i.e., the third detection frame) corresponding to the anti-collision detection object, it can be determined that the cargo box B protrudes from the carrier and there is no collision risk. In the case where the first boundary of the detection frame 73 (i.e., the first detection frame) corresponding to the cargo box C is located outside the detection frame 74 corresponding to the carrier, and there is an overlapping area with the detection frame 75 corresponding to the anti-collision detection object, it can be determined that the cargo box C protrudes from the carrier and there is a collision risk.

[0282] When the protrusion information indicates that the target object protrudes from the carrier and there is no collision risk, the control device may control the workstation to continue working and generate warning information. The warning information may be configured to indicate to the staff that there is currently a cargo box that protrudes without collision risk.

[0283] If the salutation indicates that a target object is protruding from the vehicle and there is a risk of collision, the control device can suspend the workstation and generate an alarm message. This alarm message may include audible and / or visual signals, configured to indicate to the worker the presence of a protruding cargo container with a potential collision risk. After the worker has dealt with the protruding cargo container, the control device can receive a work instruction triggered by the worker and, in response, control the workstation to resume operation.

[0284] At least one target image further includes a second target image; the second target image is acquired based on a second direction among at least one acquisition direction, wherein the first direction and the second direction are different acquisition directions; when acquiring the second target image based on the second direction, the reference object includes a vehicle; the detection box corresponding to the target object includes a fourth detection box, and the detection box corresponding to the reference object includes a fifth detection box; wherein, detecting the target object and the reference object in at least one target image through a target detection model to obtain the detection box corresponding to the target object and the detection box corresponding to the reference object may include: detecting the target object and the reference object in the second target image through a target detection model to obtain the fourth detection box and the fifth detection box.

[0285] It should be noted that the acquisition direction in this embodiment can be one acquisition direction or at least two acquisition directions. At least two acquisition directions include a first direction and a second direction. The first direction and the second direction form a preset angle.

[0286] The second direction can be from left to right or from right to left, and the second target image is a side view of the target object and the reference object. That is, the target object and the reference object can be photographed from left to right or from right to left using an image acquisition device to obtain the second target image.

[0287] Based on the positional relationship between the detection frame corresponding to the target object and the detection frame corresponding to the reference object, the protrusion information of the target object relative to the reference object is determined, including: based on the positional relationship between the first detection frame and the second detection frame, the horizontal protrusion position of the target object relative to the vehicle is determined; based on the positional relationship between the fourth detection frame and the fifth detection frame, the vertical protrusion position of the target object relative to the reference object is determined; wherein, the protrusion information includes the horizontal protrusion position and the vertical protrusion position.

[0288] Based on the positional relationship between the first and second detection boxes in the first target image, the horizontal protrusion position of the target object relative to the vehicle can be determined, i.e., which column of the vehicle the target object is located in; based on the positional relationship between the fourth and fifth detection boxes, the vertical protrusion position of the target object relative to the reference object can be determined, i.e., which row of the vehicle the target object is located in.

[0289] As shown in Figure 15C, in the first target image shown in Figure 15Ca, it can be determined that the cargo box D is located in the second column from left to right of the vehicle; in the second target image shown in Figure 15Cb, it can be determined that the cargo box D is located in the second row from top to bottom of the vehicle.

[0290] If the second boundary of the fourth detection frame is outside the fifth detection frame, it can be determined that the target object protrudes from the vehicle.

[0291] The warning and / or alarm information generated by the control device can carry the horizontal and vertical protrusion positions of the target object relative to a reference object. This allows staff to quickly locate and handle protruding objects based on their horizontal and vertical protrusion positions, improving anomaly resolution efficiency and thus enhancing the overall operational efficiency of the warehousing system.

[0292] Understandably, combining image acquisition equipment with reference objects to perform salient detection of the target object can not only reduce installation accuracy but also improve the applicability of this method.

[0293] Figure 16 shows a schematic diagram of a target detection model training method. As shown in Figure 16, the target detection model training method provided in this embodiment includes S801-S803.

[0294] S801, Obtain sample data.

[0295] Sample data can be selected from historical data according to requirements, or it can be selected from a big data platform. This disclosure does not limit the method of obtaining sample data.

[0296] The sample data includes sample images, as well as ground truth bounding boxes corresponding to the target objects and reference objects in the sample images. Specifically, the ground truth bounding boxes for the target objects can be obtained by manually annotating them, and the ground truth bounding boxes for the reference objects can be obtained by manually annotating them.

[0297] S802. Based on the initial detection model, detect the target objects and reference objects in the sample image to obtain the predicted detection boxes corresponding to the target objects and the predicted detection boxes corresponding to the reference objects.

[0298] As shown in Figure 17, S802 may include S8021-S8023.

[0299] S8021. Perform convolution activation processing on the sample image using the initial detection model to obtain the processed image.

[0300] S8022. The processed image is segmented using the initial detection model to obtain multi-layer sample feature maps.

[0301] S8023. The multi-layer sample feature map is sampled and processed by the initial detection model to obtain the predicted detection box corresponding to the target object and the predicted detection box corresponding to the reference object.

[0302] It should be noted that S8021-S8023 are similar to S3021-S3023, and will not be described again here.

[0303] The initial detection model samples the multi-layer sample feature map to obtain the predicted detection box corresponding to the target object and the predicted detection box corresponding to the reference object. This can include downsampling the multi-layer sample feature map using the initial detection model to determine the features corresponding to the target object and the reference object from the multi-layer sample feature map; and upsampling the features corresponding to the target object and the reference object using the initial detection model to obtain the predicted detection box corresponding to the target object and the predicted detection box corresponding to the reference object.

[0304] S803. Using the predicted detection boxes corresponding to the target object and the predicted detection boxes corresponding to the reference object as the initial training output of the initial detection model, and the real detection boxes corresponding to the target object and the reference object as the supervision information, the initial detection model is iteratively trained to obtain the target detection model.

[0305] The initial detection model can be trained iteratively based on mean squared error or cross-entropy loss. The loss function can be selected according to actual needs for iterative training of the initial detection model. This disclosure does not limit the loss function used for iterative training.

[0306] It should be noted that the executing entity for S801-S803 can be the same as or different from the executing entity for S301-S303. For example, S301-S303 can be executed by the control device in the warehousing system 100, and S801-S803 can also be executed by the control device in the warehousing system 100. Alternatively, S301-S303 can be executed by the control device in the warehousing system 100, and S801-S803 can be executed by other control devices. This disclosure does not limit this aspect.

[0307] Figure 18a is a schematic diagram of another detection mechanism and reference mark provided in an embodiment of the present disclosure, and Figure 18b is a schematic diagram of another detection mechanism and reference mark provided in an embodiment of the present disclosure. Referring to Figures 18a and 18b, the detection mechanism 14 includes a lidar 142, and the reference mark 15 includes a scanning surface 152 formed by the lidar 142 above the operating surface 50 of the movable carrier 20. The lidar 142 is configured to emit a laser beam toward the detection area to form the scanning surface 152, and to determine whether at least a portion of the scanning surface 152 is obstructed by a protruding object on the movable carrier 20 based on the received laser beam.

[0308] For example, the lidar 142 can emit a series of parallel or angled laser beams that form a scanning surface 152 above the ground. When an object on the movable vehicle 20 protrudes and partially blocks the scanning surface 152, the corresponding laser beam is reflected back to the lidar 142 through the surface of the protruding object. By detecting and calculating the time difference between the emitted and received beams, the lidar 142 determines the degree and location of the obstruction of the scanning surface 152, thereby determining that an object protrudes from the movable vehicle 20 and the position of the protruding object in the height direction.

[0309] Similar to camera 141, lidar 142 can also compare detected occlusion information with preset thresholds or models to determine whether an alarm needs to be triggered or a preset processing procedure needs to be executed. This lidar 142-based detection method also offers advantages such as high precision, high efficiency, and low maintenance costs. Furthermore, lidar 142 is not limited by lighting conditions and can operate stably in various lighting environments, further improving the reliability and flexibility of item management in the warehouse system. In some advanced applications, the warehouse system can also combine the detection data from camera 141 and lidar 142 for multimodal information fusion to improve detection accuracy and robustness. For example, by using image information from camera 141 and distance information from lidar 142, three-dimensional reconstruction and precise positioning of protruding items can be achieved, providing stronger support for the automated and intelligent management of the warehouse system.

[0310] Referring to FIG18a, the lidar 142 can emit a laser beam along the length direction of the movable vehicle 20 (refer to the x direction in FIG18a) toward one side of the docking surface B of the movable vehicle 20 to form a vertical scanning surface 152 above the ground.

[0311] It is understood that the scanning angle of the lidar 142 can reach about 270°. Therefore, in this example, the lidar 142 can be set below the top of the movable carrier 20 and scan from one side to the other along the length of the movable carrier 20. The scanning surface 152 can cover each layer of the cargo bay 21a of the movable carrier 20.

[0312] By setting the lidar 142 below the top of the movable carrier 20 to emit a laser beam along the length of the movable carrier 20, the assembly stability of the lidar 142 can be improved, reducing or avoiding the situation where the lidar 142 is set too high and shakes, thereby affecting the position and angle of the scanning surface 152, and improving the detection accuracy of the lidar 142 for protruding objects.

[0313] Referring to FIG18b, the lidar 142 can be positioned above the top of the movable vehicle 20 and emits a laser beam from top to bottom along the height direction to form a scanning surface 152 on the docking surface B side of the movable vehicle 20.

[0314] For example, the larger the absolute value of the time difference between emitting and receiving the beam, the lower the position of the protruding object; the smaller the absolute value of the time difference between emitting and receiving the beam, the higher the position of the protruding object. This can accurately indicate the approximate height of the protruding object to the operator, making it easier to push the protruding object back accurately and quickly.

[0315] In other examples, the lidar 142 may be positioned below the bottom of the movable vehicle 20, for example, directly on the ground, and emit a laser beam from bottom to top to form a scanning surface 152 on the docking surface B side of the movable vehicle 20.

[0316] This embodiment does not limit the location of the lidar 142 or the direction of the emitted laser beam, as long as the formed scanning surface 152 is located on the docking surface B side of the movable carrier 20.

[0317] During the movement of the mobile carrier 20 toward the docking area 12, the scanning surface 152 is parallel to the surface where the cargo opening 21a of the mobile carrier 20 is located. This ensures that when the size of the protruding part of an item at any height of the mobile carrier 20 is greater than or equal to H1, it can block the scanning surface 152 and be detected by the lidar 142, thus avoiding the problem caused by the scanning surface 152 being tilted to the surface where the cargo opening 21a of the mobile carrier 20 is located.

[0318] For example, if the scanning surface 152 is parallel to the surface where the storage slot 21a of the movable carrier 20 is located, it can avoid the problem that an item protruding a short length can be detected by the lidar 142, and it can also avoid the problem that an item protruding a long length can not be detected by the lidar 142. That is, the above setting can improve the accuracy and reliability of item management in the warehousing system.

[0319] Figure 19 is a schematic diagram of another detection mechanism and reference mark provided in an embodiment of this disclosure. Referring to Figure 19, the detection mechanism 14 includes a signal transmitting component 143, and the reference mark 15 includes a plurality of signal receiving components 153, which are arranged along the length direction of the movable carrier 20. For example, when the movable carrier 20 moves toward the docking area 12 along the direction shown in s2, the plurality of signal receiving components 153 can be arranged along the direction shown in s2 on one side of the loading and unloading equipment 11, so that when the movable carrier 20 moves toward the docking area 12 along the direction shown in s2, the movable carrier 20 can pass by each signal receiving component 153.

[0320] The signal transmitting component 143 is configured to transmit a detection signal, and the signal receiving component 153 is configured to receive the detection signal. When at least one signal receiving component 153 fails to receive the detection signal, it is determined that the signal receiving component 153 is blocked by a protruding object on the movable vehicle 20.

[0321] For example, the signal transmitting component 143 can be a wireless signal transmitter, such as an radio frequency transmitter or an infrared transmitter, while the signal receiving component 153 can be a corresponding wireless signal receiver. When the detection signal emitted by the signal transmitting component 143 is blocked by a protruding object, causing at least one signal receiving component 153 to be unable to receive the signal, the storage system can determine that an object is protruding and take appropriate measures. This wireless signal-based detection method has the advantages of simple installation and high flexibility, and is suitable for various complex storage environments.

[0322] To improve the accuracy and stability of detection, the warehousing system can also employ a strategy that combines multiple detection methods. For example, multiple means such as camera 141, lidar 142, and wireless signal detection can be used simultaneously to monitor items on the mobile carrier 20 from all angles. By integrating multiple detection data, the warehousing system can achieve accurate identification, positioning, and tracking of protruding items, providing stronger technical support for the automation and intelligence of logistics operations.

[0323] Figure 20 is a schematic diagram of the structure of a detection mechanism and a positioning mechanism provided in an embodiment of this disclosure. Referring to Figure 20, a storage system, such as workstation 10, may further include a positioning mechanism 17, which is configured to form a positioning mark 19 on or above the operating surface 50. The detection mechanism 14 determines the installation position and angle based on the positioning mark 19 and the positioning mechanism 17.

[0324] The positioning mark 19 includes a visible mark 191 formed by the positioning mechanism 17 on the operating surface 50. For example, a reference mark 15 is set on the operating surface 50, such as the ground, and the positioning mechanism 17 forms a visible mark 191 on the operating surface 50. By controlling the relative position between the visible mark 191 and the reference mark 15, the installation position and installation angle of the detection mechanism 14 relative to the positioning mechanism 17 are determined.

[0325] For example, by adjusting the angle and position of the positioning mechanism 17 so that the visible mark 191 formed on the ground coincides with or is parallel to the reference mark 15, the operator can use the positioning mechanism 17 as an installation reference to quickly and accurately determine the specific installation position and angle of the detection mechanism 14, thereby ensuring that the detection signal of the detection mechanism 14 can accurately cover the reference mark 15 and effectively avoid missed detection or false detection.

[0326] In this example, the reference mark 15 may be a structure such as a reference line 151 set on the operating surface 50, or a scanning surface 152 formed by the lidar 142 above the operating surface 50.

[0327] When the visible mark 191 coincides with the reference mark 15, and the center of the visible mark 191 coincides with the center of the reference mark 15, the detection mechanism 14 can be set at a position that overlaps with the positioning mechanism 17 in the height direction. For example, the detection mechanism 14 can be set on top of the positioning mechanism 17, with the detection surface of the detection mechanism 14 parallel to the positioning surface of the positioning mechanism 17, and ensuring that the positioning mechanism 17 does not obstruct the detection surface of the detection mechanism 14.

[0328] Referring to FIG20, when the visible mark 191 is located on one side of the reference mark 15 along the y-direction, for example, the left side, and is parallel to the reference mark 15, the detection mechanism 14 can be set on the right side of the positioning mechanism 17 along the y-direction. The detection surface of the detection mechanism 14 is parallel to the positioning surface of the positioning mechanism 17 and can be at the same height as the positioning mechanism 17.

[0329] It should be noted that the detection surface of the detection mechanism 14 refers to the surface on which the detection mechanism 14 is configured to capture the reference mark 15, and the positioning surface of the positioning mechanism 17 refers to the emitting surface on which the positioning mechanism 17 is configured to form the visible mark 191.

[0330] Figure 21 is a second structural schematic diagram of the detection mechanism and positioning mechanism provided in an embodiment of this disclosure. Referring to Figure 21, the positioning mark 19 includes a positioning surface 192 formed by the positioning mechanism 17 above the running surface 50. The installation angle of the detection mechanism 14 is determined based on the angle between the positioning surface 192 and the running surface 50 and the installation angle of the positioning mechanism 17.

[0331] For example, by adjusting the installation angle of the positioning mechanism 17 so that the positioning surface 192 is perpendicular to the running surface 50, the operator can use the positioning mechanism 17 as an installation reference to quickly and accurately determine the installation angle of the detection mechanism 14 relative to the running surface 50, so that when the detection mechanism 14 is an image acquisition device (e.g., camera 141), the target plane C of the camera 141 is perpendicular to the reference mark 15.

[0332] Alternatively, when the detection mechanism 14 is a lidar 142, the scanning surface 152 formed by the lidar 142 can be perpendicular to the running surface 50, so that during the movement of the movable carrier 20 towards the docking area 12, the scanning surface 152 can be parallel to the surface where the cargo port 21a of the movable carrier 20 is located (i.e., the docking surface B).

[0333] It is understood that Figure 21 shows a calibration schematic of the positioning mechanism 17 in an example where the lidar 142 is positioned above the top of the movable vehicle 20 to emit a laser beam from top to bottom (see Figure 18b), i.e., in this example, the positioning mechanism 17, for example, emits a laser beam from top to bottom.

[0334] In an example where the lidar 142 is positioned below the top of the movable vehicle 20 to emit a laser beam along the length of the movable vehicle 20, the positioning mechanism 17, for example, a laser emits a laser beam (not shown) along the length of the movable vehicle 20 (i.e., the x-direction).

[0335] The positioning mechanism 17 can also be integrated with the control system of the warehousing system to achieve automated installation position calibration and angle adjustment. Thus, during the operation of the warehousing system, even if the position or angle of the detection mechanism 14 deviates slightly due to changes in the external environment or equipment wear, the control system can promptly detect and automatically adjust the positioning mechanism 17, ensuring that the detection mechanism 14 remains in optimal working condition. This intelligent adjustment mechanism not only improves the operational efficiency and accuracy of the warehousing system but also reduces the frequency and difficulty of manual intervention, providing strong support for the automation and intelligence of warehousing and logistics.

[0336] The positioning mechanism 17 may include visual indicators such as LED lights or lasers. For example, the positioning mechanism 17 includes a laser configured to project multiple laser dots or laser lines onto the operating surface 50, and the detection mechanism 14 determines the installation position or angle based on the positional relationship between the laser dots or laser lines and the reference mark 15. The laser dots or laser lines are configured as visual marks 191.

[0337] The laser can also be configured to form a laser scanning surface above the operating surface 50, which is configured as the positioning surface 192. In this way, the installation angle of the detection mechanism 14 can be determined based on the angle between the laser scanning surface and the operating surface 50 and the installation angle of the positioning mechanism 17. By setting the positioning mechanism 17 as a laser, the position and direction of the laser point or laser line, or the position and angle of the laser scanning surface, can be precisely controlled, thereby further improving the accuracy of the installation position and angle of the detection mechanism 14.

[0338] In addition, laser pointers offer advantages such as good directionality, high brightness, and good monochromaticity, clearly displaying the visible mark 191 even in complex lighting environments, ensuring operators can accurately install the device according to the mark 191. Furthermore, the laser's long projection distance meets the needs of long-distance positioning of the detection mechanism 14 in large-scale warehousing systems. The laser can also be equipped with adjustable spot size and brightness to adapt to different working environments and installation requirements.

[0339] When the detection mechanism 14 is a lidar 142, the positioning mechanism 17 is not required to calibrate the position. The lidar 142 directly emits a laser beam. By observing the position and angle of the laser beam, the installation height and angle of the lidar 142 are adjusted until the scanning surface 152 formed by the laser beam is located on the docking surface B of the movable carrier 20 and can completely cover the cargo positions 21a of each layer of the movable carrier 20.

[0340] Referring to Figures 1A and 20, the storage system, such as workstation 10, may also include a mounting frame 18, on which the detection mechanism 14 is mounted.

[0341] The mounting bracket 18 can be fixed to the frame of equipment such as loading and unloading equipment 11 at the workstation 10, so that the detection mechanism 14 is mounted on the loading and unloading equipment 11. For example, when the movable carrier 20 moves toward the docking area 12 in the direction shown in s2, the mounting bracket 18 can be mounted on the frame of the loading and unloading equipment 11, and the mounting bracket 18 can extend out of the frame in a first direction, so that the detection mechanism 14 is mounted on one side of the loading and unloading equipment 11 in the first direction.

[0342] The mounting bracket 18 is detachably mounted on the running surface 50 of the handling equipment 30. In this way, the inspection mechanism 14 can be flexibly arranged in the workstation 10 or other areas, thereby adapting to different movement paths and storage system layouts and improving the flexibility of use of the inspection mechanism 14.

[0343] Figure 22 is a schematic diagram of another storage system provided in an embodiment of this disclosure. Referring to Figure 22, a limiting mechanism 60 is provided on the movement path of the movable carrier 20 to the receiving area 12. When the detection mechanism 14 detects that an item protrudes from the movable carrier 20, the limiting mechanism 60 can restrict the item outside the working plane A to prevent the item from interfering with the picking and placing mechanism 111 in the working plane A.

[0344] Understandably, in this example, the limiting mechanism 60 is the same as the anti-protrusion mechanism mentioned above.

[0345] The limiting mechanism 60 can be a protrusion or a column disposed on the moving path of the movable carrier 20. It is understood that the distance between the protrusion or column and the working plane A is within the range of the distance between the pick-and-place mechanism 111 and the movable carrier 20, and the distance for transferring the target item. For example, referring to FIG22, in this embodiment of the present disclosure, the limiting mechanism 60 can be disposed in the area between the docking surface B and the working plane A. Furthermore, the side of the limiting mechanism 60 facing the movable carrier 20 protrudes from the working plane. Thus, when the movable carrier 20 moves toward the loading / unloading device 11, the limiting mechanism 60 can contact the target item on the movable carrier 20, thereby limiting the target item outside the working plane.

[0346] It is understood that the distance between the movable carrier 20 and the working surface can typically be controlled by the handling equipment 30. For example, location markers (usually QR codes in some examples) can be set on the movement path of the handling equipment 30 (usually the ground). The handling equipment 30 adjusts the distance between the movable carrier 20 and the working surface by recognizing the location markers, so that the distance between the movable carrier 20 and the working surface is within the picking distance range of the picking and placing mechanism 111, and does not affect the movement of the picking and placing mechanism 111 within the working surface.

[0347] It is understood that the limiting mechanism 60 can also be set in an empty position in the warehouse, such as at the edge of the warehouse; the handling equipment 30 can first move the movable carrier 20 to the limiting mechanism 60, and push the target item (such as a material box) protruding from the movable carrier 20 into the movable carrier 20 through the limiting mechanism 60, and then move the movable carrier 20 to the receiving area 12 through the handling equipment 30, and carry out the handling and transfer of the target item on the movable carrier 20 in the receiving area 12.

[0348] Alternatively, the limiting mechanism 60 can also be located on one side of the loading and unloading equipment 11, for example, on the outside of the working plane of the loading and unloading equipment 11, as shown in FIG22. For example, when the handling equipment 30 moves the movable carrier 20 along the path shown by the hollow arrow in FIG3, the limiting mechanism 60 located on the outside of the loading and unloading equipment 11 contacts the target item on the movable carrier 20, thereby limiting the target item outside the working plane.

[0349] Additionally, in this embodiment, the other limiting mechanism 60 can also contact the side of the movable carrier 20 facing the working plane. That is, the limiting mechanism 60 can also confine the movable carrier 20 outside the working plane.

[0350] This disclosure also provides a target object protrusion detection device for implementing the target object protrusion detection method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more control module embodiments provided below can be found in the limitations of the target object protrusion detection method described above, and will not be repeated here. Figure 23 is a schematic diagram of the structure of a target object protrusion detection device according to an embodiment of this disclosure. As shown in Figure 23, the target object protrusion detection device 1000 includes:

[0351] The first acquisition module 1010 is configured to determine at least one target image; wherein the at least one target image is acquired based on at least one acquisition direction, and the target image includes a target object and a reference object;

[0352] The first detection module 1020 is configured to detect the target object and the reference object in the at least one target image, and obtain a detection box corresponding to the target object and a detection box corresponding to the reference object.

[0353] The determination module 1030 is configured to determine the prominence information of the target object relative to the reference object based on the positional relationship between the detection frame corresponding to the target object and the detection frame corresponding to the reference object.

[0354] At least one target image includes a first target image; the first target image is acquired based on a first direction among at least one acquisition direction; when the first target image is acquired based on the first direction, the reference objects include a vehicle and a collision detection object; the detection box corresponding to the target object includes a first detection box, and the detection box corresponding to the reference object includes a second detection box corresponding to the vehicle and a third detection box corresponding to the collision detection object; the first detection module 1020 is configured to: detect the target object and the reference object in the first target image through a target detection model to obtain a first detection box, a second detection box, and a third detection box.

[0355] The determination module 1030 is further configured to determine that the protrusion information is a target object protruding from the vehicle and there is no collision risk when the first boundary of the first detection frame is outside the second detection frame and there is no overlapping area between the first detection frame and the third detection frame; and to determine that the protrusion information is a target object protruding from the vehicle and there is a collision risk when the first boundary of the first detection frame is outside the second detection frame and there is an overlapping area between the first detection frame and the third detection frame.

[0356] At least one target image further includes a second target image; the second target image is acquired based on a second direction in at least one acquisition direction, wherein the first direction and the second direction are different acquisition directions; when the second target image is acquired based on the second direction, the reference object includes a vehicle; the detection box corresponding to the target object includes a fourth detection box, and the detection box corresponding to the reference object includes a fifth detection box; the first detection module 1020 is further configured to: detect the target object and the reference object in the second target image through a target detection model to obtain the fourth detection box and the fifth detection box.

[0357] The determining module 1030 is further configured to determine the horizontal protrusion position of the target object relative to the carrier based on the positional relationship between the first detection frame and the second detection frame; and to determine the vertical protrusion position of the target object relative to the reference object based on the positional relationship between the fourth detection frame and the fifth detection frame; wherein the protrusion information includes the horizontal protrusion position and the vertical protrusion position.

[0358] The first detection module 1020 is further configured to perform convolutional activation processing on at least one target image through a target detection model to obtain a first image; to perform segmentation processing on the first image through a target detection model to obtain a multi-layer feature map; and to perform sampling processing on the multi-layer feature map through a target detection model to obtain a detection box corresponding to the target object and a detection box corresponding to the reference object.

[0359] The target object protrusion detection device 1010 also includes: a processing module;

[0360] The first acquisition module 1010 is also configured to acquire the initial image acquired by the image acquisition device;

[0361] The processing module is also configured to perform distortion correction processing on the initial image based on the intrinsic parameter matrix and distortion parameters corresponding to the image acquisition device, so as to obtain the target image.

[0362] The processing module is also configured to convert the pixel coordinates of multiple pixels to be processed in the image to be processed into multiple normalized coordinates based on the intrinsic parameter matrix; wherein, the image to be processed refers to a blank image with the same resolution as the initial image; based on multiple normalized coordinates, distortion parameters and preset distortion formulas, multiple distortion coordinates are determined; the multiple distortion coordinates are mapped to the initial image to obtain multiple target coordinates; and the pixel values ​​at the target coordinates in the initial image are assigned to the multiple pixels to be processed through interpolation to obtain the target image.

[0363] For specific limitations and beneficial effects of the target object protrusion detection device, please refer to the limitations of the target object protrusion detection method mentioned above, which will not be repeated here.

[0364] This disclosure also provides a target detection model training device for implementing the target detection model training method described above. The solution provided by this device is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more control module embodiments provided below can be found in the limitations of the target detection model training method described above, and will not be repeated here. Figure 24 is a schematic diagram of the structure of a target detection model training device according to an embodiment of this disclosure. As shown in Figure 24, the target detection model training device 1100 includes:

[0365] The second acquisition module 1110 is configured to acquire sample data; wherein, the sample data includes sample images, and the true detection boxes corresponding to the sample target objects and the true detection boxes corresponding to the sample reference objects in the sample images;

[0366] The second detection module 1120 is configured to detect the sample target object and the sample reference object in the sample image through the initial detection model, and obtain the predicted detection box corresponding to the sample target object and the predicted detection box corresponding to the sample reference object.

[0367] The iterative training module 1130 is configured to use the predicted detection boxes corresponding to the sample target object and the predicted detection boxes corresponding to the sample reference object as the initial training output of the initial detection model, and the real detection boxes corresponding to the sample target object and the real detection boxes corresponding to the sample reference object as supervision information to iteratively train the initial detection model and obtain the target detection model.

[0368] The second detection module 1120 is further configured to perform convolution activation processing on the sample image through the initial detection model to obtain the processed image; to perform segmentation processing on the processed image through the initial detection model to obtain multi-layer sample feature maps; and to perform sampling processing on the multi-layer sample feature maps through the initial detection model to obtain the predicted detection box corresponding to the sample target object and the predicted detection box corresponding to the sample reference object.

[0369] The second detection module 1120 is also configured to perform downsampling processing on the multi-layer sample feature map through the initial detection model, determine the features corresponding to the sample target object and the features corresponding to the sample reference object from the multi-layer sample feature map; and perform upsampling processing on the features corresponding to the sample target object and the features corresponding to the sample reference object through the initial detection model to obtain the predicted detection box corresponding to the sample target object and the predicted detection box corresponding to the sample reference object.

[0370] For specific limitations on the training device for the object detection model and the beneficial effects it can achieve, please refer to the limitations on the training method for the object detection model mentioned above, which will not be repeated here.

[0371] Figure 25 shows a schematic diagram of the internal structure of a robot according to an embodiment of this disclosure. This robot, also known as a mobile device, includes a processor, memory, communication interface, display screen, and input device connected via a system bus. The robot's processor is configured to provide computing and control capabilities. The robot's memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The robot's communication interface is configured to communicate with external terminals via wired or wireless means; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a target object protrusion detection method. The robot's display screen can be an electronic display screen, an LCD screen, or an e-ink display screen. The robot's input device can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the robot's shell, or an external keyboard, touchpad, or mouse, etc.

[0372] Figure 26 shows a schematic diagram of the internal structure of a control device according to an embodiment of this disclosure. This control device can also be called an electronic device, which may be a server. The electronic device includes a processor, a memory, and a network interface connected via a system bus. The processor of the electronic device is configured to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the electronic device is configured to store height parameters and 3D map data. The network interface of the electronic device is configured to communicate with an external terminal via a network connection. When the computer program is executed by the processor, it implements a target protrusion detection method or a target detection model training method.

[0373] Those skilled in the art will understand that the structures shown in Figures 25 and 26 are merely block diagrams of some structures related to the present disclosure and do not constitute a limitation on the robots and electronic devices to which the present disclosure is applied. Specific robots and electronic devices may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements.

[0374] In a specific implementation, this disclosure provides an electronic device, including a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, it implements the steps of the method in any of the above embodiments.

[0375] In a specific implementation, this disclosure provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method in any of the above embodiments.

[0376] In a specific implementation, this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method in any of the above embodiments.

[0377] In a specific implementation, this disclosure provides a computer program that, when executed by a processor, implements the steps of the method in any of the above embodiments.

[0378] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties. Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this disclosure can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in this disclosure may include at least one of relational databases and non-relational databases. Non-relational databases may include, but are not limited to, blockchain-based distributed databases, etc. The processors involved in the embodiments provided in this disclosure may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logic devices based on quantitative computation, etc., etc., and are not limited to these.

[0379] It is readily understood that those skilled in the art can combine, split, or reorganize the embodiments provided in this disclosure to obtain other embodiments, none of which exceed the protection scope of this disclosure.

[0380] The above detailed embodiments further illustrate the purpose, technical solutions, and beneficial effects of the embodiments of this disclosure. It should be understood that the above are merely specific embodiments of the embodiments of this disclosure and are not intended to limit the protection scope of the embodiments of this disclosure. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solutions of the embodiments of this disclosure should be included within the protection scope of the embodiments of this disclosure.

Claims

1. A warehousing system, comprising: Reference mark (15); A mobile vehicle (20) is configured to carry items; A handling device (30) is configured to handle the movable vehicle (20) to transport the movable vehicle (20) to a target area (70); The detection mechanism (14) is configured to determine, based on a reference marker (15), whether an item on the movable vehicle (20) protrudes from the movable vehicle (20), or to determine whether there is a deviation in the pose of the movable vehicle (20).

2. The warehousing system according to claim 1, wherein, The reference mark (15) is located on one side of the moving path of the movable vehicle (20) towards the target area (70), the reference mark (15) is located within the detection area of ​​the detection mechanism (14), and the cargo port (21a) of the movable vehicle (20) faces the detection area during the movement of the movable vehicle (20) towards the target area (70); The detection mechanism (14) is configured to detect whether at least a portion of the reference mark (15) is obscured in order to determine whether an item on the movable vehicle (20) protrudes from the movable vehicle (20) or to determine whether there is a deviation in the pose of the movable vehicle (20).

3. The warehousing system according to claim 2, wherein, The detection mechanism (14) is higher than or level with the top of the top shelf on the target mobile carrier (20), and the reference mark (15) is lower than or level with the bottom of the lowest shelf on the target mobile carrier (20); wherein the target mobile carrier (20) is the mobile carrier (20) located on the handling equipment (30).

4. The warehousing system according to claim 2, wherein, The detection mechanism (14) includes an image acquisition device, and the reference mark (15) is located on the running surface (50) of the movable vehicle (20); The image acquisition device is configured to acquire image information of the reference mark (15) to determine whether the reference mark (15) is occluded based on the image information.

5. The warehousing system according to claim 4, wherein, The image acquisition device includes a camera (141), and the reference mark (15) includes a reference line (151); the image information of the reference mark (15) includes the image information of the reference line (151), and the camera (141) is configured to acquire the image information of the reference line (151) to determine whether the reference line (151) is occluded based on the degree of missing image information.

6. The warehousing system according to claim 4, wherein, The target plane (C) of the image acquisition device is perpendicular to the reference mark (15).

7. The warehousing system according to claim 4, wherein, The reference mark (15) is a reflective element affixed to the running surface (50).

8. The warehousing system according to claim 2, wherein, The detection mechanism (14) includes a lidar (142), and the reference mark (15) includes a scanning surface (152) formed by the lidar (142) above the running surface (50) of the movable vehicle (20); The lidar (142) is configured to emit a laser beam toward the detection area to form the scanning surface (152), and to determine whether at least a portion of the scanning surface (152) is obstructed based on the received laser beam; During the movement of the movable vehicle (20) toward the target area (12), the scanning surface (152) is parallel to the surface where the cargo bay (21a) of the movable vehicle (20) is located.

9. The warehousing system according to claim 2, wherein, The detection mechanism (14) includes a signal transmitting component (143), and the reference mark (15) includes a plurality of signal receiving components (153), which are arranged along the length of the movable vehicle (20). The signal transmitting component (143) is configured to transmit a detection signal, and the signal receiving component (153) is configured to receive the detection signal. When at least one of the signal receiving components (153) fails to receive the detection signal, it is determined that the signal receiving component (153) is blocked.

10. The warehousing system according to claim 2, wherein, The warehousing system also includes a docking device (40), the target area (70) includes a docking area (12) of the docking device (40), and the mobile vehicle (20) is configured to dock with the docking device (40) in the docking area (12); The reference mark (15) is located on the side of the docking device (40) facing the docking area (12), and the distance between the reference mark (15) and the docking device (40) in the direction of the docking device (40) taking and returning objects is greater than or equal to zero.

11. The warehousing system according to claim 10, wherein, The reference mark (15) is located on one side of the docking device (40) along a first direction; wherein the first direction is parallel to the running surface (50) of the movable carrier (20) and intersects with the object retrieval direction of the docking device (40); During the process of the movable vehicle (20) moving along the first direction to the docking area (12) via the reference mark (15), the distance between the movable vehicle (20) and the reference mark (15) along the retrieval direction is less than or equal to the distance between the movable vehicle (20) and the docking device (40).

12. The warehousing system according to claim 10, wherein the reference mark (15) is parallel to the working plane (A) of the docking device (40); wherein, The working plane (A) is the surface of the docking device (40) facing the docking area (12).

13. The warehousing system according to any one of claims 1-12, wherein, It also includes a positioning mechanism (17) configured to send a positioning mark (19) to the operating surface (50) of the movable vehicle (20) or above the operating surface (50), and the detection mechanism (14) determines the installation position and angle based on the positioning mark (19) and the positioning mechanism (17).

14. The warehousing system according to claim 13, wherein, The positioning mark (19) includes a visible mark (191) formed by the positioning mechanism (17) on the running surface (50). By controlling the relative position between the visible mark (191) and the reference mark (15), the installation position and installation angle of the detection mechanism (14) relative to the positioning mechanism (17) are determined.

15. The warehousing system according to claim 14, wherein, The positioning mechanism (17) includes a laser configured to project laser dots or laser lines onto the running surface (50), wherein the laser dots or laser lines are configured as visible marks (191) in the positioning marks (19).

16. The warehousing system according to any one of claims 1-12, 14, and 15, wherein, The target area (70) includes the docking area (12) of the loading and unloading equipment (11) in the workstation, which is configured to remove items from the mobile vehicle (20) or place items in the mobile vehicle (20). The docking equipment (40) of the warehousing system includes the loading and unloading equipment (11).

17. A method for detecting protruding objects, applied to the warehousing system according to any one of claims 1-16, the method comprising: Acquire at least one target image; wherein the at least one target image is acquired based on at least one acquisition direction, and the target image includes a target object and a reference object; The target object and the reference object in the at least one target image are detected by the target detection model to obtain the detection box corresponding to the target object and the detection box corresponding to the reference object; Based on the positional relationship between the detection box corresponding to the target object and the detection box corresponding to the reference object, the prominence information of the target object relative to the reference object is determined.

18. The target object protrusion detection method according to claim 17, wherein, The at least one target image includes a first target image; the first target image is acquired based on a first direction among the at least one acquisition direction; when the first target image is acquired based on the first direction, the reference object includes a vehicle and a collision detection object; the detection frame corresponding to the target object includes a first detection frame, and the detection frame corresponding to the reference object includes a second detection frame corresponding to the vehicle and a third detection frame corresponding to the collision detection object; The step of detecting the target object and the reference object in the at least one target image using a target detection model to obtain the detection box corresponding to the target object and the detection box corresponding to the reference object includes: The target object and the reference object in the first target image are detected by the target detection model to obtain the first detection box, the second detection box and the third detection box.

19. The target object protrusion detection method according to claim 17, wherein, The step of determining the prominence information of the target object relative to the reference object based on the positional relationship between the detection frame corresponding to the target object and the detection frame corresponding to the reference object includes: If the first boundary of the first detection frame is outside the second detection frame and there is no overlapping area between the first detection frame and the third detection frame, the protrusion information is determined to be that the target object protrudes from the vehicle and there is no risk of collision. If the first boundary of the first detection frame is outside the second detection frame and there is an overlapping area between it and the third detection frame, the protrusion information is determined to be that the target object protrudes from the vehicle and there is a risk of collision.

20. The target object protrusion detection method according to claim 18 or 19, wherein, The at least one target image further includes a second target image; the second target image is acquired based on a second direction among the at least one acquisition direction, wherein the first direction and the second direction are different acquisition directions; when the second target image is acquired based on the second direction, the reference object includes the vehicle; the detection box corresponding to the target object includes a fourth detection box, and the detection box corresponding to the reference object includes a fifth detection box; The step of detecting the target object and the reference object in the at least one target image using a target detection model to obtain the detection box corresponding to the target object and the detection box corresponding to the reference object includes: The target object and the reference object in the second target image are detected by the target detection model to obtain the fourth detection box and the fifth detection box.

21. The target object protrusion detection method according to claim 20, wherein, The step of determining the prominence information of the target object relative to the reference object based on the positional relationship between the detection frame corresponding to the target object and the detection frame corresponding to the reference object includes: Based on the positional relationship between the first detection frame and the second detection frame, the horizontal protrusion position of the target object relative to the vehicle is determined; Based on the positional relationship between the fourth and fifth detection frames, the vertical protrusion position of the target object relative to the reference object is determined; wherein, the protrusion information includes the horizontal protrusion position and the vertical protrusion position.

22. The target object protrusion detection method according to claim 17, wherein, The step of detecting the target object and the reference object in the at least one target image using a target detection model to obtain the detection box corresponding to the target object and the detection box corresponding to the reference object includes: The target detection model is used to perform convolutional activation processing on the at least one target image to obtain a first image; The first image is segmented using the target detection model to obtain a multi-layer feature map; The target detection model samples the multi-layer feature map to obtain the detection box corresponding to the target object and the detection box corresponding to the reference object.

23. The target object protrusion detection method according to claim 17, wherein, The target object protrusion detection method also includes: Acquire the initial image captured by the image acquisition device; Based on the intrinsic parameter matrix and distortion parameters corresponding to the image acquisition device, the initial image is subjected to distortion removal processing to obtain the target image.

24. The target object protrusion detection method according to claim 23, wherein, The step of performing distortion correction processing on the initial image based on the intrinsic parameter matrix and distortion parameters corresponding to the image acquisition device to obtain the target image includes: Based on the intrinsic parameter matrix, the pixel coordinates of multiple pixels to be processed in the image to be processed are converted into multiple normalized coordinates; wherein, the image to be processed refers to a blank image with the same resolution as the initial image; Based on the multiple normalized coordinates, the distortion parameters, and the preset distortion formula, multiple distortion coordinates are determined; The multiple distorted coordinates are mapped onto the initial image to obtain multiple target coordinates; The pixel value at the target coordinate in the initial image is assigned to the plurality of pixels to be processed by interpolation to obtain the target image.

25. A workstation, comprising: Reference mark (15); Loading and unloading equipment (11) having a docking area (12) on one side, the loading and unloading equipment (11) being configured to dock with a mobile vehicle (20) located in the docking area (12) to transfer articles between the mobile vehicle (20) and the loading and unloading equipment (11); The detection mechanism (14) is configured to determine, based on the reference mark (15), whether an item on the movable vehicle (20) protrudes from the movable vehicle (20), or whether the pose of the movable vehicle (20) is skewed.

26. The workstation according to claim 25, wherein, The reference mark (15) is located on one side of the moving path of the movable vehicle (20) towards the docking area (12), and the reference mark (15) is located within the detection area of ​​the detection mechanism (14); during the movement of the movable vehicle (20) towards the docking area (12), the cargo port (21a) of the movable vehicle (20) faces the detection area; The detection mechanism (14) is configured to detect whether at least a portion of the reference mark (15) is obscured in order to determine whether an item on the movable vehicle (20) protrudes from the movable vehicle (20) or to determine whether the pose of the movable vehicle (20) is skewed.

27. The workstation according to claim 26, wherein, The reference mark (15) is located on the side of the loading and unloading equipment (11) facing the docking area (12), and the distance between the reference mark (15) and the loading and unloading equipment (11) in the direction of picking up and returning the goods is greater than or equal to zero.

28. The workstation according to claim 27, wherein, The reference mark (15) is located on one side of the loading and unloading equipment (11) along the first direction; During the process of the movable vehicle (20) moving along the first direction past the reference mark (15) to the docking area (12), the distance between the movable vehicle (20) and the reference mark (15) is less than the distance between the reference mark (15) and the loading and unloading equipment (11); The first direction is parallel to the running surface (50) of the movable vehicle (20) and intersects with the loading and unloading equipment (11) in the direction of picking up and returning the goods.

29. The workstation according to claim 25, wherein, The detection mechanism (14) includes an image acquisition device, and the reference mark (15) is located on the running surface (50) of the movable vehicle (20). The image acquisition device is configured to acquire image information of the reference mark (15) and determine, based on the image information, whether the reference mark (15) is obscured by a protruding object on the movable vehicle (20).

30. The workstation according to claim 25, wherein, The detection mechanism (14) includes a lidar (142), and the reference mark (15) includes a scanning surface (152) formed by the lidar (142) above the running surface (50) of the movable vehicle (20); The lidar (142) is configured to emit a laser beam toward the detection area to form the scanning surface (152), and to determine, based on the received laser beam, whether at least a portion of the scanning surface (152) is obstructed by an object protruding from the movable vehicle (20). During the movement of the movable vehicle (20) toward the docking area (12), the scanning surface (152) is parallel to the surface where the cargo bay (21a) of the movable vehicle (20) is located.

31. The workstation according to claim 25, wherein, The detection mechanism (14) includes a signal transmitting component (143), and the reference mark (15) includes a plurality of signal receiving components (153), which are arranged along the length of the movable vehicle (20). The signal transmitting component (143) is configured to transmit a detection signal, the signal receiving component (153) is configured to receive the detection signal, and when at least one of the signal receiving components (153) fails to receive the detection signal, it is determined that the signal receiving component (153) is blocked by a protruding object on the movable vehicle (20).

Images