Method and device for confirming material cage stacking, computer device and storage medium

Abstract

The application relates to a method and device for confirming stacking of material cages, a computer device and a storage medium. The method comprises the following steps: before a first material cage is stacked on a second material cage, the stacking devices of the first material cage and the second material cage are photographed respectively, so that corresponding material cage images are obtained. Then, two detection models are used to identify the stacking devices of the two material cages in the material cage images. The first detection model is used to position the stacking devices of the two material cages, and a first stacking result is obtained. The second detection model is used to directly output a second stacking result corresponding to the stacking devices of the two material cages. In this way, the stacking results obtained by the two detection models in different ways can be combined to obtain a final stacking result. Therefore, the two detection models can quickly and accurately determine whether the material cages can be stacked without manual confirmation, and the efficiency of confirming the stacking of the material cages is greatly improved.

Description

Technical Field

[0001] This application relates to the field of warehousing and logistics technology, and in particular to a method, apparatus, computer equipment, and storage medium for confirming the stacking of material cages. Background Technology

[0002] With the development of warehousing and logistics technology, cages are often used to transport and store goods during warehousing operations. Stacking cages can achieve three-dimensional warehousing and reduce the space occupied in the warehouse.

[0003] In traditional warehousing scenarios, whenever a driver uses a forklift to stack cages, they need to stick their head out of the forklift to check the status of the cages and determine if they can be stacked. However, this manual operation by the driver to confirm the stacking each time reduces the efficiency of cage stacking. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, computer equipment, and storage medium for confirming the stacking of material cages to address the aforementioned technical problems.

[0005] A method for confirming cage stacking, the method comprising:

[0006] Before stacking the first cage onto the second cage, images of the stacking devices for the first and second cages are acquired to obtain corresponding cage images. A first detection model is used to perform first target detection on the cage images to identify the first stacking device corresponding to the first cage and the second stacking device corresponding to the second cage. First position information of the first stacking device and second position information of the second stacking device are determined, and a first stacking result is determined based on the first and second position information. A second detection model is used to perform second target detection on the cage images, and a second stacking result is directly output based on the image features extracted by the second target detection. A final stacking result is determined based on the first and second stacking results. The final stacking result includes a result that triggers the stacking of the first cage onto the second cage and a result that prevents the stacking of the first cage onto the second cage.

[0007] In one embodiment, the cage image includes a first cage image and a second cage image; before stacking the first cage onto the second cage, image acquisition is performed on the stacking devices of the first and second cages respectively to obtain corresponding cage images, including:

[0008] Before stacking the first cage onto the second cage, an image of the first cage is obtained by photographing the stacking device in the first and second cages in a first position, and an image of the second cage is obtained by photographing the stacking device in the first and second cages in a second position; wherein the first position and the second position are two different positions.

[0009] In one embodiment, the stacking device includes a foot cup and a foot block that mates with the foot cup. Before stacking the first cage onto the second cage, the stacking device at a first position in the first and second cages is photographed to obtain an image of the first cage, and the stacking device at a second position in the first and second cages is photographed to obtain an image of the second cage. This includes:

[0010] Before stacking the first cage on top of the second cage, images of the first cage's foot cup in a first position and the second cage's block in a first position are taken to obtain an image of the first cage; images of the first cage's foot cup in a second position and the second cage's block in a second position are taken to obtain an image of the second cage.

[0011] In one embodiment, a first target detection is performed on the cage image using a first detection model to identify a first stacking device corresponding to the first cage and a second stacking device corresponding to the second cage from the cage image, including:

[0012] The first detection model is used to obtain the feature information of the cage image, and the first stacking device corresponding to the first cage and the second stacking device corresponding to the second cage are identified based on the feature information.

[0013] In one embodiment, a first foot cup is provided on a first surface of the first cage, and a first block is provided on a second surface; a second foot cup is provided on a first surface of the second cage, and a second block is provided on a second surface; determining a first stacking result based on first position information of the first stacking device and second position information of the second stacking device, and determining a first stacking result based on the first position information and the second position information, includes:

[0014] The first position information corresponding to the first foot cup in the first stacking device and the second position information corresponding to the second block in the second stacking device are determined; based on the first position information and the second position information, the distance between the first foot cup and the second block in each stacking combination is obtained; wherein the first foot cup and the second block in the stacking combination are located in the same vertical direction; the distance between the first foot cup and the second block in each stacking combination is compared with a distance threshold to obtain the comparison result of each stacking combination; the first stacking result is determined based on the comparison result of each stacking combination.

[0015] In one embodiment, before performing second target detection on the cage image using a second detection model and directly outputting the second stacking result based on the image features extracted by the second target detection, the method further includes:

[0016] Obtain cage image samples; the cage image samples include first cage sample images corresponding to two cages in a stackable state, and second cage sample images corresponding to two cages in a non-stackable state; train a second detection model using the cage image samples and the labels corresponding to whether the cage images are in a stackable state, and obtain the trained second detection model.

[0017] In one embodiment, determining the final stacking result based on the first stacking result and the second stacking result includes:

[0018] By using a stacking verification model, the first stacking result and the second stacking result are weighted and summed to obtain stacking data; the stacking data is compared with the stacking threshold, and the final stacking result is obtained based on the comparison result.

[0019] A device for confirming the stacking of material cages, the device comprising:

[0020] The acquisition module is used to acquire images of the stacking devices of the first and second cages respectively before stacking the first cage on the second cage, and obtain corresponding cage images.

[0021] The first processing module is used to perform first target detection on the image of the material cage using a first detection model, and to identify the first stacking device corresponding to the first material cage and the second stacking device corresponding to the second material cage from the image of the material cage.

[0022] The first determining module is used to determine the first stacking result based on the first position information of the first stacking device and the second position information of the second stacking device;

[0023] The second processing module is used to perform second target detection on the cage image through the second detection model, and directly output the second stacking result based on the image features extracted by the second target detection.

[0024] The second determining module is used to determine a final stacking result based on the first stacking result and the second stacking result; the final stacking result includes a result that triggers the stacking of the first cage onto the second cage, and a result that triggers the prevention of stacking the first cage onto the second cage.

[0025] A computer device includes a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement a method for confirming cage stacking as described above.

[0026] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for confirming cage stacking as described above.

[0027] The aforementioned method, apparatus, computer equipment, and storage medium for confirming cage stacking capture images of the stacking devices of both cages before stacking the first cage on top of the second cage. This allows for the identification of the stacking devices of the two cages in the images using two separate detection models. The first detection model locates the positions of the stacking devices to obtain a first stacking result, while the second detection model directly outputs a second stacking result corresponding to the stacking devices. This allows for the synthesis of stacking results obtained from the two detection models using different methods to arrive at a final stacking result. Thus, manual confirmation of whether the two cages can be stacked is unnecessary; the two detection models can quickly and accurately determine this, significantly improving the efficiency of cage stacking confirmation. Attached Figure Description

[0028] Figure 1 This is a diagram illustrating the application environment of a method for verifying cage stacking in one embodiment.

[0029] Figure 2 This is a flowchart illustrating a method for confirming cage stacking in one embodiment;

[0030] Figure 3 This is a schematic diagram of the cage structure in one embodiment;

[0031] Figure 4 This is a schematic diagram of the structure of an unmanned warehouse forklift in one embodiment;

[0032] Figure 5 This is a flowchart illustrating the steps for obtaining a cage image in one embodiment;

[0033] Figure 6 This is a flowchart illustrating the step of determining the first stacking result in one embodiment;

[0034] Figure 7 This is a flowchart illustrating the step of obtaining the second detection model in one embodiment;

[0035] Figure 8 This is a flowchart illustrating the steps for obtaining the final stacking result in one embodiment;

[0036] Figure 9 A structural block diagram of a device for confirming the stacking of material cages in one embodiment;

[0037] Figure 10This is a structural block diagram of a device for confirming the stacking of material cages in another embodiment;

[0038] Figure 11 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0040] The method for verifying cage stacking provided in this application can be applied to, for example... Figure 1 In the application environment shown, the camera device 102 communicates with the computer device 104 via a network. Before stacking the first cage onto the second cage, the camera device 102 captures images of the stacking devices of the first and second cages respectively, obtaining corresponding cage images, and transmits them to the computer device 104. The computer device 104 performs a first target detection on the cage images using a first detection model, identifying the first stacking device corresponding to the first cage and the second stacking device corresponding to the second cage from the cage images. Then, the computer device 104 determines the first position information of the first stacking device and the second position information of the second stacking device, and determines a first stacking result based on the first and second position information. The computer device 104 then performs a second target detection on the cage images using a second detection model, and directly outputs a second stacking result based on the image features extracted by the second target detection. Finally, the computer device 104 determines a final stacking result based on the first and second stacking results. The final stacking result includes a result that triggers the stacking of the first cage onto the second cage, and a result that prevents the stacking of the first cage onto the second cage. The camera device 102 can be, but is not limited to, various image acquisition devices, such as high-definition cameras, high-definition vehicle cameras, charge-coupled device cameras (CCD cameras), scanners, or mobile phones with camera functions. The computer device 104 can specifically be a terminal or a server. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, portable wearable devices, and industrial control computers on warehousing and transportation vehicles. The server can be a standalone server or a server cluster composed of multiple servers.

[0041] In one embodiment, such as Figure 2 As shown, a method for confirming cage stacking is provided, which is applied to... Figure 1 Taking a computer device as an example, the explanation includes the following steps:

[0042] Step 202: Before stacking the first cage onto the second cage, image acquisition is performed on the stacking devices of the first and second cages respectively to obtain corresponding cage images.

[0043] The material cage is a logistics container welded from raw steel materials, used to store goods. Stacking material cages can improve the effective utilization of storage space. In one embodiment, each material cage includes four feet and four supports; please refer to the following for details. Figure 3 The diagram shows the structure of the feed cage. It is understood that the feed cage may also include more or fewer foot cups and blocks, for example, each feed cage may include 6 foot cups and 6 blocks, etc., and the embodiments of this application do not limit this.

[0044] Specifically, before stacking the first cage onto the second cage, the automated warehouse forklift acquires images of the stacking devices of both cages, and the industrial control computer within the forklift obtains the acquired images. The automated warehouse forklift includes an industrial control computer and a camera system, which are located on the same intranet. For details, please refer to [reference needed]. Figure 4 The diagram shows the structure of an automated warehouse forklift. For example, when the automated warehouse forklift needs to stack a first cage on the forklift to a second cage, the camera device in the automated warehouse forklift captures images of the stacking device of the first and second cages, and then the industrial control computer in the automated warehouse forklift acquires the captured images of the cages. The camera device can be, but is not limited to, various image acquisition devices, such as high-definition cameras, high-definition vehicle-mounted cameras, charge-coupled device cameras (CCD cameras), scanners, or devices with camera functions such as mobile phones and tablets. The automated warehouse forklift can be, but is not limited to, various unmanned driving devices, such as unmanned vehicles or drones.

[0045] Step 204: Perform first target detection on the cage image using the first detection model to identify the first stacking device corresponding to the first cage and the second stacking device corresponding to the second cage from the cage image.

[0046] The first detection model is a detection model for computer equipment, such as the detection model YOLOv5 (You Only Look Once v5, which belongs to the YOLO algorithm family, is a deep neural network-based object recognition and localization algorithm used for faster and more efficient target detection) deployed on the industrial control computer of an unmanned warehouse forklift using OpenVINO (Open Visual Inference & Neural Network Optimization, OpenVINO is a comprehensive toolkit for rapid development of applications and solutions). This detection model is run by a program set by the industrial control computer.

[0047] Specifically, the computer equipment inputs the acquired image of the material cage into a first detection model to perform first target detection. Based on the image of the material cage, it identifies the first stacking device corresponding to the first material cage and the second stacking device corresponding to the second material cage. For example, the industrial control computer of a warehouse unmanned forklift inputs the image of the material cage into a first detection model Yolov5 deployed through OpenVINO to perform first target detection. Through this target detection, it identifies the first stacking device corresponding to the first material cage and the second stacking device corresponding to the second material cage.

[0048] Step 206: Determine the first position information of the first stacking device and the second position information of the second stacking device, and determine the first stacking result based on the first position information and the second position information.

[0049] The first position information is the coordinate information of the first stacking device, and the second position information is the coordinate information of the second stacking device. The first stacking result is whether the first cage can be stacked on top of the second cage. If it can be stacked, the result is Success and can be recorded as 1; if it cannot be stacked, the first stacking result is Fail and can be recorded as 0.

[0050] Specifically, the computer equipment, based on the first stacking device and the second stacking device identified by the first detection model, determines the first coordinate information of the first stacking device and the second coordinate information of the second stacking device, and determines the first stacking result based on the first coordinate information and the second coordinate information. For example, the industrial control computer in the warehouse unmanned forklift, based on the first stacking device and the second stacking device identified by the first detection model, calculates the first coordinate information of the first stacking device and the second coordinate information of the second stacking device, and determines the first stacking result based on the first coordinate information and the second coordinate information. When the first cage can be stacked on the second cage, the first stacking result is Success, which can be recorded as 1; when the first cage cannot be stacked on the second cage, the first stacking result is Fail, which can be recorded as 0.

[0051] Step 208: Perform second target detection on the cage image using the second detection model, and directly output the second stacking result based on the image features extracted by the second target detection.

[0052] The second detection model is a YOLOv5 (You Only Look Once v5, for faster and more efficient object detection) detection model deployed on an industrial control computer using OpenVINO (Open Visual Inference & Neural Network Optimization, a comprehensive toolkit for rapid application and solution development). This detection model runs through a program set on the industrial control computer. This second detection model can directly obtain detection results from the input image. The first stacking result indicates whether the first cage can be stacked on top of the second cage. If stacking is possible, the result is "Success" and can be recorded as 1; if stacking is not possible, the first stacking result is "Fail" and can be recorded as 0.

[0053] Specifically, the computer equipment inputs the acquired cage image into a second detection model for second target detection, extracts image features corresponding to the cage image, and directly obtains the second stacking result based on these image features. For example, the industrial control computer of a warehouse unmanned forklift inputs the cage image into a second detection model (Yolov5) deployed via OpenVINO for second target detection, extracts image features corresponding to the cage image, and directly obtains the second stacking result based on these image features. When the first cage can be stacked on top of the second cage, the second stacking result is "Success," which can be recorded as 1; when the first cage cannot be stacked on top of the second cage, the second stacking result is "Fail," which can be recorded as 0.

[0054] Step 210: Based on the first stacking result and the second stacking result, determine the final stacking result; the final stacking result includes the result of triggering the stacking of the first cage onto the second cage, and the result of triggering the prevention of stacking the first cage onto the second cage.

[0055] Specifically, the computer equipment determines the final stacking result of the first and second cages based on the first and second stacking results. This final stacking structure includes the result of the automated warehouse forklift's control computer triggering the stacking of the first cage onto the second cage, and the result of the automated warehouse forklift preventing the stacking of the first cage onto the second cage. This preventing operation can be a prompt message or a direct prohibition of the automated warehouse forklift from stacking the first cage onto the second cage.

[0056] In the aforementioned method for confirming cage stacking, before stacking the first cage on top of the second cage, images of the stacking devices of both cages are captured, resulting in corresponding cage images. Two detection models can then be used to identify the stacking devices of the two cages in the images. The first detection model locates the positions of the stacking devices to obtain a first stacking result, while the second detection model directly outputs a second stacking result corresponding to the stacking devices of the two cages. This allows for the integration of stacking results obtained from the two detection models using different methods to arrive at the final stacking result. Thus, manual confirmation of whether the two cages can be stacked is unnecessary; the two detection models can quickly and accurately determine this, significantly improving the efficiency of cage stacking confirmation.

[0057] In one embodiment, the cage image includes a first cage image and a second cage image; before stacking the first cage on the second cage, image acquisition is performed on the stacking devices of the first and second cages respectively to obtain corresponding cage images, including: before stacking the first cage on the second cage, taking a picture of the stacking device in a first position of the first and second cages to obtain a first cage image, and taking a picture of the stacking device in a second position of the first and second cages to obtain a second cage image; wherein the first position and the second position are two different positions.

[0058] The camera equipment is placed on both sides of the automated warehouse forklift, namely the first camera and the second camera, respectively. Please refer to [reference needed] for details. Figure 4 The installation positions of the camera devices on the automated warehouse forklift are as follows: the first position and the second position are on both sides of the automated warehouse forklift. The first camera device is for acquiring the image corresponding to the second position, and the second camera device is for acquiring the image corresponding to the first position. The first position can be the left side of the automated warehouse forklift, and the second position can be the right side of the automated warehouse forklift.

[0059] Specifically, before stacking the first cage onto the second cage, the second camera of the automated warehouse forklift captures an image of the first cage by photographing the stacking device in a first position within both cages. The second camera then photographs an image of the second cage by photographing the stacking device in a second position within both cages. For example, when the automated warehouse forklift needs to stack the first cage on the forklift onto the second cage, the second camera on the left side of the forklift photographs the stacking device on the right side of both cages to capture an image of the first cage, while the first camera on the right side photographs the stacking device on the left side of both cages to capture an image of the second cage.

[0060] In this embodiment, before stacking the first cage onto the second cage, an image of the first cage is obtained by taking a picture of the stacking device in the first position of the first cage and the second cage, and an image of the second cage is obtained by taking a picture of the stacking device in the second position of the first cage and the second cage. This allows for obtaining more detailed images of the cages, which helps to determine the final stacking result and improves the reliability and accuracy of the final stacking result.

[0061] In one embodiment, such as Figure 5 As shown, the stacking device includes a foot cup and a foot block that matches the foot cup. Before stacking the first cage onto the second cage, the stacking device at a first position in the first and second cages is photographed to obtain an image of the first cage, and the stacking device at a second position in the first and second cages is photographed to obtain an image of the second cage. This includes:

[0062] Step 502: Before stacking the first cage on top of the second cage, take pictures of the foot cup of the first cage in the first position and the block of the second cage in the first position to obtain an image of the first cage.

[0063] The stacking device consists of foot cups and blocks located on the same material cage, with the foot cups positioned below the blocks within the same cage. Please refer to [reference needed]. Figure 3 The diagram shows the structure of the feed cage.

[0064] Specifically, before stacking the first cage onto the second cage, the second camera device of the automated warehouse forklift captures images of the first cage's foot cup in a first position and the second cage's support in the first position, obtaining an image of the first cage. For example, when the automated warehouse forklift needs to stack the first cage on the forklift onto the second cage, the second camera device on the left side of the automated warehouse forklift captures images of the first cage's right foot cup and the second cage's right support, obtaining an image of the first cage.

[0065] Step 504: Take pictures of the foot cup of the first cage in the second position and the pier of the second cage in the second position to obtain an image of the second cage.

[0066] The stacking device consists of foot cups and blocks located on the same material cage, with the foot cups positioned below the blocks within the same cage. Please refer to [reference needed]. Figure 3 The diagram shows the structure of the feed cage.

[0067] Specifically, the first camera device of the automated warehouse forklift captures images of the foot cup of the first cage in a second position and the block of the second cage in a second position to obtain an image of the second cage. For example, the first camera device on the right side of the automated warehouse forklift captures images of the foot cup on the left side of the first cage and the block on the left side of the second cage to obtain an image of the first cage.

[0068] In this embodiment, before stacking the first cage on top of the second cage, images of the foot cup of the first cage in a first position and the block of the second cage in a first position are taken to obtain an image of the first cage; images of the foot cup of the first cage in a second position and the block of the second cage in a second position are taken to obtain an image of the second cage. This allows for obtaining more detailed images of the cages, which helps to determine the final stacking result and thus improves the reliability and accuracy of the final stacking result.

[0069] In one embodiment, the first target detection of the cage image using a first detection model, and the identification of a first stacking device corresponding to the first cage and a second stacking device corresponding to the second cage from the cage image, includes: obtaining feature information of the cage image using the first detection model, and identifying the first stacking device corresponding to the first cage and the second stacking device corresponding to the second cage based on the feature information.

[0070] The first detection model is the Yolov5 (You Only Look Once v5, used for faster and more efficient target detection) detection model deployed on an industrial control computer using OpenVINO (which is a comprehensive toolkit for rapid development of applications and solutions). This detection model runs through a program set on the industrial control computer.

[0071] Specifically, the computer device inputs the acquired image of the material cage into a first detection model. Based on this first detection model, it acquires the feature information of each object in the image of the material cage in real time, and classifies each object based on this feature information, thereby identifying the first stacking device corresponding to the first material cage and the second stacking device corresponding to the second material cage. For example, the industrial control computer of a warehouse unmanned forklift inputs the image of the material cage into a first detection model (Yolov5) deployed via OpenVINO. This first detection model acquires the feature information of each object in the image of the material cage in real time, classifies each object based on this feature information, and thereby identifies the first stacking device corresponding to the first material cage and the second stacking device corresponding to the second material cage.

[0072] In this embodiment, feature information of the cage image is obtained through a first detection model, and a first stacking device corresponding to the first cage and a second stacking device corresponding to the second cage are identified based on the feature information. The first stacking device and the second stacking device obtained according to the first detection model can be acquired in real time, which helps to determine the final stacking result of the cage stacking, thereby improving the reliability and accuracy of the final stacking result.

[0073] In one embodiment, such as Figure 6 As shown, a first foot cup is provided on the first surface of the first cage, and a first block is provided on the second surface; a second foot cup is provided on the first surface of the second cage, and a second block is provided on the second surface; the determination of the first stacking result based on the first position information of the first stacking device and the second position information of the second stacking device includes:

[0074] Step 602: Determine the first position information corresponding to the first foot cup in the first stacking device, and determine the second position information corresponding to the second block in the second stacking device.

[0075] In the first stacking device, the first position information corresponding to the first foot cup is the first coordinate information corresponding to the first foot cup in the first face of the first cage, and in the second stacking device, the second position information corresponding to the second block is the second coordinate information corresponding to the second block in the second face of the second cage.

[0076] Specifically, the computer equipment uses the midpoint of the rectangular frame formed by the first foot cup and the second block as the positioning point of the first foot cup and the second block. Based on the camera's intrinsic and extrinsic parameters and the equations of the ground in the coordinate systems of each camera device, it obtains in real time the first position information corresponding to the first foot cup in the first stacking device, and determines the second position information corresponding to the second block in the second stacking device. For example, the industrial control computer of the warehouse unmanned forklift uses the midpoint of the rectangular frame formed by the first foot cup and the second block as the positioning point of the first foot cup and the second block. Based on the camera's intrinsic parameters such as focal length and pixels, extrinsic parameters such as the camera's position and rotation direction, and the plane equations of the ground in the coordinate systems of each camera device, that is, any plane in the spatial coordinate system can be represented by a ternary linear equation Ax + By + Cz + D = 0 (where A, B, C, and D are constants), it obtains in real time the first coordinate information corresponding to the first foot cup in the first stacking device and the second coordinate information corresponding to the second block in the second stacking device.

[0077] Step 604: Based on the first position information and the second position information, obtain the distance between the first foot cup and the second block in each stacking combination; wherein the first foot cup and the second block in the stacking combination are located in the same vertical direction.

[0078] Among them, reference Figure 3 The schematic diagram of the cage structure shown includes four blocks and four foot cups for the same cage. The first foot cup in the first stacking device and the second block in the second stacking device form a stacking combination with the blocks and foot cups located in the same vertical orientation, resulting in four stacking combinations.

[0079] Specifically, the computer device calculates the distance between the first foot cup and the second foot cup in each stack combination based on the first location information and the second location information acquired in real time.

[0080] In one embodiment, for each stack combination, the computer device performs a subtraction operation on the coordinate information of the first foot cup in the first stacking device and the coordinate information of the second block in the second stacking device to obtain the difference information between the first foot cup and the first block, which is the distance between the first foot cup and the second block in each stack combination.

[0081] Step 606: Compare the distance between the first foot cup and the second foot cup in each stack combination with the distance threshold to obtain the comparison results of each stack combination.

[0082] Specifically, the computer equipment compares the distance between the first foot cup and the second pier in each stacked combination with a distance threshold to obtain the comparison result of each stacked combination. When the distance between the first foot cup and the second pier in each stacked combination is less than the distance threshold, the comparison result of the stacked combination is that the stacked combination can be stacked. When the distance between the first foot cup and the second pier in each stacked combination is greater than or equal to the distance threshold, the comparison result of the stacked combination is that the stacked combination cannot be stacked. The distance threshold can be modified according to the actual shape of the cage on site, and can generally be set to 1cm.

[0083] Step 608: Determine the first stacking result based on the comparison results of each stacking combination.

[0084] Specifically, the computer device determines a first stacking result based on the comparison results of each stacking combination, wherein the first stacking result includes the comparison results of each stacking combination.

[0085] In one embodiment, the computer device can directly use the comparison results of each stack combination as the first stack result, that is, the first stack result includes whether each stack combination can achieve stacking.

[0086] In another embodiment, the computer device may determine the first stacking result based on the comparison results of each stacking combination. For example, when all the comparison results of all stacking combinations indicate that they can be stacked, the first stacking result is stackable; when at least one comparison result indicates that they cannot be stacked, the first stacking result is not stackable.

[0087] In this embodiment, firstly, the first position information corresponding to the first foot cup in the first stacking device and the second position information corresponding to the second block in the second stacking device are determined; then, based on the first position information and the second position information, the distance between the first foot cup and the second block in each stacking combination is obtained; then, the distance between the first foot cup and the second block in each stacking combination is compared with a distance threshold to obtain the comparison result of each stacking combination; finally, based on the comparison result of each stacking combination, the first stacking result is determined, which helps to combine the second stacking result to determine the final cage stacking result, thereby improving the efficiency of cage stacking.

[0088] In one embodiment, such as Figure 7 As shown, before directly outputting the second stacking result based on the image features extracted by the second target detection and performing second target detection on the cage image using the second detection model, the process also includes:

[0089] Step 702: Obtain cage image samples; the cage image samples include first cage sample images corresponding to two cages in a stackable state, and second cage sample images corresponding to two cages in a non-stackable state.

[0090] Specifically, based on the camera equipment of the warehouse unmanned forklift, the computer equipment acquires multiple cage image samples, which include a first cage sample image corresponding to two cages in a stackable state and a second cage sample image corresponding to two cages in a non-stackable state.

[0091] Step 704: Train the second detection model using the image sample of the cage and the label corresponding to whether the cage image is in a stackable state, and obtain the trained second detection model.

[0092] Specifically, the computer device trains a second detection model based on Yolov5 using multiple cage image samples and labels indicating whether the cage image samples are stackable, thus obtaining a trained second detection model. This second detection model can directly obtain the result of cage stacking based on the cage image samples.

[0093] In this embodiment, cage image samples are first obtained. These cage image samples include first cage sample images corresponding to two cages in a stackable state and second cage sample images corresponding to two cages in a non-stackable state. Finally, using these cage image samples and the corresponding labels indicating whether the cages are in a stackable state, a second detection model is trained to obtain the trained second detection model. Therefore, in practical scenarios, inputting the acquired cage images into the trained second detection model can directly obtain the second stacking result, which helps to combine the first stacking result to determine the final cage stacking result, thereby improving the efficiency of cage stacking.

[0094] In one embodiment, such as Figure 8 As shown, the final stacking result is determined based on the first stacking result and the second stacking result, including:

[0095] Step 802: Confirm the model by stacking, and obtain stacked data by weighted summation of the first stacking result and the second stacking result.

[0096] The stacking confirmation model is a trained classifier. Based on the trained classifier, the weights of the first stacking result and the second stacking result can be obtained, and the stacking threshold can be determined.

[0097] Specifically, the computer device inputs the first stacking result and the second stacking result into the stacking confirmation model, and performs a weighted summation of the first stacking result and the second stacking result according to the weights of the first stacking result and the second stacking result in the stacking confirmation model to obtain the stacking data.

[0098] In one embodiment, the stacking confirmation model is a trained classifier. The computer device inputs the comparison results of each stack combination in the first stacking result into the classifier, and inputs the comparison results of each stack result in the second stacking result into the classifier. Based on the weight distribution of the two stacking results in the classifier, the first stacking result is multiplied by the weight corresponding to the first stacking result to obtain the first stacking data, and the second stacking result is multiplied by the weight corresponding to the stacking result to obtain the second stacking data. Finally, the first stacking data and the second stacking data are added together to obtain the stacking data.

[0099] Step 804: Compare the stacked data with the stacking threshold, and obtain the final stacking result based on the comparison result.

[0100] Specifically, the computer device compares the stacking data with the stacking threshold based on the stacking confirmation model, and obtains the final stacking result based on the comparison result. When all stacking results are in a state of "can be stacked," the final stacking result is confirmed as "can be stacked." When the final stacking result is "can be stacked," the perception program outputs the final stacking status of the two cages to the main program of the automated warehouse forklift via JsonRPC (JSON-based cross-language remote call protocol). The automated warehouse forklift receives the final stacking result. If the final stacking result is "can be stacked," it triggers the operation of stacking the first cage onto the second cage, directly placing the first cage on top of the second cage. If the final stacking result is "cannot be stacked," it triggers the operation to prevent stacking the first cage onto the second cage, the automated warehouse forklift will report an error, and stop the current action.

[0101] In this embodiment, firstly, based on the stacking confirmation model, the first stacking result and the second stacking result are weighted and summed to obtain stacking data; then, the stacking data is compared with the stacking threshold, and the final stacking result is obtained based on the comparison result. Therefore, based on the stacking confirmation model, the two stacking results can be neutralized, thereby quickly and accurately determining whether the two cages are stacked, thus improving the efficiency of cage stacking.

[0102] It should be understood that, although Figure 2 , 5 The steps in flowchart -8 are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order requirement for the execution of these steps, and they can be executed in other orders. Furthermore, Figure 2 , 5 At least some of the steps in -8 may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.

[0103] In one embodiment, such as Figure 9 As shown, a device for confirming the stacking of material cages is provided. The device 900 includes: an acquisition module 902, a first processing module 904, a first determination module 906, a second processing module 908, and a second determination module 910, wherein:

[0104] The module 902 is used to acquire images of the stacking devices of the first and second cages respectively before stacking the first cage on the second cage, and obtain corresponding cage images.

[0105] The first processing module 904 is used to perform first target detection on the cage image using a first detection model, and to identify a first stacking device corresponding to the first cage and a second stacking device corresponding to the second cage from the cage image.

[0106] The first determining module 906 is used to determine the first stacking result based on the first position information of the first stacking device and the second position information of the second stacking device.

[0107] The second processing module 908 is used to perform second target detection on the cage image through the second detection model, and directly output the second stacking result based on the image features extracted by the second target detection.

[0108] The second determining module 910 is used to determine a final stacking result based on the first stacking result and the second stacking result; the final stacking result includes a result that triggers the stacking of the first cage onto the second cage, and a result that triggers the prevention of stacking the first cage onto the second cage.

[0109] In one embodiment, the acquisition module 902 is used to take a picture of the stacking device in the first cage and the second cage in a first position before stacking the first cage on the second cage to obtain an image of the first cage, and to take a picture of the stacking device in the first cage and the second cage in a second position to obtain an image of the second cage; wherein the first position and the second position are two different positions.

[0110] In one embodiment, the acquisition module 902 is used to take pictures of the foot cup of the first cage in a first position and the block of the second cage in a first position before stacking the first cage on the second cage, to obtain an image of the first cage; and to take pictures of the foot cup of the first cage in a second position and the block of the second cage in a second position, to obtain an image of the second cage.

[0111] In one embodiment, the first processing module 904 is used to acquire feature information of the cage image through a first detection model, and identify a first stacking device corresponding to the first cage and a second stacking device corresponding to the second cage based on the feature information.

[0112] In one embodiment, the first determining module 906 is used to determine the first position information corresponding to the first foot cup in the first stacking device, and to determine the second position information corresponding to the second block in the second stacking device; based on the first position information and the second position information, to obtain the distance between the first foot cup and the second block in each stacking combination; wherein the first foot cup and the second block in the stacking combination are located in the same vertical direction; to compare the distance between the first foot cup and the second block in each stacking combination with a distance threshold, and to obtain the comparison result of each stacking combination; and to determine the first stacking result based on the comparison result of each stacking combination.

[0113] In one embodiment, such as Figure 10 As shown, the device 900 further includes a training module 912 for obtaining cage image samples; the cage image samples include a first cage sample image corresponding to two cages in a stackable state, and a second cage sample image corresponding to two cages in a non-stackable state; the second detection model is trained using the cage image samples and the labels corresponding to whether the cage images are in a stackable state, to obtain the trained second detection model.

[0114] In one embodiment, the second determining module is used to obtain stacked data by weighted summation of the first stacking result and the second stacking result through a stacking confirmation model; compare the stacked data with a stacking threshold, and obtain the final stacking result based on the comparison result.

[0115] Specific limitations regarding the cage stacking verification device can be found in the limitations of the cage stacking verification method described above, and will not be repeated here. Each module in the aforementioned cage stacking verification device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0116] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 11As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores confirmation data for cage stacking. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a cage stacking confirmation method.

[0117] Those skilled in the art will understand that Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0118] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0119] Before stacking the first cage onto the second cage, images of the stacking devices for the first and second cages are acquired to obtain corresponding cage images. A first detection model is used to perform first target detection on the cage images to identify the first stacking device corresponding to the first cage and the second stacking device corresponding to the second cage. First position information of the first stacking device and second position information of the second stacking device are determined, and a first stacking result is determined based on the first and second position information. A second detection model is used to perform second target detection on the cage images, and a second stacking result is directly output based on the image features extracted by the second target detection. A final stacking result is determined based on the first and second stacking results. The final stacking result includes a result that triggers the stacking of the first cage onto the second cage and a result that prevents the stacking of the first cage onto the second cage.

[0120] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0121] Before stacking the first cage onto the second cage, an image of the first cage is obtained by photographing the stacking device in the first and second cages in a first position, and an image of the second cage is obtained by photographing the stacking device in the first and second cages in a second position; wherein the first position and the second position are two different positions.

[0122] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0123] Before stacking the first cage on top of the second cage, images of the first cage's foot cup in a first position and the second cage's block in a first position are taken to obtain an image of the first cage; images of the first cage's foot cup in a second position and the second cage's block in a second position are taken to obtain an image of the second cage.

[0124] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0125] The first detection model is used to obtain the feature information of the cage image, and the first stacking device corresponding to the first cage and the second stacking device corresponding to the second cage are identified based on the feature information.

[0126] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0127] The first position information corresponding to the first foot cup in the first stacking device and the second position information corresponding to the second block in the second stacking device are determined; based on the first position information and the second position information, the distance between the first foot cup and the second block in each stacking combination is obtained; wherein the first foot cup and the second block in the stacking combination are located in the same vertical direction; the distance between the first foot cup and the second block in each stacking combination is compared with a distance threshold to obtain the comparison result of each stacking combination; the first stacking result is determined based on the comparison result of each stacking combination.

[0128] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0129] Obtain cage image samples; the cage image samples include first cage sample images corresponding to two cages in a stackable state, and second cage sample images corresponding to two cages in a non-stackable state; train a second detection model using the cage image samples and the labels corresponding to whether the cage images are in a stackable state, and obtain the trained second detection model.

[0130] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0131] By using a stacking verification model, the first stacking result and the second stacking result are weighted and summed to obtain stacking data; the stacking data is compared with the stacking threshold, and the final stacking result is obtained based on the comparison result.

[0132] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0133] Before stacking the first cage onto the second cage, images of the stacking devices for the first and second cages are acquired to obtain corresponding cage images. A first detection model is used to perform first target detection on the cage images to identify the first stacking device corresponding to the first cage and the second stacking device corresponding to the second cage. First position information of the first stacking device and second position information of the second stacking device are determined, and a first stacking result is determined based on the first and second position information. A second detection model is used to perform second target detection on the cage images, and a second stacking result is directly output based on the image features extracted by the second target detection. A final stacking result is determined based on the first and second stacking results. The final stacking result includes a result that triggers the stacking of the first cage onto the second cage and a result that prevents the stacking of the first cage onto the second cage.

[0134] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0135] Before stacking the first cage onto the second cage, an image of the first cage is obtained by photographing the stacking device in the first and second cages in a first position, and an image of the second cage is obtained by photographing the stacking device in the first and second cages in a second position; wherein the first position and the second position are two different positions.

[0136] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0137] Before stacking the first cage on top of the second cage, images of the first cage's foot cup in a first position and the second cage's block in a first position are taken to obtain an image of the first cage; images of the first cage's foot cup in a second position and the second cage's block in a second position are taken to obtain an image of the second cage.

[0138] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0139] The first detection model is used to obtain the feature information of the cage image, and the first stacking device corresponding to the first cage and the second stacking device corresponding to the second cage are identified based on the feature information.

[0140] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0141] The first position information corresponding to the first foot cup in the first stacking device and the second position information corresponding to the second block in the second stacking device are determined; based on the first position information and the second position information, the distance between the first foot cup and the second block in each stacking combination is obtained; wherein the first foot cup and the second block in the stacking combination are located in the same vertical direction; the distance between the first foot cup and the second block in each stacking combination is compared with a distance threshold to obtain the comparison result of each stacking combination; the first stacking result is determined based on the comparison result of each stacking combination.

[0142] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0143] Obtain cage image samples; the cage image samples include first cage sample images corresponding to two cages in a stackable state, and second cage sample images corresponding to two cages in a non-stackable state; train a second detection model using the cage image samples and the labels corresponding to whether the cage images are in a stackable state, and obtain the trained second detection model.

[0144] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0145] By using a stacking verification model, the first stacking result and the second stacking result are weighted and summed to obtain stacking data; the stacking data is compared with the stacking threshold, and the final stacking result is obtained based on the comparison result.

[0146] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0147] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0148] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for confirming the stacking of material cages, characterized in that, The method includes: Before stacking the first cage onto the second cage, images of the stacking devices of the first and second cages are captured to obtain corresponding cage images. The first detection model acquires feature information of the cage image and identifies a first stacking device corresponding to the first cage and a second stacking device corresponding to the second cage based on the feature information. The first cage has a first foot cup on its first surface and a first block on its second surface; the second cage has a second foot cup on its first surface and a second block on its second surface. Determine the first position information corresponding to the first foot cup in the first stacking device, and determine the second position information corresponding to the second block in the second stacking device; Based on the first location information and the second location information, the distance between the first foot cup and the second block in each stacking combination is obtained; wherein, the first foot cup and the second block in the stacking combination are located in the same vertical direction; The distance between the first foot cup and the second foot cup in each stack combination is compared with the distance threshold to obtain the comparison results of each stack combination; The first stacking result is determined based on the comparison results of each stacking combination; The cage image is input into the second detection model for second target detection, and the image features corresponding to the cage image are extracted. Based on the image features, the second stacking result is directly output through the second detection model. Based on the first stacking result and the second stacking result, a final stacking result is determined; the final stacking result includes a result that triggers the stacking of the first cage onto the second cage, and a result that triggers the prevention of the stacking of the first cage onto the second cage.

2. The method according to claim 1, characterized in that, The cage image includes a first cage image and a second cage image; the step of acquiring images of the stacking devices of the first and second cages respectively before stacking the first cage on the second cage to obtain corresponding cage images includes: Before stacking the first cage onto the second cage, an image of the first cage is obtained by photographing the stacking device in the first and second cages in a first position, and an image of the second cage is obtained by photographing the stacking device in the first and second cages in a second position; wherein the first position and the second position are two different positions.

3. The method according to claim 2, characterized in that, The stacking device includes a foot cup and a block that matches the foot cup. Before stacking the first cage onto the second cage, the process of photographing the stacking device in a first position within the first and second cages to obtain an image of the first cage, and photographing the stacking device in a second position within the first and second cages to obtain an image of the second cage, includes: Before stacking the first cage on top of the second cage, take pictures of the foot cup of the first cage in the first position and the block of the second cage in the first position to obtain an image of the first cage. The foot cup of the first cage in the second position and the pier of the second cage in the second position are photographed to obtain an image of the second cage.

4. The method according to claim 1, characterized in that, Before inputting the cage image into the second detection model for second target detection, the method further includes: Obtain material cage image samples; the material cage image samples include a first material cage sample image corresponding to two material cages in a stackable state, and a second material cage sample image corresponding to two material cages in a non-stackable state; The second detection model is trained using the cage image samples and the corresponding labels indicating whether the cage image is in a stackable state, resulting in a trained second detection model.

5. The method according to claim 1, characterized in that, Determining the final stacking result based on the first stacking result and the second stacking result includes: By confirming the stacking model, the first stacking result and the second stacking result are weighted and summed to obtain the stacked data; The stacked data is compared with the stacking threshold, and the final stacking result is obtained based on the comparison result.

6. A device for confirming the stacking of material cages, characterized in that, The device includes: The acquisition module is used to acquire images of the stacking devices of the first and second cages respectively before stacking the first cage on the second cage, and obtain corresponding cage images. The first processing module is used to acquire feature information of the cage image through a first detection model, and identify a first stacking device corresponding to the first cage and a second stacking device corresponding to the second cage based on the feature information. The first cage has a first foot cup on its first surface and a first block on its second surface; the second cage has a second foot cup on its first surface and a second block on its second surface. A first determining module is used to determine the first position information corresponding to the first foot cup in the first stacking device, and to determine the second position information corresponding to the second block in the second stacking device; based on the first position information and the second position information, to obtain the distance between the first foot cup and the second block in each stacking combination; wherein the first foot cup and the second block in the stacking combination are located in the same vertical direction; to compare the distance between the first foot cup and the second block in each stacking combination with a distance threshold, and to obtain the comparison result of each stacking combination; and to determine a first stacking result based on the comparison result of each stacking combination. The second processing module is used to input the cage image into the second detection model for second target detection, extract the image features corresponding to the cage image, and output the second stacking result directly through the second detection model based on the image features; The second determining module is used to determine a final stacking result based on the first stacking result and the second stacking result; the final stacking result includes a result that triggers the stacking of the first cage onto the second cage, and a result that triggers the prevention of the stacking of the first cage onto the second cage.

7. The apparatus according to claim 6, characterized in that, The cage image includes a first cage image and a second cage image; the obtaining module is used to take a picture of the stacking device in the first cage and the second cage in a first position before stacking the first cage on the second cage to obtain a first cage image, and to take a picture of the stacking device in the first cage and the second cage in a second position to obtain a second cage image; wherein the first position and the second position are two different positions.

8. The apparatus according to claim 6, characterized in that, The stacking device includes a foot cup and a block that matches the foot cup. The obtaining module is used to take pictures of the foot cup of the first cage in a first position and the block of the second cage in a first position before stacking the first cage on the second cage, so as to obtain an image of the first cage. The foot cup of the first cage in the second position and the pier of the second cage in the second position are photographed to obtain an image of the second cage.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images