Material processing method and device, electronic equipment and storage medium

Abstract

The invention provides a material processing method and device, electronic equipment and a storage medium, and relates to the technical field of intelligent warehousing, and the method comprises the steps: obtaining an image of a material box containing a to-be-processed material; identifying target information of the material box based on the image of the material box; the target information comprises at least one of indication information of the material box, the type of the to-be-processed material, form information of the to-be-processed material, the volume ratio of the material box and sorting pose information of the to-be-processed material; the indication information is used for indicating that the material box is divided into grids or not divided into grids; and processing the to-be-processed material based on a target task and the target information. By identifying the image of the material box containing the to-be-processed material, the to-be-processed material can be automatically processed based on the identified target information and the target task, and the material processing efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent warehousing technology, and in particular to a material processing method, device, electronic equipment and storage medium. Background Art

[0002] As the number and types of materials stored in storage warehouses increase, the requirements for material handling become increasingly higher, especially for inventory or picking of materials.

[0003] For example, the material inventory process in a traditional storage warehouse generally includes two types: 1. The inventory personnel use a personal digital assistant (PDA) or computer to query the storage location of the materials to be counted, and then go to the corresponding storage location to count the quantity, model and other inventory information of the materials. Finally, the inventory information such as quantity and model is uploaded to the system via the PDA or computer to complete the inventory; 2. The inventory personnel use a PDA or computer to query the storage location of the materials to be counted, and then transport the materials in the corresponding storage location to the inventory workbench by other manual and automatic methods such as carts, automated guided vehicles (AGVs), conveyor lines, etc. Finally, the inventory personnel upload the inventory information such as quantity and model to the system via the PDA or computer to complete the inventory.

[0004] Traditional material picking methods in storage warehouses primarily include person-to-goods picking and goods-to-person picking. In the case of person-to-goods picking, the picker uses a PDA or computer to query the picking order and material storage location information. They then push a cart or picking box to the corresponding picking location to pick the items. Once the items are picked, they place the cart or box on the packing table for packing. In the goods-to-person picking process, the upstream order management system sends the picking order information to the equipment management system. The equipment management system then transports the corresponding material turnover boxes to the picking table via automated guided vehicles (AGVs), conveyor lines, or other manual and automated methods. Finally, the picker completes the picking process at the picking table.

[0005] Currently, the end-of-line material processing work in storage warehouses (such as end-of-line inventory or picking) is mainly completed manually. This method results in low personnel utilization and low degree of automation, resulting in low material processing efficiency. Summary of the Invention

[0006] The present invention provides a material processing method, device, electronic equipment and storage medium, which are used to solve the problem of low efficiency in material processing.

[0007] The present invention provides a material processing method, comprising: Acquire an image of a bin containing material to be processed; Based on the image of the material box, target information of the material box is identified; the target information includes at least one of the following: indication information of the material box, type of the material to be processed, shape information of the material to be processed, volume ratio of the material box, and picking posture information of the material to be processed; the indication information is used to indicate whether the material box is divided or not divided; Based on the target task and the target information, the material to be processed is processed.

[0008] According to a material processing method provided by the present invention, the material to be processed is processed based on the target task and the target information, including: In the case where the target task is an inventory task, performing an inventory of the materials to be processed based on the target information; In the case where the target task is a picking task, the materials to be processed are picked based on the target information.

[0009] According to a material processing method provided by the present invention, when the target information includes the indication information of the material box and the type of the material to be processed, the inventory of the material to be processed based on the target information includes: When the indication information indicates that the material box is not divided into grids, based on the type of the material to be processed, calling a database to determine the weight of a single material of the material to be processed; the database pre-stores the weight of a single material corresponding to a plurality of types of materials; Based on the weight of the individual materials, the materials to be processed are counted.

[0010] According to a material processing method provided by the present invention, the inventory of the material to be processed based on the weight of the single material includes: When the weight of the single material is greater than a first preset threshold, determining the quantity of the material to be processed based on the weighing information of the material to be processed and the weight of the single material; Based on the quantity of the materials to be processed, an inventory is taken of the materials to be processed.

[0011] According to a material processing method provided by the present invention, when the target information further includes the volume ratio of the material box and the form information of the material to be processed, the inventory of the material to be processed based on the weight of the individual material includes: When the weight of the single material is not greater than a first preset threshold and the volume ratio is less than a second preset threshold, determining whether the material to be processed is easy to grasp based on the type of the material to be processed and the morphological information of the material to be processed; Based on the determination result, an inventory is taken of the materials to be processed.

[0012] According to a material processing method provided by the present invention, when the target information includes indication information of the material box, the type of the material to be processed, the volume ratio of the material box, and the form information of the material to be processed, the inventory of the material to be processed based on the target information includes: When the indication information indicates that the material box is divided into compartments and the volume ratio of the material box is less than a third preset threshold, determining whether the material to be processed is easy to grasp based on the type of the material to be processed and the morphological information of the material to be processed; Based on the determination result, an inventory is taken of the materials to be processed.

[0013] According to a material processing method provided by the present invention, the inventory of the material to be processed based on the determination result includes: In the case where the material to be processed is easy to grasp, the first robotic arm is controlled to grasp the material to be processed and place the material to be processed on a visual workbench; the visual workbench is used to take inventory of the material to be processed.

[0014] According to a material processing method provided by the present invention, the inventory of the material to be processed based on the determination result includes: When the material box is not divided and the material to be processed is not easy to grab, or the material box division and the number of divisions of the material box are not greater than the fourth preset threshold and the material to be processed is not easy to grab, the second robotic arm is controlled to dump the material to be processed onto the visual workbench, and the material destacking module is controlled to destacking the material to be processed and then perform inventory.

[0015] According to a material processing method provided by the present invention, the method further comprises: When the indication information indicates that the material box is divided into compartments, the volume ratio of the material box is not less than the third preset threshold, and the number of compartments of the material box is not greater than the fourth preset threshold, the second robotic arm is controlled to dump the material to be processed onto the visual workbench, and the material destacking module is controlled to destacking the material to be processed and then perform inventory.

[0016] According to a material processing method provided by the present invention, when the target information includes the type of the material to be processed, the shape information of the material to be processed, and the picking posture information of the material to be processed, picking the material to be processed based on the target information includes: Determining whether the material to be processed is easy to grasp based on the type of the material to be processed and the morphological information of the material to be processed; When the material to be processed is easy to grasp, the picking posture information of the material to be processed is sent to the first robotic arm; the first robotic arm is used to pick the material to be processed to a distribution vehicle based on the picking posture information of the material to be processed; the distribution vehicle is used to transport the picked material to be processed to the location of the collection order box, and transfer the picked material to be processed to the collection order box.

[0017] According to a material processing method provided by the present invention, the method further comprises: Determine whether the picking collection sheet corresponding to the picking task has been picked; In the case that the picking set list has not been picked up, the step of sending the picking posture information of the material to be processed to the first robotic arm is repeated until all the picking sets are picked up.

[0018] The present invention also provides a material processing device, comprising: an acquisition module for acquiring an image of a bin containing a material to be processed; an identification module, configured to identify target information of the material box based on the image of the material box; the target information including at least one of the following: indication information of the material box, type of the material to be processed, form information of the material to be processed, volume ratio of the material box, and picking posture information of the material to be processed; the indication information is used to indicate whether the material box is divided or not divided; A processing module is used to process the material to be processed based on the target task and the target information.

[0019] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any of the above-described material processing methods is implemented.

[0020] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which implements any of the above-mentioned material processing methods when executed by a processor.

[0021] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned material processing methods.

[0022] The material processing method, device, electronic device, and storage medium provided by the present invention obtain an image of a material box containing materials to be processed; based on the image of the material box, identify the target information of the material box; the target information includes at least one of the following: the indication information of the material box, the type of the material to be processed, the shape information of the material to be processed, the volume ratio of the material box, and the picking posture information of the material to be processed; the indication information is used to indicate whether the material box is divided or not divided; based on the target task and the target information, the material to be processed is processed. By identifying the image of the material box containing materials to be processed, the materials to be processed can be automatically processed based on the identified target information and target task, thereby improving the efficiency of material processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0024] Figure 1 This is one of the flow diagrams of the material processing method provided by the present invention.

[0025] Figure 2 It is a schematic diagram of the warehousing workstation provided by the present invention.

[0026] Figure 3 It is a system schematic diagram of the material processing workstation provided by the present invention.

[0027] Figure 4 This is the second flow chart of the material processing method provided by the present invention.

[0028] Figure 5 This is the third flow chart of the material processing method provided by the present invention.

[0029] Figure 6 It is a structural schematic diagram of the material processing device provided by the present invention.

[0030] Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0031] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0032] The following combination Figure 1-Figure 5 The material processing method of the present invention is described.

[0033] Figure 1 This is one of the flow diagrams of the material processing method provided by the present invention, such as Figure 1 As shown, the method includes steps 101 to 103.

[0034] Step 101: Acquire an image of a material box containing material to be processed.

[0035] It should be noted that the material handling method provided by the present invention can be applied to material handling scenarios in storage warehouses, for example, inventory or picking of materials; the executor of the method can be a material handling device, such as an electronic device or a control module in the material handling device for executing the material handling method.

[0036] Specifically, the materials to be processed can be set up based on the actual scenario, without any restrictions. Images of bins containing the materials to be processed can be captured using a two-dimensional (2D) camera or a three-dimensional (3D) camera. The 2D / 3D camera can be installed at the material handling workstation, for example, at the weighing module of the material handling workstation or in front of the first robotic arm. The weighing module has a visual recognition function for identifying the type of material to be processed, whether the bin is compartmentalized, the shape of the material to be processed, the volume ratio of the bin, and the picking posture information of the material to be processed. Bins of the materials to be processed can be transported from the storage warehouse to the material handling workstation using turnover box transport modules such as transport vehicles (AGVs) and conveyor lines.

[0037] It should be noted that before the materials are stored in the storage warehouse, the plastic film of the materials needs to be removed at the storage workstation. Figure 2 It is a schematic diagram of the warehousing workstation provided by the present invention, such as Figure 2As shown, the incoming workstation is equipped with an electronic scale, which is used to weigh empty bins and filled bins. The scale then calculates the weight of each item based on the weight of the empty bin, the total weight of the filled bins, and the number of items. This information is then sent to an upstream management system, such as a warehouse control system (WCS) or warehouse execution system (WES). The upstream management system then records the reported weight of the empty bin, the total weight of the filled bins, the weight of each item, and the item dimensions in a database.

[0038] Step 102: Based on the image of the material box, identify the target information of the material box; the target information includes at least one of the following: the indication information of the material box, the type of the material to be processed, the shape information of the material to be processed, the volume ratio of the material box and the picking posture information of the material to be processed; the indication information is used to indicate whether the material box is divided or not divided.

[0039] Specifically, based on the image of the material box, the target information of the material box can be identified by using an image recognition method; wherein the target information includes at least one of the following: the indication information of the material box, the type of material to be processed, the shape information of the material to be processed, the volume ratio of the material box and the picking posture information of the material to be processed; the indication information is used to indicate whether the material box is divided or not divided.

[0040] Step 103: Process the material to be processed based on the target task and the target information.

[0041] Specifically, the target task is an inventory task or a picking task. The upstream management system generates these tasks. These tasks are generated based on maintenance, cycle, or dynamic inventory. Maintenance inventory refers to routine maintenance or damage checks on materials. Based on the target task and target information, inventory or picking can be performed on the materials to be processed.

[0042] The material processing method provided by the present invention obtains an image of a bin containing material to be processed; based on the image of the bin, identifies target information of the bin; the target information includes at least one of the following: indication information of the bin, the type of material to be processed, the morphological information of the material to be processed, the volume ratio of the bin, and the picking posture information of the material to be processed; the indication information is used to indicate whether the bin is divided or not divided; based on the target task and the target information, the material to be processed is processed. By identifying the image of the bin containing material to be processed, the material to be processed can be automatically processed based on the identified target information and target task, thereby improving the efficiency of material processing.

[0043] Optionally, a specific implementation of step 103 includes: When the target task is an inventory task, the materials to be processed are inventory-taken based on the target information; when the target task is a picking task, the materials to be processed are picked based on the target information.

[0044] Specifically, if the target task generated by the upstream management system is an inventory task, the materials to be processed are counted based on the target information; or if the target task generated by the upstream management system is a picking task, the materials to be processed are picked based on the target information. Through inventory tasks or picking tasks, the automated inventory or picking of materials is achieved, improving the efficiency of inventory or picking.

[0045] Optionally, when the target information includes the indication information of the material box and the type of the material to be processed, the inventorying of the material to be processed based on the target information includes: When the indication information indicates that the material box is not divided into compartments, based on the type of the material to be processed, the database is called to determine the individual material weights of the material to be processed; the database pre-stores the individual material weights corresponding to multiple types of materials; and based on the individual material weights, the material to be processed is inventoryed.

[0046] Specifically, the database pre-stores the weights of individual materials corresponding to multiple types of materials. The database also stores weight information of empty material boxes, total weight information of material boxes filled with materials, and size information of the materials.

[0047] If the target information includes bin information and the type of material to be processed, and the bin information indicates that the bin is undivided, the database can be used to determine the individual weights of the materials to be processed based on the type of material to be processed. The individual weights are used to determine whether the materials are relatively heavy or light. Based on the individual weights, an inventory of the materials to be processed can then be performed.

[0048] Optionally, the inventorying of the materials to be processed based on the weight of the individual materials includes: When the weight of the single material is greater than a first preset threshold, the quantity of the materials to be processed is determined based on the weighing information of the materials to be processed and the weight of the single material; and the materials to be processed are inventoryed based on the quantity of the materials to be processed.

[0049] Specifically, if the weight of a single material exceeds a first preset threshold, indicating that the material to be processed is relatively heavy, a weighing inventory is performed. This ratio of the material weighing information to the weight of each material is used to determine the quantity of material to be processed. This weight information is obtained by the weighing module. Based on the quantity of material to be processed, an inventory is then taken and reported to the upstream management system. This automated inventory of materials, using the weighing information and the weight of each material, improves both accuracy and efficiency.

[0050] Optionally, when the target information further includes the volume ratio of the material box and the form information of the material to be processed, the inventorying of the material to be processed based on the weight of the single material includes: When the weight of the single material is not greater than a first preset threshold and the volume ratio is less than a second preset threshold, determine whether the material to be processed is easy to grasp based on the type of the material to be processed and the morphological information of the material to be processed; and based on the determination result, take inventory of the material to be processed.

[0051] Specifically, when the weight of a single material is not greater than the first preset threshold and the volume ratio is less than the second preset threshold, it means that the material to be processed is a relatively light material and the material box is not full of the material to be processed. At this time, based on the type of the material to be processed and the morphological information of the material to be processed, determine whether the material to be processed is easy to grasp; based on the determination result, the material to be processed is inventoryed.

[0052] Optionally, when the target information includes indication information of the material box, the type of the material to be processed, the volume ratio of the material box, and the form information of the material to be processed, the inventorying of the material to be processed based on the target information includes: When the indication information indicates that the material box is divided and the volume ratio of the material box is less than a third preset threshold, determine whether the material to be processed is easy to grasp based on the type of the material to be processed and the morphological information of the material to be processed; and based on the determination result, take inventory of the material to be processed.

[0053] Specifically, when the target information includes the indication information of the material box, the type of material to be processed, the volume ratio of the material box and the morphological information of the material to be processed, and the indication information indicates that the material box is divided and the volume ratio of the material box is less than the third preset threshold, that is, in the case of the material box being divided and the material box being not full, it can be determined whether the material to be processed is easy to grab based on the type of material to be processed and the morphological information of the material to be processed; and based on the determination result, the material to be processed is inventoryed.

[0054] Optionally, taking inventory of the materials to be processed based on the determination result includes: In the case where the material to be processed is easy to grasp, the first robotic arm is controlled to grasp the material to be processed and place the material to be processed on a visual workbench; the visual workbench is used to take inventory of the material to be processed.

[0055] Specifically, when the materials to be processed are easy to grab, that is, when the material box is not divided, the material box is not full and is easy to grab, or when the material box is divided, the material box is not full and is easy to grab, the first robot arm is controlled to grab the materials to be processed and place them on the visual workbench; the visual workbench is used to inventory the materials to be processed, that is, the visual workbench identifies the grabbed materials to be processed, determines whether the number of materials to be processed on the visual workbench is the same as the number grabbed by the first robot arm, and completes the review of the number grabbed by the first robot arm. The number grabbed by the first robot arm is determined based on the grab success rate reported by the first robot arm; when the number of materials to be processed on the visual workbench is the same as the number grabbed by the first robot arm, it means that the number grabbed reported by the first robot arm is accurate. At the same time, the visual workbench completes the inventory of the materials to be processed, the first robot arm stacks the materials to be processed into the material box, and reports the number of materials to be processed to the upstream management system. Through the mutual cooperation of the first robot arm and the visual workbench, the automated inventory of materials to be processed is realized, and the inventory efficiency is improved.

[0056] It should be noted that in the case of compartmentalized bins, partially full bins, and easy to grab, the same materials to be processed may be distributed in different bins. In this case, the different compartmented bins can be merged. The first robotic arm is then controlled to place the materials to be processed into the merged bins, and the quantity of materials to be processed is reported to the upstream management system.

[0057] Optionally, taking inventory of the materials to be processed based on the determination result includes: When the material box is not divided and the material to be processed is not easy to grab, or the material box division and the number of divisions of the material box are not greater than the fourth preset threshold and the material to be processed is not easy to grab, the second robotic arm is controlled to dump the material to be processed onto the visual workbench, and the material destacking module is controlled to destacking the material to be processed and then perform inventory.

[0058] Specifically, when the material box is not divided and the material to be processed is not easy to grab, or the material box is divided and the number of compartments is not greater than the fourth preset threshold and the material to be processed is not easy to grab, that is, when the material box is not divided, the material box is not full and it is not easy to grab, or the material box is not divided, the material box is not full, it is not easy to grab and the number of compartments is small, the second robot arm can be controlled to dump the material to be processed onto the visual workbench. Since materials may be stacked on the visual workbench, the material destacking module is controlled to destacking the material to be processed so that the material to be processed is spread flat on the visual workbench, and the visual workbench can then take inventory of the material to be processed.

[0059] Optionally, the method further includes: When the indication information indicates that the material box is divided into compartments, the volume ratio of the material box is not less than the third preset threshold, and the number of compartments of the material box is not greater than the fourth preset threshold, the second robotic arm is controlled to dump the material to be processed onto the visual workbench, and the material destacking module is controlled to destacking the material to be processed and then perform inventory.

[0060] Specifically, when the indication information indicates that the material box is divided into compartments, the volume ratio of the material box is not less than the third preset threshold, and the number of compartments of the material box is not greater than the fourth preset threshold, that is, when the material box is divided into compartments, the material box is full, the number of compartments is small and the material to be processed is not easy to grab, the second robotic arm is controlled to dump the material to be processed onto the visual workbench, and the material destacking module is controlled to perform inventory after destacking the material to be processed.

[0061] Optionally, after the visual workbench completes the inventory of the materials to be processed, it controls the second robotic arm to dump the materials to be processed into a material box, and then reports the quantity of the materials to be processed to an upstream management system.

[0062] Optionally, when the material box is not divided into compartments, the weight of a single material is not greater than the first preset threshold, and the volume ratio is not less than the second preset threshold, it means that the material to be processed is a relatively light material, and the material box is full of the material to be processed. At this time, the material box containing the material to be processed is transported to the manual inventory station for manual inventory through turnover box transport carrying modules such as handling AGVs and conveyor lines.

[0063] Optionally, when the material box is divided into compartments, the volume ratio of the material box is not less than the third preset threshold, and the number of compartments of the material box is greater than the fourth preset threshold, it means that the material box is full of materials to be processed and the number of compartments is large. At this time, the material box containing the materials to be processed is transported to the manual inventory station for manual inventory through turnover box transport carrying modules such as handling AGVs and conveyor lines.

[0064] Table 1 is the inventory method of the material handling workstation provided by the present invention. As shown in Table 1, according to the weight and shape of the material box and the weight, shape and material of the material, the inventory method of the material handling workstation can be divided into weighing inventory, robot arm picking-stacking box-visual recognition composite inventory, visual recognition inventory and manual inventory. Among them, for undivided material boxes containing relatively heavy materials, weighing inventory can be controlled; for undivided, partially full material boxes and easy to grab materials, robot arm picking-stacking box-visual recognition composite inventory can be controlled; for undivided, partially full material boxes and difficult to grab materials, visual recognition inventory can be controlled; for divided, partially full material boxes and easy to grab materials, robot arm picking-stacking box-visual recognition composite inventory can be controlled; for divided, partially full material boxes, difficult to grab materials with a small number of divided boxes, visual recognition inventory can be controlled; for divided, full material boxes and a small number of divided boxes, visual recognition inventory can be controlled; for materials with other shapes, materials, carrier characteristics, etc., manual inventory is performed.

[0065] Table 1. Inventory methods for material handling workstations

[0066] Optionally, when the target information includes the type of the material to be processed, the shape information of the material to be processed, and the picking posture information of the material to be processed, the picking of the material to be processed based on the target information includes: Based on the type of the material to be processed and the morphological information of the material to be processed, determine whether the material to be processed is easy to grasp; if the material to be processed is easy to grasp, send the picking posture information of the material to be processed to the first robotic arm; the first robotic arm is used to pick the material to be processed to a distribution vehicle based on the picking posture information of the material to be processed; the distribution vehicle is used to transport the picked material to be processed to the location of the collective order box, and transfer the picked material to be processed to the collective order box.

[0067] Specifically, when the target information includes the type of material to be processed, the shape information of the material to be processed and the picking posture information of the material to be processed, it can be determined whether the material to be processed is easy to grasp; when the material to be processed is easy to grasp, the picking posture information of the material to be processed is sent to the first robotic arm; the first robotic arm receives the picking posture information of the material to be processed and picks the material to be processed to the distribution vehicle; the distribution vehicle transports the picked material to be processed to the location of the collection order box based on the movement position information of the distribution vehicle, and transfers the picked material to be processed to the collection order box.

[0068] It should be noted that after the first robotic arm picks the materials to be processed and puts them on the distribution vehicle, it returns to its initial position; after the distribution vehicle transfers the picked materials to be processed to the collection order box, it returns to its initial position.

[0069] Optionally, determine whether the picking collection list corresponding to the picking task has been picked; if the picking collection list has not been picked, repeat the step of sending the picking posture information of the material to be processed to the first robotic arm until all the picking collection lists are picked.

[0070] Specifically, after the sorting vehicle transfers the picked materials to be processed to the collection order box, it determines whether the picking collection order corresponding to the picking task has been completed; if the picking collection order has not been completed, repeat the step of sending the picking posture information of the materials to be processed to the first robotic arm until all the picking collection orders are completed; wherein, in the process of executing each step, the picking posture information of the materials to be processed is obtained based on the image recognition of the currently acquired material box.

[0071] When the picking of the picking set is completed, the inventory of the materials to be processed is completed, and the picking information is reported to the upstream management system, where the picking information includes at least one of the following: the type and quantity of the picked materials, abnormal information during the picking process, and picking completion information.

[0072] Optionally, when the materials to be processed are not easy to grab, the boxes containing the materials to be processed are transported to the manual inventory station for manual inventory by using turnover box transport carrying modules such as handling AGVs and conveyor lines.

[0073] Figure 3 This is a system diagram of the material processing workstation provided by the present invention. Figure 3As shown, the material handling workstation includes a material box, a material box handling module with visual recognition function, a weighing module with visual recognition function, a first robotic arm, a second robotic arm, a material destacking robotic arm, a visual inventory workbench, a sorting vehicle, a collection order box (such as collection order box 1, collection order box 2, collection order box 3, collection order box 4 and collection order box 5) and a robotic arm quick-change end pick-up device. Among them, the material boxes containing materials to be processed are transported to the material processing workstation through turnover box transport carrying modules such as handling AGVs and conveyor lines; the material box handling module with visual recognition function is used to identify the shape and posture information of the material boxes, and perform the material box carrying action between turnover box transmission carrying modules such as handling AGVs and conveyor lines and the weighing module; the weighing platform is used to weigh the material boxes containing materials to be processed, and recognize the image of the material boxes containing materials to be processed to obtain the target information of the material boxes; the visual inventory workbench is used to place the materials successfully grasped by the first robotic arm or the materials to be boxed, and to identify the quantity and posture information of the materials successfully grasped by the first robotic arm; the first robotic arm is used to perform material grasping, picking and boxing actions, so that the visual inventory workbench can inventory and verify the quantity of grasped materials according to the grasping success rate, and is also used to perform end picker quick change according to the material size or material, thereby improving the success rate of material grasping and boxing. The second robotic arm is used to dump the hard-to-grasp materials in the material box onto the visual inventory workbench, and dump the hard-to-grasp materials into the material box after the visual workbench completes the inventory; the material destacking module is used to destacking the materials so that the materials are spread flat on the visual inventory workbench to facilitate visual identification and inventory. The sorting vehicle is used to perform picking tasks using the first robotic arm when the workstation is not performing inventory tasks, and transport the materials to the corresponding collection order box. The material handling workstation also includes a manual inventory station, which transports the material boxes to the manual inventory station for manual inventory through turnover box transmission and carrying modules such as handling AGVs and conveyor lines.

[0074] Figure 4 This is the second flow diagram of the material processing method provided by the present invention, such as Figure 4 As shown, the method includes steps 401 to 426.

[0075] Step 401: Remove the plastic film from the materials and prepare for storage.

[0076] Step 402: Weigh the empty material box to obtain the weight of the empty material box.

[0077] Step 403: After the material is placed in the material box, it is weighed to obtain the total weight of the material and the material box.

[0078] Step 404: Report the empty bin weight, the total weight of the materials and bin, and the weight of each item. Specifically, the weight of each item is calculated based on the empty bin weight, the total weight of the materials and bin, and the number of items. The empty bin weight, the total weight of the materials and bin, and the weight of each item are then uploaded to the upstream management system.

[0079] Step 405: Store in database. Specifically, the upstream management system records the weight of the empty material box, the total weight of the material and the material box, the weight of each material, and the size of the material into the database.

[0080] Step 406: The material boxes are put into storage. Specifically, the material boxes containing the materials to be processed are transported to the storage warehouse by using turnover box transport carrying modules such as transport AGVs and conveyor lines.

[0081] Step 407: The upstream management system generates an inventory task, wherein the inventory task is generated based on maintenance inventory, cycle inventory, or dynamic inventory.

[0082] Step 408: The materials are transported to the material processing workstation. The material boxes containing the materials to be processed are transported to the material processing workstation by a turnover box transport carrying module such as a transport AGV or a conveyor line.

[0083] Step 409: Weigh the bin, take a photo, and identify the target information. Specifically, the weighing module is controlled to weigh the bin and take a photo. Based on the image of the bin, image recognition methods are used to identify the target information of the bin. The target information includes at least one of the following: bin indication information, type of material to be processed, shape information of the material to be processed, volume ratio of the bin, and picking posture information of the material to be processed. The indication information is used to indicate whether the bin is divided or not divided.

[0084] Step 410: Determine whether the material box is divided into compartments. If the material box is not divided into compartments, go to step 411; if the material box is divided into compartments, go to step 422.

[0085] Step 411: Determine the weight of a single material. Specifically, based on the type of the material to be processed, call a database to determine the weight of a single material of the material to be processed; the database pre-stores the weight of a single material corresponding to multiple types of materials.

[0086] Step 412: Determine whether the weight of the single material is greater than a first preset threshold. If the weight of the single material is greater than the first preset threshold, go to step 413; if the weight of the single material is not greater than the first preset threshold, go to step 414.

[0087] Step 413: Determine the quantity of materials to be processed. Specifically, determine the quantity of materials to be processed based on the weighing information of the materials to be processed and the weight of each material; and perform an inventory of the materials to be processed based on the quantity of the materials to be processed.

[0088] Step 414: Determine whether the floor area ratio is less than a second preset threshold. If the floor area ratio is less than the second preset threshold, proceed to step 415; if the floor area ratio is not less than the second preset threshold, proceed to step 426.

[0089] Step 415: Determine whether the material to be processed is easy to grasp. Specifically, based on the type and morphology of the material to be processed, determine whether the material to be processed is easy to grasp. If the material to be processed is easy to grasp, proceed to step 416. If the material to be processed is not easy to grasp, proceed to step 420 if the bin is not divided into compartments, or proceed to step 425 if the bin is divided into compartments.

[0090] Step 416: Control the first robotic arm to grab the material to be processed and place the material to be processed on the visual workbench.

[0091] Step 417: Perform an inventory based on the first robotic arm's grasping success rate and the number of visually recognized items. Specifically, the visual workbench identifies the grasped materials to be processed, determines whether the number of materials on the visual workbench is the same as the number grasped by the first robotic arm, and completes a review of the number grasped by the first robotic arm. The number grasped by the first robotic arm is determined based on the grasping success rate reported by the first robotic arm. If the number of materials on the visual workbench is the same as the number grasped by the first robotic arm, the number grasped reported by the first robotic arm is accurate, and the visual workbench completes the inventory of the materials to be processed.

[0092] Step 418: Control the first robotic arm to stack the materials to be processed into the material box.

[0093] Step 419: Report the inventory information to the upstream management system. The inventory information includes at least one of the following: quantity, type, and signal of the materials to be processed.

[0094] Step 420 , control the second robotic arm to dump the material to be processed onto the visual workbench, and control the material de-stacking module to de-stacking the material to be processed, and execute step 417 , and then execute step 421 .

[0095] Step 421: Control the second robotic arm to dump the material to be processed into the material box.

[0096] Step 422: Determine whether the floor area ratio is less than a third preset threshold. If the floor area ratio is less than the third preset threshold, proceed to step 415 and execute steps 415-417 before executing step 423. If the floor area ratio is not less than the third preset threshold, proceed to step 425.

[0097] Step 423: Merge the compartmented bins. Specifically, since the same materials to be processed may be distributed in different bins, the different compartmented bins may be merged.

[0098] Step 424 : Control the first robotic arm to stack the materials to be processed into the material box, and execute step 419 .

[0099] Step 425: Determine whether the number of grids is less than a fourth preset threshold. If the number of grids is less than the fourth preset threshold, go to step 420; if the number of grids is not less than the fourth preset threshold, go to step 426. Step 426: Manual inventory. Specifically, the boxes containing the materials to be processed are transported to the manual inventory station for manual inventory by using turnover box transport carrying modules such as transport AGVs and conveyor lines.

[0100] Figure 5 This is the third flow diagram of the material processing method provided by the present invention, such as Figure 5 As shown, the method includes steps 501 to 513.

[0101] Step 501: The upstream management system generates a picking task.

[0102] Step 502: The material box is transported to the picking position. Specifically, the material box containing the material to be processed is transported to the picking position of the material processing workstation by a turnover box transport carrying module such as a transport AGV or a conveyor line.

[0103] Step 503: Take a photo of the bin and identify the target information. Specifically, a 2D / 3D camera is used to take a photo of the bin. Based on the image of the bin, image recognition methods are used to identify the target information of the bin. The target information includes at least one of the following: the type of material to be processed, the shape of the material to be processed, and the picking posture information of the material to be processed.

[0104] Step 504: Determine whether the material to be processed is easy to grasp. Specifically, based on the type and morphology of the material to be processed, determine whether the material to be processed is easy to grasp. If the material to be processed is easy to grasp, proceed to step 505; if the material to be processed is not easy to grasp, proceed to step 513.

[0105] Step 505: Take a photo of the material box and identify the picking posture information of the material to be processed.

[0106] Step 506: Send the picking posture information of the material to be processed to the first robotic arm.

[0107] Step 507: Control the first robotic arm to pick the materials to be processed to the distribution vehicle.

[0108] Step 508: Send the movement position information of the distribution vehicle to the distribution vehicle.

[0109] Step 509: Control the distribution vehicle to transport the materials to be processed to the location of the collective order box, and then transfer the materials to be processed to the collective order box.

[0110] Step 510: The first robotic arm and the distribution vehicle return to their initial positions.

[0111] Step 511: Determine whether the picking set list corresponding to the picking task has been picked. If the picking set list has not been picked, go to step 505; if the picking set list has been picked, go to step 512.

[0112] Step 512: Report the picking information to the upstream management system. The picking information includes at least one of the following: the type and quantity of the picked materials, abnormal information during the picking process, and picking completion information.

[0113] Step 513: Manual picking. Specifically, the boxes containing the materials to be processed are transported to the manual inventory station for manual picking by using turnover box transport carrying modules such as transport AGVs and conveyor lines.

[0114] This invention integrates a 2D / 3D camera, a weighing module (load cell), a first robotic arm, a second robotic arm, and a sorting cart, utilizing visual recognition, weight calculation, robotic gripping, picking, and stacking to achieve automated inventory and sorting of warehouse materials. Furthermore, this invention develops different inventory and sorting methods based on the weight and shape of the bins and materials, improving both the success rate and efficiency of these processes.

[0115] The material processing device provided by the present invention is described below. The material processing device described below and the material processing method described above can be referenced to each other.

[0116] Figure 6 It is a structural diagram of the material processing device provided by the present invention, such as Figure 6 As shown, the material processing device 600 includes an acquisition module 601, an identification module 602 and a processing module 603; wherein, An acquisition module 601 is used to acquire an image of a material box containing a material to be processed; The identification module 602 is configured to identify target information of the material box based on the image of the material box; the target information includes at least one of the following: indication information of the material box, type of the material to be processed, form information of the material to be processed, volume ratio of the material box, and picking posture information of the material to be processed; the indication information is used to indicate whether the material box is divided or not divided; The processing module 603 is used to process the material to be processed based on the target task and the target information.

[0117] The material processing device provided by the present invention obtains an image of a bin containing material to be processed; based on the image of the bin, identifies target information for the bin; the target information includes at least one of the following: indication information of the bin, the type of material to be processed, the morphological information of the material to be processed, the volume ratio of the bin, and the picking posture information of the material to be processed; the indication information is used to indicate whether the bin is divided or not divided; and the material to be processed is processed based on a target task and the target information. By identifying the image of the bin containing material to be processed, the material to be processed can be automatically processed based on the identified target information and target task, thereby improving the efficiency of material processing.

[0118] Optionally, the processing module 603 is specifically configured to: In the case where the target task is an inventory task, performing an inventory of the materials to be processed based on the target information; In the case where the target task is a picking task, the materials to be processed are picked based on the target information.

[0119] Optionally, the processing module 603 is further configured to: When the indication information indicates that the material box is not divided into grids, based on the type of the material to be processed, calling a database to determine the weight of a single material of the material to be processed; the database pre-stores the weight of a single material corresponding to a plurality of types of materials; Based on the weight of the individual materials, the materials to be processed are counted.

[0120] Optionally, the processing module 603 is further configured to: When the weight of the single material is greater than a first preset threshold, determining the quantity of the material to be processed based on the weighing information of the material to be processed and the weight of the single material; Based on the quantity of the materials to be processed, an inventory is taken of the materials to be processed.

[0121] Optionally, when the target information further includes the volume ratio of the material box and the form information of the material to be processed, the processing module 603 is further configured to: When the weight of the single material is not greater than a first preset threshold and the volume ratio is less than a second preset threshold, determining whether the material to be processed is easy to grasp based on the type of the material to be processed and the morphological information of the material to be processed; Based on the determination result, an inventory is taken of the materials to be processed.

[0122] Optionally, when the target information includes the indication information of the material box, the type of the material to be processed, the volume ratio of the material box, and the form information of the material to be processed, the processing module 603 is further configured to: When the indication information indicates that the material box is divided into compartments and the volume ratio of the material box is less than a third preset threshold, determining whether the material to be processed is easy to grasp based on the type of the material to be processed and the morphological information of the material to be processed; Based on the determination result, an inventory is taken of the materials to be processed.

[0123] Optionally, the processing module 603 is further configured to: In the case where the material to be processed is easy to grasp, the first robotic arm is controlled to grasp the material to be processed and place the material to be processed on a visual workbench; the visual workbench is used to take inventory of the material to be processed.

[0124] Optionally, the processing module 603 is further configured to: When the material box is not divided and the material to be processed is not easy to grab, or when the material box is divided, the number of compartments in the material box is not greater than a fourth preset threshold, and the material to be processed is not easy to grab, the second robotic arm is controlled to dump the material to be processed onto the visual workbench, and the material destacking module is controlled to destacking the material to be processed and then perform inventory.

[0125] Optionally, the material processing device 600 further includes: The dumping module is used to control the second robotic arm to dump the material to be processed onto the visual workbench when the indication information indicates that the material box is divided, the volume ratio of the material box is not less than the third preset threshold, and the number of compartments of the material box is not greater than the fourth preset threshold, and control the material destacking module to destacking the material to be processed and then conduct inventory.

[0126] Optionally, when the target information includes the type of the material to be processed, the shape information of the material to be processed, and the picking posture information of the material to be processed, the processing module 603 is further configured to: Determining whether the material to be processed is easy to grasp based on the type of the material to be processed and the morphological information of the material to be processed; When the material to be processed is easy to grasp, the picking posture information of the material to be processed is sent to the first robotic arm; the first robotic arm is used to pick the material to be processed to a distribution vehicle based on the picking posture information of the material to be processed; the distribution vehicle is used to transport the picked material to be processed to the location of the collection order box, and transfer the picked material to be processed to the collection order box.

[0127] Optionally, the material processing device 600 further includes: A judgment module, used to judge whether the picking collection list corresponding to the picking task has been picked; The repeated execution module is used to repeatedly execute the step of sending the picking posture information of the material to be processed to the first robotic arm when the picking of the picking set list is not completed until all the picking of the picking set list is completed.

[0128] Figure 7 This is a schematic diagram of the physical structure of an electronic device provided by the present invention, such as Figure 7 As shown, the electronic device 700 may include: a processor 710, a communications interface 720, a memory 730, and a communications bus 740, wherein the processor 710, the communications interface 720, and the memory 730 communicate with each other via the communications bus 740. The processor 710 may invoke logic instructions in the memory 730 to execute a material processing method, which includes: acquiring an image of a bin containing materials to be processed; identifying target information of the bin based on the image of the bin; the target information includes at least one of the following: indication information of the bin, the type of the materials to be processed, the form information of the materials to be processed, the volume ratio of the bin, and the picking posture information of the materials to be processed; the indication information is used to indicate whether the bin is divided or not divided; and processing the materials to be processed based on the target task and the target information.

[0129] Furthermore, the logic instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0130] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the material processing method provided by the above methods, which includes: obtaining an image of a box containing the material to be processed; based on the image of the box, identifying the target information of the box; the target information includes at least one of the following: indication information of the box, the type of the material to be processed, the shape information of the material to be processed, the volume ratio of the box and the picking posture information of the material to be processed; the indication information is used to indicate whether the box is divided or not divided; based on the target task and the target information, the material to be processed is processed.

[0131] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the material processing method provided by the above-mentioned methods, the method comprising: obtaining an image of a material box containing the material to be processed; based on the image of the material box, identifying the target information of the material box; the target information comprises at least one of the following: indication information of the material box, the type of the material to be processed, the morphological information of the material to be processed, the volume ratio of the material box and the picking posture information of the material to be processed; the indication information is used to indicate whether the material box is divided or not divided; based on the target task and the target information, the material to be processed is processed.

[0132] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0133] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0134] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A material processing method, characterized in that: include: Acquire an image of a bin containing material to be processed; Based on the image of the material box, identifying target information of the material box; The target information includes at least one of the following: indication information of the material box, type of the material to be processed, shape information of the material to be processed, volume ratio of the material box, and picking posture information of the material to be processed; The indication information is used to indicate whether the material box is divided or not divided; Based on the target task and the target information, the material to be processed is processed.

2. The material processing method according to claim 1, characterized in that: The processing of the material to be processed based on the target task and the target information includes: In the case where the target task is an inventory task, performing an inventory of the materials to be processed based on the target information; In the case where the target task is a picking task, the materials to be processed are picked based on the target information.

3. The material processing method according to claim 2, characterized in that: In a case where the target information includes the indication information of the material box and the type of the material to be processed, the inventorying of the material to be processed based on the target information includes: When the indication information indicates that the material box is not divided into grids, based on the type of the material to be processed, calling a database to determine the weight of a single material of the material to be processed; the database stores the weights of single materials corresponding to multiple types of materials; Based on the weight of the individual materials, the materials to be processed are counted.

4. The material processing method according to claim 3, characterized in that: The inventorying of the materials to be processed based on the weight of the individual materials includes: When the weight of the single material is greater than a first preset threshold, determining the quantity of the material to be processed based on the weighing information of the material to be processed and the weight of the single material; Based on the quantity of the materials to be processed, an inventory is taken of the materials to be processed.

5. The material processing method according to claim 3, characterized in that: In a case where the target information further includes the volume ratio of the material box and the form information of the material to be processed, the inventorying of the material to be processed based on the weight of the individual materials includes: When the weight of the single material is not greater than a first preset threshold and the volume ratio is less than a second preset threshold, determining whether the material to be processed is easy to grasp based on the type of the material to be processed and the morphological information of the material to be processed; Based on the determination result, an inventory is taken of the materials to be processed.

6. The material processing method according to claim 2, characterized in that: In a case where the target information includes indication information of the material box, the type of the material to be processed, the volume ratio of the material box, and the form information of the material to be processed, the inventorying of the material to be processed based on the material box information includes: When the indication information indicates that the material box is divided into compartments and the volume ratio of the material box is less than a third preset threshold, determining whether the material to be processed is easy to grasp based on the type of the material to be processed and the morphological information of the material to be processed; Based on the determination result, an inventory is taken of the materials to be processed.

7. The material processing method according to claim 5 or 6, characterized in that: The method of taking inventory of the materials to be processed based on the determination result includes: In the case where the material to be processed is easy to grasp, the first robotic arm is controlled to grasp the material to be processed and place the material to be processed on a visual workbench; the visual workbench is used to take inventory of the material to be processed.

8. The material processing method according to claim 5 or 6, characterized in that: The method of taking inventory of the materials to be processed based on the determination result includes: When the material box is not divided and the material to be processed is not easy to grab, or when the material box is divided, the number of compartments in the material box is not greater than a fourth preset threshold, and the material to be processed is not easy to grab, the second robotic arm is controlled to dump the material to be processed onto the visual workbench, and the material destacking module is controlled to destacking the material to be processed and then perform inventory.

9. The material processing method according to claim 6, characterized in that: The method further comprises: When the indication information indicates that the material box is divided into compartments, the volume ratio of the material box is not less than the third preset threshold, and the number of compartments of the material box is not greater than the fourth preset threshold, the second robotic arm is controlled to dump the material to be processed onto the visual workbench, and the material destacking module is used to destacking the material to be processed and then perform inventory.

10. The material processing method according to claim 2, characterized in that: In a case where the target information includes the type of the material to be processed, the shape information of the material to be processed, and the picking posture information of the material to be processed, the picking of the material to be processed based on the target information includes: Determining whether the material to be processed is easy to grasp based on the type of the material to be processed and the morphological information of the material to be processed; When the material to be processed is easy to grasp, the picking posture information of the material to be processed is sent to the first robotic arm; the first robotic arm is used to pick the material to be processed to a distribution vehicle based on the picking posture information of the material to be processed; the distribution vehicle is used to transport the picked material to be processed to the location of the collection order box, and transfer the picked material to be processed to the collection order box.

11. The material processing method according to claim 10, characterized in that: The method further comprises: Determine whether the picking collection sheet corresponding to the picking task has been picked; In the case that the picking set list has not been picked up, the step of sending the picking posture information of the material to be processed to the first robotic arm is repeated until all the picking sets are picked up.

12. A material processing device, characterized in that: include: an acquisition module for acquiring an image of a bin containing a material to be processed; an identification module, configured to identify target information of the material box based on the image of the material box; The target information includes at least one of the following: indication information of the material box, type of the material to be processed, shape information of the material to be processed, volume ratio of the material box, and picking posture information of the material to be processed; The indication information is used to indicate whether the material box is divided or not divided; A processing module is used to process the material to be processed based on the target task and the target information.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the material processing method according to any one of claims 1 to 11 is implemented.

14. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the material processing method according to any one of claims 1 to 11 is implemented.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the material processing method according to any one of claims 1 to 11 is implemented.

Images