Material surplus management method, device, equipment and storage medium

Abstract

The present application discloses a management method, device, equipment and storage medium for the remaining amount of materials, relates to the field of warehouse management technology, and discloses a management method for the remaining amount of materials, including: after determining that the user has completed the collection of materials, calling up the shooting picture in the warehouse robot transport warehouse and the monitoring picture at the location of the warehouse where the materials are stored; extracting the material quantity of the user's missing materials from the shooting picture, and extracting the packaging quantity of the packaged materials and the stacking picture of the stacked materials from the monitoring picture; extracting the stacking features of the stacking picture based on a preset decomposition model, and predicting the number of scattered materials of the stacked materials according to the feature extraction results; integrating the material quantity, the scattered material quantity and the packaging quantity to determine the remaining quantity of the remaining materials in the warehouse. That is, by reducing the data that needs to be managed and accurately decomposing the number of materials stacked in the warehouse through the preset decomposition model, the management efficiency of the remaining amount of materials is improved.

Description

Technical Field

[0001] The present application relates to the field of warehouse management technology, and in particular to a management method, device, equipment and storage medium for remaining material quantities. Background Art

[0002] Warehouse management requires accurate statistics of the remaining materials in the warehouse to avoid delays in the project due to insufficient remaining materials when users collect materials from the warehouse.

[0003] Currently, after users pick up materials from the warehouse, the warehouse management platform usually checks the remaining materials in the warehouse based on the user's demand list for materials to take inventory. However, when the storage robot retrieves materials from the shelf, it may scatter scattered materials under the shelf or the user may take less materials when picking up materials. Therefore, errors occur in the verification of the remaining materials in the warehouse, making the warehouse's management of the remaining materials inefficient.

[0004] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of this application is to provide a method for managing the remaining amount of materials, aiming to solve the technical problem that errors occur in the verification of the remaining materials in the warehouse, resulting in low efficiency in the management of the remaining amount of materials in the warehouse.

[0006] To achieve the above purpose, the present application proposes a method for managing the remaining amount of materials, the method comprising:

[0007] After confirming that the user has completed picking up the materials, retrieve the captured images in the storage robot transport warehouse and the monitoring images at the location where the picked up materials are stored in the warehouse;

[0008] Extracting the quantity of materials omitted by the user from the shooting picture, and extracting the packing quantity of the packing materials and the stacking picture of the stacking materials from the monitoring picture;

[0009] Extracting stacking features from the stacking picture based on a preset decomposition model, and predicting the quantity of scattered materials of the stacked materials according to the feature extraction result, wherein the preset decomposition model is obtained by training the stacking conditions of various materials at different quantities, and the stacking conditions are obtained after simulating the stacking of different quantities according to the appearance characteristics of various materials;

[0010] The material quantity, the scattered material quantity and the packaged quantity are integrated to determine the remaining quantity of the remaining materials in the warehouse.

[0011] In one embodiment, after determining that the user has completed picking up the material, before the step of retrieving the captured image in the storage robot transport warehouse and the monitoring image at the location of the picked-up material storage warehouse comprises:

[0012] Acquire a stacking image of a sample material and the stacking quantity of the sample material;

[0013] extracting stacking features of the sample material from the stacked image;

[0014] Determining a placement method of each of the sample materials based on the stacking characteristics;

[0015] Simulating stacking conditions of different quantities of sample materials according to the placement method and the appearance characteristics of the sample materials;

[0016] The placement manner and the appearance features are used as stacking features of the stacking situation, and a preset model to be trained is trained to obtain a preset decomposition model.

[0017] In one embodiment, the step of simulating stacking conditions of different quantities of sample materials according to the placement manner and the appearance characteristics of the sample materials includes:

[0018] Determining, according to the appearance features and the placement methods, the forces to which adjacent samples are subjected when placed in different ways;

[0019] Based on the forces acting on the sample materials, the stacking conditions of different quantities of sample materials when the forces reach equilibrium are simulated.

[0020] In one embodiment, the step of determining the placement of each sample material based on the stacking feature includes:

[0021] Extracting contact points between adjacent sample materials from the stacking features;

[0022] Simulating a stacking process of the sample materials based on the contact points to determine a balance value of each of the sample materials after stacking;

[0023] Determine a target balance value at which the balance value satisfies a preset balance threshold requirement;

[0024] The sample placement method corresponding to the target balance value is used as the placement method of each of the sample materials.

[0025] In one embodiment, the step of extracting the packaging quantity of packaging materials and the stacking picture of stacking materials from the monitoring picture includes:

[0026] Identifying the package quantity of the packaged materials from the monitoring picture, and extracting a distribution map of unpackaged scattered materials from the monitoring picture;

[0027] Extracting the accumulation characteristics of each of the scattered materials from the distribution map;

[0028] Based on the accumulation characteristics, determining whether the scattered materials are accumulated;

[0029] If it is determined that the scattered materials are piled up, the piled up scattered materials are determined to be piled up materials, and a stacking picture of the piled up materials is extracted from the distribution map.

[0030] In one embodiment, after the step of integrating the material quantity, the scattered material quantity and the packaged quantity to determine the remaining quantity of the remaining material in the warehouse, the step further includes:

[0031] Retrieving a recording screen of the user taking materials from the transport bin of the storage robot and the storage quantity before the user picks up the materials;

[0032] Extracting the quantity of materials taken by the user from the record screen;

[0033] Add the received quantity to the remaining quantity and compare it with the storage quantity to determine whether the remaining quantity is correct;

[0034] If the remaining quantity is incorrect, determine whether the claimed quantity is correct;

[0035] If the quantity picked up is correct, record the material scattered and the quantity scattered.

[0036] In addition, to achieve the above purpose, the present application also proposes a material remaining amount management device, the material remaining amount management device comprising:

[0037] The retrieval module is used to retrieve the captured images in the storage robot transport warehouse and the monitoring images at the storage warehouse where the materials are stored after the user has completed the material collection;

[0038] An extraction module, used to extract the material quantity of the material omitted by the user from the shooting picture, and to extract the packing quantity of the packing material and the stacking picture of the stacking material from the monitoring picture;

[0039] A prediction module, used for extracting stacking features from the stacking picture based on a preset decomposition model, and predicting the quantity of scattered materials of the stacked materials according to the feature extraction result, wherein the preset decomposition model is obtained by training the stacking conditions of various materials at different quantities, and the stacking conditions are obtained after simulating the stacking of different quantities according to the appearance characteristics of various materials;

[0040] The integration module is used to integrate the material quantity, the scattered material quantity and the packaged quantity to determine the remaining quantity of the remaining materials in the warehouse.

[0041] In addition, to achieve the above-mentioned purpose, the present application also proposes a material remaining quantity management device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the material remaining quantity management method as described above.

[0042] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the material remaining quantity management method as described above are implemented.

[0043] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the material remaining quantity management method as described above.

[0044] One or more technical solutions proposed in this application have at least the following technical effects:

[0045] Since users may miss some materials when taking them from the transport warehouse of the storage robot, in order to ensure that the source of each material in the warehouse can be ensured when the user misses some materials, the user needs to retrieve the captured images in the transport warehouse of the storage robot after picking up the materials. In addition, since different types of materials are stored in the warehouse, in order to avoid repeated inventory of materials whose quantities have not changed, wasting material management resources and affecting material management efficiency, it is also necessary to retrieve the monitoring images of the warehouse where the materials are stored, so as to extract the packaging quantity of the packaged materials and the stacking images of the stacked materials from the monitoring images, and extract the material quantity of the materials missed by the user from the captured images, so as to avoid being unable to Accurately identify the material data of stacked materials. Therefore, use the preset decomposition model trained by the stacking conditions of various materials in different quantities to decompose the stacked materials and determine the number of scattered materials in the stacked materials. When training the preset decomposition model, the stacking conditions are obtained after simulating stacking of different quantities based on the appearance characteristics of various materials. Therefore, the preset decomposition model can be used to accurately decompose various stacked materials. Finally, the remaining number of materials in the warehouse is determined by integrating the material quantity, the scattered material quantity and the packaged quantity. That is, by reducing the data that needs to be managed and accurately decomposing the number of stacked materials in the warehouse through the preset decomposition model, the management efficiency of the remaining material quantity is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0047] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0048] Figure 1 A flow chart of the first embodiment of the method for managing the remaining amount of materials in this application;

[0049] Figure 2 A flow chart of the second embodiment of the method for managing the remaining amount of materials in this application;

[0050] Figure 3 A flow chart of the third embodiment of the method for managing the remaining amount of materials in this application;

[0051] Figure 4 This is a schematic diagram of the module structure of the material remaining amount management device of the embodiment of the present application;

[0052] Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the material remaining quantity management method in the embodiment of the present application.

[0053] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0054] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0055] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0056] The main solution of the embodiment of the present application is: after determining that the user has completed collecting materials, the warehouse management platform retrieves the shooting picture in the warehouse robot transport warehouse and the monitoring picture of the warehouse location where the collected materials are stored; extracts the material quantity of the user's omitted materials from the shooting picture, and extracts the packing quantity of the packaged materials and the stacking picture of the stacked materials from the monitoring picture; performs stacking feature extraction on the stacking picture based on a preset decomposition model, and predicts the number of scattered materials of the stacked materials according to the feature extraction result, the preset decomposition model is trained by the stacking conditions of various materials in different quantities, and the stacking condition is obtained after simulating the stacking of different quantities according to the appearance characteristics of various materials; the material quantity, the scattered material quantity and the packing quantity are integrated to determine the remaining quantity of the remaining materials in the warehouse.

[0057] In this embodiment, for ease of description, the following description is made with the warehouse management platform as the execution entity.

[0058] After users pick up materials from the warehouse, the warehouse management platform usually checks the remaining materials in the warehouse based on the user's demand list for materials to take inventory. However, when the storage robot retrieves materials from the shelf, it may scatter scattered materials under the shelf or the user may take less materials when picking up materials. Therefore, errors occur in the verification of the remaining materials in the warehouse, making the warehouse's management of the remaining materials inefficient.

[0059] The present application provides a solution. Since the user may miss some materials when taking them from the transport warehouse of the storage robot, in order to ensure that the source of each material in the warehouse can be ensured when the user misses some materials, after the user completes picking up the materials, it is necessary to retrieve the captured images in the transport warehouse of the storage robot. Moreover, since different kinds of materials are stored in the warehouse, in order to avoid repeated inventory of materials whose quantity has not changed, wasting material management resources and affecting material management efficiency, it is also necessary to retrieve the monitoring images of the warehouse where the materials are stored, so as to extract the packaging quantity of the packaged materials and the stacking images of the stacked materials from the monitoring images, and extract the material quantity of the materials missed by the user from the captured images. In order to avoid the inaccurate identification of material data of stacked materials, the stacked materials are decomposed by a preset decomposition model trained using the stacking conditions of various materials in different quantities to determine the number of scattered materials in the stacked materials. When training the preset decomposition model, the stacking conditions are obtained after simulating stacking of different quantities based on the appearance characteristics of various materials. Therefore, the preset decomposition model can be used to accurately decompose various stacked materials. Finally, the remaining number of materials in the warehouse is determined by integrating the material quantity, the scattered material quantity and the packaged quantity. That is, by reducing the data that needs to be managed and accurately decomposing the number of stacked materials in the warehouse through the preset decomposition model, the management efficiency of the remaining material quantity is improved.

[0060] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a warehouse management platform, etc. The following takes the warehouse management platform as an example to illustrate this embodiment and the following embodiments.

[0061] Based on this, the present application embodiment provides a method for managing the remaining amount of materials, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the material remaining quantity management method of the present application.

[0062] In this embodiment, the method for managing the remaining amount of materials includes steps S10 to S40:

[0063] Step S10, after determining that the user has completed picking up the materials, retrieve the captured images in the transport warehouse of the storage robot and the monitoring images at the location of the warehouse where the materials are stored;

[0064] It should be noted that the captured images are images of the transport warehouse taken by the storage robot when transporting materials. The monitoring images can be monitoring images of the shelves where materials are picked up. If there are no monitoring images of the shelves, they can also be images of the shelves where materials are placed when the storage robot obtains materials from the shelves. They can also be images taken by the transfer robot when taking materials from the shelves. They can also be images selected from various monitoring images of the entire warehouse, including images of the storage location where materials are picked up.

[0065] It is understandable that in the material collection system, when the user completes the relevant operations of collecting materials, for example, clicking Confirm on the confirmation interface of collecting materials, the subsequent screen retrieval process is triggered. This can be achieved through the interactive interface with the material collection system to ensure that it is started after the user has collected the materials, so as to avoid the user not being able to collect all the materials at once due to a large number of materials, and obtaining useless screens.

[0066] It is understandable that the shooting images in the storage robot transport warehouse can reflect the situation where the user may miss materials, and the monitoring images at the warehouse location contain relevant information of packaged materials and scattered materials. By calling up the shooting images and monitoring images, the remaining materials can be managed in real time to avoid increasing the amount of data that needs to be processed when there are many work orders or a large number of material storage and retrieval times, thereby improving the management efficiency of the remaining materials.

[0067] In the specific implementation, if the monitoring screen is selected from the monitoring screens of the entire warehouse, including the screen of the storage location where the materials are collected, the monitoring screen needs to be processed for clarity to avoid the situation where the large-scale monitoring cannot clearly capture the characteristics of the remaining materials at the storage location where the materials are collected. Therefore, the monitoring screen needs to be processed for clarity to ensure that the information of the remaining materials can be accurately extracted from the monitoring screen to improve the management efficiency of the remaining materials.

[0068] Step S20, extracting the quantity of the material that the user has omitted from the shooting picture, and extracting the packing quantity of the packing material and the stacking picture of the stacking material from the monitoring picture;

[0069] It should be noted that missing materials are materials that are left in the transport bin when the user takes the materials. Packaged materials can be materials packaged according to a certain quantity, and each package of materials will be marked with the corresponding quantity, weight, type and other information. Stacked materials can be a general term for scattered materials after one or more materials are stacked together. Among them, scattered materials are unpackaged materials. The stacking screen is a monitoring screen for stacked materials, which includes information such as the stacking method, height, shape, etc. of the materials.

[0070] It is understandable that, since the packaging materials are packaged according to a certain quantity during packaging, and the number of packages of the packaging materials can be intuitively determined in the monitoring screen, the packaging quantity of the packaging materials can be directly determined based on the monitoring screen.

[0071] It is understandable that when materials are stacked together, there will be materials that cannot be captured by the surveillance camera, that is, materials covered by multiple scattered materials. Therefore, it is necessary to obtain a stacking picture of the stacked materials from the surveillance picture to facilitate the subsequent prediction of the number of stacked materials based on the stacking picture.

[0072] It can be understood that the image recognition technology is used to extract the quantity of materials missed by the user from the captured images of the storage robot transport warehouse. For example, the shape, color and other appearance features of the materials in the image are identified and matched with pre-stored material templates to determine the types of materials in the image and count their quantities. The image can also be processed based on existing deep learning algorithms, such as convolutional neural networks (CNN), to accurately identify and count materials.

[0073] It is understandable that, from the monitoring screen of the material storage warehouse, the image segmentation technology can be used to segment the packaged material area in the screen, and then count the number of packages of the packaged materials; at the same time, the stacking screen of the stacked materials in the monitoring screen can be extracted. Among them, the image segmentation technology can use a threshold-based segmentation method or a semantic-based segmentation method to accurately obtain target information from the monitoring screen.

[0074] Furthermore, step S20 may also include:

[0075] Identifying the package quantity of the packaged materials from the monitoring picture, and extracting a distribution map of unpackaged scattered materials from the monitoring picture;

[0076] Extracting the accumulation characteristics of each of the scattered materials from the distribution map;

[0077] Based on the accumulation characteristics, determining whether the scattered materials are accumulated;

[0078] If it is determined that the scattered materials are piled up, the piled up scattered materials are determined to be piled up materials, and a stacking picture of the piled up materials is extracted from the distribution map.

[0079] It should be noted that the stacking characteristics may include information such as the shape, size, arrangement, and stacking height of the material.

[0080] It can be understood that the shape, size, arrangement, stacking height and other stacking characteristics of each scattered material are extracted from the distribution map, and the judgment criteria are set. For example, when the stacking height of the material exceeds a certain threshold, the material is considered to be stacked; when the arrangement of the material shows obvious stacking characteristics (such as stacking layers, close arrangement, etc.), the material is considered to be stacked. Specifically, the judgment criteria can be determined through experimental data and experience accumulation. If it is determined that the scattered materials are stacked, the stacked scattered materials are determined to be stacked materials. The determination of stacked materials can be carried out by comprehensively analyzing the stacking characteristics in the distribution map to confirm the stacking situation of the materials, so as to improve the efficiency and accuracy of material management.

[0081] In the specific implementation, the edge contour of the material can be extracted by analyzing the changes in pixel values ​​in the distribution map; the shape characteristics of the material can be obtained by calculating the area and perimeter of the material area; and whether the material is stacked can be determined by analyzing the arrangement of the material.

[0082] Step S30, extracting stacking features from the stacking picture based on a preset decomposition model, and predicting the quantity of scattered materials of the stacked materials according to the feature extraction result, wherein the preset decomposition model is obtained by training the stacking conditions of various materials at different quantities, and the stacking conditions are obtained by simulating the stacking of different quantities according to the appearance characteristics of various materials;

[0083] It is understandable that during the model training process, different numbers of stacking simulations are performed according to the appearance characteristics of various materials, and the image data of these stacking situations and the corresponding actual scattered material quantity data are collected. For example, for materials in the shape of cubes, the stacking situations of different numbers ranging from 1 to 100 are simulated, and the image features (such as the number of layers, the number of each layer, etc.) and the actual number of scattered materials under different stacking methods are recorded, and the model is trained using these data, wherein the model can be a regression model based on machine learning, such as a support vector regression (SVR) model or a neural network regression model, so that the trained model can accurately predict the number of scattered materials of the stacked materials according to the characteristics of the input stacking picture.

[0084] It can be understood that predicting the quantity of scattered materials in stacked materials through a preset decomposition model can more accurately take into account the actual status of the materials, improve the accuracy of calculating the remaining material quantity, and the data-driven preset decomposition model can adapt to the appearance characteristics and stacking conditions of different materials and has strong versatility.

[0085] Step S40, integrating the material quantity, the scattered material quantity and the packaged quantity to determine the remaining quantity of the remaining materials in the warehouse.

[0086] It can be understood that the number of materials extracted from the shooting screen, the number of scattered materials predicted by the preset decomposition model, and the number of packages extracted from the monitoring screen are integrated to determine the remaining number of materials remaining in the warehouse. The integration process can be to set the number of user omitted materials extracted from the shooting screen as A, the number of scattered materials predicted as B, and the number of packages extracted from the monitoring screen as C, then the integrated quantity relationship is A+B+C.

[0087] The present embodiment provides a method for managing the remaining amount of materials. Since the user may miss some materials when taking them from the transport bin of the storage robot, in order to ensure that the source of each material in the warehouse can be ensured when the user misses some materials, after the user completes picking up the materials, it is necessary to retrieve the captured images in the transport bin of the storage robot. Moreover, since different kinds of materials are stored in the warehouse, in order to avoid repeated inventory of materials whose quantities have not changed, wasting material management resources and affecting material management efficiency, it is also necessary to retrieve the monitoring images of the warehouse where the materials are stored, so as to extract the packaging quantity of the packaged materials and the stacking images of the stacked materials from the monitoring images, and to extract the quantity of the materials missed by the user from the captured images. In order to avoid the inability to accurately identify the material data of stacked materials, the stacked materials are decomposed by a preset decomposition model trained using the stacking conditions of various materials in different quantities to determine the number of scattered materials in the stacked materials. When training the preset decomposition model, the stacking conditions are obtained after simulating stacking of different quantities based on the appearance characteristics of various materials. Therefore, the preset decomposition model can be used to accurately decompose various stacked materials. Finally, the remaining number of materials in the warehouse is determined by integrating the material quantity, the scattered material quantity and the packaged quantity. That is, by reducing the data that needs to be managed and accurately decomposing the number of materials stacked in the warehouse through the preset decomposition model, the management efficiency of the remaining materials is improved.

[0088] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can refer to the above introduction, and will not be repeated later. Figure 2 Before step S10, the method for managing the remaining amount of materials further includes steps S01 to S05:

[0089] Step S01, obtaining a stacking image of a sample material and the stacking quantity of the sample material;

[0090] Step S02, extracting stacking features of the sample material from the stacked image;

[0091] Step S03, determining a placement method of each of the sample materials based on the stacking characteristics;

[0092] Step S04, simulating the stacking of different quantities of sample materials according to the placement method and the appearance characteristics of the sample materials;

[0093] Step S05: Using the placement method and the appearance feature as stacking features of the stacking situation, training a preset model to be trained, and obtaining a preset decomposition model.

[0094] It should be noted that the sample materials may be various materials stored in the warehouse, and the stacking conditions may include whether the materials are placed horizontally, vertically, in a single layer, or in multiple layers.

[0095] It can be understood that by obtaining the stacking images and stacking quantities of sample materials, extracting stacking features, determining the placement method, simulating the stacking conditions of different quantities of materials, and training the preset decomposition model based on the stacking conditions, the accuracy of the preset decomposition model in identifying the quantity of stacked materials can be improved, thereby improving the efficiency and accuracy of material management.

[0096] Furthermore, step S04 further includes:

[0097] Determining, according to the appearance features and the placement methods, the forces to which adjacent samples are subjected when placed in different ways;

[0098] Based on the forces acting on the sample materials, the stacking conditions of different quantities of sample materials when the forces reach equilibrium are simulated.

[0099] It should be noted that the action force is the force exerted by one object on another object, and the object receiving the force will in turn exert a reaction force on the object exerting the force. The balance of action forces can be understood as the action force when the material is in a static state under the action forces of each material and its own gravity.

[0100] It is understandable that since the forces acting on materials when they are stacked are in a state of equilibrium, and the forces acting on materials from other materials are different depending on how they are placed, it is necessary to first determine the possible placement methods of the materials when they are stacked, and then determine the forces acting on each sample material based on the appearance characteristics and placement methods, so as to scientifically simulate the situations of different numbers of sample materials based on the forces, and obtain all situations in which the sample materials can be stacked, so as to improve the decomposition accuracy and universality of the preset decomposition model.

[0101] In the specific implementation, computer simulation technology can be used to simulate the stacking of different quantities of materials according to the force of the materials. The simulation process should take into account factors such as the size, weight, friction, etc. of the materials to truly reflect the stacking of the materials.

[0102] Furthermore, step S03 may also include:

[0103] Extracting contact points between adjacent sample materials from the stacking features;

[0104] Simulating a stacking process of the sample materials based on the contact points to determine a balance value of each of the sample materials after stacking;

[0105] Determine a target balance value at which the balance value satisfies a preset balance threshold requirement;

[0106] The sample placement method corresponding to the target balance value is used as the placement method of each of the sample materials.

[0107] It should be noted that the contact point may be a point of contact between adjacent sample materials, and the contact point may be obtained through image processing technology, for example, using an edge detection algorithm to identify the contour of the material, and then extracting the contact point between the adjacent materials; the contact point between two sample materials may be one or more, and there is no specific limitation. The balance value may be the possibility that the forces acting on each sample material after stacking are balanced so that the stacked materials will not fall apart.

[0108] It can be understood that by simulating the stacking process, the balance value of the material can be accurately determined, avoiding the subjectivity and uncertainty of manual judgment, and by determining the target balance value, the simulated placement method can be made more in line with the actual situation, making the simulated stacking situation more stable, reducing the learning of unstable stacking situations, thereby improving the learning efficiency of the preset decomposition model, and improving the efficiency of extracting stacking features of the preset decomposition model during use, thereby improving the management efficiency of the remaining materials in the warehouse.

[0109] Based on the first and second embodiments of the present application, in the third embodiment of the present application, the same or similar contents as those in the above embodiments can be referred to the above introduction, and will not be described in detail later. Figure 3 After step S40, the method for managing the remaining amount of materials further includes steps S1 to S5:

[0110] Step S1, retrieve the record screen of the user taking materials from the transport warehouse of the storage robot, and the storage quantity before the user takes the materials;

[0111] Step S2, extracting the quantity of materials taken by the user from the record screen;

[0112] Step S3, adding the received quantity to the remaining quantity and comparing it with the storage quantity to determine whether the remaining quantity is correct;

[0113] Step S4, if the remaining quantity is incorrect, determining whether the received quantity is correct;

[0114] Step S5, if the picked-up quantity is correct, then record the scattered materials and the scattered quantity.

[0115] It should be noted that the collection quantity can be the number of materials actually taken by the user from the transport warehouse, including the number of materials that fall to the ground after the user takes them from the transport warehouse, that is, the number of materials that have been taken out of the warehouse.

[0116] It is understandable that by judging whether the quantity collected by the user is correct, the user can be prevented from taking too much. The scattered collected materials outside the shelves of the warehouse can also be recorded in time to avoid the number of materials that can actually be monitored in the warehouse not corresponding to the data recorded by the material management platform, which requires user intervention in management, thereby improving the management efficiency of the remaining materials in the warehouse.

[0117] In the specific implementation, if the remaining quantity is correct, it means that the user has picked up the correct quantity of materials and the materials have not been scattered off the shelf. If the remaining quantity is incorrect and the picked-up quantity is incorrect, it is necessary to record the type and corresponding quantity of materials taken by the user according to the actual situation, and further detect whether the materials are scattered outside the shelf when taking materials from the shelf. The user can also be reminded in time to take more materials.

[0118] In the specific implementation, the material quantity, scattered material quantity and packaged quantity are integrated to determine the remaining quantity of the remaining materials in the warehouse, and then the record screen of the user taking materials from the transport warehouse of the storage robot and the storage quantity before the user picks up the materials are automatically retrieved, and the specific collection quantity of the materials taken by the user is extracted from the record screen. The extracted collection quantity is added to the previously determined remaining quantity and then compared with the original storage quantity to determine whether the remaining quantity is correct. If the remaining quantity is found to be incorrect, the collection quantity will be further determined to determine whether it is correct. If the collection quantity is correct, the quantity may be inconsistent due to material scattering. At this time, it is necessary to record the scattering of materials and the scattered quantity. According to the judgment result, the material quantity record in the system is updated to ensure the real-time and accuracy of the data.

[0119] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the management method of the remaining material quantity of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0120] This application also provides a material remaining amount management device, please refer to Figure 4 , the material remaining amount management device comprises:

[0121] The retrieval module 10 is used to retrieve the captured images in the storage robot transport warehouse and the monitoring images at the storage warehouse location of the picked-up materials after determining that the user has completed picking up the materials;

[0122] An extraction module 20, configured to extract the quantity of materials omitted by the user from the shooting picture, and to extract the packing quantity of the packing materials and the stacking picture of the stacking materials from the monitoring picture;

[0123] A prediction module 30 is used to extract stacking features from the stacking picture based on a preset decomposition model, and predict the quantity of scattered materials of the stacked materials according to the feature extraction result, wherein the preset decomposition model is obtained by training the stacking conditions of various materials at different quantities, and the stacking conditions are obtained by simulating the stacking of different quantities according to the appearance characteristics of various materials;

[0124] The integration module 40 is used to integrate the material quantity, the scattered material quantity and the packaged quantity to determine the remaining quantity of the remaining materials in the warehouse.

[0125] Optionally, the calling module 10 is also used to obtain a stacking image of a sample material and the stacking quantity of the sample material; extract stacking features of the sample material from the stacking image; determine a placement method of each sample material based on the stacking features; simulate the stacking conditions of different quantities of sample materials according to the placement method and the appearance features of the sample material; use the placement method and the appearance features as stacking features of the stacking condition, train a preset model to be trained, and obtain a preset decomposition model.

[0126] Optionally, the calling module 10 is further used to determine the forces acting on adjacent samples when placed in different ways according to the appearance characteristics and the placement methods; and based on the forces acting on each of the sample materials, simulate the stacking conditions of different numbers of sample materials when the forces reach equilibrium.

[0127] Optionally, the calling module 10 is also used to extract contact points between adjacent sample materials from the stacking features; simulate the stacking process of the sample materials based on the contact points, and determine the balance value of each sample material after stacking; determine a target balance value at which the balance value meets a preset balance threshold requirement; and use the sample placement method corresponding to the target balance value as the placement method of each sample material.

[0128] Optionally, the extraction module 20 is also used to identify the package quantity of packaged materials from the monitoring screen, and extract a distribution map of unpackaged scattered materials from the monitoring screen; extract the stacking characteristics of each of the scattered materials from the distribution map; based on the stacking characteristics, determine whether the scattered materials are stacked; if it is determined that the scattered materials are stacked, determine that the stacked scattered materials are stacked materials, and extract a stacking screen of the stacked materials from the distribution map.

[0129] Optionally, the extraction module 20 is also used to retrieve the record screen of the user taking materials from the transport warehouse of the storage robot and the storage quantity before the user picks up the materials; extract the picked-up quantity of the materials taken by the user from the record screen; add the picked-up quantity to the remaining quantity and compare them with the storage quantity to determine whether the remaining quantity is correct; if the remaining quantity is incorrect, determine whether the picked-up quantity is correct; if the picked-up quantity is correct, record the scattered materials and the scattered quantity.

[0130] The management device for the remaining amount of materials provided in the present application adopts the management method for the remaining amount of materials in the above-mentioned embodiment, which can solve the technical problem that errors occur in the verification of the remaining materials in the warehouse, making the management efficiency of the remaining amount of materials in the warehouse low. Compared with the prior art, the beneficial effects of the management device for the remaining amount of materials provided in the present application are the same as the beneficial effects of the management method for the remaining amount of materials provided in the above-mentioned embodiment, and the other technical features of the management device for the remaining amount of materials are the same as the features disclosed in the above-mentioned embodiment method, which will not be repeated here.

[0131] The present application provides a material remaining quantity management device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the material remaining quantity management method in the above-mentioned embodiment one.

[0132] Reference below Figure 5 , which shows a schematic diagram of the structure of a material remaining quantity management device suitable for implementing the embodiment of the present application. The material remaining quantity management device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The material remaining quantity management device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0133] like Figure 5 As shown, the material remaining quantity management device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the material remaining quantity management device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the material remaining quantity management device to communicate wirelessly or wired with other devices to exchange data. Although the figure shows a material remaining quantity management device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided instead.

[0134] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0135] The material remaining quantity management device provided by the present application adopts the material remaining quantity management method in the above-mentioned embodiment, which can solve the technical problem that errors occur in the verification of the remaining materials in the warehouse, making the warehouse's management of the material remaining quantity inefficient. Compared with the prior art, the beneficial effects of the material remaining quantity management device provided by the present application are the same as the beneficial effects of the material remaining quantity management method provided by the above-mentioned embodiment, and the other technical features of the material remaining quantity management device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0136] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0137] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0138] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the material remaining quantity management method in the above-mentioned embodiment.

[0139] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0140] The computer-readable storage medium may be included in the material remaining quantity management device; or may exist independently without being assembled into the material remaining quantity management device.

[0141] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the material remaining quantity management device, the material remaining quantity management device: after determining that the user has completed collecting materials, retrieves the shooting picture in the storage robot transport warehouse and the monitoring picture of the warehouse location where the collected materials are stored; extracts the material quantity of the user's omitted materials from the shooting picture, and extracts the packaged quantity of packaged materials and the stacking picture of stacked materials from the monitoring picture; extracts stacking features of the stacking picture based on a preset decomposition model, and predicts the scattered material quantity of the stacked material according to the feature extraction result, the preset decomposition model is obtained by training the stacking conditions of various materials at different quantities, and the stacking conditions are obtained after simulating the stacking of different quantities according to the appearance characteristics of various materials; integrates the material quantity, the scattered material quantity and the packaged quantity to determine the remaining quantity of the remaining materials in the warehouse.

[0142] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0143] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0144] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0145] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned material remaining quantity management method, and can solve the technical problem that errors occur in the verification of the remaining materials in the warehouse, making the warehouse's management of the remaining materials inefficient. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the material remaining quantity management method provided by the above-mentioned embodiment, and will not be repeated here.

[0146] The present application also provides a computer program product, including a computer program, which implements the steps of the material surplus management method as described above when the computer program is executed by a processor.

[0147] The computer program product provided by the present application can solve the technical problem that errors occur in checking the remaining materials in the warehouse, resulting in low efficiency in the management of the remaining materials in the warehouse. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the material remaining management method provided by the above embodiment, and will not be repeated here.

[0148] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for managing the remaining amount of materials, characterized in that: The method includes: After confirming that the user has completed picking up the materials, retrieve the captured images in the storage robot transport warehouse and the monitoring images at the location where the picked up materials are stored in the warehouse; Extracting the quantity of materials omitted by the user from the shooting picture, and extracting the packing quantity of the packing materials and the stacking picture of the stacking materials from the monitoring picture; Extracting stacking features from the stacking picture based on a preset decomposition model, and predicting the quantity of scattered materials of the stacked materials according to the feature extraction result, wherein the preset decomposition model is obtained by training the stacking conditions of various materials at different quantities, and the stacking conditions are obtained after simulating the stacking of different quantities according to the appearance characteristics of various materials; The material quantity, the scattered material quantity and the packaged quantity are integrated to determine the remaining quantity of the remaining materials in the warehouse.

2. The method according to claim 1, characterized in that After determining that the user has completed picking up the materials, the step of retrieving the captured images in the storage robot transport warehouse and the monitoring images at the location of the picked-up materials storage warehouse includes: Acquire a stacking image of a sample material and the stacking quantity of the sample material; extracting stacking features of the sample material from the stacked image; Determining a placement method of each of the sample materials based on the stacking characteristics; Simulating stacking conditions of different quantities of sample materials according to the placement method and the appearance characteristics of the sample materials; The placement manner and the appearance features are used as stacking features of the stacking situation, and a preset model to be trained is trained to obtain a preset decomposition model.

3. The method according to claim 2, characterized in that The step of simulating stacking conditions of different quantities of sample materials according to the placement method and the appearance characteristics of the sample materials comprises: Determining, according to the appearance features and the placement methods, the forces to which adjacent samples are subjected when placed in different ways; Based on the forces acting on the sample materials, the stacking conditions of different quantities of sample materials when the forces reach equilibrium are simulated.

4. The method according to claim 2, characterized in that The step of determining the placement of each sample material based on the stacking feature comprises: Extracting contact points between adjacent sample materials from the stacking features; Simulating a stacking process of the sample materials based on the contact points to determine a balance value of each of the sample materials after stacking; Determine a target balance value at which the balance value satisfies a preset balance threshold requirement; The sample placement method corresponding to the target balance value is used as the placement method of each of the sample materials.

5. The method according to any one of claims 1 to 4, characterized in that The step of extracting the packaging quantity of packaging materials and the stacking picture of stacking materials from the monitoring picture includes: Identifying the package quantity of the packaged materials from the monitoring picture, and extracting a distribution map of unpackaged scattered materials from the monitoring picture; Extracting the accumulation characteristics of each of the scattered materials from the distribution map; Based on the accumulation characteristics, determining whether the scattered materials are accumulated; If it is determined that the scattered materials are piled up, the piled up scattered materials are determined to be piled up materials, and a stacking picture of the piled up materials is extracted from the distribution map.

6. The method according to any one of claims 1 to 4, characterized in that After the step of integrating the material quantity, the scattered material quantity and the packaged quantity to determine the remaining quantity of the remaining material in the warehouse, the method further includes: Retrieving a recording screen of the user taking materials from the transport bin of the storage robot and the storage quantity before the user picks up the materials; Extracting the quantity of materials taken by the user from the record screen; Add the received quantity to the remaining quantity and compare it with the storage quantity to determine whether the remaining quantity is correct; If the remaining quantity is incorrect, determine whether the claimed quantity is correct; If the quantity picked up is correct, record the material scattered and the quantity scattered.

7. A device for managing the remaining amount of materials, characterized in that: The device comprises: The retrieval module is used to retrieve the captured images in the storage robot transport warehouse and the monitoring images at the storage warehouse where the materials are stored after the user has completed the material collection; An extraction module, used to extract the material quantity of the material omitted by the user from the shooting picture, and to extract the packing quantity of the packing material and the stacking picture of the stacking material from the monitoring picture; A prediction module, used for extracting stacking features from the stacking picture based on a preset decomposition model, and predicting the quantity of scattered materials of the stacked materials according to the feature extraction result, wherein the preset decomposition model is obtained by training the stacking conditions of various materials at different quantities, and the stacking conditions are obtained after simulating the stacking of different quantities according to the appearance characteristics of various materials; The integration module is used to integrate the material quantity, the scattered material quantity and the packaged quantity to determine the remaining quantity of the remaining materials in the warehouse.

8. A material remaining quantity management device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the material surplus management method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the material remaining quantity management method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the material surplus management method according to any one of claims 1 to 6 are implemented.

Images