Container inventory management method, device and system and computer program product

Abstract

The invention discloses a container inventory management method, device and system and a computer program product, the method is applied to a control end, at least one camera is installed in a container, the camera is used for collecting a plurality of frames of images in the container when the container is in an open state, and the plurality of frames of images are sent to the control end; the method comprises the following steps: processing the plurality of frames of images to obtain a visual identification result of first inventory information; the first inventory information comprises the state and the category of target goods and the quantity of the category of goods; the state comprises warehouse-out or warehouse-in; when the cargo quantity meets a preset quantity condition, updating second inventory information of the target cargo according to the cargo quantity and the category; and / or, when the cargo quantity does not meet the preset quantity condition, performing exception handling. According to the invention, the inventory information can be automatically counted when the goods are taken or put in, and the management efficiency can be improved.

Description

Technical Field

[0001] The present invention relates to the field of warehousing technology, and in particular to a container inventory management method and device, system, and computer program product. Background Art

[0002] In order to avoid the waste of consumables, it is necessary to manage the taking and putting of goods (entry and exit) and monitor the inventory situation in time. At present, it is necessary to arrange special management personnel to handle the management and registration of goods. When the goods are frequently taken out of the warehouse or put into the operation warehouse, the manual processing efficiency is not high. Summary of the invention

[0003] In order to solve the above technical problems, the present application provides a container inventory management method and device, system, and computer storage medium.

[0004] Specifically, the technical solution of the present invention is as follows:

[0005] In a first aspect, the present application provides a method for managing container inventory, which is applied to a control terminal, wherein at least one camera is installed inside the container, and the camera is used to collect a plurality of frames of images inside the container when the container is in an open state. The method comprises:

[0006] Processing the plurality of frames of images to obtain a visual recognition result of first inventory information; the first inventory information includes the state, category and quantity of target goods; the state includes outbound or inbound;

[0007] When the category and the quantity of goods meet a preset condition, the second inventory information of the target goods is updated according to the quantity of goods and the category.

[0008] In one implementation, the processing of the plurality of frames of images to obtain a visual recognition result of the first inventory information includes:

[0009] Performing a first recognition process on the plurality of frames of images according to a preset action recognition model to determine the state;

[0010] Extracting image features from the plurality of frames of images using a preset feature extraction model; and determining the category according to the image features;

[0011] Perform a second recognition process on the plurality of frames of images to determine the quantity of the goods.

[0012] In one implementation, performing a first recognition process on the plurality of frames of images to determine the state includes:

[0013] Extracting action features from the plurality of frames of images, and determining whether the action features are standard features based on the preset action recognition model; the standard features include standard picking features or standard putting features;

[0014] When the action feature is the standard feature, determining the state as outbound or inbound;

[0015] When the action feature is not the standard feature, exception processing is performed.

[0016] In one implementation, it further includes:

[0017] When the category and the quantity of goods do not meet the preset conditions, exception processing is performed.

[0018] In one implementation, it further includes:

[0019] Before the container is opened, obtaining the preset first inventory information;

[0020] Verifying the visual recognition result according to the preset first inventory information;

[0021] When the preset first inventory information does not match the visual recognition result, exception processing is performed.

[0022] In one implementation, a radio frequency identification device is further installed inside the container, and when the container is in the open state, the radio frequency identification device collects radio frequency information of the goods in real time, and the method further includes:

[0023] Processing the radio frequency information to generate radio frequency identification results of the category and quantity of the goods;

[0024] Verifying the category and the quantity of goods in the visual identification result according to the radio frequency identification result;

[0025] When the category and the quantity of goods in the radio frequency identification result do not match the category and the quantity of goods in the visual identification result, an exception process is performed.

[0026] In one implementation, the exception handling includes:

[0027] Send an identification request to the user terminal, so that the user terminal can collect the image of the target goods according to the identification request and send it to the control terminal, and the control terminal processes the image of the target goods and re-acquires the first inventory information; wherein the user terminal matches the user information, and the user information is used for security verification before the container is opened. When the user information passes the security verification, the container is controlled to be opened.

[0028] In a second aspect, the present application provides a container inventory management device, including a processor for calling instructions stored in a memory, wherein when the instructions are called by the processor, the processor executes any of the management methods described above.

[0029] In a third aspect, the present application provides a container inventory management system, comprising a control end, a container, and a user end, wherein the control end comprises the management device as described in the second aspect above; the user end is used to collect an image of the target cargo and send it to the control end according to the identification request sent by the container when performing exception processing; the control end is also used to process the image of the target cargo and update the second inventory information.

[0030] In a fourth aspect, the present application provides a computer program product, comprising a computer program / instruction, which implements the steps of any of the above-mentioned management methods when executed by a processor.

[0031] Compared with the prior art, the present invention has at least one of the following beneficial effects:

[0032] (1) This application collects images when goods are taken or put in, and automatically manages the container inventory based on visual recognition. Compared with the manual inventory counting method, it effectively improves work efficiency and reduces labor costs.

[0033] (2) The present application recognizes the state and category of target goods through a preset action recognition model and a preset feature recognition model, thereby improving the speed of image processing and recognition accuracy, thereby further improving management efficiency.

[0034] (3) The present application combines radio frequency identification with visual identification, which can supplement the visual identification solution and improve the recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The following is a brief introduction to the drawings used in describing the embodiments of the present application:

[0036] Figure 1 A structural block diagram of a container inventory management system provided in an embodiment of the present application;

[0037] Figure 2 A flowchart of a container inventory management method provided in an embodiment of the present application;

[0038] Figure 3 A flowchart of another container inventory management method provided in an embodiment of the present application;

[0039] Figure 4 A structural block diagram of another container inventory management system provided in an embodiment of the present application;

[0040] Figure 5 This is a structural diagram of a container storage management device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the specific implementation methods of the present application will be described below with reference to the accompanying drawings. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings and other implementation methods can be obtained based on these drawings without creative work.

[0042] In order to simplify the drawings, only the parts related to the present application are schematically shown in each figure, and they do not represent the actual structure of the product. In addition, in order to simplify the drawings and facilitate understanding, in some figures, only one of the parts with the same structure or function is schematically drawn or marked.

[0043] In the present application, unless otherwise clearly specified and limited, ordinal numbers, such as "first", "second", etc., are only used to distinguish and describe associated objects, and cannot be understood as indicating or implying the relative importance or order between associated objects; in addition, they do not represent the number of associated objects. " / " is used to describe the relationship between associated objects, which indicates the "or" relationship between associated objects. "And / or" is used to describe the relationship between associated objects, which includes any combination relationship between associated objects, for example, "a and / or b" includes: "alone a", "alone b", or "a and b". "At least one" of multiple objects refers to any object or any combination of multiple objects, for example, "one or more of a1, a2, a3" or "at least one of a1, a2, a3" includes: "alone a1", "alone a2", "alone a3", "a1 and a2", "a1 and a3", "a2 and a3", or "a1, a2 and a3". In the present application, "installation" includes direct or indirect positional relationship, for example, "installation" can be direct installation or installation through other components.

[0044] In order to avoid waste of consumables, it is necessary to manage the entry and exit of goods and monitor the inventory of containers in a timely manner. At present, it is necessary to arrange special management personnel to handle matters such as the management and registration of goods. When goods are frequently sent out of the warehouse or into the operation warehouse, manual processing efficiency is not high. In addition, in some application scenarios, there are strict restrictions on the taking of goods. For example, in order to reduce the abuse of equipment and waste of consumables, hospitals have strict quantity requirements for each use (or outbound) of medical equipment. It is necessary to take it in compliance with the quantitative requirements and regularly check the inventory of medical equipment. If the number of goods to be taken is too large, it will increase the workload of the staff and reduce work efficiency. In addition, there are certain errors in manual operation, which makes it easy to take more or less, affecting the statistics of inventory status, resulting in discrepancies between the inventory status and the actual situation, causing inconvenience to management.

[0045] Therefore, the embodiment of the present application provides a method for managing container inventory, which can capture images when taking out or putting in (or warehousing) goods, and by processing the captured images, identify whether it is a warehouse-out operation or a warehouse-in operation and identify the type and quantity of goods, thereby updating the inventory information of the container to achieve automated management of the container inventory, thereby improving management efficiency and reducing labor costs. In the embodiment of the present application, the container is described as a medical equipment cabinet, and it can also be a container used in other application scenarios, such as a vending machine, a safe, etc., and the present application does not limit this.

[0046] The following is a description with reference to the accompanying drawings. Figure 1 , which is a structural block diagram of a container inventory management system provided by an embodiment of the present application. Figure 1 As shown, the container inventory management system 100 includes a container 110 and a control terminal 120. The container 110 has two states: open and closed. Specifically, it refers to the doors or logistics channels installed on the container being open or closed. At least one camera is installed inside the container 110. When the container 110 is in the open state, the staff puts or takes out goods into or out of the container (the goods put in or taken out are the target goods). During the process, the camera collects several frames of images inside the container, that is, the images when the goods are put in or taken out. The control terminal 120 processes these images to obtain a visual recognition result of the first inventory information. The first inventory information is the state, category and quantity of the target goods corresponding to the category of goods. The control terminal 120 automatically updates the inventory information of the container according to the visual recognition result. The specific process is as follows: Figure 2 As shown, the steps include:

[0047] S100 processes a plurality of frames of images captured by the camera to obtain a visual recognition result of the first inventory information.

[0048] S200 When the quantity of the goods meets the preset quantity condition, the second inventory information of the target goods is updated according to the quantity and category of the goods.

[0049] Wherein, the first inventory information refers to the state, category and quantity of goods of the target goods, and the state refers to storage or warehousing. For example, assuming that a box of gauze needs to be taken from the container, this box of gauze is the target goods, and the state of the box of gauze is taken out of the warehouse. The category is used to distinguish different goods, for example, including the name of the goods (such as gauze, alcohol, etc.), and also including the specifications, models, etc. of the goods, which are not limited by this application. The quantity of goods refers to the quantity of the target goods when leaving the warehouse (when being taken) or warehousing (when being put into), and still taking a box of gauze as an example, the quantity of goods is 1 box, i.e. the quantity taken out.

[0050] The camera inside the container collects images during the process of taking or putting goods in. In one implementation, for example, in a vending machine, when the vending machine is turned on, the camera also starts working, but this method increases the data processing pressure on the control end and easily causes additional resource waste. Therefore, in some implementations, the camera starts working after the container is opened.

[0051] In addition, the camera can continuously capture images, for example, by setting a capture frequency, and after the container is opened and before it is closed, the camera continuously captures multiple frames of images at the capture frequency. In one implementation, the camera can also capture video of the entire process. Furthermore, the camera can also capture images at fixed times, for example, capturing one frame of image after the container is opened and another frame of image after the container is closed. Continuous capture is conducive to improving the accuracy of visual recognition, while fixed capture is more resource-saving.

[0052] It should be noted that there is no limit on the number of cameras in the embodiments of the present application, and the number of cameras can be increased or decreased according to actual inventory management needs. For example, when the container is large or has multiple layers, cameras can be installed at different locations of the container to avoid monitoring the four corners and affecting the recognition results. The installation position of the camera depends on the internal structure of the container. For example, it can be installed at the four corners of the container to achieve a wider coverage; or it can be installed at the door, exit, entrance and other locations. At these locations, images of taking or putting in goods can be collected more accurately, further improving processing efficiency, and the space utilization rate is high, which will not hinder the normal placement of goods.

[0053] Furthermore, the control terminal will store images of various types of goods in advance, and various image features can be extracted from the images, such as the appearance, color, shape, size, etc. of the goods, and further, the barcode, name, and other information on the goods. After acquiring several frames of images collected by the camera, the control terminal performs visual recognition processing on these images, such as extracting corresponding image features from them and comparing them with the image features pre-stored in the control terminal, so as to obtain the status, category, and quantity of the target goods. The information obtained through image processing is the visual recognition result.

[0054] The control end can update the current inventory information of the container based on the visual recognition results, such as the category and quantity of goods contained in the container after taking or putting in a certain amount of goods (i.e., the second inventory information). In order to improve the accuracy of the inventory information, the category and quantity of goods in the visual recognition results need to meet the preset conditions. For example, the quantity of goods cannot exceed the preset quantity. For example, in a medical equipment cabinet, each time the quantity is taken, it cannot exceed the prescribed amount. Assuming that the single quantity taken is 1, and the number of goods in the visual recognition result exceeds 1, it is considered an abnormal situation. At this time, the control end does not update the second inventory information, but performs an exception processing. Of course, within the capacity limit of the container, the quantity of goods can be set to any value, and this application does not limit this.

[0055] In addition, the category also needs to meet the preset conditions. For example, the item to be put in is A, but the category in the visual recognition result is item B. Another example is that the item name is the same but the specification or model does not meet the requirements. In this case, the category does not meet the preset conditions and exception processing is also required. In short, if at least one of the category and the quantity of goods does not meet the preset conditions, exception processing is required; if both the category and the quantity meet the preset conditions, the visual recognition result is considered to be the correct result, and the second inventory information can be updated according to the result.

[0056] In this way, by collecting images when taking or putting goods, the container inventory can be automatically managed based on visual recognition. Compared with the manual inventory counting method, it effectively improves work efficiency and reduces labor costs.

[0057] In one implementation, Figure 3 As shown, a container inventory management method provided in an embodiment of the present application includes the following steps:

[0058] S110 performs a first recognition process on a plurality of frames of images according to a preset action recognition model to determine a state.

[0059] S120 extracts image features from a plurality of image frames using a preset feature extraction model, and determines a category based on the image features.

[0060] S130 performs a second recognition process on a plurality of frames of images to determine the quantity of the goods.

[0061] S210 determines whether the quantity of the goods meets the preset quantity condition. If it is determined to be yes, execute step S220; if it is determined to be no, execute step S230.

[0062] S220 updates the second inventory information of the target goods according to the quantity and category of the goods.

[0063] S230 exception handling.

[0064] Specifically, in the embodiments of the present application, for example, a large number of images of picking up or putting goods are collected, and action recognition models and feature extraction models are pre-trained based on these images, and the trained models are used as preset action recognition models and preset feature extraction models.

[0065] The preset action recognition model is used to perform recognition processing (first recognition processing) on ​​several frames of images collected by the camera to determine whether the action being performed in the image is a picking action or a putting action. If it is a picking action, it is determined to be in the outbound state; if it is a putting action, it is determined to be in the inbound state. In one implementation, the action features are first extracted from several frames of images collected by the camera, and then input into the preset action recognition model for judgment. If the action feature is a standard feature (standard picking feature or standard putting feature), the state of the goods is determined to be outbound or inbound; if the action feature is identified as not a standard feature, an exception is handled. The preset feature extraction model is used to extract image features to identify what specific items the target goods in the image are. In addition, the second recognition processing is performed on several frames of images to determine the quantity of goods, which can be achieved through various image recognition algorithms, which will not be repeated here. It should be understood that the order of S110-S130 can be interchanged, and this application does not limit this.

[0066] This embodiment improves the speed of image processing and recognition accuracy by presetting the action recognition model and the feature recognition model, thereby further improving the management efficiency.

[0067] The following is an explanation of the exception handling mentioned in the above embodiment. Figure 4 , which is a structural block diagram of a container inventory management system in an embodiment of the present application. Figure 3 As shown, the container inventory management system 100 includes a container 110, a control terminal 120, and a user terminal 130. The user terminal 130 is, for example, a mobile terminal equipped with a camera, and the user terminal 130 can communicate and interact with the control terminal 120. For example, the user terminal 130 can obtain inventory information and various outbound and inbound tasks (for example, N items A need to be taken or put in) from the control terminal 120.

[0068] When an abnormality occurs in the identification process, the control terminal 120 sends an identification request to the user terminal 130; the user terminal 130 collects images of the target goods and sends them to the control terminal 120 according to the received identification request, for example, the user collects images of the goods that have been taken through the user terminal 130; then the control terminal 120 processes the images collected by the user terminal 130 and re-acquires the first inventory information, and the processing method refers to the description in the aforementioned embodiment. Among them, the control terminal 120 will match the user terminal 130 through user information, and the user information is, for example, the user's identity authentication information. The user information is mainly sent to the control terminal before the container is opened. The control terminal performs security verification on the user information. When the user information passes the security verification, the container is controlled to be opened; if the user information verification fails, the container is not controlled to be opened. In addition, the abnormal processing also includes issuing alarm information, for example, the control terminal sends an alarm prompt to the user terminal to serve as a reminder.

[0069] Furthermore, exception handling is also required when at least one of the following situations occurs: for example, the user end / control end stores preset first inventory information, which is used to indicate the quantity and category of goods to be taken or put in. Before the container is opened, the control end has obtained this information. When the control end obtains the visual recognition result through image processing, the visual recognition result is verified according to the preset information to determine whether each information in the visual recognition result matches the preset information. If at least one item does not match, such as the state does not match but the quantity and category match, it is considered that this is an abnormal situation and exception handling is required. In addition, in order to improve the efficiency of item management, the current radio frequency identification method is used for automated management, such as binding radio frequency tags on the goods by posting or embedding, and obtaining radio frequency information by identifying the radio frequency tags through the radio frequency identification device installed inside the container. The control end further processes the radio frequency information to generate a radio frequency identification result, which includes the category and quantity of the target goods. Although this method can improve the degree of automation, as the number of goods increases, it is difficult to set a radio frequency tag on each item for identification. At the same time, the cost of radio frequency tags is always high, and radio frequency tags are easily blocked, and the possibility of abnormality is high.

[0070] Therefore, in the embodiment of the present application, RFID is used as an auxiliary solution. For goods that have been bound with RFID tags, RFID can be performed in the above manner, and the visual recognition result is verified according to the RFID result. When the category and the quantity of goods in the RFID result do not match the category and the quantity of goods in the visual recognition result, an exception is handled. The relevant content of the exception handling refers to the description in the above embodiment, which is not repeated here. In this embodiment, RFID is combined with visual recognition, which can supplement the visual recognition solution and improve the recognition accuracy.

[0071] The present application also provides a container inventory management device, such as Figure 5 As shown, the management device 500 includes a processor 510, which is used to call instructions stored in a memory 520. When the instructions are called by the processor 510, the processor 510 executes any one of the management methods in the above embodiments. The control end in any of the aforementioned embodiments includes the above management device, which is used to implement any of the management methods in the above embodiments. In the embodiments of the present application, the control end may include a processor, such as a microcontroller unit (MCU), or a digital signal processor (DSP), etc.; in other implementations, a central processing unit (CPU), or a graphics processing unit (GPU), etc. may also be selected; or a combination of at least two of these processor forms.

[0072] In addition, an embodiment of the present application further provides a computer-readable storage medium on which a computer program / instruction is stored, and when the computer program / instruction is executed by a processor, the steps of any of the above management methods are implemented. An embodiment of the present application further provides a computer program product, including a program / instruction, and when the program / instruction is executed by a processor, the steps of any of the above management methods are implemented.

[0073] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments. In addition, the above embodiments can be freely combined as needed.

Claims

1. A method for managing container inventory, characterized in that: Applied to the control end, at least one camera is installed inside the container, and the camera is used to collect several frames of images inside the container when the container is in an open state. The method includes: Processing the plurality of frames of images to obtain a visual recognition result of first inventory information; the first inventory information includes the state, category and quantity of target goods; the state includes outbound or inbound; When the category and the quantity of goods meet a preset condition, the second inventory information of the target goods is updated according to the quantity of goods and the category.

2. The management method according to claim 1, characterized in that: The step of processing the plurality of frames of images to obtain a visual recognition result of the first inventory information includes: Performing a first recognition process on the plurality of frames of images according to a preset action recognition model to determine the state; Extracting image features from the plurality of frames of images using a preset feature extraction model; and determining the category according to the image features; Perform a second recognition process on the plurality of frames of images to determine the quantity of the goods.

3. The management method according to claim 2, characterized in that: The performing the first recognition processing on the plurality of frames of images to determine the state includes: Extracting action features from the plurality of frames of images, and determining whether the action features are standard features based on the preset action recognition model; the standard features include standard picking features or standard putting features; When the action feature is the standard feature, determining the state as outbound or inbound; When the action feature is not the standard feature, exception processing is performed.

4. The management method according to claim 1, characterized in that: Also includes: When the category and the quantity of goods do not meet the preset conditions, exception processing is performed.

5. The management method according to claim 1, characterized in that: Also includes: Before the container is opened, obtaining the preset first inventory information; Verifying the visual recognition result according to the preset first inventory information; When the preset first inventory information does not match the visual recognition result, exception processing is performed.

6. The management method according to claim 1, characterized in that: A radio frequency identification device is also installed inside the container. When the container is in the open state, the radio frequency identification device collects radio frequency information of the goods in real time. The method further includes: Processing the radio frequency information to generate radio frequency identification results of the category and quantity of the goods; Verifying the category and the quantity of goods in the visual identification result according to the radio frequency identification result; When the category and the quantity of goods in the radio frequency identification result do not match the category and the quantity of goods in the visual identification result, an exception process is performed.

7. The management method according to any one of claims 3 to 6, characterized in that: The exception handling includes: Sending a recognition request to a user terminal, so that the user terminal collects an image of the target goods according to the recognition request and sends the image to the control terminal, and the control terminal processes the image of the target goods and re-acquires the first inventory information; The user terminal is matched with the user information, and the user information is used for security verification before the container is opened. When the user information passes the security verification, the container is controlled to be opened.

8. A container inventory management device, characterized in that: It includes a processor, which is used to call instructions stored in a memory, and when the instructions are called by the processor, the processor executes the management method according to any one of claims 1 to 7.

9. A container inventory management system, characterized in that: Including control terminal, container, user terminal, The control end comprises the management device as claimed in claim 8; The user end is used to collect the image of the target goods according to the identification request sent by the container and send it to the control end when performing exception processing; the control end is also used to process the image of the target goods and update the second inventory information.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the management method according to any one of claims 1 to 7 are implemented.

Images