Cargo management method and apparatus

Abstract

The application provides a kind of goods management method and device, it is related to warehouse management technical field.The goods management method includes acquisition and obtain the video stream of the warehouse to be monitored, every frame video image in video stream includes goods shelf;When detecting that there is character and goods car in video image, mark the first area containing the goods shelf located on the side of character, and mark the second area containing the goods car located on the other side of character;When detecting that the goods of first area between adjacent two frame video images change, record first time;And when detecting that the goods of second area between adjacent two frame video images change, record second time;According to the order of first time and second time, determine the in-out operation behavior of goods;According to the in-out operation behavior determined, the goods of the warehouse to be monitored are managed correspondingly, do not need to set goods sensor on goods placement site, cost is very high.

Description

Technical Field

[0001] This application relates to the field of warehouse management technology, and in particular to a cargo management method and apparatus. Background Technology

[0002] Typically, goods move very frequently in some companies' warehouses, so it is necessary to effectively record the inbound and outbound operations of goods in the warehouse for subsequent traceability.

[0003] Currently, one way to record the entry and exit operations of goods in a warehouse is to install a goods sensor at each goods placement position on the shelf. When the goods sensor detects that the goods in the goods placement position have been taken away, it is recorded as an outbound operation; when the goods sensor detects that a new goods have been added to the goods placement position, it is recorded as an inbound operation. This is how to manage goods, such as recording the entry and exit behavior of goods.

[0004] However, the above technical solution requires the installation of cargo sensors at each cargo placement location, which is very costly. Summary of the Invention

[0005] This application provides a cargo management method and apparatus to solve the problem that the management of cargo in warehouses in the prior art requires the installation of cargo sensors at each cargo placement location, which results in high costs.

[0006] Firstly, this application provides a goods management method applied to a server. The goods management method provided by this application includes: the server acquiring a video stream of a warehouse to be monitored, each frame of the video stream including a shelf; when the server detects the presence of a person and a vehicle in the video image, marking a first area containing a shelf located on one side of the person, and marking a second area containing a vehicle located on the other side of the person; when the server detects a change in the goods in the first area between two adjacent video frames, recording a first moment; and when the server detects a change in the goods in the second area between two adjacent video frames, recording a second moment; the server determining the inbound / outbound operation behavior of the goods based on the order of the first and second moments; and the server managing the goods in the warehouse to be monitored accordingly based on the determined inbound / outbound operation behavior.

[0007] The goods management method provided in this application involves a server marking a first area containing shelves located on one side of the person and a second area containing vehicles located on the other side of the person when the server detects the presence of people and vehicles in a video image of a warehouse under monitoring. Furthermore, the method can determine the inbound / outbound operations of goods based on the sequence of detected changes in goods in the second area between two adjacent video frames. This allows for the management of goods in the monitored warehouse based on the determined inbound / outbound operations, eliminating the need for goods sensors at each placement location and reducing costs.

[0008] In one optional implementation, the server determines the inbound / outbound operation of goods based on the order of the first moment and the second moment, including: when the first moment is earlier than the second moment, the server determines that the inbound / outbound operation of goods is an outbound operation.

[0009] When the first moment is earlier than the second moment, it indicates that the person takes the goods from the first area of ​​the shelf and puts them into the second area of ​​the cargo vehicle. Therefore, the operation of goods entering and leaving the warehouse is an outbound operation.

[0010] In one optional implementation, the server determines the inbound / outbound operation of goods based on the order of the first moment and the second moment, including: when the first moment is later than the second moment, the server determines that the inbound / outbound operation of goods is an inbound operation.

[0011] When the second moment is earlier than the first moment, it means that the person takes the goods from the second area of ​​the cargo vehicle and puts them on the first area of ​​the cargo shelf. Therefore, the operation of goods entering and leaving the warehouse is an entry operation.

[0012] In one optional implementation, detecting the presence of people and goods in a video image includes: comparing every two adjacent video images to determine whether the image content in the video image has changed; when the image content in the video image changes, selecting a target area in the video image where the image content has changed; and detecting the presence of people and goods in the target area based on a pre-trained image detection model.

[0013] Understandably, determining whether the content of a video image has changed is less computationally intensive than detecting the presence of people or vehicles in a target region. Therefore, when the content of a video image changes, selecting the target region where the content has changed, and then using a pre-trained image detection model to detect whether people or vehicles exist in that target region, can save computational resources.

[0014] In one optional implementation, comparing every two adjacent video frames to determine whether the image content in the video frames has changed includes: performing a difference calculation on every two adjacent video frames; and determining that the image content in the video frames has changed when the area of ​​the connected components contained in the result of the difference calculation is greater than a set area threshold.

[0015] In one alternative implementation, the first region is obtained by expanding the region containing the person by a preset multiple.

[0016] Typically, the person stands on one side of the shelf, and the area occupied by the person is expanded by a preset multiple to include all or part of the shelf area. Since the person is included in the first area, this area is where the person is likely to load or unload goods, which can improve the accuracy of subsequent determination of inbound and outbound behaviors.

[0017] In one alternative implementation, the goods in the warehouse to be monitored are managed accordingly, including: recording inbound and outbound operations, identifying the cameras that capture video streams, and recording the time when inbound and outbound operations occur.

[0018] Secondly, this application also provides a cargo management device, comprising: a video acquisition unit for acquiring a video stream of a warehouse to be monitored, wherein each frame of the video stream includes a shelf; an image marking unit for marking a first area containing a shelf located on one side of a person and a cargo vehicle located on the other side of a person when a person and a cargo vehicle are detected in the video image; an image recognition unit for recording a first moment when a change in cargo is detected in the first area between two adjacent video frames; and recording a second moment when a change in cargo is detected in the second area between two adjacent video frames; an entry / exit determination unit for determining the entry / exit operation behavior of cargo based on the order of the first moment and the second moment; and a cargo management unit for managing the cargo in the warehouse to be monitored according to the determined entry / exit operation behavior.

[0019] Thirdly, this application provides a server including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it causes the server to perform the method provided in the first aspect.

[0020] Fourthly, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the computer to perform the method provided in the first aspect.

[0021] Fifthly, this application also provides a computer program product, including a computer program that, when run, causes a computer to perform the method provided in the first aspect. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram illustrating the interaction between the server and the camera, provided in an embodiment of this application.

[0024] Figure 2 A flowchart of a cargo management method provided in an embodiment of this application;

[0025] Figure 3 This is one of the schematic diagrams of a warehouse to be monitored provided in an embodiment of this application;

[0026] Figure 4 The second schematic diagram of a warehouse to be monitored provided in an embodiment of this application;

[0027] Figure 5 for Figure 2 The detailed flowchart of S202 in the document;

[0028] Figure 6 This is a schematic diagram of the functional units of the cargo management device provided in the embodiments of this application. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments made by those skilled in the art under the guidance of these embodiments are within the scope of protection of this application.

[0030] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] Currently, one way to record the entry and exit operations of goods in a warehouse is to install a goods sensor at each goods placement position on the shelf. When the goods sensor detects that the goods in the goods placement position have been taken away, it is recorded as an outbound operation; when the goods sensor detects that a new goods have been added to the goods placement position, it is recorded as an inbound operation. This is how to manage goods, such as recording the entry and exit behavior of goods.

[0032] However, the above technical solution requires the installation of cargo sensors at each cargo placement location, which is very costly.

[0033] Based on the aforementioned technical problems, the inventive concept of this application is to: acquire video streams of the warehouse to be monitored, and determine the inbound and outbound behaviors of goods by performing image recognition on the video streams. Furthermore, based on the determined inbound and outbound operations, the goods in the monitored warehouse are managed accordingly, eliminating the need to install goods sensors at the goods placement locations, which would otherwise be very costly.

[0034] The technical solutions of this application and how they solve the aforementioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0035] This application provides a goods management system for managing warehouse goods. Figure 1 As shown, the system includes a server and multiple cameras capable of capturing images of the locations of the shelves, enabling the cameras to photograph the shelves and the goods on them within the monitored warehouse. Specifically, the server 100 is communicatively connected to the cameras 200.

[0036] On the aforementioned system, Figure 2 A flowchart of the cargo management method provided in the embodiments of this application is shown below. Figure 2As shown, the cargo management method provided in this application includes:

[0037] S201: Server 100 acquires video streams of the warehouse to be monitored. Each frame of the video stream includes the shelves.

[0038] Understandably, the camera 200 located within the warehouse to be monitored can capture video streams of the warehouse and send them to the server 100. The server 100 can then receive the video streams from the warehouse to be monitored. Each frame of the video stream includes a shelf, which is used to store goods.

[0039] like Figure 3 As shown, the warehouse to be monitored in the video image includes shelves with goods placed on them (no people or vehicles entered the warehouse to be monitored).

[0040] S202: When the server 100 detects the presence of a person and a cargo vehicle in the video image, it marks a first area 301 containing the cargo shelf located on one side of the person and a second area 302 containing the cargo vehicle located on the other side of the person.

[0041] When a warehouse manager needs to bring goods into or out of the warehouse, they need to push a cart into the warehouse. At this time, the server 100 can detect the presence of people and carts in the video image. It should be noted that the cart can be a handcart, electric pallet truck, or small forklift, and the goods are placed on the pallet or plug of the cart.

[0042] Optionally, the first region 301 is obtained by expanding the area containing the person by a preset multiple (such as 1.5 times or 1.2 times, which is not limited here). Typically, the person stands on one side of the shelf, and the first region 301, expanded by the preset multiple, includes all or part of the shelf area. Since the first region 301 includes the person, it is an area where the person may load or unload goods, improving the accuracy of subsequent determination of inbound and outbound actions. For example, the first region 301 and the second region 302 marked by the server 100 can be as follows: Figure 4 As shown.

[0043] Specifically, such as Figure 5 As shown, S202 may include the following steps:

[0044] S401: Compare every two adjacent video frames to determine whether the content of the video images has changed. If so, execute S402.

[0045] Specifically, differential calculation can be performed on every two adjacent video frames; when the area of ​​the connected components contained in the result of the differential calculation is greater than a set area threshold, it is determined that the image content in the video image has changed.

[0046] S402: Select the target area in the video image where the image content changes 303.

[0047] For example, when a warehouse manager pushes a cart into the warehouse, such as Figure 3 As shown, the target area 303, including people and cargo vehicles, can be selected as a whole.

[0048] Furthermore, the position and size of the target region 303 can be recorded. In subsequent video images, the target region 303 can be tracked. If people and vehicles are also present in subsequent video images, the position and size of the dynamic target region 303 can be updated. In addition, to reduce redundancy, the target region 303 can be marked as a preset multiple of the bounding rectangle that includes people and vehicles. For example, the preset multiple could be 1.2 times, 1.5 times, etc., which is not limited here.

[0049] S403: Based on the pre-trained image detection model, people and cargo vehicles were detected in target region 303.

[0050] The pre-trained image detection model can be obtained by pre-training the network by inputting images containing people and images containing goods as positive samples, and images not containing people or goods as negative samples. The network to be trained can be a convolutional neural network, a random forest network, a decision tree network, etc., and is not limited here.

[0051] Understandably, based on the above S401-S403, the computational cost of determining whether the image content in the video image has changed is less than that of detecting whether people or goods / vehicles exist in the target region 303. Therefore, when the image content in the video image changes, selecting the target region 303 where the image content has changed, and then using the pre-trained image detection model to detect whether people or goods / vehicles exist in the target region 303, can save computational resources.

[0052] S203: When the server 100 detects a change in the goods in the first region 301 between two adjacent video frames, it records a first moment; and when it detects a change in the goods in the second region 302 between two adjacent video frames, it records a second moment.

[0053] For example, S203 can be implemented as follows: First, determine the size and position w1×h1 of the area where the person is located in the video image. Then, enlarge the area by a preset factor to obtain the size w2×h2 of the first area 301, and determine the geometric center P1 of the first area 301. Next, perform a difference between the first area 301 of the video image in the Nth frame and the first area 301 of the video image in the N+1th frame, with P1 as the geometric center and a size of w1×h1. If there is no changing area, it indicates that no goods have been added or removed from the shelf; if there is a changing area, it indicates that goods have been added or removed from the shelf. Then, record the time when the N+1th frame of the video image was received (i.e., the first moment).

[0054] The size and position (m1×n1) of the area where the cargo vehicle is located in the video image are determined, and the geometric center (Q1) of the second area 302 is determined. The first area 301 of the video image in the Nth frame and the first area 301 of the N+1th frame with Q1 as the geometric center and a size (m1×n1) are subtracted. If there is no changing area, it means that there is no addition or subtraction of goods on the shelf; if there is a changing area, it means that there is an addition or subtraction of goods on the shelf. Then, the time when the N+1th frame of the video image is received (i.e., the second moment) is recorded.

[0055] S204: Server 100 determines the inbound and outbound operations of goods based on the chronological order of the first and second moments.

[0056] The specific implementation methods of S204 include, but are not limited to, the following two:

[0057] The first scenario: For example, when the first moment is earlier than the second moment, the server 100 determines that the operation of entering or leaving the warehouse for goods is an outbound operation. Understandably, when the first moment is earlier than the second moment, it means that a person has taken goods from the first area 301 of the shelf and placed them in the second area 302 of the cart; therefore, the operation of entering or leaving the warehouse for goods is an outbound operation.

[0058] The second scenario: When the first moment is later than the second moment, server 100 determines that the operation of moving goods in and out of the warehouse is an inbound operation. When the second moment is earlier than the first moment, it means that the character takes the goods from the second area 302 of the cargo vehicle and puts them on the first area 301 of the cargo shelf. Therefore, the operation of moving goods in and out of the warehouse is an inbound operation.

[0059] S205: Server 100 manages the goods in the monitored warehouse according to the determined inbound and outbound operations.

[0060] Specifically, managing the goods in the monitored warehouse can involve recording detailed information about the goods entering and leaving the warehouse. This detailed information may include, but is not limited to, the entry and exit operations (such as inbound and outbound), the identifier of the camera 200 that collects the video stream, and the time when the entry and exit operations occur, which is not limited here.

[0061] Furthermore, warehouse managers can view the recorded details of goods entering and leaving the warehouse through terminal devices, enabling efficient inventory checks. These terminal devices can be, but are not limited to, mobile phones or computers.

[0062] Specifically, warehouse managers can input search criteria (such as camera 200 identifier or time period) into the terminal device, triggering the terminal device to send a data request carrying the search criteria to the server 100. The server 100 then sends detailed information that meets the search criteria to the terminal device. In this way, the terminal device can view the recorded details of goods entering and leaving the warehouse.

[0063] In summary, the goods management method provided in this application involves server 100 marking a first area 301 containing shelves located on one side of the person and a second area 302 containing vehicles located on the other side of the person when the server detects the presence of people and vehicles in a video image of the warehouse to be monitored. Furthermore, the order in which changes in goods are detected in the second area 302 between two adjacent video frames can be used to determine the inbound / outbound operations for goods. Consequently, the goods in the monitored warehouse can be managed accordingly based on the determined inbound / outbound operations, eliminating the need for goods sensors at each placement location and reducing costs.

[0064] Please see Figure 6 This application also provides a cargo management device 500. It should be noted that the cargo management device 500 provided in this application has the same basic principle and technical effects as the above embodiments. For the sake of brevity, any parts not mentioned in this application can be referred to the corresponding content in the above embodiments. The cargo management device 500 includes a video acquisition unit 501, an image marking unit 502, an image recognition unit 503, and a cargo management unit 505.

[0065] The video acquisition unit 501 is used to acquire video streams of the warehouse to be monitored. Each frame of the video stream includes the shelves.

[0066] The image marking unit 502 is used to mark a first area containing a shelf located on one side of the person and a second area containing a vehicle located on the other side of the person when a person and a vehicle are detected in a video image.

[0067] In one alternative implementation, the image recognition unit 503 is used to determine, when the first moment is earlier than the second moment, that the server determines that the inbound / outbound operation of goods is an outbound operation.

[0068] In another alternative implementation, the image recognition unit 503 is used to determine, when the first moment is later than the second moment, that the server determines that the inbound / outbound operation of goods is an inbound operation.

[0069] In one alternative implementation, the first region is obtained by expanding the region containing the person by a preset multiple.

[0070] The image recognition unit 503 is configured to record a first moment when a change in the goods in a first region between two adjacent video frames is detected; and to record a second moment when a change in the goods in a second region between two adjacent video frames is detected.

[0071] In an optional implementation, the image recognition unit 503 is further configured to compare every two adjacent video images to determine whether the image content in the video image has changed; when the image content in the video image changes, to select the target area in the video image where the image content has changed; and to detect the presence of people and cargo vehicles in the target area based on a pre-trained image detection model.

[0072] In one optional implementation, the image recognition unit 503 is specifically used to perform differential calculation on every two adjacent video images; when the area of ​​the connected components contained in the result of the differential calculation is greater than a set area threshold, it is determined that the image content in the video image has changed.

[0073] The inbound / outbound determination unit 504 is used to determine the inbound / outbound operation behavior of goods based on the order of the first moment and the second moment.

[0074] The goods management unit 505 is used to manage the goods in the warehouse to be monitored based on the determined inbound and outbound operations.

[0075] In one alternative implementation, the goods management unit is specifically used to record inbound and outbound operations, the identification of the camera that captures the video stream, and the time when the inbound and outbound operations occur.

[0076] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the computer to perform the method provided in the above embodiments.

[0077] This application also provides a server, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it causes the server to perform the method provided in the above embodiments.

[0078] This application also provides a computer program product, including a computer program that, when run, causes a computer to perform the methods provided in the above embodiments.

[0079] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0080] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A cargo management method characterized by, Applied to a server, the method comprises: The server acquires a video stream of a warehouse to be monitored through a camera located in the warehouse, each frame of video image in the video stream comprising a goods shelf; When the server detects that a person and a goods vehicle exist in the video image, the server marks a first area comprising the goods shelf on one side of the person and a second area comprising the goods vehicle on the other side of the person; The detection of the person and the goods vehicle in the video image comprises: Difference calculation is performed on every two adjacent frames of the video image; When the area of a connected domain contained in the result of the difference calculation is greater than a set area threshold, it is determined that the image content in the video image has changed; when the image content in the video image has changed, a target area in which the image content has changed in the video image is framed; According to a pre-trained image detection model, it is detected that the person and the goods vehicle exist in the target area; The server records a first time when it is detected that the goods in the first area between two adjacent frames of the video image have changed, and records a second time when it is detected that the goods in the second area between two adjacent frames of the video image have changed; The server determines an in-out warehouse operation behavior on the goods according to the order of the first time and the second time; The server manages the goods in the warehouse to be monitored according to the determined in-out warehouse operation behavior; The management of the goods in the warehouse to be monitored comprises: Recording the in-out warehouse operation behavior, the identifier of the camera that acquires the video stream, and the time at which the in-out warehouse operation behavior occurs.

2. The method of claim 1, wherein, The determination of the in-out warehouse operation behavior on the goods according to the order of the first time and the second time comprises: When the first time is earlier than the second time, the server determines that the in-out warehouse operation behavior on the goods is an out-warehouse operation behavior.

3. The method of claim 1, wherein, The determination of the in-out warehouse operation behavior on the goods according to the order of the first time and the second time comprises: When the first time is later than the second time, the server determines that the in-out warehouse operation behavior on the goods is an in-warehouse operation behavior.

4. The method of claim 1, wherein, The first area is obtained by expanding the area comprising the person by a preset multiple.

5. A cargo management device, characterized by, The device comprises: A video acquisition unit configured to acquire a video stream of a warehouse to be monitored through a camera located in the warehouse, each frame of video image in the video stream comprising a goods shelf; An image marking unit configured to, when a person and a goods vehicle are detected in the video image, mark a first area comprising the goods shelf on one side of the person and a second area comprising the goods vehicle on the other side of the person; An image recognition unit configured to, when the goods in the first area between two adjacent frames of the video image have changed, record a first time; and when the goods in the second area between two adjacent frames of the video image have changed, record a second time; The image recognition unit is specifically configured to: perform difference calculation on every two adjacent video images; determine that image content in the video image changes when an area of a connected domain contained in a result of the difference calculation is greater than a set area threshold; when the image content in the video image changes, frame the target region in which the image content in the video image changes; and detect, according to a pre-trained image detection model, that a person and a cargo vehicle exist in the target region. The warehouse in-out determination unit is configured to determine an in-out operation behavior on the cargo according to the first time and the second time. The cargo management unit is configured to perform corresponding management on the cargo of the warehouse to be monitored according to the determined in-out operation behavior. The cargo management unit is specifically configured to record the in-out operation behavior, an identifier of a camera that collects the video stream, and a time at which the in-out operation behavior occurs.

6. A server comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The computer program is executed by the processor, so that the server executes the method in any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, wherein the computer program comprises the following steps of: receiving a request for a resource from a client; determining whether the client is authorized to access the resource; and if the client is authorized to access the resource, providing the resource to the client. The computer program is executed by the processor, so that the computer executes the method in any one of claims 1 to 4.

Images