Goods quantity determination method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a goods quantity determination method and device, electronic equipment and a storage medium, and the method comprises the steps: carrying out the detection of a monitoring video of a target goods location in a warehouse, obtaining the track data of a monitoring object, enabling the monitoring object to comprise a carrier for transporting goods, and enabling the track data to comprise the goods quantity data of the carrier at each track point; a critical track point is determined from all the track points according to the goods quantity data of the carrier at all the track points, and the critical track point is located in the goods allocation area of the target goods allocation and is the track point with the changed goods quantity data in all the track points; selecting a target trajectory point from the trajectory points according to the critical trajectory point; and according to the cargo quantity data of the target track point, determining the cargo quantity variation of the target cargo allocation. According to the invention, the moving track of the carrier and the goods direction are tracked based on the monitoring video of the goods allocation in the warehouse, so that the goods allocation in which the goods are placed is determined, the goods quantity condition of each goods allocation can be known in real time, the existing video monitoring facility can be reused, and the cost is reduced.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of warehousing technology, and in particular, to a method, device, electronic device and storage medium for determining cargo quantity. Background Art

[0002] Warehouse cargo counting usually adopts methods such as entrance and exit counting and sensor counting. Among them, entrance and exit counting usually sets a counting line at the entrance and exit. When goods enter the warehouse through this line, the count increases by one, and when goods leave the warehouse through this line, the count decreases by one. Sensor counting is to attach a radio frequency tag to each cargo, and identify how many radio frequency tags there are through the tag identifier to achieve the purpose of counting.

[0003] In the process of implementing the present invention, the inventors found that the entrance and exit counting method cannot determine which cargo location the goods have entered, which may cause confusion in logistics and storage and affect the accurate tracking of goods. In addition, some goods transactions require precision to the cargo location, and the entrance and exit counting method can no longer meet the actual application needs; the sensor counting method requires a large number of radio frequency tags, which is costly. Summary of the invention

[0004] The embodiments of the present invention provide a method, device, electronic device and storage medium for determining the quantity of goods, which can accurately determine the quantity of goods in each cargo location and save costs.

[0005] In a first aspect, a method for determining cargo quantity provided by an embodiment of the present invention includes:

[0006] Detect the surveillance video of the target cargo location in the warehouse to obtain the trajectory data of the monitored object, which includes the carrier transporting the goods, and the trajectory data includes the cargo volume data of the carrier at each trajectory point;

[0007] Determine a critical trajectory point from each trajectory point according to the cargo volume data of the carrier at each trajectory point, the critical trajectory point is a trajectory point where the cargo volume data in each trajectory point changes, and the critical trajectory point is located in the cargo location area of ​​the target cargo location;

[0008] Selecting a target trajectory point from each trajectory point according to a critical trajectory point;

[0009] Determine the change in cargo volume at the target cargo location based on the cargo volume data at the target trajectory point.

[0010] In a second aspect, an embodiment of the present invention provides a cargo quantity determination device, comprising:

[0011] A data detection module is used to detect the surveillance video of the target cargo location in the warehouse to obtain the trajectory data of the monitored object, the monitored object includes the carrier for transporting the goods, and the trajectory data includes the cargo volume data of the carrier at each trajectory point;

[0012] A track point determination module is used to determine a critical track point from each track point according to the cargo volume data of the carrier at each track point. The critical track point is a track point where the cargo volume data in each track point changes. The critical track point is located in the cargo area of ​​the target cargo location.

[0013] A trajectory point selection module is used to select a target trajectory point from various trajectory points according to a critical trajectory point;

[0014] The cargo quantity determination module is used to determine the cargo quantity change of the target cargo location based on the cargo quantity data of the target trajectory point.

[0015] In a third aspect, an electronic device provided by an embodiment of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the program, a method for determining cargo quantity as in any embodiment of the present invention is implemented.

[0016] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, a method for determining cargo quantity as in any embodiment of the present invention is implemented.

[0017] The scheme of the embodiment of the present invention can detect the monitoring video of the target cargo position in the warehouse to obtain the trajectory data of the monitored object, the monitored object includes the carrier for transporting goods, and the trajectory data includes the cargo quantity data of the carrier at each trajectory point; according to the cargo quantity data of the carrier at each trajectory point, a critical trajectory point is determined from each trajectory point, the critical trajectory point is the trajectory point where the cargo quantity data in each trajectory point changes, and the critical trajectory point is located in the cargo position area of ​​the target cargo position; according to the critical trajectory point, a target trajectory point is selected from each trajectory point; according to the cargo quantity data of the target trajectory point, the cargo quantity change of the target cargo position is determined. The present invention is equivalent to tracking the moving trajectory of the carrier and the destination of the goods based on the monitoring video of the cargo position in the warehouse, so as to determine which cargo position the goods are specifically placed in the warehouse, which is convenient for real-time understanding of the cargo quantity of each cargo position, and helps warehouse managers make accurate inventory management decisions. This counting method can meet the actual application needs; in addition, the present invention can reuse existing video surveillance facilities, does not need to attach radio frequency tags to each cargo, does not need to invest a lot of extra costs to purchase and maintain tags, and reduces costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solution of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1is a flow chart of a method for determining cargo quantity provided by an embodiment of the present invention;

[0020] Figure 2 is another flow chart of the method for determining cargo quantity provided by an embodiment of the present invention;

[0021] Figure 3 is an example diagram of a cargo area provided by an embodiment of the present invention;

[0022] Figure 4 is an example diagram of a target detection model provided by an embodiment of the present invention;

[0023] Figure 5 is an example diagram of an object group provided by an embodiment of the present invention;

[0024] Figure 6 is an example diagram of a warehouse-out operation provided by an embodiment of the present invention;

[0025] Figure 7 This is an example diagram of a target cargo location not being operated provided by an embodiment of the present invention;

[0026] Figure 8 is a structural schematic diagram of a cargo quantity determination device provided by an embodiment of the present invention;

[0027] Fig. 9 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] Figure 1 The flowchart of the cargo quantity determination method provided by the embodiment of the present invention is a schematic diagram. The cargo quantity determination method provided by the embodiment of the present invention can be applied to the scene where the cargo quantity of each cargo location in the warehouse needs to be monitored or determined. The cargo quantity determination method can be executed by the cargo quantity determination device provided by the embodiment of the present invention, and the device can be implemented in software and / or hardware. In a specific embodiment, the cargo quantity determination device can be integrated into an electronic device, and the electronic device can be a computer or a server. The following embodiment is described by taking the cargo quantity determination device integrated into an electronic device as an example. Please refer to Figure 1 The method of this embodiment may include the following steps:

[0031] S110: Detect the surveillance video of the target cargo location in the warehouse to obtain trajectory data of the monitored object.

[0032] Among them, the cargo location is a specific spatial location in the warehouse for storing goods; the target cargo location is a cargo location that needs to be paid attention to and monitored in warehouse management, and the target cargo location can be any cargo location in the warehouse. The goods in this embodiment can refer to bulk goods, which can be goods with a relatively large transaction volume and volume, such as some building materials, construction equipment, etc. The transaction of bulk goods often relies heavily on the cargo location, and it is necessary to clarify the amount of goods on each cargo location.

[0033] The monitoring object is a target that needs to be monitored in real time in the warehouse. The monitoring object can be a moving object. The monitoring object can cause changes in the amount of goods on the cargo space. The monitoring object can include a carrier for transporting goods. For example, the carrier can be a forklift, a logistics vehicle, a robot, etc.

[0034] Trajectory data refers to data that indicates the movement path and status information of the monitored object in the warehouse. Trajectory data can be represented by videos, pictures, arrays, etc. Trajectory data can include cargo volume data of the carrier at each trajectory point; a trajectory point is a record point in the trajectory data, which represents the specific location of the monitored object at a certain moment. Through a series of trajectory points, the complete movement trajectory of the monitored object can be depicted; cargo volume data can be data on the amount of cargo carried by the carrier at a certain trajectory point. The cargo volume can be the number of cargo, the volume of cargo, etc.

[0035] Optionally, multiple cameras are installed in the warehouse to monitor each cargo location in the warehouse in real time; one camera can monitor one or more cargo locations.

[0036] Specifically, when it is necessary to determine the quantity of goods in each cargo location in the warehouse, the current cargo location whose quantity needs to be determined can be selected from each cargo location, and the current cargo location can be determined as the target cargo location. Then, the surveillance video taken by the camera monitoring the target cargo location is obtained to obtain the surveillance video of the target cargo location. Then, the surveillance video of the target cargo location is detected. For example, the surveillance video can be analyzed using technologies such as video analysis, image recognition, and target detection, and the movement trajectory of the monitored object can be detected. The cargo quantity data carried by the monitored object at each trajectory point can be determined, thereby obtaining the trajectory data of the monitored object.

[0037] S120, determining a critical trajectory point from each trajectory point according to the cargo volume data of the carrier at each trajectory point.

[0038] The critical trajectory point is a trajectory point where the cargo volume data changes among the trajectory points, and the critical trajectory point is located within the cargo area of ​​the target cargo location. The cargo volume data changes, which may be an increase in cargo volume or a decrease in cargo volume.

[0039] The cargo area is an area corresponding to the cargo location that needs to be monitored, that is, an area for storing goods. Optionally, a corresponding cargo area can be set in advance for each cargo location in the warehouse.

[0040] Specifically, after obtaining the trajectory data of the monitored object, the critical trajectory point can be determined from each trajectory point according to the cargo volume data of the carrier at each trajectory point. For example, all trajectory points can be traversed, and the cargo volume data of adjacent trajectory points can be compared, and then the trajectory point where the cargo volume data begins to change can be found, and the trajectory point can be determined as a candidate trajectory point, and then it can be determined whether the candidate trajectory point is located in the cargo area of ​​the target cargo location, and when the candidate trajectory point is located in the cargo area of ​​the target cargo location, the candidate trajectory point is determined as a critical trajectory point, and if the candidate trajectory point is not in the cargo area of ​​the target cargo location, the candidate trajectory point is not determined as a critical trajectory point. If the candidate trajectory point is in the cargo area of ​​the target cargo location, it means that the change in the cargo volume data of the carrier will affect the cargo volume of the target cargo location; for example, at this time, the carrier may be performing warehousing or outbound operations at the target cargo location, which will cause the cargo volume data of the carrier to change, and then affect the cargo volume of the target cargo location. If the candidate trajectory point is not in the cargo area of ​​the target cargo location, it means that changes in the carrier's cargo volume data will not affect the cargo volume of the target cargo location; for example, at this time the carrier only adjusts the loading position of its own cargo, temporarily unloads cargo, etc. outside the target cargo location. Although the carrier's cargo volume data has changed, this change does not affect the cargo volume of the target cargo location.

[0041] S130 , selecting a target trajectory point from each trajectory point according to the critical trajectory point.

[0042] The target trajectory point may be a trajectory point among the trajectory points that is close to the critical trajectory point in time or space dimension.

[0043] Specifically, after obtaining the critical trajectory point, the target trajectory point can be selected from each trajectory point according to the critical trajectory point. For example, based on the time dimension, the trajectory points whose generation time is close to and continuous with the critical trajectory point can be determined from each trajectory point, and these trajectory points are determined as target trajectory points. Alternatively, based on the spatial dimension, the trajectory points whose positions are close to and continuous with the critical trajectory point can be determined from each trajectory point, and these trajectory points are determined as target trajectory points. There may be multiple selected target trajectory points, and the multiple trajectory points may include trajectory points located in front of and / or behind the critical trajectory point, and the trajectory points before and after can be measured according to the time or space dimension. In order to accurately determine the changes in shipment volume in the future, points can be selected from before and after the critical trajectory point to obtain the target trajectory point.

[0044] S140. Determine the change in cargo volume at the target cargo location based on the cargo volume data at the target trajectory point.

[0045] The change in cargo volume refers to the increase or decrease in the quantity of cargo at the target cargo location over a period of time.

[0046] Specifically, after obtaining the target trajectory point, the change in cargo volume of the target cargo location can be determined based on the cargo volume data of the target trajectory point. For example, the cargo volume data of each target trajectory point can be obtained and compared to determine the change in cargo volume between the target trajectory points. Then, the change in cargo volume of the target cargo location can be determined based on the change in cargo volume between the target trajectory points. For example, the absolute value of the change in cargo volume between the target trajectory points can be determined as the absolute value of the change in cargo volume of the target cargo location, and when the cargo volume between the target trajectory points increases, it is determined that the cargo volume of the target cargo location decreases; when the cargo volume between the target trajectory points decreases, it is determined that the cargo volume of the target cargo location increases, thereby determining the current cargo volume of the target cargo location.

[0047] The technical solution provided in this embodiment can detect the monitoring video of the target cargo position in the warehouse to obtain the trajectory data of the monitored object, the monitored object includes the carrier for transporting goods, and the trajectory data includes the cargo quantity data of the carrier at each trajectory point; according to the cargo quantity data of the carrier at each trajectory point, a critical trajectory point is determined from each trajectory point, the critical trajectory point is the trajectory point where the cargo quantity data in each trajectory point changes, and the critical trajectory point is located in the cargo position area of ​​the target cargo position; according to the critical trajectory point, a target trajectory point is selected from each trajectory point; according to the cargo quantity data of the target trajectory point, the cargo quantity change of the target cargo position is determined. The present invention is equivalent to tracking the moving trajectory of the carrier and the destination of the goods based on the monitoring video of the cargo position in the warehouse, so as to determine which cargo position the goods are specifically placed in the warehouse, which is convenient for real-time understanding of the cargo quantity of each cargo position, and helps warehouse managers make accurate inventory management decisions. This counting method can meet the actual application needs; in addition, the present invention can reuse existing video surveillance facilities, does not need to attach radio frequency tags to each cargo, does not need to invest a lot of extra costs to purchase and maintain tags, and reduces costs.

[0048] The method for determining the cargo quantity provided by the embodiment of the present invention is further described below. Figure 2 As shown, the method of this embodiment includes:

[0049] S201, detecting a carrier, goods transported by the carrier, and a dividing line between the goods in a monitoring video to obtain a detection result.

[0050] In a specific implementation, a camera can be configured for each cargo location in the warehouse, and the camera can be used to monitor the corresponding cargo location, so as to obtain a monitoring video stream of the corresponding cargo location. A camera can monitor one or more cargo locations. The monitoring video stream of each cargo location is displayed to the management personnel through the front end, so that the management personnel can set a cargo location outline for the corresponding cargo location in the monitoring video screen of the cargo location, and the electronic device can determine the cargo location area of ​​the corresponding cargo location based on the cargo location outline set for each cargo location. For example, the management personnel can manually frame a polygon on the corresponding cargo location in the monitoring video screen displayed on the front end to set a cargo location outline for each cargo location, and the electronic device can determine the area framed by each cargo location outline as the cargo location area of ​​the corresponding cargo location. By way of example, a cargo location area that is set can be as follows: Figure 3 As shown, the area selected by the polygon is the cargo area of ​​the corresponding cargo location. Setting the cargo area for each cargo location can provide scenario support for the subsequent determination of the change in the cargo volume of each cargo location.

[0051] When it is necessary to determine the quantity of goods in each cargo position in the warehouse, the current cargo position for which the quantity of goods needs to be determined can be selected from each cargo position, and the current cargo position is determined as the target cargo position. Then, the surveillance video taken by the camera monitoring the target cargo position is obtained to obtain the surveillance video of the target cargo position, and then the surveillance video of the target cargo position is subjected to multi-target detection to obtain the detection result. The multiple targets may include carriers, goods transported by carriers, dividing lines of goods, and carrier identification, etc. The dividing line of goods, which may also be referred to as interval lines, connecting lines, etc., is the physical boundary between goods and is used to distinguish the quantity of goods. For example, the dividing line of goods may be the boundary between two boxes of goods. When the goods are placed on a pallet, the dividing line of goods may be the boundary between two pallets. The carrier identification is used to mark the information of different carriers, which may be a digital card, carrier name, number, and serial number, etc. The carrier identification may be installed at any position of the carrier.

[0052] Exemplarily, the surveillance video of the target cargo location can be divided into frames to obtain multiple frames of surveillance images, and multi-target detection can be performed on each frame of surveillance image. The detection results may include the carrier, the goods transported by the carrier, the dividing line of the goods and the carrier identification, etc. The carrier, the goods transported by the carrier, the dividing line of the goods and the carrier identification can be marked with a detection box in the corresponding surveillance image.

[0053] Optionally, in a possible implementation, a target detection model may be used to detect each frame of the surveillance video, that is, to detect the carrier, the goods transported by the carrier, the dividing line of the goods, and the carrier logo.

[0054] Among them, the target detection model is a pre-trained model, and the target detection model may include a feature extraction module and a detection module; the feature extraction module is used to extract features from the monitoring image, and the feature extraction module may include at least two convolution blocks, and the number of channels of at least two convolution blocks is greater than the preset number of channels, and the number of channels of at least two convolution blocks increases successively. The preset number of channels is the number of channels of a conventional convolution block, which is usually small, that is, in this embodiment, the number of channels of each convolution block can be greater than the number of channels of a conventional convolution block. The number of channels of a conventional convolution block can be, for example, 6, 16, etc., and the number of channels of at least two convolution blocks of the present invention can be 112, 224, etc.; the detection module is used to perform target detection based on the features extracted by the feature extraction module.

[0055] Specifically, the current image frame can be input into the target detection model, and then the feature extraction module of the target detection model can be used to extract features from the current image frame, and the extracted features can be input into the detection module, that is, the feature extraction module can use at least two convolution blocks to perform convolution and pooling operations on the current image frame in sequence to extract features from the current image frame; then, the detection module of the target detection model can be used to perform target detection on the features extracted by the feature extraction module to identify and locate the extracted features to obtain the detection results of the current image frame, such as the position information of the carrier in the current image frame, the position information of the goods transported by the carrier, the position information of the dividing line of the goods, and the position information of the carrier logo.

[0056] For example, Figure 4 As shown, Figure 4 is an example diagram of a target detection model provided by an embodiment of the present invention. Figure 4 The target detection model in includes a feature extraction module and a detection module, wherein the feature extraction module includes at least two convolution blocks, and the number of channels of the at least two convolution blocks increases successively. For example, the feature extraction module can be a standard convolution block (Convolution Block with Leaky ReLU, CBL), and the CBL consists of a convolution layer, a batch normalization layer, and an activation function. For example, the feature extraction module can include 4 standard convolution blocks, the number of channels of the first standard convolution block is 112, the number of channels of the second standard convolution block is 224, the number of channels of the third standard convolution block is 448, and the number of channels of the fourth standard convolution block is 896.

[0057] Since the targets to be detected by the present invention include carriers, goods, dividing lines of goods, carrier logos, etc., there are many categories, and the dividing lines of goods, carrier logos, etc. are relatively small, the number of channels of at least two convolution blocks in the feature extraction module is increased successively, and the number of channels of each convolution block is greater than the number of channels of the conventional convolution block, so that the target detection model has a larger learning space, and thus the target detection model can better capture and learn the features in the input data, thereby increasing the capacity of the target detection model; at the same time, by increasing the number of channels, more feature maps can be constructed, thereby ensuring the amount of information in the feature maps to a certain extent, thereby ensuring the integrity of the information; and increasing the number of channels can enable the feature extraction module to generate more different types of feature maps, so that the target detection model can extract more complex and richer features, thereby improving the performance of the target detection model; using the feature extraction module and the detection module in the target detection model, the carrier, the goods transported by the carrier, the dividing line of the goods and the carrier logo in the monitoring video are detected, realizing the multi-target detection function, improving the detection accuracy, and providing data support for the subsequent determination of the object group.

[0058] S202: Associating the carrier, the cargo and the dividing line according to the detection result to obtain an object group.

[0059] Among them, the object group is a logical unit composed of interrelated carriers, goods and dividing lines.

[0060] Specifically, when a carrier is transporting goods, the carrier, the goods transported by the same carrier, and the dividing line of the aforementioned goods can be regarded as a whole. Thus, after obtaining the detection results, the detection results can be analyzed to determine which carriers, goods, and dividing lines are interrelated, and the related carriers, goods, and dividing lines can be combined into an object group.

[0061] Optionally, in a possible implementation, the matching degree between the carrier and the goods can be calculated based on the detection results, and the matching degree between the goods and the dividing line can be calculated based on the detection results; based on the matching degree between the carrier and the goods and the matching degree between the goods and the dividing line, the carrier, the goods and the dividing line are associated to obtain an object group.

[0062] Among them, the matching degree is the similarity or consistency between two or more objects. In the embodiment of the present invention, the matching degree is used to measure the correlation between the carrier and the goods, and the correlation between the goods and the dividing line; the higher the matching degree, the higher the corresponding correlation.

[0063] Specifically, the matching degree calculation function can be used to analyze the carrier, goods and dividing lines in the detection results. For example, the Hungarian algorithm and intersection over union (IOU) can be used to determine the matching degree between the carrier and the goods, and the matching degree between the goods and the dividing line, and determine which carriers, goods and dividing lines are related to each other.

[0064] Specifically, for any pair of carriers and goods, the IOU value between their detection frames can be calculated to obtain the IOU value between the carriers and goods, and for any pair of goods and dividing lines, the IOU value between their detection frames can be calculated to obtain the IOU value between the goods and the dividing line; secondly, the carrier and goods can be used as nodes, and the line between the carrier and the goods can be used as an edge to construct a carrier-goods bipartite graph, in which the edge indicates that there is a possible matching relationship between the carrier and the goods, and the weight of the edge is 1 minus the IOU value between the carrier and the goods (to facilitate the Hungarian algorithm to find the minimum weight match); then, the goods and the dividing line can be used as nodes, and the line between the goods and the dividing line can be used as an edge to construct a goods-dividing line bipartite graph, in which the edge indicates that there is a possible matching relationship between the goods and the dividing line, and the weight of the edge is 1 minus the IOU value between the goods and the dividing line; then, the Hungarian algorithm can be used to process the carrier-goods bipartite graph and the goods-dividing line bipartite graph respectively to determine the minimum weight match. The carrier and cargo corresponding to the weight (i.e., the maximum IOU value) are obtained to obtain multiple carrier-cargo matching pairs (i.e., matched carrier-cargo pairs), and the cargo and dividing line corresponding to the minimum weight (i.e., the maximum IOU value) are obtained to obtain multiple cargo-dividing line matching pairs (i.e., matched cargo-dividing line pairs); thereafter, based on the preset IOU threshold, all carrier-cargo matching pairs and cargo-dividing line matching pairs can be traversed, and if the IOU value of the carrier-cargo matching pair is less than the preset IOU threshold, the matching pair is released; if the IOU value of the cargo-dividing line matching pair is less than the preset IOU threshold, the matching pair is released; finally, based on the remaining matching pairs (i.e., the carrier-cargo matching pairs whose IOU values ​​are greater than or equal to the preset IOU threshold, and the cargo-dividing line matching pairs whose IOU values ​​are greater than or equal to the preset IOU threshold), the carrier, cargo and dividing line can be associated to obtain an object group, that is, the carrier-cargo matching pairs and the cargo-dividing line matching pairs belonging to the same cargo are taken as one object group. Among them, the IOU value reflects the degree of matching between objects, and the larger the value, the higher the matching degree; the preset IOU threshold is a pre-set value used to determine whether the carrier matches the goods, and whether the goods match the dividing line. The user can adjust and set the preset IOU threshold according to actual usage requirements, and the embodiment of the present invention does not make specific limitations on this; illustratively, the preset IOU threshold can be 0.5.

[0065] For example, Figure 5 FIG. 1 is an example diagram of an object group provided by an embodiment of the present invention. Figure 5 Two object groups are included, and different colors are used to identify different object groups, that is, the carrier, goods and dividing line marked by the blue frame are one object group, and the carrier, goods and dividing line marked by the red frame are another object group.

[0066] In an embodiment of the present invention, the matching degree between the carrier and the goods is calculated based on the detection results, and the matching degree between the goods and the dividing line is calculated based on the detection results. Then, based on the matching degree between the carrier and the goods and the matching degree between the goods and the dividing line, the carrier, the goods and the dividing line are associated to obtain an object group, thereby realizing the object group determination function; by calculating the matching degree, the carrier, the goods carried by the carrier and the dividing line of the goods can be more accurately associated, thereby improving the accuracy of determining the object group, and realizing the automation and intelligence of warehouse management; by associating the carrier, the goods and the dividing line, the movement trajectory of the object group can be more effectively determined, thereby improving the accuracy of determining the trajectory data of the monitored object.

[0067] S203: Track the object group to obtain trajectory data of the monitored objects.

[0068] In this embodiment, once a carrier is detected in a certain frame of the target cargo location monitoring video, the carrier is associated with the corresponding target as a whole, that is, an object group, and the object group is continuously monitored.

[0069] Specifically, after obtaining the object group, an object group can be determined as a monitoring object, and then the object group can be tracked, that is, the carrier identification of the object group can be obtained, and the object group can be tracked according to the carrier identification; an object group includes a carrier, so the carrier identification can be regarded as the identification of the object group, thereby the object group can be tracked according to the carrier identification, which improves the tracking accuracy of the object group and reduces the tracking complexity of the object group. That is, a monitoring object of the present invention may include a carrier, the goods transported by the carrier, and the dividing line of the goods. Among them, the dividing line of the goods is used to determine the number of goods transported by the carrier; illustratively, in the case where the goods are counted in pallets, if the monitoring object includes 1 dividing line, the number of goods transported by the carrier is 2 pallets; if the monitoring object includes 2 dividing lines, the number of goods transported by the carrier is 3 pallets.

[0070] When tracking the object group according to the carrier identification, the position information, image frame and cargo volume data of the object group at each track point can be obtained, and the aforementioned data can be combined into the track data of the object group, that is, the track data of the monitored object.

[0071] In a possible implementation, the cargo quantity data of the object group at each track point can be determined based on the cargo transported by the carrier and the dividing line of the cargo, that is, any track point can be selected as the current track point, and the cargo transported by the carrier and the dividing line of the cargo corresponding to the current track point can be obtained. Then, when it is determined that the cargo transported by the carrier exists, the cargo quantity data of the object group at the current track point can be determined based on the dividing line of the cargo. For example, when the cargo is counted in units of pallets, it can be considered that 1 dividing line corresponds to two pallets of cargo, and 2 dividing lines correspond to three pallets of cargo, that is, when the number of dividing lines of the cargo is 1, the cargo quantity data of the current track point is 2 pallets; when the number of dividing lines of the cargo is 2, the cargo quantity data of the current track point is 3 pallets. Afterwards, other track points can be selected as the current track point, and the above process can be repeated to obtain the cargo quantity data of the object group at each track point.

[0072] In an embodiment of the present invention, the cargo quantity data of the object group at each trajectory point is determined according to the cargo transported by the carrier and the dividing line of the cargo, thereby realizing the function of determining the cargo quantity data; determining the cargo quantity data by the dividing line of the cargo can improve the calculation efficiency, reduce the implementation complexity, and thereby improve the determination efficiency and accuracy of the cargo quantity data, thereby improving the accuracy of the trajectory data.

[0073] S204, determining a critical trajectory point from each trajectory point according to the cargo volume data of the carrier at each trajectory point.

[0074] The critical trajectory point is a trajectory point where the cargo quantity data changes among the trajectory points, and the critical trajectory point is located within the cargo area of ​​the target cargo location. For example, if there are 10 consecutive trajectory points all in the cargo area of ​​the target cargo location, and the cargo quantity data of the first 5 trajectory points are all 3, and the cargo quantity data of the sixth trajectory point is 0, then the sixth trajectory point can be determined as the critical trajectory point.

[0075] S205 , selecting a first number of trajectory points from the critical trajectory point forward to obtain a first trajectory point, and selecting a second number of trajectory points from the critical trajectory point backward to obtain a second trajectory point.

[0076] Among them, the first number is a preset value used to represent the number of trajectory points that need to be selected from the critical trajectory point forward; the second number is a preset value used to represent the number of trajectory points that need to be selected from the critical trajectory point backward; the first number and the second number can be the same or different.

[0077] The first trajectory point is a trajectory point located before the critical trajectory point in terms of time or space among all trajectory points of the monitored object; the second trajectory point is a trajectory point located after the critical trajectory point in terms of time or space among all trajectory points of the monitored object.

[0078] Specifically, after obtaining the critical trajectory point, you can start from the critical trajectory point, trace back along the movement direction of the monitored object (i.e., the object group), check the previous trajectory points one by one, and record these points as the first trajectory points until the set first number is reached. If the movement trajectory of the monitored object ends before this, only record to the end point; then, start from the critical trajectory point, move forward along the movement direction of the monitored object (i.e., the object group), check the subsequent trajectory points one by one, and record these points as the second trajectory points until the set second number is reached. If the movement trajectory of the monitored object ends before this, only record to the end point.

[0079] Exemplarily, when a trajectory point is recorded every 10 seconds, if the time corresponding to the critical trajectory point is 8:00:00, the first number is 6, and the second number is 6, then the first trajectory point includes the trajectory points corresponding to the times 7:59:50, 7:59:40, 7:59:30, 7:59:20, 7:59:10 and 7:59:00, and the second trajectory point includes the trajectory points corresponding to the times 8:00:10, 8:00:20, 8:00:30, 8:00:40, 8:00:50 and 8:01:00.

[0080] S206: Determine the first trajectory point and the second trajectory point as target trajectory points.

[0081] S207: Identify the operation type according to the trajectory data of the monitored object.

[0082] Among them, the operation type can include inbound operation and outbound operation.

[0083] Specifically, after obtaining the target trajectory point, the operation type can be identified according to the trajectory data of the monitored object, that is, if the monitored object drives into the cargo area of ​​the target cargo location from outside the cargo area of ​​the target cargo location, and the cargo quantity data corresponding to the trajectory point of the monitored object when entering the cargo area is greater than the cargo quantity data corresponding to the trajectory point when leaving the cargo area, then it indicates that the operation type is a warehousing operation. At this time, the cargo quantity data of the target cargo location has increased, while the cargo quantity data transported by the monitored object has decreased; if the monitored object drives into the cargo area of ​​the target cargo location from outside the cargo area of ​​the target cargo location, and the trajectory of the monitored object when entering the cargo area If the cargo quantity data corresponding to the trajectory point when the monitored object enters the cargo area of ​​the target cargo location is less than the cargo quantity data corresponding to the trajectory point when the cargo location is driven out of the cargo area, it indicates that the operation type is outbound operation. At this time, the cargo quantity data of the target cargo location is reduced, while the cargo quantity data transported by the monitored object is increased; if the monitored object enters the cargo area of ​​the target cargo location from outside the cargo area of ​​the target cargo location, and the cargo quantity data corresponding to the trajectory point when the monitored object enters the cargo area is equal to the cargo quantity data corresponding to the trajectory point when the cargo location is driven out of the cargo area, it indicates that the monitored object has not performed any operation on the target cargo location. At this time, the cargo quantity data of the target cargo location has not changed, and the cargo quantity data transported by the monitored object has not changed.

[0084] S208. Determine the change in the cargo volume of the monitored object according to the cargo volume data of the target trajectory point.

[0085] Specifically, after obtaining the operation type, the change in the cargo volume of the monitored object can be determined according to the cargo volume data of the target trajectory point, that is, the difference between the cargo volume data of the first trajectory point and the cargo volume data of the second trajectory point in the target trajectory point can be calculated, and the difference can be determined as the change in the cargo volume of the monitored object. The cargo volume data of the first trajectory point can be represented by the mode of the cargo volume data of the first trajectory point, and the cargo volume data of the second trajectory point can be represented by the mode of the cargo volume data of the second trajectory point.

[0086] Exemplarily, when the cargo quantity data of the first trajectory point 1 in the target trajectory point is 3, the cargo quantity data of the first trajectory point 2 is 3, the cargo quantity data of the first trajectory point 3 is 3, the cargo quantity data of the first trajectory point 4 is 3, the cargo quantity data of the first trajectory point 5 is 3, the cargo quantity data of the first trajectory point 6 is 3, the cargo quantity data of the second trajectory point 1 is 2, the cargo quantity data of the second trajectory point 2 is 2, the cargo quantity data of the second trajectory point 3 is 2, the cargo quantity data of the second trajectory point 4 is 2, the cargo quantity data of the second trajectory point 5 is 2, and the cargo quantity data of the second trajectory point 6 is 2, then the cargo quantity change of the monitored object is 1.

[0087] S209: Determine the change in the quantity of cargo at the target cargo location according to the operation type and the change in the quantity of cargo of the monitored object.

[0088] Specifically, after obtaining the change in the quantity of goods of the monitored object, the change in the quantity of goods of the target cargo location can be determined according to the operation type and the change in the quantity of goods of the monitored object. In a possible implementation, when the operation type is an incoming operation, the increase in the quantity of goods of the target cargo location is determined according to the decrease in the quantity of goods of the monitored object; when the operation type is an outgoing operation, the decrease in the quantity of goods of the target cargo location is determined according to the increase in the quantity of goods of the monitored object. That is, when the operation type is an incoming operation, the change in the quantity of goods of the monitored object is the decrease in quantity of goods, and the quantity data of the target cargo location will increase, that is, the carrier puts the goods on the target cargo location, at this time, the decrease in the quantity of goods of the monitored object can be determined as the increase in the quantity of goods of the target cargo location; when the operation type is an outgoing operation, the change in the quantity of goods of the monitored object is the increase in quantity of goods, and the quantity data of the target cargo location will decrease, that is, the carrier takes away the goods in the target cargo location, at this time, the increase in the quantity of goods of the monitored object can be determined as the decrease in the quantity of goods of the target cargo location. The change in the quantity of cargo at the target cargo location can be displayed in real time in the monitoring video of the target cargo location, so that the management personnel can understand the change in the quantity of cargo at the target cargo location in time.

[0089] For example, Figure 6 FIG. 1 is an example diagram of a warehouse outbound operation provided by an embodiment of the present invention. Figure 6 The green box in the figure is the target cargo area. Since the initial cargo quantity data of the target cargo is not maintained, the default initial cargo quantity data of the target cargo is 0. Figure 6 The red number "-2" in the figure indicates that the change in the cargo volume of the target cargo location is the decrease in cargo volume, that is, the decrease in cargo volume is 2. In other words, the monitored object drives into the cargo volume area of ​​the target cargo location from outside the cargo volume area of ​​the target cargo location, takes away 2 trailers of cargo in the target cargo location, and then drives out of the cargo volume area of ​​the target cargo location. At this time, the change in the cargo volume of the monitored object is the increase in cargo volume, that is, the increase in cargo volume is 2.

[0090] For example, Figure 7 The figure shows an example of a target cargo location not being operated provided by an embodiment of the present invention. Figure 7 The green box in the figure is the target cargo area. Since the initial cargo quantity data of the target cargo is not maintained, the default initial cargo quantity data of the target cargo is 0. Figure 7 The red number "0" in the figure indicates that the change in the cargo volume of the target cargo location is 0, that is, the monitored object drives into the cargo area of ​​the target cargo location from outside the cargo area of ​​the target cargo location, and then directly drives out of the cargo area of ​​the target cargo location. It does not place the goods on the target cargo location, nor does it take away the goods on the target cargo location. At this time, the change in the cargo volume of the monitored object is 0.

[0091] The monitored object is continuously monitored until the monitored object no longer generates new track points, and the monitoring of the monitored object can be stopped. For example, if the monitored object does not generate new track points for more than a preset time, or the monitored object is not detected in a preset number of consecutive image frames, the monitoring of the monitored object is stopped. The preset time is, for example, 5 minutes, 10 minutes, etc., and the preset number is, for example, 10 frames, 20 frames, etc.

[0092] In the embodiment of the present invention, when the operation type is an incoming operation, the increase in the quantity of cargo at the target cargo location is determined according to the decrease in the quantity of cargo of the monitored object; when the operation type is an outgoing operation, the decrease in the quantity of cargo at the target cargo location is determined according to the increase in the quantity of cargo of the monitored object, thereby realizing the function of determining the change in the quantity of cargo at the target cargo location. By determining the change in the quantity of cargo at the target cargo location through the change in the quantity of cargo of the monitored object and the operation type, the efficiency and accuracy of determining the change in the quantity of cargo are improved, data support is provided for determining the quantity of cargo at the target cargo location, and the quantity of cargo at the target cargo location can be accurately determined.

[0093] Optionally, after the change in the cargo volume of the target cargo location is determined, the current cargo volume of the target cargo location may be determined based on the cargo volume data historically stored in the target cargo location and the change in the cargo volume of the target cargo location.

[0094] S210: Determine whether the change in the quantity of goods at the target cargo location matches the preset operation task. If not, execute S211; if so, execute S201.

[0095] The preset operation task is a pre-set operation task, such as a warehouse-out task and a warehouse-in task. The preset operation task may include a carrier identification, a cargo location, an operation type corresponding to the cargo location, and an estimated cargo volume change corresponding to the cargo location.

[0096] Specifically, after obtaining the change in cargo volume of the target cargo location, the preset operation task can be queried based on the target cargo location to obtain the expected change in cargo volume corresponding to the target cargo location, and then the actual change in cargo volume of the target cargo location (i.e., the change in cargo volume obtained in S209) is compared with the expected change in cargo volume to determine whether the change in cargo volume of the target cargo location matches the preset operation task. If not, S211 can be executed; otherwise, S201 can be executed to continue to determine the change in cargo volume of other cargo locations.

[0097] S211. If there is no match, an abnormal operation prompt is given.

[0098] Among them, the operation abnormality prompt is a prompt information about the operation abnormality generated by the electronic device to prompt the staff to deal with the abnormality in time.

[0099] Specifically, if the change in the amount of cargo at the target cargo location does not match the preset operation task, it indicates that the operation of the monitored object is abnormal, and an operation abnormality prompt can be generated at this time. If the electronic device is a computer with a display screen, an operation abnormality prompt can be displayed to prompt the staff to deal with the abnormality in time; if the electronic device is a server without a display screen, an operation abnormality prompt can be sent to the front end, and the front end can be a terminal used by staff and managers. For example, the preset operation task indicates that 100 pallets need to be shipped from cargo location 1 using carrier X, and the monitoring result shows that 120 pallets are shipped from cargo location 1 using carrier X, then an operation abnormality prompt can be issued to prompt the staff that there is an abnormality in the shipment.

[0100] The technical solution provided in this embodiment can detect the carrier, the goods transported by the carrier and the dividing line of the goods in the monitoring video to obtain the detection result, and associate the carrier, the goods and the dividing line according to the detection result to obtain the object group, and then track the object group to obtain the trajectory data of the monitored object, thereby realizing the function of determining the trajectory data. By accurately detecting and associating the carrier, the goods and the dividing line, it is possible to ensure that the position and identity of the monitored object are accurately identified, thereby improving the accuracy of determining the trajectory data of the monitored object, and providing a data basis for the subsequent determination of the change in the amount of cargo; and after associating the carrier, the goods and the dividing line as an object group, the entire object group can be tracked at one time without the need to track each individual entity separately, thereby simplifying the tracking process, improving the monitoring efficiency, and thereby improving the efficiency of determining the trajectory data of the monitored object; thereafter, According to the cargo volume data of the carrier at each trajectory point, a critical trajectory point is determined from each trajectory point, and then a first number of trajectory points are selected forward from the critical trajectory point to obtain a first trajectory point, and a second number of trajectory points are selected backward from the critical trajectory point to obtain a second trajectory point, and then the first trajectory point and the second trajectory point are determined as target trajectory points, thereby realizing the function of determining the target trajectory point. Through the selection of the first trajectory point and the second trajectory point, more comprehensive information about the change in cargo volume can be provided, and then the trend and amplitude of the change in cargo volume can be more accurately captured, thereby improving the accuracy of determining the change in cargo volume; and by selecting the trajectory points before and after the critical trajectory point as the target trajectory point, the process of gradual change in cargo volume can be better simulated, so that the determination of the change in cargo volume is more in line with the actual situation, thereby improving the stability and reliability of determining the change in cargo volume.

[0101] Afterwards, the operation type can be identified according to the trajectory data of the monitored object, and the operation performed by the monitored object at the target cargo location can be understood more accurately. Then, the change in the cargo volume of the monitored object is determined according to the cargo volume data of the target trajectory point, and then the change in the cargo volume of the target cargo location is determined according to the operation type and the change in the cargo volume of the monitored object, realizing the determination function of the change in the cargo volume of the target cargo location. By combining the operation type and the change in the cargo volume of the monitored object, the change in the cargo volume of the target cargo location can be accurately calculated according to different types of operations, thereby improving the accuracy of determining the change in the cargo volume, and by distinguishing the operation type, the change in the cargo on the cargo location can be accurately grasped, thereby managing the warehouse more finely; afterward, it can be determined whether the change in the cargo volume of the target cargo location matches the preset operation task, and when the change in the cargo volume of the target cargo location does not match the preset operation task, an abnormal operation prompt is given, realizing the abnormal prompt function, and abnormal situations can be prompted in time, so that the staff can take corresponding measures to deal with them in time, and by automatically monitoring and prompting abnormalities, the workload of manual inspections can be greatly reduced, avoiding possible omissions or errors in human operations, thereby improving the efficiency and accuracy of warehouse management, thereby improving the user experience.

[0102] Figure 8 is a structural diagram of a device for determining cargo quantity provided by an embodiment of the present invention, such as Figure 8 As shown, the device may specifically include:

[0103] The data detection module 810 is used to detect the monitoring video of the target cargo location in the warehouse to obtain the trajectory data of the monitored object, the monitored object includes the carrier for transporting the goods, and the trajectory data includes the cargo quantity data of the carrier at each trajectory point;

[0104] A track point determination module 820 is used to determine a critical track point from each track point according to the cargo volume data of the carrier at each track point. The critical track point is a track point where the cargo volume data of each track point changes. The critical track point is located in the cargo area of ​​the target cargo location.

[0105] A trajectory point selection module 830, for selecting a target trajectory point from each trajectory point according to a critical trajectory point;

[0106] The cargo quantity determination module 840 is used to determine the cargo quantity change of the target cargo location according to the cargo quantity data of the target trajectory point.

[0107] In one embodiment, the data detection module 810 is specifically used for:

[0108] Detect the carrier, the goods transported by the carrier and the dividing line of the goods in the monitoring video to obtain the detection result;

[0109] According to the detection results, the carrier, the goods and the dividing line are associated to obtain an object group;

[0110] Track the object group and obtain the trajectory data of the monitored objects;

[0111] Among them, the cargo volume data of each trajectory point is determined according to the cargo transported by the carrier and the dividing line of the cargo.

[0112] In one embodiment, the data detection module 810 detects the carrier, the goods transported by the carrier, and the dividing line of the goods in the surveillance video, including:

[0113] The target detection model is used to detect the carrier, the cargo transported by the carrier, and the dividing line between the cargo in the surveillance video;

[0114] Among them, the target detection model includes a feature extraction module and a detection module. The feature extraction module is used to extract features from the surveillance video, and the detection module is used to perform target detection based on the features extracted by the feature extraction module. The feature extraction module includes at least two convolution blocks, and the number of channels of at least two convolution blocks is greater than the preset number of channels, and the number of channels of at least two convolution blocks increases successively.

[0115] In one embodiment, the data detection module 810 associates the carrier, the goods, and the dividing line according to the detection result to obtain an object group, including:

[0116] Calculate the matching degree between the carrier and the goods according to the test results, and calculate the matching degree between the goods and the dividing line according to the test results;

[0117] According to the matching degree between the carrier and the goods and the matching degree between the goods and the dividing line, the carrier, the goods and the dividing line are associated to obtain an object group.

[0118] In one embodiment, the data detection module 810 tracks the object group, including:

[0119] Get the identifier of the carrier in the object group to obtain the carrier identifier;

[0120] Track groups of objects based on carrier identification.

[0121] In one embodiment, the cargo area of ​​the target cargo location in the track point determination module 820 is determined by the following method:

[0122] Get the cargo location outline set for the target cargo location;

[0123] The area corresponding to the cargo location outline is determined as the cargo location area of ​​the target cargo location.

[0124] In one embodiment, the trajectory point selection module 830 is specifically used for:

[0125] Selecting a first number of trajectory points from the critical trajectory point forward to obtain a first trajectory point, and selecting a second number of trajectory points from the critical trajectory point backward to obtain a second trajectory point;

[0126] The first trajectory point and the second trajectory point are determined as target trajectory points.

[0127] In one embodiment, the cargo quantity determination module 840 is specifically used to:

[0128] Identify the type of operation based on the trajectory data of the monitored object;

[0129] Determine the change in the volume of cargo of the monitored object according to the cargo volume data of the target trajectory point;

[0130] The change in cargo volume of the target cargo location is determined based on the operation type and the change in cargo volume of the monitored object.

[0131] In one embodiment, the cargo quantity determination module 840 determines the cargo quantity change of the target cargo location according to the operation type and the cargo quantity change of the monitored object, including:

[0132] When the operation type is a warehousing operation, the increase in the quantity of goods in the target cargo location is determined according to the decrease in the quantity of goods of the monitored object;

[0133] When the operation type is an outbound operation, the reduction in the quantity of goods at the target cargo location is determined based on the increase in the quantity of goods of the monitored object.

[0134] In one embodiment, the device further includes an abnormality prompt module, which is specifically used to:

[0135] Determine whether the change in the quantity of cargo at the target cargo location matches the preset operation task;

[0136] If they do not match, an abnormal operation prompt will be given.

[0137] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example for illustration. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the functional modules described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0138] The device of the embodiment of the present invention can detect the monitoring video of the target cargo position in the warehouse to obtain the trajectory data of the monitored object, the monitored object includes the carrier for transporting goods, and the trajectory data includes the cargo quantity data of the carrier at each trajectory point; according to the cargo quantity data of the carrier at each trajectory point, a critical trajectory point is determined from each trajectory point, the critical trajectory point is the trajectory point where the cargo quantity data in each trajectory point changes, and the critical trajectory point is located in the cargo position area of ​​the target cargo position; according to the critical trajectory point, a target trajectory point is selected from each trajectory point; according to the cargo quantity data of the target trajectory point, the cargo quantity change of the target cargo position is determined. The present invention is equivalent to tracking the moving trajectory of the carrier and the destination of the goods based on the monitoring video of the cargo position in the warehouse, so as to determine which cargo position the goods are specifically placed in the warehouse, which is convenient for real-time understanding of the cargo quantity of each cargo position, and helps warehouse managers make accurate inventory management decisions. This counting method can meet the actual application needs; in addition, the present invention can reuse existing video surveillance facilities, does not need to attach radio frequency tags to each cargo, does not need to invest a lot of extra costs to purchase and maintain tags, and reduces costs.

[0139] An embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for determining the quantity of goods provided in any of the above embodiments is implemented.

[0140] Reference below Fig. 9 , which shows a schematic diagram of the structure of a computer system 900 of an electronic device suitable for implementing an embodiment of the present invention. Fig. 9 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0141] like Fig. 9 As shown, the computer system 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the computer system 900 are also stored. The CPU 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0142] Typically, the following components are connected to the I / O interface 905: an input section 906 including a keyboard, a mouse, etc.; an output section 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, a modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage section 908 as needed.

[0143] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the central processing unit (CPU) 901, the above-mentioned functions defined in the system of the present invention are executed.

[0144] It should be noted that the computer-readable medium shown in the present invention can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0145] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0146] The modules and / or units involved in the embodiments of the present invention may be implemented in software or hardware. The modules and / or units described may also be set in a processor, for example, it may be described as: a processor includes a data detection module, a track point determination module, a track point selection module, and a cargo quantity determination module. The names of these modules do not, in some cases, constitute limitations on the modules themselves.

[0147] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes:

[0148] Detect the surveillance video of the target cargo location in the warehouse to obtain the trajectory data of the monitored object, which includes the carrier transporting the goods, and the trajectory data includes the cargo volume data of the carrier at each trajectory point;

[0149] Determine a critical trajectory point from each trajectory point according to the cargo volume data of the carrier at each trajectory point, the critical trajectory point is a trajectory point where the cargo volume data in each trajectory point changes, and the critical trajectory point is located in the cargo location area of ​​the target cargo location;

[0150] Selecting a target trajectory point from each trajectory point according to a critical trajectory point;

[0151] Determine the change in cargo volume at the target cargo location based on the cargo volume data at the target trajectory point.

[0152] The technical solution of the embodiment of the present invention can detect the monitoring video of the target cargo position in the warehouse to obtain the trajectory data of the monitored object, the monitored object includes the carrier for transporting goods, and the trajectory data includes the cargo quantity data of the carrier at each trajectory point; according to the cargo quantity data of the carrier at each trajectory point, a critical trajectory point is determined from each trajectory point, the critical trajectory point is the trajectory point where the cargo quantity data in each trajectory point changes, and the critical trajectory point is located in the cargo position area of ​​the target cargo position; according to the critical trajectory point, a target trajectory point is selected from each trajectory point; according to the cargo quantity data of the target trajectory point, the cargo quantity change of the target cargo position is determined. The present invention is equivalent to tracking the moving trajectory of the carrier and the destination of the goods based on the monitoring video of the cargo position in the warehouse, so as to determine which cargo position the goods are specifically placed in the warehouse, which is convenient for real-time understanding of the cargo quantity of each cargo position, and helps warehouse managers make accurate inventory management decisions. This counting method can meet the actual application needs; in addition, the present invention can reuse existing video surveillance facilities, does not need to attach radio frequency tags to each cargo, does not need to invest a lot of extra costs to purchase and maintain tags, and reduces costs.

[0153] An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the method for determining the quantity of goods provided in any embodiment of the present invention.

[0154] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0155] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0156] It should be noted that the collection, collection, updating, analysis, processing, use, transmission, storage and other aspects of user personal information involved in the technical solution of this disclosure are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security and national security.

[0157] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for determining cargo volume, characterized in that: include: Detecting the surveillance video of the target cargo location in the warehouse to obtain the trajectory data of the monitored object, wherein the monitored object includes a carrier for transporting goods, and the trajectory data includes the cargo quantity data of the carrier at each trajectory point; Determine a critical track point from each track point according to the cargo volume data of the carrier at each track point, wherein the critical track point is a track point where the cargo volume data of each track point changes, and the critical track point is located in a cargo location area of ​​the target cargo location; Selecting a target trajectory point from each of the trajectory points according to the critical trajectory point; The change in cargo volume of the target cargo location is determined according to the cargo volume data of the target trajectory point.

2. The method for determining cargo volume according to claim 1, characterized in that: The monitoring video of the target cargo location in the warehouse is detected to obtain the trajectory data of the monitored object, including: Detecting the carrier, the goods transported by the carrier, and the dividing line of the goods in the monitoring video to obtain a detection result; Associating the carrier, the cargo, and the dividing line according to the detection result to obtain an object group; Tracking the object group to obtain trajectory data of the monitored objects; The cargo volume data of each trajectory point is determined according to the cargo transported by the carrier and the dividing line of the cargo.

3. The method for determining cargo volume according to claim 2, characterized in that: The detecting of the carrier, the cargo transported by the carrier and the dividing line of the cargo in the monitoring video includes: Using a target detection model to detect the carrier, the goods transported by the carrier, and the dividing line of the goods in the surveillance video; Among them, the target detection model includes a feature extraction module and a detection module, the feature extraction module is used to extract features from the surveillance video, and the detection module is used to perform target detection based on the features extracted by the feature extraction module, the feature extraction module includes at least two convolution blocks, the number of channels of the at least two convolution blocks are greater than the preset number of channels, and the number of channels of the at least two convolution blocks increases successively.

4. The method for determining cargo volume according to claim 2, characterized in that: The method of associating the carrier, the cargo and the dividing line according to the detection result to obtain an object group includes: Calculating the matching degree between the carrier and the goods according to the detection results, and calculating the matching degree between the goods and the dividing line according to the detection results; According to the matching degree between the carrier and the goods and the matching degree between the goods and the dividing line, the carrier, the goods and the dividing line are associated to obtain the object group.

5. The method for determining cargo volume according to claim 2, characterized in that: The tracking of the object group includes: Obtaining the identification of the carrier in the object group to obtain the carrier identification; The group of objects is tracked based on the carrier identification.

6. The method for determining cargo volume according to claim 1, characterized in that: The cargo area of ​​the target cargo location is determined by the following method: Obtaining a cargo location profile set for the target cargo location; The area corresponding to the cargo location outline is determined as the cargo location area of ​​the target cargo location.

7. The method for determining cargo volume according to claim 1, characterized in that: The selecting a target trajectory point from each trajectory point according to the critical trajectory point comprises: Selecting a first number of trajectory points from the critical trajectory point forward to obtain a first trajectory point, and selecting a second number of trajectory points from the critical trajectory point backward to obtain a second trajectory point; The first trajectory point and the second trajectory point are determined as the target trajectory points.

8. The method for determining cargo volume according to claim 1, characterized in that: The determining the change in the cargo volume of the target cargo location according to the cargo volume data of the target trajectory point includes: identifying the type of operation according to the trajectory data of the monitored object; Determine the change in the volume of cargo of the monitored object according to the cargo volume data of the target trajectory point; The change in the amount of cargo at the target cargo location is determined according to the operation type and the change in the amount of cargo of the monitored object.

9. The method for determining cargo volume according to claim 8, characterized in that: The determining the change in the quantity of cargo of the target cargo location according to the operation type and the change in the quantity of cargo of the monitored object comprises: When the operation type is a warehousing operation, determining the increase in the quantity of cargo at the target cargo location according to the decrease in the quantity of cargo of the monitored object; When the operation type is a warehouse-out operation, the decrease in the quantity of goods in the target cargo location is determined according to the increase in the quantity of goods of the monitored object.

10. The method for determining cargo quantity according to any one of claims 1 to 9, characterized in that: The method further comprises: Determine whether the change in the cargo volume of the target cargo location matches the preset operation task; If they do not match, an abnormal operation prompt will be given.

11. A cargo quantity determination device, characterized in that: include: A data detection module is used to detect the surveillance video of the target cargo location in the warehouse to obtain the trajectory data of the monitored object, wherein the monitored object includes a carrier for transporting goods, and the trajectory data includes the cargo quantity data of the carrier at each trajectory point; A track point determination module, used to determine a critical track point from each track point according to the cargo volume data of the carrier at each track point, wherein the critical track point is a track point where the cargo volume data of each track point changes, and the critical track point is located in a cargo location area of ​​the target cargo location; A trajectory point selection module, used for selecting a target trajectory point from each trajectory point according to the critical trajectory point; The cargo quantity determination module is used to determine the cargo quantity change of the target cargo location according to the cargo quantity data of the target trajectory point.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the cargo quantity determination method according to any one of claims 1 to 10 is implemented.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the cargo quantity determination method according to any one of claims 1 to 10 is implemented.

Images