A warehouse storage location anomaly screening method and system

Abstract

The application provides a warehouse storage abnormality screening method and system, comprising: at a preset reference sensing point, a first sensing device is used to acquire reference data of goods to be stored in a warehouse, and the reference data is stored in association with a target storage location of the goods to be stored in the warehouse; a mobile device in a lane carrying a second sensing device moves to a checking position above a target storage location to be checked in response to a checking instruction; at the checking position, real-time data of the target storage location is collected by the second sensing device; the real-time data and the reference data associated with the target storage location are compared to determine whether the goods state of the target storage location is abnormal; wherein the first sensing device and the second sensing device are visual collection devices of the same model and have the same physical baseline distance from the target storage location. Through the application, the problem of visual blind area is effectively solved, the physical obstruction between multi-layer shelves is broken through, and automatic checking of any storage location in the warehouse system without handling is realized, thereby significantly reducing the operating cost.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of warehouse automation, in particular to a warehouse storage location abnormality screening method and system. BACKGROUND

[0002] The four-way vehicle dense storage system significantly improves the warehouse space utilization rate and operation efficiency due to its excellent automation level and high-density storage capacity. However, this highly optimized design also brings unique inventory management challenges: in order to maximize space, the rack structure usually has a vertical depth of tens of meters or even higher, and the aisle width is only slightly larger than the body of the four-way vehicle. This extreme space compression design fundamentally blocks the possibility of manual access to the aisle for close-range inventory, not only posing a high safety risk, but also making the traditional manual visual method physically completely infeasible.

[0003] Currently, the inventory method for such dense storage systems usually relies on moving the goods on the target storage location to a designated workstation outside the warehouse by a mobile device (such as a four-way vehicle) in the aisle, and then manually checking and confirming before moving them back to the original storage location. This process not only includes multiple steps such as warehouse-out, manual checking, and warehouse-in, but also requires moving the goods on external storage locations one by one when checking internal storage locations, causing a chain of movements and resulting in extremely long single-storage-location inventory time, low inventory efficiency, and a large amount of invalid movement, causing serious waste of mobile devices and energy. Some automated inventory solutions propose using mobile components equipped with sensing devices to take pictures of the racks and compare the acquired real-time data with preset goods information to determine the goods status. However, in the four-way vehicle dense storage environment of multi-layer racks, the line of sight of the mobile device will be physically blocked by the upper racks or goods when it travels along the track in the aisle, making it impossible to effectively observe the goods status of the middle and lower storage locations, resulting in a serious inventory blind area. In addition, relying only on preset general goods information as a comparison benchmark also makes it difficult to cope with the situation where actual warehouse-in goods have different shapes, affecting inventory accuracy.

[0004] Therefore, how to overcome physical obstruction and achieve rapid and accurate inventory of internal storage locations in a dense storage system without large-scale material movement is a technical problem that needs to be solved in the field. SUMMARY

[0005] To this end, the present application provides a warehouse storage location abnormality screening method and system, aiming to solve the technical problems of tedious process, low efficiency, high operating cost, and difficulty in effectively detecting obstructed internal storage locations when inventorying a dense storage system in the prior art.

[0006] To achieve the above purpose, the present application adopts the following technical solutions:

[0007] According to a first aspect of the present application, the present application provides a warehouse storage location anomaly screening method, the method comprising:

[0008] At a preset reference collection point, reference data of a to-be-warehoused cargo is acquired by a first sensing device, and the reference data is stored in association with a target storage location of the to-be-warehoused cargo;

[0009] A laneway mobile device carrying a second sensing device moves to an inventorying position above a target storage location to be inventoried in response to an inventorying instruction;

[0010] At the inventorying position, real-time data of the target storage location is collected by the second sensing device;

[0011] The real-time data is compared with reference data associated with the target storage location to determine whether the cargo state of the target storage location is abnormal;

[0012] The first sensing device and the second sensing device are visual collection devices of the same model and have the same physical baseline distance from the target storage location.

[0013] Further, the first sensing device is fixedly installed at a first position of a warehousing end conveyor that carries the to-be-warehoused cargo by a pallet, and the first position is a target height directly above a center point of the pallet;

[0014] The second sensing device is fixedly installed at a second position of the laneway mobile device, and the second position is below the laneway mobile device; when the second sensing device is directly above the center point of the pallet of the target storage location, the distance from the center point of the pallet is the target height.

[0015] Further, the acquisition of the reference data of the to-be-warehoused cargo at the preset reference collection point by the first sensing device comprises:

[0016] When the warehousing end conveyor carries the to-be-warehoused cargo to the warehousing end, the warehousing end conveyor is controlled to be positioned at the preset reference collection point, triggering the first sensing device to vertically collect reference data on the top surface of the to-be-warehoused cargo;

[0017] And / or,

[0018] The laneway mobile device carrying the second sensing device moves to the inventorying position above the target storage location to be inventoried in response to the inventorying instruction, comprising:

[0019] The laneway mobile device positions the second sensing device directly above the center point of the pallet of the target storage location at a position adjacent to the target storage location above the target storage location in response to the inventorying instruction.

[0020] Further, the first sensing device and the second sensing device are a dual-focus camera set or a laser radar;

[0021] When the first sensing device and the second sensing device are both dual-focus camera sets, the reference data and the real-time data are both 2D image data collected vertically to the top surface of the to-be-warehoused cargo;

[0022] and / or,

[0023] When the first sensing device and the second sensing device are both laser radars, the reference data and the real-time data are both point cloud data collected vertically to the top surface of the target storage location.

[0024] Further, when the reference data and the real-time data are both 2D image data collected vertically to the top surface of the to-be-warehoused cargo, the comparison of the real-time data with the reference data associated with the target storage location to determine whether the cargo state of the target storage location is abnormal includes:

[0025] predefining a reference point on the top of the cargo box stack of the target storage location;

[0026] locating the pixel coordinates of the reference point in the two sets of 2D image pairs obtained by the dual-focus camera set in the reference data and the real-time data respectively, to determine a first pixel distance of the reference point in the reference data and a second pixel distance of the reference point in the real-time data;

[0027] calculating the distance difference between the first pixel distance and the second pixel distance, and determining that the cargo state of the target storage location is abnormal when the distance difference exceeds a preset distance threshold.

[0028] Further, when the reference data and the real-time data are both point cloud data collected vertically to the top surface of the target storage location, the comparison of the real-time data with the reference data associated with the target storage location to determine whether the cargo state of the target storage location is abnormal includes:

[0029] respectively performing data preprocessing on the reference data and the real-time data to obtain reference point cloud data and real-time point cloud data; wherein the data preprocessing includes ROI extraction, filtering and / or down-sampling;

[0030] registering the reference point cloud data and the real-time point cloud data, and calculating the coincidence degree between the registered reference point cloud data and real-time point cloud data;

[0031] comparing the coincidence degree with a preset coincidence threshold, and determining that the cargo state of the target storage location is abnormal when the coincidence degree is lower than the preset coincidence threshold;

[0032] when the coincidence degree reaches the preset coincidence threshold, retaining a translation transformation of the real-time point cloud data in the tray plane;

[0033] dividing the translation-transformed real-time point cloud data and the reference point cloud data into a plurality of grid blocks, calculating and comparing the average height of each grid block to obtain a height difference ratio;

[0034] comparing the height difference ratio with a preset difference threshold, and when the height difference ratio exceeds the preset difference threshold, determining that the state of the goods in the target storage location is abnormal.

[0035] Further, the method further comprises:

[0036] based on the to-be-counted task, dynamically scheduling one or more of the intrabay mobile devices, so that the intrabay mobile devices preferentially process the to-be-counted task in a parallel carrying-counting mode without conflict with daily carrying tasks.

[0037] Further, the method further comprises:

[0038] when a preset condition is triggered, updating the reference data in real time, specifically including:

[0039] when the goods are re-stored after being taken out or the goods are processed back to the warehouse after being abnormally handled, updating the reference data using the collection data at the time of storage; and / or,

[0040] when the state of the goods in the target storage location is normal after being counted, updating the reference data using the real-time data of the target storage location.

[0041] Further, the method further comprises:

[0042] when the state of the goods in the target storage location is abnormal, marking the target storage location as a to-be-reviewed storage location;

[0043] The intrabay mobile device transports the to-be-reviewed goods in the to-be-reviewed storage location to a review end in response to a review instruction, so as to review the to-be-reviewed goods.

[0044] According to the second aspect of the present application, the present application provides a warehouse storage location abnormality screening system, which comprises: a first sensing device, a second sensing device, an intrabay mobile device and a control module; the control module is in communication connection with the first sensing device, the second sensing device and the intrabay mobile device respectively;

[0045] The control module comprises a reference data acquisition unit, a real-time data acquisition unit and a goods state judgment unit;

[0046] The reference data collection unit is configured to control the first sensing device to collect reference data of the goods to be stored in the warehouse at a preset reference collection point, and store the reference data in association with a target storage location of the goods to be stored in the warehouse.

[0047] The real-time data collection unit is configured to control the in-lane mobile device carrying the second sensing device to move to a checking position above the target storage location to be checked in response to a checking instruction, and control the second sensing device to collect real-time data of the target storage location at the checking position.

[0048] The goods state judgment unit is configured to compare the real-time data with the reference data associated with the target storage location to determine whether the goods state of the target storage location is abnormal.

[0049] The first sensing device and the second sensing device are visual collection devices of the same type and have the same physical baseline distance from the target storage location.

[0050] The technical scheme of the present application has at least the following beneficial effects:

[0051] The present application provides a warehouse storage location abnormality screening method, which comprises the following steps: collecting reference data of goods to be stored in the warehouse at a preset reference collection point by a first sensing device, and storing the reference data in association with a target storage location of the goods to be stored in the warehouse; moving an in-lane mobile device carrying a second sensing device to a checking position above a target storage location to be checked in response to a checking instruction; collecting real-time data of the target storage location by the second sensing device at the checking position; comparing the real-time data with the reference data associated with the target storage location to determine whether the goods state of the target storage location is abnormal; and the first sensing device and the second sensing device are visual collection devices of the same type and have the same physical baseline distance from the target storage location. The present application effectively solves the problem of visual blind area, breaks through the physical obstruction between multi-layer shelves, realizes automatic checking of any storage location in the warehouse system without manual handling, and significantly reduces operating costs.

[0052] It should be understood that the general description above and the detailed description below are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiment or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0054] Figure 1 Fig. 1 shows a flowchart of a warehouse storage location anomaly screening method according to an embodiment of the present application;

[0055] Figure 2 Fig. 2 shows a schematic diagram of real-time data acquisition according to an embodiment of the present application;

[0056] Figure 3 Fig. 3 shows a flowchart of cargo state anomaly judgment according to an embodiment of the present application;

[0057] Figure 4 Fig. 4 shows a flowchart of cargo state anomaly judgment according to another embodiment of the present application;

[0058] Figure 5 Fig. 5 shows a structural diagram of a warehouse storage location anomaly screening system according to an embodiment of the present application.

[0059] In the drawings, reference numerals: 210 - mobile device in the aisle; 220 - second sensing device; 230 - target storage location; 240 - pallet; 250 - cargo box stack. DETAILED DESCRIPTION

[0060] Exemplary embodiments of the present disclosure will be described more fully hereinafter with reference to the accompanying drawings; however, they are not limited to the embodiments set forth herein but can be implemented in various forms. The embodiments are described in such a manner that a more complete and thorough understanding of the present disclosure can be achieved and, to enable the range of the present disclosure to be conveyed completely to those skilled in the art.

[0061] It should be noted that, in this document, relational terms such as first and second, and the like can be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element preceded by "comprises... a" does not, without more constraints, foreclose the existence of additional identical elements in the process, method, article, or apparatus that comprises the listed element.

[0062] An embodiment of the present application provides a warehouse storage location anomaly screening method, as shown in Fig. 1, which can at least include the following steps S101-S104: Figure 1

[0063] ​Step S101, at a preset reference collection point, the first sensing device is used to collect the reference data of the goods to be stored in the warehouse, and the reference data is stored in association with the target storage location of the goods to be stored in the warehouse.

[0064] The first sensing device in the embodiment of the application can be fixedly installed at a first position of a warehouse-inlet end conveyor that carries the goods to be stored in the warehouse by a pallet. The first position is a target height directly above the center point of the pallet.

[0065] For example, a horizontal aluminum profile gantry is arranged on the warehouse-inlet end conveyor, and the first sensing device of the same type as the mobile device in the aisle (such as a four-way vehicle in the warehouse) is installed on the gantry. It can be understood that, in order to ensure the spatial positional relationship between the first sensing device and the goods, especially the height parameter, and to ensure that the spatial positional relationship between the mobile device in the aisle and the goods during the inventory operation is consistent, thereby ensuring that the reference data collected during storage is directly comparable with the real-time data collected during subsequent inventory operations, and providing a basis for subsequent image or point cloud comparison, the height parameter of the first sensing device needs to be strictly matched with the inventory operation state of the mobile device in the aisle, and the position thereof should be located directly above the center point of the pallet carrying the goods to be stored in the warehouse as much as possible.

[0066] The preset reference collection point can be arranged on the path that the goods to be stored in the warehouse must pass through, for example, on the warehouse-inlet end. When the warehouse-inlet end conveyor carries the goods to be stored in the warehouse to the warehouse-inlet end, the warehouse-inlet end conveyor is controlled to be positioned at the preset reference collection point, and the first sensing device is triggered to collect the reference data vertically to the top surface of the goods to be stored in the warehouse. In actual operation, after the photoelectric sensor is triggered to stop when the goods to be stored in the warehouse arrive at the warehouse-inlet end, the warehouse control system WCS can control the warehouse-inlet end conveyor to be accurately positioned at the preset reference collection point, and then trigger the first sensing device to vertically take a panoramic picture of the top surface of the goods, thereby obtaining the reference data (Data_base).

[0067] The warehouse control system WCS sets the target storage location information corresponding to the goods to be stored in the warehouse, and after the reference data is collected, the reference data is stored in association with the target storage location information (for example, the target storage location number “X, Y, Z”) and the storage time stamp (T_base).

[0068] Step S102, the mobile device in the aisle carrying the second sensing device moves to an inventory position above the target storage location to be inventoried in response to an inventory instruction.

[0069] In this embodiment of the invention, the second sensing device is a visual acquisition device of the same model as the first sensing device and with the same physical baseline distance to the target storage location. Specifically, the second sensing device can be fixedly installed under a mobile device (such as a four-way vehicle) in the aisle; when the second sensing device is located directly above the center point of the pallet at the target storage location (i.e., the inventory counting position), the distance from the center point of the pallet is the target height. In actual operation, the mobile device in the aisle can respond to the inventory counting command from the warehouse management system (WMS) and position the second sensing device directly above the center point of the pallet at the target storage location on an adjacent layer above the target storage location.

[0070] Step S103: At the inventory location, real-time data of the target storage location is collected through the second sensor device.

[0071] like Figure 2 The diagram illustrates the principle of real-time data acquisition. In response to an inventory call, the mobile device 210 moves along an adjacent layer above the target location 230. After precisely positioning itself directly above the center point of the pallet 240 where the target location 230 is located, it waits for the warehouse control system (WCS) to send a shooting command. The second sensor 220 responds to the shooting command, vertically scanning the top of the stack of goods 250 to obtain real-time data (Data_current). Simultaneously, it records the target location information (e.g., target storage location number "X, Y, Z") and a real-time timestamp (T_current) for associated storage.

[0072] It should be noted that, for screening abnormal storage locations, this embodiment of the invention provides two specific implementations of visual acquisition devices: the first sensing device and the second sensing device are either a dual-focal-length camera group or a LiDAR. When both the first and second sensing devices are dual-focal-length camera groups, the reference data and real-time data are both 2D image data vertically acquired from the top surface of the goods to be stored; when both the first and second sensing devices are LiDAR, the reference data and real-time data are both point cloud data vertically acquired from the top surface of the target storage location.

[0073] In practical operation, it is important to note that for a dual-focal-length camera group solution, a fixed-baseline camera group can be selected, consisting of two independent cameras of the same specifications, equipped with a larger focal length lens and a smaller focal length lens. The focal length lenses must ensure complete coverage of the stacked container surface, and the physical baseline distance between the two cameras must be strictly fixed. For a LiDAR solution, a short-range wide-angle solid-state LiDAR can be selected to ensure that its scanning range covers the top layer of the stacked container surface, obtaining complete inventory information.

[0074] Generally, the four-way vehicle dense storage system comprises a plurality of intrabay mobile devices, when a to-be-counted task needs to be executed, a counting instruction is generated by a warehouse management system (WMS), and a warehouse control system (WCS) schedules the task to a single or multiple intrabay mobile devices. Specifically, the embodiment of the present application adopts a dynamic scheduling rule cooperating with the daily handling task, so that the intrabay mobile device preferentially processes the to-be-counted task in the handling-counting parallel mode without conflict with the daily handling task. That is, the intrabay mobile device can execute the to-be-counted task in the gap of the daily handling task, or specially execute the to-be-counted task when there is no handling task or the counting area is different from the handling area. Thus, the working efficiency of the four-way vehicle dense storage system is improved, and the operation cost is saved.

[0075] In step S104, the real-time data is compared with the reference data associated with the target storage location to determine whether the state of the goods in the target storage location is abnormal.

[0076] In an optional embodiment, when the reference data and the real-time data are both 2D image data vertically collected for the top surface of the to-be-warehoused goods, as shown in FIG. 1, the abnormality judgment process for the state of the goods in the target storage location is as follows: Figure 3

[0077] In step S104-1-1, a reference point is preset on the top of the goods box stack in the target storage location.

[0078] It can be understood that the parallax effect causes the image similarity from the top surface to be high for the box stacks of the same stacking form but different heights (or quantities), and it is difficult to reliably distinguish the difference, which is easy to cause misjudgment of the inventory state. The embodiment of the present application is based on the visual detection of the fixed baseline double-focus camera group, and the relative distance change of the imaging position of the specific reference point in the field of view of the double cameras is measured to determine whether the state of the storage location is abnormal.

[0079] First, a physical point with high visual salience, stability, and not easy to be blocked or moved is fixed on the top of the box stack as a reference point. In actual operation, the reference point can be a special mark first pasted on a specific position (such as the center) of the top surface of the box stack.

[0080] In step S104-1-2, the pixel coordinates of the reference point in the two groups of 2D image pairs obtained by the double-focus camera group are located in the reference data and the real-time data respectively, to determine the first pixel distance of the reference point in the reference data and the second pixel distance of the reference point in the real-time data.

[0081] For the two groups of reference data obtained by the double-focus camera group camera1 and camera2 I_base 1 and I_base ​2, respectively, using image processing algorithms (such as feature matching, template matching, landmark detection) to accurately identify the pixel coordinates of the reference points, obtaining I_base 1 in the coordinates of P_base 1=( x 1 b , y 1 b ); I_base 2 in the coordinates of P_base 2=( x 2 b , y 2 b )。Then, the pixel relative distance vector is D_base = P_base 1- P_base 2; the pixel absolute distance can be obtained by using the Euclidean distance formula:

[0082]

[0083] Further, the first pixel distance ( P_base 1, P_base 2, Distance_base ) can be associated with the target storage information and stored.

[0084] For the two sets of real-time data obtained by the dual-focus camera group camera1 and camera2 I_current 1, I_ current 2, the pixel coordinates of the reference points are identified by the same image processing algorithm as described above, obtaining I_current 1 in the coordinates of P_current 1=( x 1 c , y 1 c ); I_current 2 in the coordinates of P_current 2=( x 2 c , y 2 c )。Then, the pixel relative distance vector is D_current = P_current 1-P_current2; the pixel absolute distance can be obtained by using the Euclidean distance formula:

[0085]

[0086] Further, the second pixel distance ( P_current 1, P_current 2, Distance_current ) can be associated with the target storage information and stored.

[0087] Step S104-1-3: Calculate the distance difference value between the first pixel distance and the second pixel distance, and determine that the target storage location is in an abnormal state when the distance difference value exceeds a first preset distance threshold.

[0088] In the embodiment of the present application, the distance difference value D is the absolute difference between the first pixel distance (reference distance) and the second pixel distance (real-time distance), and the mathematical expression is:

[0089]

[0090] It can be understood that if the distance difference value D is less than or equal to the first preset distance threshold, the target storage location can be marked as "normal", indicating that the target storage location is consistent with the inventory quantity or state at the time of warehousing, and the inventory verification time can be updated. If the distance difference value D is greater than the first preset distance threshold, the target storage location can be marked as "abnormal", indicating that the target storage location is inconsistent with the inventory quantity or state at the time of warehousing, and can be added to the manual review queue.

[0091] It should be noted that the specific value of the first preset distance threshold can be set according to actual needs, and the present application does not limit this.

[0092] In another optional embodiment, when the reference data and the real-time data are both point cloud data collected vertically on the top surface of the target storage location, as shown in FIG. 4B, the abnormality judgment process for the target storage location is as follows: Figure 4

[0093] Step S104-2-1: Perform data preprocessing on the reference data and the real-time data respectively to obtain reference point cloud data and real-time point cloud data.

[0094] The data preprocessing includes ROI extraction, filtering and / or downsampling. Specifically, when performing ROI extraction, a cuboid region greater than 20mm of the tray boundary and with a height range from the upper surface of the tray to the upper surface of the goods can be set as the ROI, and the point cloud in the ROI corresponding to the reference data is taken as the reference point cloud data Pcd_dst, and the point cloud in the ROI corresponding to the real-time data is taken as the real-time point cloud data Pcd_src.

[0095] Step S104-2-2: Register the reference point cloud data and the real-time point cloud data, and calculate the coincidence degree between the registered reference point cloud data and the real-time point cloud data.

[0096] ​In actual application, when performing the to-be-counted task, the corresponding reference point cloud data Pcd_dst is retrieved and loaded according to the target storage location number information corresponding to the real-time point cloud data Pcd_src. Due to the offset of the goods placement position and the camera installation, the real-time point cloud data Pcd_src and the reference point cloud data Pcd_dst do not absolutely coincide. In order to solve the technical problem, the embodiment of the application takes the reference point cloud data Pcd_dst as a reference to register the real-time point cloud data Pcd_src. First, the fast point feature histogram FPFH of the two groups of point clouds is calculated, and the real-time point cloud data Pcd_src is globally roughly registered by taking the FPFH as a feature descriptor. Then, the real-time point cloud data Pcd_src is precisely registered by ICP by taking the rough registration transformation matrix as an initial transformation matrix.

[0097] After registering the two groups of point clouds, each point in the real-time point cloud data Pcd_src after matrix transformation is traversed, and the point cloud distance of the nearest neighbor point in the reference point cloud data Pcd_dst is found and calculated. If the distance is less than a second preset distance threshold, it is considered that the points coincide; and the coincidence degree can be obtained by calculating the proportion of the coincident points in the total number of points.

[0098] Step S104-2-3: Compare the coincidence degree with a preset coincidence threshold, and determine that the goods state of the target storage location is abnormal when the coincidence degree is lower than the preset coincidence threshold.

[0099] It can be understood that if the coincidence degree is less than the preset coincidence threshold, it indicates that the target storage location is inconsistent with the inventory quantity or state when warehousing, and the target storage location is marked as "abnormal" and added to the artificial review queue.

[0100] Step S104-2-4: When the coincidence degree reaches the preset coincidence threshold, the translation transformation of the real-time point cloud data in the tray plane is retained.

[0101] In the embodiment of the application, since the point cloud registration coincidence degree cannot distinguish the target with inconsistent layer height but consistent top surface goods, even if the coincidence degree reaches the preset coincidence threshold, it cannot indicate that the target storage location is completely "normal". Therefore, further comparison is needed.

[0102] Specifically, the rotation component in the transformation matrix during precise registration is removed, and only the translation transformation of the real-time point cloud data Pcd_src in the tray plane (XOY plane) is performed.

[0103] Step S104-2-5: The real-time point cloud data after translation transformation and the reference point cloud data are divided into a plurality of grid blocks, the average height of each grid block is calculated and compared, and the height difference ratio is obtained.

[0104] Further, the real-time point cloud data after translation transformation and the reference point cloud data are divided into 9 blocks by a 3x3 grid. The average height of each block of the two sets of point clouds is calculated, and the height values of the relative positions are compared and the difference ratio is calculated, and the mathematical expression is:

[0105]

[0106] wherein, diff represents the height difference ratio; srcz represents the real-time point cloud average height; dstz represents the reference point cloud average height; i represents grid block number.

[0107] Step S104-2-6: Compare the height difference ratio with the preset difference threshold value, and when the height difference ratio exceeds the preset difference threshold value, determine that the target goods status is abnormal.

[0108] Specifically, the embodiment compares each block, and if there is a block with a height difference ratio higher than the preset difference threshold value, it is determined that the inventory and the warehouse information are inconsistent, i.e. "abnormal goods location"; otherwise, if there is no block with a height difference ratio higher than the preset difference threshold value, it is determined that the inventory and the warehouse information are consistent, i.e. "normal goods location".

[0109] It should be noted that the specific values of the second preset distance threshold value, the preset coincidence threshold value and the preset difference threshold value in the embodiment can be set according to actual needs, and the present application does not limit this.

[0110] Further, when the target goods status is abnormal, the target goods location is marked as a to-be-reviewed goods location; the mobile device in the aisle responds to the review instruction to transport the to-be-reviewed goods of the to-be-reviewed goods location to the review end for review. The warehouse management system WMS can also automatically generate an abnormal goods location review list after each round of to-be-inspected task, and manually visually inspect the actual data / state and update the inventory record.

[0111] In order to ensure the accuracy of the warehouse goods location abnormal screening, the embodiment can also update the reference data in real time during the operation of the four-way vehicle dense storage system. For example, when the goods are re-stored after being taken out of the warehouse or the goods are processed back to the warehouse, the reference data is updated using the collection data when the goods are stored; when the target goods status is normal after being inspected, the reference data is updated using the real-time data of the target goods location.

[0112] The embodiment provides a warehouse goods location abnormal screening method, which proposes a machine vision preliminary screening and manual fine review paradigm for the inventory business, and reconstructs the traditional inventory process, and at least has the following beneficial effects:

[0113] 1) Dual-source vision architecture: Collect baseline data through fixed same-parameter vision equipment deployed at the warehouse entrance, and install vision equipment at the bottom of the four-way vehicle to collect real-time data. Through post-processing, the perspective deviation is solved to ensure the consistency of the comparison basis;

[0114] 2) Vision consistency judgment mechanism: Use the vertical penetration feature of the four-way vehicle (move to the target location directly above the adjacent layer), break through the physical obstruction of the multi-layer shelf to the lower layer goods, realize the non-handling goods location state detection, and do not need to change the shelf structure or add track equipment;

[0115] 3) Graded inventory business paradigm: Only the machine pre-screening abnormal goods location (accounting for less than 10%) is executed by manual outbound review, 90% of normal goods location is zero manual intervention, which overturns the traditional full-warehouse handling inventory mode, and the comprehensive inventory efficiency is improved by multiple times;

[0116] 4) Dual-focal camera group collaborative application: A fixed baseline dual-camera combination with the same sensor specification + different focal length lenses (wide angle + long focus) is adopted, and the relative position vector difference (not absolute coordinates) of the reference point in the dual-camera imaging plane is calculated as the state criterion, which solves the problem that monocular 2D vision is not sensitive to the change of stacking height.

[0117] Further, as a specific implementation of Figure 1 , the embodiment of the present application provides a warehouse goods location abnormal screening system, as shown in Figure 5 , the system can include: a first sensing device 510, a second sensing device 520, a roadway mobile device 530 and a control module 540; the control module 540 is in communication connection with the first sensing device 510, the second sensing device 520 and the roadway mobile device 530 respectively; the control module 540 includes a reference data acquisition unit 541, a real-time data acquisition unit 542 and a goods state judgment unit 543.

[0118] The reference data acquisition unit 541 can be used to control the first sensing device to acquire the reference data of the goods to be warehoused at the preset reference sensing point, and store the reference data in association with the target location of the goods to be warehoused;

[0119] The real-time data acquisition unit 542 can be used to control the roadway mobile device carrying the second sensing device to move to a detection position above the target location to be detected in response to a detection instruction; at the detection position, the second sensing device is controlled to collect real-time data of the target location;

[0120] The goods state judgment unit 543 can be used to compare the real-time data and the reference data associated with the target location to determine whether the goods state of the target location is abnormal;

[0121] The first sensing device and the second sensing device are visual acquisition devices of the same model and have the same physical baseline distance from the target storage location.

[0122] It should be noted that other corresponding descriptions of the functions of the warehouse storage location anomaly screening system provided in the embodiments of the present application can be referred to the corresponding descriptions of the method shown in the above Figure 1 The corresponding descriptions of the method are not repeated here.

[0123] Those skilled in the art can clearly understand the specific working process of the system, device, module and unit described above, and the corresponding process in the foregoing method embodiments can be referred to for brevity. Here, no further description is made.

[0124] In addition, each functional unit in each embodiment of the present application can be physically independent of each other, or two or more functional units can be integrated together, and all functional units can be integrated in one processing unit. The integrated functional units can be realized in the form of hardware or in the form of software or firmware.

[0125] Those skilled in the art can understand that the integrated functional units, if realized in the form of software and sold or used as independent products, can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computing device (such as a personal computer, a server, or a network device) to execute all or part of the steps of the method described in the embodiments of the present application when the instructions are executed. The foregoing storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program code storage media.

[0126] Alternatively, all or part of the steps of the foregoing method embodiments can be completed by program instruction related hardware (such as a computing device of a personal computer, a server, or a network device), and the program instruction can be stored in a computer readable storage medium. When the program instruction is executed by the processor of the computing device, the computing device executes all or part of the steps of the method described in the embodiments of the present application.

[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that, within the spirit and principle of the present application, the technical solutions recorded in the foregoing embodiments can still be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not cause the corresponding technical solutions to deviate from the protection scope of the present application.

Claims

1. A warehouse storage location anomaly screening method, characterized in that, The method comprises: At a preset reference access point, a first sensing device acquires reference data of the goods to be stored in the warehouse, and stores the reference data in association with the target storage location of the goods to be stored in the warehouse; A second sensing device mounted on a mobile device in the aisle moves to a counting position above the target storage location to be counted in response to a counting instruction; At the counting position, real-time data of the target storage location is collected by the second sensing device; The real-time data and the reference data associated with the target storage location are compared to determine whether the state of the goods in the target storage location is abnormal; The first sensing device and the second sensing device are the same model and have the same physical baseline distance from the target storage location. The first sensing device and the second sensing device are a dual-focus camera group or a laser radar. When the first sensing device and the second sensing device are both dual-focus camera groups, the reference data and the real-time data are both 2D image data collected vertically on the top surface of the goods to be stored in the warehouse. And / or, When the first sensing device and the second sensing device are both laser radars, the reference data and the real-time data are both point cloud data collected vertically on the top surface of the target storage location. When the reference data and the real-time data are both 2D image data collected vertically on the top surface of the goods to be stored in the warehouse, the comparison of the real-time data and the reference data associated with the target storage location to determine whether the state of the goods in the target storage location is abnormal comprises: A preset reference point is provided on the top of the goods box stack in the target storage location; In the reference data and the real-time data, respectively, the pixel coordinates of the reference point in the two sets of 2D image pairs obtained by the dual-focus camera group are located to determine the first pixel distance of the reference point in the reference data and the second pixel distance of the reference point in the real-time data; The distance difference between the first pixel distance and the second pixel distance is calculated, and when the distance difference exceeds a preset distance threshold, it is determined that the state of the goods in the target storage location is abnormal. When the reference data and the real-time data are both point cloud data collected vertically on the top surface of the target storage location, the comparison of the real-time data and the reference data associated with the target storage location to determine whether the state of the goods in the target storage location is abnormal comprises: Data preprocessing is performed on the reference data and the real-time data respectively to obtain reference point cloud data and real-time point cloud data; wherein the data preprocessing includes ROI extraction, filtering and / or downsampling; The reference point cloud data and the real-time point cloud data are registered, and the coincidence degree between the registered reference point cloud data and real-time point cloud data is calculated; The coincidence degree is compared with a preset coincidence threshold, and when the coincidence degree is lower than the preset coincidence threshold, it is determined that the state of the goods in the target storage location is abnormal; When the coincidence degree reaches the preset coincidence threshold, the translation transformation of the real-time point cloud data in the tray plane is retained. The real-time point cloud data after the translation transformation and the reference point cloud data are divided into multiple grid blocks, the average height of each grid block is calculated and compared, and a height difference ratio is obtained; The height difference ratio is compared with a preset difference threshold, and when the height difference ratio exceeds the preset difference threshold, it is determined that the state of the goods in the target storage location is abnormal.

2. The method of claim 1, wherein, The first sensing device is fixedly installed at a first position of an in-warehouse end conveyor that carries the to-be-warehoused goods by pallets, and the first position is a target height directly above a center point of the pallet; The second sensing device is fixedly installed at a second position of the intrabay mobile device, and the second position is below the intrabay mobile device; when the second sensing device is directly above the center point of the pallet at the target storage location, the distance from the center point of the pallet is the target height.

3. The method of claim 2, wherein, The first sensing device acquires reference data of the to-be-warehoused goods at a preset reference sensing point, including: When the in-warehouse end conveyor carries the to-be-warehoused goods to the in-warehouse end, the in-warehouse end conveyor is controlled to be positioned at the preset reference sensing point, and the first sensing device is triggered to vertically collect reference data of the top surface of the to-be-warehoused goods; And / or, The intrabay mobile device carrying the second sensing device moves to a counting position above the target storage location to be counted in response to a counting instruction, including: The intrabay mobile device positions the second sensing device directly above the center point of the pallet at the target storage location at an adjacent layer above the target storage location in response to the counting instruction.

4. The method of claim 1, wherein, The method further includes: Based on the to-be-counted task, one or more intrabay mobile devices are dynamically scheduled, so that the intrabay mobile device processes the to-be-counted task in a parallel carrying-counting mode without conflict with daily carrying tasks.

5. The method of claim 1, wherein, The method further includes: When a preset condition is triggered, the reference data is updated in real time, specifically including: When the goods are re-warehoused after being taken out or the goods are processed and returned to the warehouse, the collection data when the goods are warehoused is used to update the reference data; and / or, When the state of the goods in the target storage location after being counted is normal, the real-time data of the target storage location is used to update the reference data.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: When the state of the goods in the target storage location is abnormal, the target storage location is marked as a to-be-reviewed storage location; The intrabay mobile device carries the to-be-reviewed goods in the to-be-reviewed storage location to a review end in response to a review instruction, so as to review the to-be-reviewed goods.

7. A warehouse storage location anomaly screening system, characterized by, The system includes a first sensing device, a second sensing device, an intrabay mobile device, and a control module; the control module is in communication connection with the first sensing device, the second sensing device, and the intrabay mobile device; The control module includes a reference data acquisition unit, a real-time data acquisition unit, and a goods state judgment unit; The reference data acquisition unit is configured to control the first sensing device to acquire reference data of to-be-warehoused goods at a preset reference sensing point, and store the reference data in association with a target storage location of the to-be-warehoused goods. The real-time data acquisition unit is configured to control the in-tunnel mobile device carrying the second sensing device to move to a checking position above a target storage location to be checked in response to a checking instruction; and control the second sensing device to acquire real-time data of the target storage location at the checking position. The goods state judgment unit is configured to compare the real-time data with reference data associated with the target storage location to determine whether the goods state of the target storage location is abnormal. The first sensing device and the second sensing device are visual acquisition devices of the same model and have the same physical baseline distance from the target storage location. The first sensing device and the second sensing device are a dual-focus camera group or a laser radar. When the first sensing device and the second sensing device are both a dual-focus camera group, the reference data and the real-time data are both 2D image data acquired vertically with respect to a top surface of the goods to be stored. When the first sensing device and the second sensing device are both a laser radar, the reference data and the real-time data are both point cloud data acquired vertically with respect to a top surface of the target storage location. The goods state judgment unit is further configured to, when the reference data and the real-time data are both 2D image data acquired vertically with respect to a top surface of the goods to be stored, preset a reference point on a top of a goods box stack of the target storage location; locate the reference point in pixel coordinates in two groups of 2D images obtained by the dual-focus camera group in the reference data and the real-time data respectively to determine a first pixel distance of the reference point in the reference data and a second pixel distance of the reference point in the real-time data; calculate a distance difference between the first pixel distance and the second pixel distance, and determine that the goods state of the target storage location is abnormal when the distance difference exceeds a preset distance threshold. The goods state judgment unit is further configured to, when the reference data and the real-time data are both point cloud data acquired vertically with respect to a top surface of the target storage location, perform data preprocessing on the reference data and the real-time data respectively to obtain reference point cloud data and real-time point cloud data; wherein the data preprocessing includes ROI extraction, filtering and / or down-sampling; perform registration on the reference point cloud data and the real-time point cloud data, and calculate a coincidence degree between the registered reference point cloud data and real-time point cloud data; compare the coincidence degree with a preset coincidence threshold, and determine that the goods state of the target storage location is abnormal when the coincidence degree is lower than the preset coincidence threshold; when the coincidence degree reaches the preset coincidence threshold, retain a translation transformation of the real-time point cloud data in a tray plane; divide the translation-transformed real-time point cloud data and the reference point cloud data into a plurality of grid blocks, calculate and compare average heights of the grid blocks to obtain a height difference ratio; compare the height difference ratio with a preset difference threshold, and determine that the goods state of the target storage location is abnormal when the height difference ratio exceeds the preset difference threshold. ​

Images