Independent visual checking device, method and system for stereoscopic warehouse

Abstract

The invention provides an independent visual checking device, method and system for a stereoscopic warehouse, and belongs to the technical field of automatic warehousing. The independent visual checking device comprises a device main body with a tray type base, a multi-dimensional movement mechanism (comprising a horizontal sliding mechanism and a lifting mechanism) arranged on the main body, a visual assembly mounted on the multi-dimensional movement mechanism, and a local controller configured in the main body. The method is executed by a local controller and comprises the following steps: receiving an inventory instruction of an upper system; the multi-dimensional movement mechanism is autonomously controlled to adjust the pose of the visual assembly and collect images; carrying out localized analysis on the image to obtain an inventory result; and a result is reported. By endowing the visual component with the two-dimensional plane movement capability, the method can actively avoid shielding, obtain the image from the optimal visual angle and remarkably improve the checking accuracy, and has the advantages of high cost effectiveness, high autonomy, easiness in maintenance and the like.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automated warehousing, and in particular to a visual inventory device, system and method for a stereoscopic warehouse. BACKGROUND

[0002] The automated stereoscopic warehouse is a core component of the modern logistics and warehousing system. In order to ensure the real-time and accuracy of the inventory data, it is necessary to periodically or irregularly inventory the goods in the warehouse. The existing inventory methods mainly include manual inventory, barcode-based inventory and fixed visual inventory, etc.

[0003] Among them, manual inventory is not only inefficient and prone to errors, but also poses a high safety risk to the workers in the high-position shelf environment of the stereoscopic warehouse. Although the barcode-based inventory method improves the automation level and efficiency to a certain extent, it highly depends on the integrity of the barcode. In the actual warehousing environment, the barcode label is prone to be stained, fallen off or blocked, resulting in recognition failure and affecting the inventory accuracy. The fixed visual inventory scheme usually integrates or installs a camera and other visual devices directly on the stacker in the aisle. This scheme has several inherent defects: first, a set of visual system is configured for each stacker in the warehouse, resulting in high hardware cost and total cost of transformation. For the old warehouse that has been built, the transformation difficulty and cost are unacceptable; second, the installation position and shooting field of view of the camera are strictly limited by the structure of the stacker itself, and the adjustment range is extremely limited. When the goods are blocked by the columns or beams of the shelf, the complete image of the goods cannot be obtained, which seriously affects the accuracy of the inventory result; finally, the maintenance or repair of the visual device requires the entire stacker to be disabled, directly affecting the normal in-out warehouse operation efficiency.

[0004] To solve the above-mentioned problems, an independent inventory device that can be carried by the stacker has been proposed in the prior art. The device is independent of the stacker in terms of machinery and control, and does not need to be transformed for each stacker, thereby reducing the cost of deployment in a multi-lane warehouse. However, the visual sensor movement capability of such existing independent inventory device is still limited, for example, its visual sensor can only realize single-direction telescopic movement. In the face of complex blocking conditions in the stereoscopic warehouse, this single-dimensional freedom of movement still cannot flexibly move the camera to the best shooting position that can avoid the blocking of the shelf columns or beams, resulting in limited ability to solve the visual blocking problem and insignificant improvement in inventory accuracy. Therefore, how to completely solve the visual blocking problem while maintaining the low-cost and easy-maintenance advantages of the independent device and realizing high-precision visual inventory is a technical problem to be solved in the field. SUMMARY

[0005] The application aims to provide an independent visual inventory device, system and method, aiming to solve the technical problem of insufficient freedom of movement of the visual sensor of the independent inventory device in the prior art, which makes it difficult to avoid structural obstructions such as shelves, cannot obtain the best shooting angle, and thus affects the accuracy of inventory.

[0006] In order to achieve the above-mentioned purpose, the application provides the following technical solutions:

[0007] In a first aspect, the application provides an independent visual inventory device for a stereoscopic warehouse, comprising:

[0008] A device body, the device body having a pallet base for transfer by a material handling equipment in the stereoscopic warehouse;

[0009] A multi-dimensional motion mechanism arranged on the device body, the multi-dimensional motion mechanism comprising a horizontal sliding mechanism and a lifting mechanism;

[0010] A visual assembly mounted on the multi-dimensional motion mechanism for image acquisition; and

[0011] A local controller configured in the device body, the local controller being configured to:

[0012] Receive an inventory instruction from an upper control system; the inventory instruction comprising a task ID, coordinate information of a to-be-inventoried storage location, and structure parameters of the warehouse;

[0013] Autonomously control the multi-dimensional motion mechanism to adjust the pose of the visual assembly based on the inventory instruction, and control the visual assembly to acquire images;

[0014] Locally analyze the acquired images to obtain an inventory result; and

[0015] Report the inventory result to the upper control system.

[0016] Based on the above device, optionally, the horizontal sliding mechanism is a planar sliding rail arranged in a horizontal direction, and the lifting mechanism is a lifting push rod mounted on a moving platform of the planar sliding rail.

[0017] Based on the above device, optionally, further comprising:

[0018] The visual assembly comprises a camera and a light source, the camera being mounted on a gimbal, and the gimbal being configured to realize the pitching and rotating movements of the camera to adjust the shooting angle in cooperation with the multi-dimensional motion mechanism.

[0019] Based on the above device, optionally, the local controller is further configured to:

[0020] After receiving the inventory instruction, a target shooting pose for avoiding occlusion is calculated according to the to-be-inventoried storage location information and preset parameters of the vision assembly.

[0021] The autonomous control of the multi-dimensional motion mechanism to adjust the pose of the vision assembly specifically includes: controlling the multi-dimensional motion mechanism to move the vision assembly to the target shooting pose.

[0022] Based on the above device, optionally, when locally analyzing the collected images, the local controller is specifically used for:

[0023] controlling the vision assembly to collect multiple images from at least two different shooting poses; and fusing the multiple images to determine the inventory result.

[0024] Based on the above device, optionally, the at least two different shooting poses include a global observation point for obtaining a top view of the goods and a fine observation point for obtaining a front view of the goods.

[0025] Based on the above device, optionally, a battery is further arranged in the device body to supply power for the device; and a charging interface is arranged on the device body to charge the device when the device is transported to a dedicated storage location.

[0026] Based on the above device, optionally, the local controller is constructed based on a robot operating system or a real-time operating system, the control of the multi-dimensional motion mechanism, the control of the vision assembly, and image analysis are implemented as independent modules or tasks, and information interaction is performed through a preset communication protocol.

[0027] In a second aspect, the embodiments of the present application further provide an independent visual inventory method for a stereoscopic warehouse, which is applied to an independent visual inventory device transported to a to-be-inventoried storage location by a material handling equipment. The inventory device includes a device body, a multi-dimensional motion mechanism arranged on the device body, a vision assembly mounted on the multi-dimensional motion mechanism, and a local controller arranged in the device body. The multi-dimensional motion mechanism includes a horizontal sliding mechanism and a lifting mechanism. The method includes the following steps performed by the local controller:

[0028] receiving an inventory instruction from an upper control system; the inventory instruction includes a task ID, coordinate information of a to-be-inventoried storage location, and structure parameters of a warehouse;

[0029] autonomously controlling the multi-dimensional motion mechanism to adjust the pose of the vision assembly based on the inventory instruction, and controlling the vision assembly to collect images;

[0030] locally analyzing the collected images to obtain an inventory result; and

[0031] reporting the inventory result to the upper control system.

[0032] In a third aspect, the embodiments of the present application further provide an independent visual inventory system, characterized in that comprising:

[0033] the independent visual inventory device according to any one of the first aspect;

[0034] a material handling equipment for transporting the independent visual inventory device in the stereoscopic warehouse; and

[0035] an upper control system, wherein the upper control system is configured to:

[0036] issue a transportation instruction to the material handling equipment to control the material handling equipment to transport the independent visual inventory device to the vicinity of the inventory location; and

[0037] issue an inventory instruction to a local controller of the independent visual inventory device after the independent visual inventory device is transported to the vicinity of the inventory location, and receive an inventory result reported by the local controller.

[0038] Compared with the prior art, the present application has the following beneficial effects:

[0039] 1. Significantly improve the inventory accuracy. The present application is provided with horizontal sliding mechanism and lifting mechanism, which gives the visual component the ability to move in a large range in two-dimensional plane, and cooperates with the gimbal rotation of the visual component itself, which can actively and flexibly avoid the obstruction of fixed structures such as shelf columns and cross beams, moves the visual component to the best shooting angle, so as to obtain high-quality images containing multiple angle information of goods, fundamentally solves the problem of inaccurate inventory caused by visual obstruction, greatly improves the accuracy and reliability of the inventory result.

[0040] 2. High cost-effectiveness and easy modification. The device of the present application is independent of the stacker, and only one set or a small number of the device is needed to serve all the aisles in the entire warehouse, which greatly reduces the hardware investment and the upgrading and modification cost of the old warehouse, and does not need to make any modification to the existing stacker, which is flexible and convenient to deploy.

[0041] 3. High autonomy and system stability. The device of the present application can autonomously complete the whole process of inventory operation from pose solving, motion control to image analysis, only interacts with the upper system at the task level instruction and the final result, significantly reduces the dependence on wireless communication bandwidth and the potential risk of communication interference, and enhances the stability and reliability of the inventory operation.

[0042] 4. Improve the efficiency of operation and maintenance. Since the device is independent of the stacker, when it needs maintenance due to failure, it can be easily taken out of the rack by the stacker for repair outside the warehouse, which does not affect the normal in-out operation of the stacker, effectively ensuring the operation efficiency of the whole warehouse system. BRIEF DESCRIPTION OF DRAWINGS

[0043] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and serve to explain the principles of the present application. Furthermore, the drawings and the associated description are not intended to limit the scope of the present application in any way, but rather serve as an example of the principles of the present application to one of ordinary skill in the art.

[0044] Figure 1 A flowchart of an independent visual inventory method provided by an embodiment of the present application;

[0045] Figure 2 An internal structure diagram of an independent visual inventory device provided by an embodiment of the present application;

[0046] Figure 3 A front view of an independent visual inventory device provided by an embodiment of the present application;

[0047] Figure 4 A top view of an independent visual inventory device provided by an embodiment of the present application;

[0048] Figure 5 A working environment diagram of an independent visual inventory system provided by an embodiment of the present application;

[0049] Figure 6 A timing diagram of signaling interaction between system entities in an embodiment of the present application;

[0050] Figure 7 A ROS-based software system architecture diagram of an inventory device in an embodiment of the present application. DETAILED DESCRIPTION

[0051] To make the purposes, technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by one of ordinary skill in the art without creative labor are within the scope of protection of the present application. The embodiments described below and the features in the embodiments can be combined with each other without conflict.

[0052] Embodiment 1

[0053] The embodiment provides an independent visual inventory device, a system where the independent visual inventory device is located, and a corresponding working method. The technical scheme aims to solve the technical problems of low visual inventory accuracy and high deployment cost caused by the shielding of the shelf structure in the automated warehouse by an intelligent device which is independent of the material handling equipment in mechanical structure and control logic.

[0054] Figure 5 A working environment of an independent visual inventory system in an embodiment of the application is shown. The system is deployed in a typical automated warehouse 100, and a plurality of high shelves 110 are arranged in the warehouse 100, and narrow aisles 120 are formed between the shelves 110. At least one material handling equipment, such as a stacker 130, is arranged in each aisle 120 to perform the storage and retrieval of goods. The core equipment of the application, that is, the independent visual inventory device 140, is stored in a special storage location 160 when not in use, and the special storage location 160 can also serve as a charging station. The upper control system 150 of the system, such as a warehouse management system or a warehouse control system, serves as the control center of the entire system, is usually deployed in the central control room, is responsible for issuing inventory tasks, and communicates with the stacker 130 and the independent visual inventory device 140 to coordinate the complete inventory process.

[0055] For reference Figures 2 to 4 , Figure 2 An internal structure diagram of an independent visual inventory device provided in an embodiment of the application is shown, Figure 3 A front view of an independent visual inventory device provided in an embodiment of the application is shown, Figure 4 A top view of an independent visual inventory device provided in an embodiment of the application is shown. The core of the design of the independent visual inventory device 140 lies in its independence and intelligence. The independence is reflected in that the device has a complete mechanical structure and control unit which does not depend on the stacker 130. Specifically, the device 140 includes a device main body which is composed of a solid main box 3 and a standardized tray-type base 14. The size and structure design of the tray-type base 14 are compatible with standard pallets, so that the forks of the stacker 130 can conveniently pick up, carry and transport the device. In this way, the independent visual inventory device 140 can be flexibly moved in the entire automated warehouse 100 as an independent unit without any structural modification of the stacker 130 itself.

[0056] Inside the device body, i.e. the main box 3, a complete set of core components for realizing autonomous inventorying is integrated. Among them, the local controller, as the intelligent hub of the device, can be specifically implemented as a high-performance embedded development board 8 in this embodiment, such as a development board based on ARM architecture, running embedded Linux operating system and robot operating system. In order to realize long-term offline autonomous work, a large-capacity rechargeable battery 4 is also configured in the main box 3 as a separate power supply for the device. The side wall of the main box 3 is provided with a charging contact 1, which is connected with the charging pile on the storage location when the device 140 is transported back to the dedicated storage location 160 by the stacker 130, and the charging battery 4 is automatically charged.

[0057] In order to realize high-precision visual inventorying of goods, the key innovation of the device 140 is a set of multi-dimensional motion mechanism arranged on the device body. The mechanism gives the visual component significant freedom of movement, enabling it to actively avoid obstacles. In this embodiment, the multi-dimensional motion mechanism is specifically composed of a horizontal sliding mechanism and a lifting mechanism. The horizontal sliding mechanism is specifically a planar slide rail 5 installed along the horizontal direction inside the main box 3, e.g. perpendicular to the extension direction of the aisle 120. The planar slide rail 5 is provided with a moving platform driven by a DC servo motor 6. The lifting mechanism is specifically an electric lifting push rod 12, the base of which is fixed to the moving platform of the planar slide rail 5. The telescopic rod of the lifting push rod 12 can pass through a long slot 15 in the top cover 13 of the main box 3 and move up and down in the vertical direction. By controlling the DC servo motor 6 to drive the moving platform of the planar slide rail 5 through the local controller, the translation of the visual component in the horizontal direction can be realized; at the same time, the telescopic rod of the lifting push rod 12 can be controlled to realize the lifting of the visual component in the vertical direction. Through the combined motion of the two, the visual component can move in a large range in a two-dimensional vertical plane.

[0058] The end of the multi-dimensional motion mechanism, i.e. the top of the lifting push rod 12, is installed with a visual component for collecting high-quality storage location images. In this embodiment, the visual component specifically includes an industrial-grade pan-tilt camera 10 and a high-power industrial light source 11. The pan-tilt camera 10 itself is installed on a two-dimensional pan-tilt head, which can realize the pitching and horizontal rotation of the camera, e.g. horizontal rotation of ±180 degrees and pitching rotation of 90 degrees. The industrial light source 11 is linked with the pan-tilt camera 10 and can provide uniform and sufficient illumination at the moment of camera exposure, so as to eliminate the influence of uneven or insufficient light in the warehouse on the imaging quality. Correspondingly, by controlling the horizontal sliding mechanism (planar slide rail 5), the lifting mechanism (lifting push rod 12) and the pan-tilt head of the pan-tilt camera 10 through the local controller, the camera lens can be accurately positioned to any target three-dimensional coordinate and attitude in the space of the aisle 120, so as to obtain the best shooting angle.

[0059] In traditional solutions, the camera is fixed in position, and its line of sight is easily blocked by the columns or beams of the shelf 110, making it impossible to obtain a complete image of the goods. However, the device 140 of the present application uses its multi-dimensional motion mechanism to move the pan-tilt camera 10 to a new position (for example, by translating it a certain distance to the side and raising it appropriately), thereby effectively avoiding obstruction and obtaining a clear and complete image of the goods.

[0060] In order to achieve the above-mentioned complex autonomous control, the local controller (embedded development board 8) is connected to each component through a variety of interfaces. For example, it communicates with the motor driver 7 through the RS485 bus, and the motor driver 7 is responsible for accurately controlling the DC servo motor 6 of the planar slide 5 and the motor of the lifting push rod 12. It is connected to the pan-tilt camera 10 through a network interface to send control instructions and receive image data. It controls the switch of the industrial light source 11 through a general input and output pin. In addition, other sensors can be integrated into the device, such as a temperature and humidity sensor 2, whose probe is exposed to the outside of the main box 3. The local controller can read the environmental data through the I2C bus and complete the monitoring of the environment in the warehouse while taking inventory. The device is connected to the wireless network in the warehouse through the built-in wireless communication module, thereby realizing information interaction with the upper control system 150.

[0061] Figure 7 The following illustrates the software system architecture running on the local controller in this embodiment. This architecture is based on the robot operating system and embodies a modular design concept. The entire software system runs around a robot operating system master node 200, which contains multiple functionally independent nodes:

[0062] 1. Control system communication node 210: used to communicate with the upper-level control system 150. This node subscribes to the inventory instruction topic from the upper-level system and publishes the parsed task information to the internal topic. At the same time, it subscribes to the internal inventory result topic and reports the results to the upper-level system.

[0063] 2. Motor drive node 220: This node is used to control the multi-dimensional motion mechanism. This node subscribes to a "target pose" topic containing the target's three-dimensional coordinates, calculates the specific positions to which the planar slide 5 and lift rod 12 must move, and executes the movement through the motor driver 7. Simultaneously, this node publishes a "current pose" topic containing the current position information.

[0064] 3. Camera Light Source Node 230: This node controls the visual component. This node subscribes to a "target angle" topic containing the target shooting angle and controls the pan / tilt camera 10 to rotate to the specified angle. After positioning is complete, this node receives a trigger command, controls the camera to take photos, synchronizes the light source 11 with the flash, and publishes the captured image data or image storage path information.

[0065] 4. Visual algorithm node 240: for performing image analysis and inventory. This node subscribes to the image information topic, upon receiving a new image, it executes a pre-set inventory algorithm, such as object detection, template matching or deep learning counting algorithm, to derive the inventory result (e.g. the number of cases, the status of each location, etc.), and publishes the result to the "inventory result" topic.

[0066] This asynchronous communication mechanism based on the publish / subscribe pattern achieves a high degree of decoupling of the functional modules, not only facilitating development, testing and upgrading, but also ensuring the stability and real-time performance of the entire control system.

[0067] In summary, the steps performed by the local controller mainly include: receiving an inventory instruction from the upper control system; the inventory instruction contains a task ID, coordinate information of the location to be inventoried, and structure parameters of the warehouse; autonomously controlling the multi-dimensional motion mechanism to adjust the pose of the visual component based on the inventory instruction, and controlling the visual component to perform image acquisition; performing local analysis on the acquired images to obtain the inventory result; and reporting the inventory result to the upper control system.

[0068] Next, the complete flow of the independent visual inventory method provided by the present embodiment will be described in detail in combination with Figure 1 and Figure 6 . Figure 1 The flowchart of the method is shown in Figure 6 , and the signaling interaction timing diagram between the entities of the system is shown in

[0069] The method starts with the upper control system 150 initiating an inventory task.

[0070] Step S1: The upper control system 150 issues a transfer instruction to the stacker crane 130 in the aisle 120, instructing it to go to the dedicated storage location 160 to pick up the independent visual inventory device 140.

[0071] Step S2: The stacker crane 130 executes the instruction, moves to the dedicated storage location 160, picks up the inventory device 140 with its forks and places it on its load platform, then transports it to the aisle 120 where the first location to be inventoried is located, and stops near the location. The stacker crane 130 reports to the upper control system 150 that the movement is complete.

[0072] Step S3: The upper control system 150 sends an inventory instruction to the local controller of the inventory device 140 through the wireless network. The instruction usually contains a task ID, coordinate information of the location to be inventoried (e.g. layer, column, depth), and structure parameters of the warehouse (e.g. aisle width, shelf size, etc.). The control system communication node 210 of the inventory device 140 receives and parses the instruction.

[0073] Step S4: After receiving the instruction, the local controller triggers its internal pose planning module (which can be integrated in the vision algorithm node 240 or a separate planning node) to calculate one or more optimal shooting poses that can avoid potential occlusions, based on the received storage location information, the preset optical parameters of the PTZ camera 10 (such as field of view, focal length), and the known structure information of the shelf 110. A pose information usually includes the three-dimensional coordinates (X, Y, Z) of the camera in space and the orientation of the camera (pitch angle, rotation angle). After the calculation is completed, the target pose information is published. The motor drive node 220 and the camera light source node 230 subscribe to this information and work together to control the planar slide rail 5, the lifting push rod 12, and the gimbal of the PTZ camera 10, so as to accurately move the camera to the calculated target pose.

[0074] Step S5: After the camera reaches the target pose and stabilizes, the local controller triggers the camera light source node 230 to perform image acquisition. The acquired high-definition image is saved in the local storage. Subsequently, the vision algorithm node 240 is activated to read the just-acquired image and perform localized analysis and processing. For example, through image preprocessing (such as noise reduction, contrast enhancement), target detection algorithms (such as identifying all the boxes in the image), and counting algorithms, the final inventory result of the storage location is obtained, such as "the number of boxes is 20" or "the storage location is full".

[0075] Step S6: The vision algorithm node 240 publishes the inventory result. After subscribing to the result, the control system communication node 210 packages and reports it to the upper-level control system 150 through the wireless network. After receiving the result, the upper-level control system 150 compares it with its own inventory database to complete an inventory. If there is a next storage location to be inventoried in the inventory task list, the steps S3 to S6 are repeated, and the upper-level control system 150 will instruct the stacker 130 to move to the next storage location and issue a new inventory instruction to the inventory device 140.

[0076] Step S7: When all inventory tasks are completed, the upper-level control system 150 issues a final transfer instruction to the stacker 130, instructing it to transport the inventory device 140 back to the dedicated storage location 160. After receiving the task completion instruction, the device controls all moving parts to return to the initial safe position and enters a low-power standby mode, waiting for the next task or charging.

[0077] Embodiment 2

[0078] This embodiment provides a variant structure of a standalone visual inventory device, the main difference being the specific implementation of the multi-dimensional motion mechanism, which is intended to illustrate that the protection scope of the present application is not limited to a specific mechanical structure combination.

[0079] In this embodiment, the overall layout of the device, the tray base 14, the main box 3, and the core components such as the local controller, the rechargeable battery 4, etc. are basically the same as in Embodiment 1. The main difference from Embodiment 1 is that this embodiment uses a combination of "rotation + lifting" to replace the "flat slide rail 5 + lifting push rod 12" combination in Embodiment 1 to achieve multi-dimensional movement.

[0080] Specifically, a rotating platform driven by a motor to rotate 360 degrees horizontally is installed at the center of the top cover 13 of the main box 3. Then, a scissor lifting mechanism is installed on the rotating platform. The scissor lifting mechanism has a large range of travel and good load stability. The visual assembly, i.e. the gimbal camera 10 and the industrial light source 11, is installed on the top platform of the scissor lifting mechanism.

[0081] Under this structure, the working mode of the local controller is also adjusted accordingly. After it calculates the Cartesian coordinates (X, Y, Z) of the target shooting pose, its motion control module needs to perform a coordinate transformation to convert the Cartesian coordinates into the corresponding polar coordinates of the mechanism, i.e. to calculate the angle (θ) that the rotating platform needs to rotate and the height (H) that the scissor lifting mechanism needs to reach. The calculation formula can be: After that, the local controller sends control instructions to the drive motor of the rotating platform and the drive motor of the scissor lifting mechanism respectively, so that they move to the calculated target angle and height. In this way, the gimbal camera 10 can also be accurately moved to the target position in the lane 120 space, and by cooperating with the pitch and rotation of the gimbal itself, it can also achieve flexible adjustment of the shooting angle, achieving the purpose of avoiding obstruction and accurate inventory.

[0082] This embodiment proves that the multi-dimensional motion mechanism implementing the inventive concept of the present application can adopt a variety of equivalent mechanical structures, including but not limited to the combination of "translation + lifting" or "rotation + lifting".

[0083] Embodiment 3

[0084] This embodiment provides an optional control architecture for the local controller of the inventory device, aiming to illustrate that the control logic of the present application does not depend on a specific software platform, and thus has wider applicability.

[0085] The hardware structure of this embodiment can be exactly the same as that of Embodiment 1, including the main box 3, the tray base 14, the flat slide rail 5, the lifting push rod 12, the gimbal camera 10, and the embedded development board 8, etc. The core difference lies in the software system running on the embedded development board 8.

[0086] In this embodiment, the local controller runs a lightweight real-time operating system, such as FreeRTOS. Under this operating system, the various "nodes" originally in the robot operating system are implemented as independent "tasks". For example, the following tasks can be created:

[0087] 1. Communication task: responsible for communication with the upper-layer control system 150.

[0088] 2. Motor control task: responsible for controlling the movement of the planar slide rail 5 and the lifting push rod 12.

[0089] 3. Camera control task: responsible for controlling the gimbal movement of the gimbal camera 10 and image acquisition.

[0090] 4. Algorithm task: responsible for executing image analysis and inventory algorithms.

[0091] These tasks are executed concurrently, and the information exchange between tasks is accomplished through the communication mechanisms provided by the real-time operating system, such as message queues or semaphores. For example, after receiving the upper-layer instruction, the communication task sends the target pose information parsed through the message queue to the motor control task.

[0092] In terms of communication protocols with the upper-layer control system 150, as an optional implementation, this embodiment can use the MQTT protocol widely used in the field of industrial Internet of Things. The inventory device 140 acts as an MQTT client and is connected to an MQTT broker played by the upper-layer control system 150 or an independent server. The device subscribes to a specific command topic, such as warehouse / inventory / device01 / command. The upper-layer system encapsulates the inventory instruction into a JSON-formatted string and publishes it to the topic. For example: {"task_id":"T12345","location":{"row":5,"column":12,"level":4},"mode":"counting"} After receiving this message, the communication task of the device parses the JSON and distributes the task information to other related tasks through the message queue. When the algorithm task completes the inventory and obtains the result, it sends the result (for example, a JSON object containing the quantity and confidence) to the communication task through the message queue. The communication task then publishes the result to a result topic, such as warehouse / inventory / device01 / result, for the upper-layer control system 150 to subscribe and receive.

[0093] This embodiment shows that the core workflow of autonomous control, local analysis, and result reporting proposed in this application can be implemented through a variety of different software technology stacks, whether it is a heavyweight robot operating system or a lightweight real-time operating system, which can effectively support the technical solutions of this application.

[0094] Embodiment 4

[0095] This embodiment is based on Embodiment 1, and further optimizes the internal logic of the vision algorithm node 240 in the local controller. In particular, for the cargo counting inventory scenario, an enhanced algorithm based on multi-view image fusion is proposed to further improve the inventory accuracy in complex stacking situations.

[0096] The hardware and basic software architecture of this embodiment remains consistent with Embodiment 1. The innovation lies in the intelligent upgrade of the image acquisition and analysis strategy in the inventory method, which aims to fully utilize the multi-dimensional motion capabilities of the device.

[0097] When the local controller receives an instruction to perform "counting inventory", its vision algorithm node 240 will start the following optimized workflow:

[0098] 1. Autonomous planning of multi-view shooting path: Unlike Embodiment 1, which only moves to a single optimal position, the algorithm in this embodiment first autonomously plans at least two, usually three or more shooting poses based on the size information of the goods location. These poses are carefully designed to capture key information of the goods from different angles. As a preferred implementation, these poses can include: one "global observation point": usually located diagonally above the goods to be inventoried, at a higher position, so that the gimbal camera 10 can overlook the top view of the goods. At least one "fine observation point": usually located in front of the goods, at a height roughly level with the middle layer of the goods stack, for clearly capturing the front or side view of the goods stack.

[0099] 2. Multi-view image acquisition: The motion control module of the local controller will drive the multi-dimensional motion mechanism and the gimbal camera 10 in sequence to accurately move to each of the planned shooting poses and acquire a high-resolution image. For example, first move to the global observation point to take a top view, then move to the fine observation point to take a front view.

[0100] 3. Multi-image information fusion and analysis: After obtaining all the images from different perspectives, the visual algorithm node 240 executes the fusion analysis algorithm. The algorithm can specifically include the following steps: first, analyze the top view taken from the global observation point. Since there is no vertical occlusion in this perspective, the algorithm can accurately calculate the number of top-level containers, denoted as N, through target segmentation and contour analysis. Second, analyze the front view taken from the fine observation point. This view can clearly show the stacking of goods in the vertical direction. The algorithm can accurately identify the number of layers of the goods stack, denoted as L, through edge detection, Hough transform or deep learning models. At the same time, it can also judge whether the stacking is neat by analyzing the gap between layers. Finally, the results are fused. The algorithm multiplies the number of top layers N and the number of layers L to get the final total number of containers Total = N*L. This method effectively overcomes the counting errors caused by perspective effect or irregular stacking in single view by obtaining two key parameters, horizontal number and vertical layer number, from the optimal view respectively.

[0101] 4. Result reporting: The final total number of containers calculated is reported to the upper control system 150 through the control system communication node 210 as the inventory result.

[0102] Through this multi-perspective scanning and information fusion strategy, the inventory device of the embodiment can achieve higher counting accuracy for complex scenarios such as irregular stacking or partial occlusion by the upper shelf beam, thereby significantly enhancing the practicality and reliability of the entire inventory system.

[0103] In addition, it can be understood that the present application can also provide a standalone visual inventory system, which can include:

[0104] The standalone visual inventory device described in any of the above embodiments;

[0105] A material handling device for transporting the standalone visual inventory device in the stereoscopic warehouse;

[0106] And an upper control system, wherein the upper control system is configured to:

[0107] Issue a transportation instruction to the material handling device to control the material handling device to transport the standalone visual inventory device to the vicinity of the inventory location to be inventoried; and

[0108] After the standalone visual inventory device is transported to the vicinity of the inventory location to be inventoried, issue an inventory instruction to the local controller of the standalone visual inventory device, and receive the inventory result reported by the local controller.

[0109] Wherein, the cooperation logic of each device or subsystem of the standalone visual inventory system can refer to the description of the foregoing embodiments, which will not be repeated here.

[0110] Based on the above example, compared with the prior art, the application has the following beneficial effects:

[0111] 1. Significantly improve the accuracy of inventory. The application sets up a horizontal sliding mechanism and a lifting mechanism, which gives the visual component the ability to move in a large range in a two-dimensional plane. Combined with the rotation of the gimbal of the visual component itself, it can actively and flexibly avoid the obstruction of fixed structures such as shelf columns and crossbeams, move the visual component to the best shooting angle, and obtain high-quality images containing multiple angle information of goods, thereby fundamentally solving the problem of inaccurate inventory caused by visual obstruction and greatly improving the accuracy and reliability of the inventory results.

[0112] 2. High cost-effectiveness and easy modification. The device of the application is independent of the stacker, and only one or a small number of the device is needed to serve all the aisles in the entire warehouse, which greatly reduces the hardware investment and the upgrading cost of old warehouses, and does not require any modification of the existing stacker, making the deployment flexible and convenient.

[0113] 3. High autonomy and system stability. The device of the application can independently complete the whole process of inventory work from pose calculation, motion control to image analysis, only with light-weight interaction of task-level instructions and final results with the upper system, which significantly reduces the dependence on wireless communication bandwidth and the potential risk of communication interference, and enhances the stability and reliability of the inventory work.

[0114] 4. Improve operation efficiency. Since the device is independent of the stacker, when it needs to be maintained due to failure, it can be conveniently taken out from the shelf to the outside of the warehouse for repair by the stacker, which does not affect the normal in-and-out warehouse operation of the stacker, effectively ensuring the operation efficiency of the entire warehouse system.

[0115] It can be understood that the same or similar parts in the above embodiments can be mutually referred to, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0116] It should be noted that in the description of the application, the terms "first", "second", etc. are only for the purpose of description, and cannot be understood as indicating or implying relative importance. In addition, in the description of the application, unless otherwise specified, "a plurality of" means at least two.

[0117] Any procedural or methodological descriptions in flow charts or otherwise described herein can be understood to represent modules, segments, or portions of code that include executable instructions for implementing the specified logical functions or steps, and the scope of preferred embodiments of the present application includes additional implementations in which the steps are performed in an order different from that shown or discussed, including substantially simultaneously or in reverse order, as appropriate to the functionality involved, as will be understood by those skilled in the art to which embodiments of the present application pertain.

[0118] It should be understood that portions of the present application can be implemented in hardware, software, firmware, or combinations thereof. In the above-described embodiments, multiple steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, and as in another embodiment, implementation can be in any one or a combination of the following technologies, which are all well known in the art: discrete logic circuitry having logic gates for implementing logic functions upon an application of data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), and the like.

[0119] Those skilled in the art can understand that all or part of the steps carried out by the above-described embodiments can be instructed by a program to relevant hardware, and the program can be stored in a computer readable storage medium, and when executed, includes one or a combination of steps of the method embodiments.

[0120] In addition, each functional unit in each embodiment of the present application can be integrated in one processing module, or each unit can be physically present separately, or two or more units can be integrated in one module. The above-mentioned integrated module can be realized in the form of hardware or in the form of a software functional module. The integrated module, if realized in the form of a software functional module and sold or used as an independent product, can also be stored in a computer readable storage medium. The above-mentioned storage medium can be a read-only memory, a magnetic disk or an optical disk, etc.

[0121] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0122] Although the embodiments of the present application have been shown and described above, it is understood that the above-described embodiments are exemplary and are not to be construed as limiting the present application, and that changes, modifications, substitutions and variations can be made by those skilled in the art without departing from the scope of the present application.

Claims

1. A stand-alone visual inventory device for a three-dimensional warehouse, characterized in that: include: A device body having a pallet-type base for transporting materials within the high-bay warehouse by material handling equipment; A multi-dimensional motion mechanism provided on the device body, the multi-dimensional motion mechanism comprising a horizontal sliding mechanism and a lifting mechanism; A visual component mounted on the multi-dimensional motion mechanism, for collecting images; as well as A local controller configured in the device body, the local controller is used to: Receive inventory counting instructions from the upper control system; the inventory counting instructions include the task ID, the coordinate information of the location to be counted, and the structural parameters of the warehouse; autonomously controlling the multi-dimensional motion mechanism based on the inventory instruction to adjust the posture of the visual component, and controlling the visual component to perform image acquisition; Perform localized analysis on the collected images to obtain inventory results; as well as Report the inventory results to the upper control system.

2. The device according to claim 1, characterized in that The horizontal sliding mechanism is a planar slide rail arranged along the horizontal direction, and the lifting mechanism is a lifting push rod installed on the moving platform of the planar slide rail.

3. The device according to claim 1, characterized in that The visual component includes a camera and a light source. The camera is installed on a pan-tilt platform. The pan-tilt platform is used to realize the pitch and rotation movement of the camera to cooperate with the multi-dimensional motion mechanism to adjust the shooting angle.

4. The device according to claim 1 or 3, characterized in that The local controller is further configured to: After receiving the inventory instruction, the target shooting posture for avoiding occlusion is calculated according to the information of the cargo location to be inventoried and the preset parameters of the visual component; Wherein, autonomously controlling the multi-dimensional motion mechanism to adjust the posture of the visual component specifically includes: controlling the multi-dimensional motion mechanism to move the visual component to the target shooting posture.

5. The device according to claim 4, characterized in that When performing localized analysis on the collected image, the local controller is specifically used to: Controlling the vision component to capture multiple images from at least two different shooting positions; and The multiple images are fused to determine the inventory result.

6. The device according to claim 5, characterized in that The at least two different shooting positions include: a global observation point for obtaining a top view of the cargo, and a fine observation point for obtaining a front view of the cargo.

7. The device according to claim 1, characterized in that A battery is also provided in the main body of the device for powering the device; a charging interface is provided on the main body of the device for charging the device when the device is transported to a dedicated storage location.

8. The device according to claim 1, characterized in that The local controller is constructed based on a robot operating system or a real-time operating system, and implements the control of the multi-dimensional motion mechanism, the control of the visual component, and the image analysis as independent modules or tasks, and performs information exchange through a preset communication protocol.

9. A standalone visual inventory method for a high-bay warehouse, applied to a standalone visual inventory device transported by material handling equipment to a location near a cargo area to be inventoried. The device comprises a device body, a multi-dimensional motion mechanism disposed on the device body, a visual component mounted on the multi-dimensional motion mechanism, and a local controller disposed within the device body. The multi-dimensional motion mechanism comprises a horizontal sliding mechanism and a lifting mechanism. The method comprises the following steps performed by the local controller: Receive inventory counting instructions from the upper control system; the inventory counting instructions include the task ID, the coordinate information of the location to be counted, and the structural parameters of the warehouse; autonomously controlling the multi-dimensional motion mechanism based on the inventory instruction to adjust the posture of the visual component, and controlling the visual component to perform image acquisition; Perform localized analysis on the collected images to obtain inventory results; as well as Report the inventory results to the upper control system.

10. A standalone visual inventory system, characterized in that: include: The standalone visual inventory device according to any one of claims 1 to 8; Material handling equipment for transporting the standalone visual inventory device within a high-bay warehouse; as well as An upper-level control system, wherein the upper-level control system is used to: issuing a transfer instruction to the material handling equipment to control the material handling equipment to transport the independent visual inventory counting device to the vicinity of the cargo location to be counted; as well as After the independent visual inventory counting device is transported to the vicinity of the cargo location to be counted, an inventory counting instruction is sent to the local controller of the independent visual inventory counting device, and an inventory counting result reported by the local controller is received.

Images