Intelligent storage system access monitoring system based on IoT technology

Abstract

The present disclosure relates to an access monitoring system for an intelligent storage system based on IoT technology, comprising: a monitoring terminal, a plurality of high-definition and ordinary cameras communicatively connected to the monitoring terminal, the plurality of high-definition cameras and a plurality of ordinary cameras being arranged along a track for a storage stacker to travel in the storage system, the ordinary camera being in a normally-on state, and the high-definition camera being in a normally-dormant state; the monitoring terminal receiving a first image taken by the ordinary camera, and inputting a plurality of adjacent first images into a pre-trained decision tree model, to obtain a first recognition result of whether the storage stacker traveling on the track has stopped running output by the decision tree model; if the first recognition result indicates that the storage stacker traveling on the track has stopped running, waking up a target high-definition camera to collect a second image of the stopped storage stacker; performing temporary stop event type recognition according to the second image, obtaining a target temporary stop event type, and executing a monitoring alarm corresponding to the target temporary stop event type.

Description

Technical Field

[0001] The present disclosure relates to the field of warehouse management and Internet of Things (IoT) technology, and in particular to an intelligent warehouse system access monitoring system based on IoT technology. Background Art

[0002] In the context of the development of intelligent manufacturing, stereoscopic warehouses have become an indispensable supporting facility for manufacturing companies. In particular, the vigorous development of my country's automobile, electronic appliances, and e-commerce industries in recent years has driven the rapid development of the entire logistics industry, and has also placed increasing demands on intelligent storage of material boxes. Intelligent warehousing is an important part of factory operation, among which space utilization, high-speed storage and retrieval, and warehouse information management are all relatively important aspects.

[0003] Existing intelligent warehousing systems often face problems such as machine failure, system errors or improper operation during operation, which may cause storage and retrieval of goods to be blocked. This may not only lead to longer storage and retrieval time, but also paralyze the storage and retrieval process. Summary of the invention

[0004] In order to overcome the problem of storage and access congestion caused by machine failure, system error or improper operation in the operation of intelligent storage systems, which may not only lead to long storage and access time but also cause the storage and access process to be paralyzed, the present invention provides an intelligent storage system access monitoring system based on IoT technology.

[0005] According to an embodiment of the present disclosure, an IoT-based intelligent storage system access monitoring system is provided, comprising:

[0006] A monitoring terminal, a plurality of high-definition cameras and a plurality of ordinary cameras in communication with the monitoring terminal, wherein the plurality of high-definition cameras and the plurality of ordinary cameras are arranged along a track for a storage stacker to travel in a storage system, the ordinary cameras are in a normally open state, and the high-definition cameras are in a normally dormant state;

[0007] The monitoring terminal is used to receive the first images captured in real time by the plurality of ordinary cameras, and input the plurality of adjacent first images captured by each ordinary camera into a pre-trained decision tree model, to obtain a first recognition result output by the decision tree model as to whether the storage stacker traveling on the track has stopped running, wherein the first recognition result is a binary classification result obtained by the decision tree model;

[0008] When the first recognition result indicates that any of the storage stacker cranes running on the track has stopped running, a second image of the stopped storage stacker crane is captured by waking up the target high-definition camera;

[0009] The type of the temporary stop event is identified according to the second image to obtain a target temporary stop event type, and a monitoring alarm corresponding to the target temporary stop event type is executed.

[0010] In a possible implementation, when the first recognition result indicates that any of the storage stacker cranes running on the track has stopped running, waking up a target high-definition camera to capture a second image of the stopped storage stacker crane includes:

[0011] In the case where the first recognition result indicates that any of the storage stacker cranes traveling on the track has stopped operating, determining a target high-definition camera for capturing a second image of the storage stacker crane that has stopped operating;

[0012] In the case where the first recognition result indicates that any of the storage stacker cranes traveling on the track has stopped running, determining any two target ordinary cameras from ordinary cameras that capture the first image used to determine the first recognition result;

[0013] Determine the shooting distance between the stopped storage stacker and the target high-definition camera according to the first distance between the target high-definition camera and the target ordinary camera, the baseline distance between the two target ordinary cameras, and the first image acquired;

[0014] The performance parameters of the target high-definition camera are determined according to the shooting distance, and after the target high-definition camera is remotely woken up, the high-definition camera is configured with the performance parameters as the target, and after the configuration is completed, a second image of the stopped storage stacker is collected through the target high-definition camera.

[0015] In a possible implementation, determining the shooting distance between the stopped storage stacker and the target high-definition camera according to the first distance between the target high-definition camera and the target ordinary camera, the baseline distance between the two target ordinary cameras, and the first image acquired includes:

[0016] Determining the number of layers of the pyramid in a process of constructing a scale space pyramid for the first image according to a baseline distance of the baselines between the two target ordinary cameras;

[0017] According to a preset initial standard deviation and a preset standard deviation step size, and according to the number of layers, Gaussian blur is performed multiple times on the first image collected by the target ordinary camera to obtain a scale space pyramid corresponding to the number of layers for each target ordinary camera;

[0018] For each layer in the scale space pyramid corresponding to each target ordinary camera, determine the gray value difference between each pixel and the pixel adjacent to it, and determine whether the pixel is a key pixel in the corresponding first image according to the gray value difference;

[0019] Determine a second distance between the stopped storage stacker and the target ordinary camera according to the key pixel corresponding to each of the target ordinary cameras and the grayscale value of the pixel in the collected first image;

[0020] The shooting distance between the stopped storage stacker and the target high-definition camera is determined according to the second distance and the first distance between the target ordinary camera and the target high-definition camera.

[0021] In a possible implementation, determining the second distance between the stopped storage stacker and the target ordinary camera according to the key pixel corresponding to each of the target ordinary cameras and the grayscale value of the pixel in the collected first image includes:

[0022] Determine, according to the grayscale value of the pixel in the first image captured by each of the target ordinary cameras, the main direction and feature description corresponding to the key pixel corresponding to each of the target ordinary cameras in the first image;

[0023] According to the main directions and feature descriptions of the key pixels corresponding to the two target ordinary cameras, feature point matching is performed on the first images captured by the two target ordinary cameras to obtain matching feature point pairs;

[0024] For each of the matching feature point pairs, determine the horizontal coordinates of the two key pixels in the matching feature point pair in the corresponding first image, and calculate the difference between the horizontal coordinates of the two key pixels;

[0025] Determine a set of target difference values ​​from the difference values, wherein the set of target difference values ​​minimizes the error between the expected image positions of all key pixels in the first image captured by the two target ordinary cameras and the actual key pixel positions calculated based on the set of target difference values ​​and the camera parameters of the target ordinary cameras;

[0026] A second distance between the stopped storage stacker and the target ordinary camera is determined according to the baseline distance of the baseline between the two target ordinary cameras and the target difference.

[0027] In a possible implementation, determining the main direction and feature description corresponding to the key pixel corresponding to each of the target ordinary cameras in the first image according to the grayscale value of the pixel in the first image captured by each of the target ordinary cameras includes:

[0028] According to the grayscale values ​​of pixels in the first image captured by each of the target ordinary cameras, calculate the gradient direction and gradient amplitude corresponding to the grayscale values ​​of pixels in a first neighborhood with a first preset number of pixels centered on the key pixel corresponding to each of the target ordinary cameras;

[0029] According to the gradient directions and gradient magnitudes corresponding to the pixels in the first neighborhood centered on each of the key pixels, construct a first histogram with the sum of the gradient magnitudes as the ordinate and the direction angle of the gradient direction as the abscissa;

[0030] Taking the peak value of the first histogram as the main direction corresponding to the key pixel;

[0031] A second histogram of pixels in a second neighborhood with a second preset number of pixels centered on the key pixel and corresponding to each of the target ordinary cameras in multiple preset directions is calculated, and multiple histogram vectors of the key pixel in the second histogram are connected to form a feature description of the key pixel.

[0032] In a possible implementation, the decision tree model performs the following steps to generate the binary classification result:

[0033] Inputting a plurality of adjacent first images captured by each of the common cameras into a position recognition node, determining the position change of the storage stacker in the plurality of first images through the position recognition node, and obtaining a first decision result;

[0034] In a case where the first decision result indicates that the position change of the storage stacker in the plurality of the first images is less than a first preset change threshold, the plurality of the first images are input into a speed recognition node, and the speed change of the storage stacker in the plurality of the first images is determined by the speed recognition node to obtain a second decision result;

[0035] When the second decision result indicates that the speed change of the storage stacker in the plurality of the first images is greater than or equal to the second preset change threshold, a first recognition result that the storage stacker has not stopped running is obtained, and the plurality of adjacent first images captured by the ordinary camera are returned to be reacquired;

[0036] When the second decision result indicates that the speed change of the storage stacker in the plurality of the first images is less than the second preset change threshold, the plurality of the first images are input into a signal recognition node, and the signal recognition node is used to determine whether the stop signal light of the storage stacker in the plurality of the first images is on, to obtain a third decision result;

[0037] When the third decision result indicates that the stop signal light of the storage stacker in the plurality of the first images is not on, a first recognition result that the storage stacker has not stopped running is obtained, and the plurality of adjacent first images captured by the common camera are returned to be reacquired;

[0038] When the third decision result indicates that the stop light of the warehouse stacker in multiple first images is on, the multiple first images are input into a specific position recognition node, and the specific position recognition node is used to determine whether the warehouse stacker in multiple first images is temporarily stopped at a specific position to obtain the first recognition result.

[0039] In a possible implementation, the decision tree model further performs the following steps to generate the binary classification result:

[0040] When the first decision result indicates that the position change of the warehouse stacker in multiple first images is greater than or equal to the first preset change threshold, a first recognition result that the warehouse stacker has not stopped running is obtained, and the multiple adjacent first images captured by the ordinary camera are returned to be re-acquired.

[0041] In a possible implementation, the temporary stop event type includes at least one of the following: cargo falling, abnormal track occupancy, artificial temporary stop of the stacker, and fire accident.

[0042] In a possible implementation, the storage system includes a multi-layered shelf, the storage stacker, the track arranged between adjacent shelves, and a storage conveying module arranged at one end of the track;

[0043] Wherein, the multi-layered shelf is constructed by column pieces and corbels, and the corbels are fixed on the column pieces;

[0044] The storage transport module is used to perform storage operations on the warehouse goods to be stored, and report the storage information of the warehouse goods to be stored to the monitoring terminal;

[0045] The storage stacker is used to receive the warehousing tasks and outbound tasks assigned by the monitoring terminal based on the warehousing information, and to stack the warehouse goods to be stored on the target shelf layer of the shelf for the warehousing tasks and to perform outbound pickup and delivery of the warehouse goods stacked on the target shelf layer for the outbound tasks.

[0046] In one possible implementation, the warehousing conveying module includes a plurality of conveying line platforms, an appearance detection device, and a code scanning device;

[0047] Wherein, the plurality of conveyor line platforms are independent power sections, corresponding to the loading section, the buffer section and the warehousing section respectively;

[0048] The code scanning device is used to scan the code to store the goods to be stored in the loading section;

[0049] The appearance detection device is used to perform appearance detection on the warehoused goods to be stored;

[0050] The storage stacker is used to fork the storage goods to be stored in the warehouse from the storage section, and stack the storage goods to be stored in the warehouse to the target shelf layer of the shelf;

[0051] The cache section is used to temporarily store the warehouse goods to be stored in the warehouse when there is a backlog of warehouse goods to be stored in the warehouse entry section.

[0052] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0053] By setting the HD camera to a permanent sleep state and waking it up only when needed, the overall energy consumption of the system is significantly reduced. This not only reduces power consumption, but also extends the life of the HD camera, thereby reducing maintenance costs. At the same time, the ordinary camera is always on, and although it consumes less power, it can continuously monitor the storage environment and ensure uninterrupted initial monitoring of the stacker's operating status, achieving a balance between energy efficiency and monitoring needs.

[0054] Furthermore, ordinary cameras capture and transmit images to the monitoring terminal in real time, allowing the system to instantly obtain the operating status of the storage stacker. By analyzing continuous images with a decision tree model, the system can quickly identify whether the stacker has stopped operating, achieving real-time monitoring of abnormal conditions. Once the stacker stops operating, the corresponding high-definition camera can be immediately awakened for more detailed image acquisition, ensuring a quick response from discovering the abnormality to obtaining detailed information.

[0055] Furthermore, the detailed images captured by the high-definition camera make it possible to accurately identify the types of temporary stop events. Through advanced image recognition technology, the specific reasons that caused the stacker to stop operating can be accurately determined, and then corresponding monitoring and alarm measures can be implemented for different types of temporary stop events to ensure that managers can quickly obtain information and take corresponding treatment measures. This allows abnormal situations in warehousing operations to be discovered and handled in a timely manner, avoiding operation delays and reduced efficiency caused by long-term stoppage of the stacker. At the same time, by accurately identifying the types of temporary stop events and executing corresponding monitoring and alarms, it can assist managers in quickly locating problems and taking measures, thereby improving the safety and stability of warehousing operations. Avoid problems such as machine failure, system errors or improper operation that cause storage and retrieval of goods to be blocked, thereby avoiding long storage and retrieval times and causing storage and retrieval process paralysis.

[0056] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0058] Figure 1 The present invention is a flowchart showing a monitoring terminal executing monitoring according to an exemplary embodiment.

[0059] Figure 2 It is a schematic diagram of a warehouse-in conveying module according to an exemplary embodiment. DETAILED DESCRIPTION

[0060] Here, exemplary embodiments will be described in detail, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present disclosure.

[0061] The exemplary embodiment of the present disclosure provides an IoT-based intelligent storage system access monitoring system, including:

[0062] A monitoring terminal, a plurality of high-definition cameras and a plurality of ordinary cameras in communication with the monitoring terminal, wherein the plurality of high-definition cameras and the plurality of ordinary cameras are arranged along a track for a storage stacker to travel in a storage system, the ordinary cameras are in a normally open state, and the high-definition cameras are in a normally dormant state;

[0063] In the disclosed embodiment, the monitoring terminal serves as the control center of the system and is responsible for receiving image data from each camera, processing and analyzing it, and issuing control instructions. High-definition cameras have high resolution and accurate image capture capabilities, but for energy-saving reasons, they are usually in a dormant state and are only awakened when needed. Ordinary cameras have moderate resolution, low power consumption, and are in a normally open state. They are used to continuously monitor the storage area and capture basic image information. The camera layout is evenly arranged along the track on which the stacker travels in the storage system to ensure coverage without blind spots and to clearly capture the operating status of the stacker.

[0064] See also Figure 1 As shown, the monitoring terminal is used to, in step S11, receive the first images taken in real time by multiple ordinary cameras, and input multiple adjacent first images collected by each ordinary camera into a pre-trained decision tree model to obtain a first recognition result output by the decision tree model as to whether the storage stacker traveling on the track has stopped running.

[0065] Wherein, the first recognition result is a binary classification result obtained by the decision tree model;

[0066] In the disclosed embodiment, an ordinary camera captures images of the storage area (first images) in real time, and transmits these data to the monitoring terminal in real time. Preliminary analysis: The monitoring terminal inputs multiple adjacent first images captured by each ordinary camera into a pre-trained decision tree model. The decision tree model determines whether the stacker has stopped running by comparing the changes in the position, shape and other features of the stacker in the image. First recognition result: The model outputs a binary classification result, that is, whether the stacker has stopped running. This step is intended to quickly screen out possible abnormal conditions and reduce unnecessary wake-ups of high-definition cameras.

[0067] In step S12, when the first recognition result indicates that any of the storage stacker cranes running on the track has stopped running, a second image of the stopped storage stacker crane is captured by waking up the target high-definition camera;

[0068] In the disclosed embodiment, when the decision tree model recognizes that any stacker has stopped operating, the monitoring terminal will automatically wake up the corresponding high-definition camera to collect detailed images (second images) of the stopped stacker and its surrounding environment.

[0069] In step S13, the type of the temporary stop event is identified according to the second image to obtain the target temporary stop event type, and a monitoring alarm corresponding to the target temporary stop event type is executed.

[0070] In the disclosed embodiment, deep learning or other advanced image recognition technologies are used to conduct an in-depth analysis of the second image captured by the high-definition camera to identify the specific reasons that caused the stacker to stop operating (such as falling goods, mechanical failure, human intervention, etc.), that is, the target temporary stop event type.

[0071] Then, according to the type of temporary stoppage event identified, the monitoring terminal will trigger the corresponding alarm mechanism, such as sound alarm, email notification, SMS reminder, etc., to notify the management personnel or maintenance personnel. Based on the alarm information, the management personnel will quickly assess the situation and initiate the corresponding emergency plan, such as remote guidance of operation, dispatching maintenance personnel to the site, etc., to ensure that the storage operation returns to normal as soon as possible.

[0072] For example, suppose that in a smart warehousing system, a stacker suddenly stops running while performing a cargo handling task. At this time, ordinary cameras arranged along the track continue to capture images and transmit data to the monitoring terminal. The decision tree model of the monitoring terminal determines that the stacker has stopped running by comparing the position changes of the stacker in several consecutive frames of images.

[0073] Subsequently, the monitoring terminal automatically woke up the nearby high-definition camera to take high-definition photos of the stopped stacker and its surroundings. The high-definition images showed that there were scattered goods under the stacker, indicating that the stop may have been caused by falling goods.

[0074] Based on the image recognition results, the monitoring terminal immediately triggers the alarm mechanism and sends an emergency notice to the management personnel, with high-definition images attached as evidence. After receiving the alarm, the management personnel quickly assess the situation and dispatch maintenance personnel to the site to deal with the cargo drop problem. At the same time, they remotely guide other stackers to avoid the fault area to ensure the overall smoothness of the warehouse operation.

[0075] The above technical solution significantly reduces the overall energy consumption of the system by setting the HD camera to a permanent sleep state and waking it up only when needed. It not only reduces power consumption, but also extends the service life of the HD camera, thereby reducing maintenance costs. At the same time, the ordinary camera is always on, and although it consumes less power, it can continuously monitor the storage environment and ensure uninterrupted initial monitoring of the stacker's operating status, achieving a balance between energy efficiency and monitoring needs.

[0076] Furthermore, ordinary cameras capture and transmit images to the monitoring terminal in real time, allowing the system to instantly obtain the operating status of the storage stacker. By analyzing continuous images with a decision tree model, the system can quickly identify whether the stacker has stopped operating, achieving real-time monitoring of abnormal conditions. Once the stacker stops operating, the corresponding high-definition camera can be immediately awakened for more detailed image acquisition, ensuring a quick response from discovering the abnormality to obtaining detailed information.

[0077] Furthermore, the detailed images captured by the high-definition camera make it possible to accurately identify the types of temporary stop events. Through advanced image recognition technology, the specific reasons that caused the stacker to stop operating can be accurately determined, and then corresponding monitoring and alarm measures can be implemented for different types of temporary stop events to ensure that managers can quickly obtain information and take corresponding treatment measures. This allows abnormal situations in warehousing operations to be discovered and handled in a timely manner, avoiding operation delays and reduced efficiency caused by long-term stoppage of the stacker. At the same time, by accurately identifying the types of temporary stop events and executing corresponding monitoring and alarms, it can assist managers in quickly locating problems and taking measures, thereby improving the safety and stability of warehousing operations. Avoid problems such as machine failure, system errors or improper operation that cause storage and retrieval of goods to be blocked, thereby avoiding long storage and retrieval times and causing storage and retrieval process paralysis.

[0078] In a possible implementation, in step S12, when the first recognition result indicates that any of the storage stacker cranes traveling on the track has stopped running, waking up a target high-definition camera to capture a second image of the stopped storage stacker crane includes:

[0079] In step S121, when the first recognition result indicates that any of the storage stacker cranes running on the track has stopped running, a target high-definition camera for capturing a second image of the stopped storage stacker crane is determined;

[0080] In the disclosed embodiment, when the monitoring terminal recognizes that any storage stacker on the track has stopped running through the pre-trained decision tree model, one or more high-definition cameras are determined to collect detailed images (i.e., second images) of the stopped stacker. The implementation of this process depends on the layout of the high-definition cameras and the accurate determination of the stacker's stopped position.

[0081] Position positioning: First, based on the location information of the stopped stacker output by the decision tree model (which may be determined by coordinates or feature points in the image), combined with the layout diagram of the HD camera, determine whether the shooting range of the HD camera can cover the location.

[0082] Optimal selection: Among the HD cameras that can cover the stopping position, the optimal camera is selected as the target HD camera based on multiple factors, such as the distance between the HD camera and the stopping position and whether there are any obstructions.

[0083] Wake up and collect: The final target HD camera will be woken up and start collecting detailed images of the stopped stacker crane and transmit these images back to the monitoring terminal for further analysis.

[0084] For example, suppose that in an intelligent warehousing system, there are 5 high-definition cameras evenly arranged along the track, and each camera can cover a certain range of areas. When the monitoring terminal recognizes that the stacker at the third position has stopped running through the decision tree model, it first determines that the second, third, and fourth high-definition cameras can cover this position based on the layout diagram of the high-definition cameras. Then, the third high-definition camera is closest to the stop position and there is no obstruction, so it is selected as the target high-definition camera. Subsequently, the high-definition camera is awakened and starts to collect detailed images of the stopped stacker and transmit the images back to the monitoring terminal.

[0085] In step S122, when the first recognition result indicates that any of the storage stacker cranes running on the track has stopped running, any two target ordinary cameras are determined from ordinary cameras that capture the first image used to determine the first recognition result;

[0086] In the disclosed embodiment, after confirming that the storage stacker has stopped running, in order to obtain more comprehensive on-site information or perform cross-verification, the system may need to determine any two target ordinary cameras from the ordinary cameras used to collect the first recognition result. The implementation of this process depends on the understanding of the ordinary camera layout and the surrounding environment of the stacker's stop position.

[0087] Coverage analysis: First, analyze which ordinary cameras around the stopping position can capture relevant images. These cameras may provide different perspectives or broader background information before and after the stacker stops.

[0088] Selection criteria: When selecting a target ordinary camera, factors such as the camera's viewing angle, shooting quality, and relative position relationship to the stop position are considered. For example, two cameras with complementary viewing angles are selected to observe the stacker's stop from different angles.

[0089] In the above-mentioned intelligent storage system, when the stacker at the third position stops running, in addition to waking up the third high-definition camera, two target cameras are selected from the ordinary cameras. These two cameras are the second ordinary camera located in front of the stop position and the fifth ordinary camera located on the side of the stop position. The second camera can capture the running status of the stacker before it stops and possible reasons for the stop (such as obstacles in front), while the fifth camera provides a clear view of the side of the stacker when it stops. The image data of these two cameras can provide a more comprehensive understanding of the situation when the stacker stops.

[0090] In step S123, the shooting distance between the stopped storage stacker and the target high-definition camera is determined according to the first distance between the target high-definition camera and the target ordinary camera, the baseline distance between the two target ordinary cameras, and the first image acquired;

[0091] In the disclosed embodiment, the actual shooting distance between the stopped storage stacker and the target HD camera is indirectly calculated by using the known physical distance (the first distance between the target HD camera and the target ordinary camera, the baseline distance between the two target ordinary cameras) and the first image collected (taken by the ordinary camera) using triangulation or image matching technology. This method relies on geometric relationships and image processing technology to estimate unknown distances through known fixed parameters and feature points in the image.

[0092] Suppose there are two ordinary cameras A and B, with a baseline distance of 10 meters between them, and their respective distances from the target HD camera C (first distance) are 5 meters and 8 meters respectively. Camera A captures a distinct feature point P of the storage stacker, which is represented by a specific pixel position in the image. The same feature point P can also be observed through camera B, but the position is slightly offset. Using the position information of point P in these two images, combined with the known baseline distance and the positional relationship between the cameras, the actual distance from point P to camera C (target HD camera), i.e. the shooting distance, can be calculated by triangulation.

[0093] In step S124, the performance parameters of the target high-definition camera are determined according to the shooting distance, and after the target high-definition camera is remotely woken up, the high-definition camera is configured with the performance parameters as the target, and after the configuration is completed, the second image of the stopped storage stacker is collected through the target high-definition camera.

[0094] In the disclosed embodiment, based on the obtained shooting distance, combined with the relationship between the performance characteristics of the high-definition camera (such as focal length, aperture, resolution, etc.) and the shooting distance, the performance parameters most suitable for the current shooting scene are selected or calculated. These parameters directly affect the image clarity, depth of field, exposure and other quality factors.

[0095] For example, assume that the maximum resolution of the target HD camera C is 4K and has an adjustable focal length function. According to the shooting distance (e.g., 8 meters), a suitable focal length value, such as 30mm, can be selected to ensure that a sufficiently clear and detailed image can be captured at a distance of 8 meters. At the same time, according to the ambient light conditions, the aperture size and ISO value are adjusted to obtain the best exposure effect.

[0096] A control signal is sent to the target HD camera through a network communication protocol (such as TCP / IP) to wake up its working state and transmit performance parameter configuration instructions. After receiving the instruction, the camera adjusts itself according to the parameters, such as adjusting the focal length, aperture, etc., to adapt to the current shooting requirements. Through the control center of the warehouse management system, a wake-up instruction is sent to the target HD camera C, and the configuration parameters (focal length 30mm, aperture F / 4, ISO 200, etc.) are attached. After receiving the instruction, camera C automatically adjusts to the specified parameter state and prepares for high-quality image acquisition.

[0097] Furthermore, after the configuration of the high-definition camera is completed, the high-definition and high-sensitivity characteristics are used to collect images of the stopped storage stacker. The collected images will be used for subsequent image analysis, recognition or storage. After the configuration is completed, the high-definition camera C immediately shoots the storage stacker and obtains a clear image with a 4K resolution. This image shows the detailed structure of the stacker, the stacking of goods and other key information.

[0098] In a possible implementation, in step S123, determining the shooting distance between the stopped storage stacker and the target high-definition camera according to the first distance between the target high-definition camera and the target ordinary camera, the baseline distance of the baseline between the two target ordinary cameras, and the acquired first image includes:

[0099] In step S1231, the number of layers of the pyramid in the process of constructing the scale space pyramid for the first image is determined according to the baseline distance of the baselines between the two target ordinary cameras;

[0100] It is understandable that different levels of blurring can preserve the main features of the image at multiple scales. The baseline distance (i.e., the distance between two ordinary cameras) determines the minimum detail difference that can be recognized in the image at different viewing angles, which in turn affects the number of pyramid layers. The longer the baseline, the finer the depth difference that can be recognized, so the number of pyramid layers may be more.

[0101] For example, assuming that the baseline distance between two target ordinary cameras is 2 meters, for example, for every 0.5 meter increase in the baseline distance, the number of pyramid layers increases by one. Therefore, the number of pyramid layers corresponding to the baseline distance of 2 meters is 4 (assuming that there is also a base layer when the baseline is 0 meters).

[0102] In step S1232, according to a preset initial standard deviation and a preset standard deviation step size, Gaussian blur is performed multiple times on the first image collected by the target ordinary camera according to the number of layers, so as to obtain a scale space pyramid corresponding to the number of layers for each target ordinary camera;

[0103] Among them, Gaussian blur is an image smoothing technology that calculates the transformation value of each pixel in the image through the Gaussian function (normal distribution) to achieve image blur processing. In the construction of the scale space pyramid, by continuously increasing the standard deviation of the Gaussian blur, the resolution and details of the image can be gradually reduced, thereby generating a multi-scale image pyramid.

[0104] For example, presetting the initial standard deviation: Assume that the initial standard deviation σ_0 = 0.5, which means that the initial blur is low.

[0105] Preset standard deviation step size: Assume that the step size Δσ = 0.5, which means that the construction of each layer of the pyramid will increase the standard deviation of 0.5 on the basis of the previous layer.

[0106] Construction process: Layer 0 (base layer): Use the original image directly without Gaussian blur (or can be regarded as blur with σ=0).

[0107] Layer 1: Gaussian blur the original image using σ=σ_0=0.5.

[0108] Layer 2: Based on layer 1, Gaussian blur is performed using σ=σ_0+Δσ=1.0.

[0109] Layer 3: Based on layer 2, Gaussian blur is performed using σ=σ_0+2*Δσ=1.5.

[0110] Layer 4 (the highest layer when the baseline distance is 2 meters according to the previous example): Based on layer 3, Gaussian blur is performed using σ=σ_0+3*Δσ=2.0.

[0111] For the first images captured by the two target ordinary cameras, the above process is repeated to construct their own scale space pyramids.

[0112] Through the above steps, the number of pyramid layers can be determined according to the baseline distance between two ordinary cameras, and a multi-scale image pyramid can be constructed by performing multiple Gaussian blurs on the collected images.

[0113] In step S1233, for each layer in the scale space pyramid corresponding to each target ordinary camera, a gray value difference between each pixel and the pixel adjacent to it is determined, and whether the pixel is a key pixel in the corresponding first image is determined according to the gray value difference;

[0114] In the disclosed embodiment, for each layer in the scale space pyramid, the gray value difference between each pixel and its adjacent pixels is calculated. These differences reflect the degree of local change of the image in the area. Generally, pixels in feature areas such as edges or corners will show large gray value changes and are therefore considered key pixels. These key pixels play an important role in subsequent depth estimation or distance measurement.

[0115] For example, suppose there is a pixel P with a grayscale value of G_P on a certain level of the scale space pyramid. Calculate the grayscale value difference between P and its four neighboring pixels above, below, left, and right (respectively denoted as P_up, P_down, P_left, P_right), namely |G_P - G_P_up|, |G_P - G_P_down|, |G_P - G_P_left|, |G_P - G_P_right|. If any of these differences exceeds the preset threshold T (indicating that there is a significant grayscale change between the pixel and its neighboring pixels), the pixel P is marked as a key pixel.

[0116] In step S1234, a second distance between the stopped storage stacker and the target ordinary camera is determined according to the key pixel corresponding to each of the target ordinary cameras and the grayscale value of the pixel in the collected first image;

[0117] In the disclosed embodiment, using the key pixels identified from each target ordinary camera and the grayscale values ​​of these pixels in the first image, stereo vision technology (such as triangulation) can be used to estimate the distance between the stacker and the camera. This step usually involves matching corresponding key pixels in the two camera views and calculating depth information based on these matching points.

[0118] For example, suppose there are two target ordinary cameras A and B, each of which captures a first image containing a stacker. First, key pixels are identified in each of the two images, and the corresponding matching points in the A and B images are found through a matching algorithm (such as feature descriptor matching). Then, using the coordinates of these matching points and the known parameters of cameras A and B (such as focal length, baseline distance, etc.), the second distance between the stacker and each camera (represented as d_A and d_B, respectively) is calculated by triangulation. If the two cameras have similar viewing angles and the stacker is fixed, these two distances may be very close.

[0119] In step S1235, the shooting distance between the stopped storage stacker and the target high-definition camera is determined according to the second distance and the first distance between the target ordinary camera and the target high-definition camera.

[0120] In the disclosed embodiment, the shooting distance between the stacker and the high-definition camera is calculated through a simple geometric relationship by using the known second distance between the stacker and the ordinary camera (such as d_A or d_B, assuming that they are similar and interchangeable) and the first distance between the ordinary camera and the high-definition camera (denoted as D_AB).

[0121] For example, let D_AB be the first distance between camera A (or B, since they are at the same distance from the HD camera) and the HD camera. Since the second distance between the stacker and camera A (or B) is known to be d_A (or d_B), and these cameras are all located on the same plane (or approximately on the same plane), the shooting distance between the stacker and the HD camera can be calculated by simple proportions or similar triangle principles. For example, if D_AB is much larger than d_A (or d_B), the shooting distance can be approximated as D_AB plus a small correction, which depends on the specific position and angle of the HD camera relative to the ordinary camera. However, in most cases, since the relative positions between the cameras are fixed, this correction can be pre-calculated and stored as a system parameter. Therefore, the shooting distance can be simply expressed as D_AB plus (or minus) a fixed offset.

[0122] In a possible implementation, in step S1234, determining the second distance between the stopped storage stacker and the target ordinary camera according to the key pixel corresponding to each of the target ordinary cameras and the grayscale value of the pixel in the collected first image includes:

[0123] Determine, according to the grayscale value of the pixel in the first image captured by each of the target ordinary cameras, the main direction and feature description corresponding to the key pixel corresponding to each of the target ordinary cameras in the first image;

[0124] Main direction determination: For each key pixel (feature point), first calculate the gradient direction and magnitude of the pixels in its neighborhood. Then, count the histogram of these gradient directions and select the direction corresponding to the peak value in the histogram as the main direction of the key pixel. The determination of the main direction makes the feature description rotationally invariant.

[0125] Feature description: Based on the main direction of the key pixel, a certain number and size of pixel blocks are selected in its neighborhood (such as a 16x16 neighborhood, which is further subdivided into 4x4 sub-blocks), and the gradient direction and magnitude of the pixels in each sub-block are calculated. Then, these gradient directions and magnitudes are rotated relative to the main direction and counted as descriptors (such as SIFT descriptors). A descriptor is usually a high-dimensional vector that uniquely identifies a feature point and its surrounding image content.

[0126] Assume that in the first image of the target ordinary camera A, a key pixel P is identified, and the gradient direction and magnitude of the surrounding pixels are known. The histogram of these gradient directions is calculated, and it is found that the direction corresponding to the main peak is θ degrees. Therefore, the main direction of P is determined to be θ degrees.

[0127] Next, a 16x16 neighborhood is selected with P as the center and subdivided into 4x4 sub-blocks. For each sub-block, the system calculates the gradient direction and magnitude of the internal pixels and rotates them relative to θ degrees. Finally, these corrected gradient directions and magnitudes are counted as a 128-dimensional SIFT descriptor (because there are 4x4 sub-blocks, each with 8 directions).

[0128] According to the main directions and feature descriptions of the key pixels corresponding to the two target ordinary cameras, feature point matching is performed on the first images captured by the two target ordinary cameras to obtain matching feature point pairs;

[0129] In the disclosed embodiment, the key pixels and their feature descriptions in the first images of the two target ordinary cameras (such as A and B) are determined, and the system can find matching feature point pairs by comparing these descriptors. This usually involves using some similarity metric (such as Euclidean distance, Hamming distance, etc.) to compare the descriptors and select the most similar descriptor pair as the matching point pair.

[0130] In order to improve the accuracy and efficiency of matching, some optimization algorithms can be used, such as fast nearest neighbor search (FLANN), ratio test, etc. Ratio test is a commonly used method, which requires that the similarity difference between the descriptor of the matching point pair and the descriptor of the suboptimal matching point pair is large enough to reduce the possibility of mismatching.

[0131] For example, the key pixels and their SIFT descriptors in the first images of cameras A and B are obtained respectively. For a key pixel P_A and its descriptor D_A in camera A, the system searches for the descriptor D_B that is most similar to D_A in the descriptor set of camera B. Assuming that the Euclidean distance between D_B and D_A is the smallest, but to ensure that this is not a false match, the system also checks the distance between the second best match (i.e., the second closest descriptor) and D_A, and calculates the distance ratio between them. If this distance ratio is less than a preset threshold (such as 0.8), D_A and D_B are considered to be matched, and their corresponding feature points P_A and P_B are regarded as a pair of matched feature point pairs.

[0132] For each of the matching feature point pairs, determine the horizontal coordinates of the two key pixels in the matching feature point pair in the corresponding first image, and calculate the difference between the horizontal coordinates of the two key pixels;

[0133] In the disclosed embodiment, for each pair of matching feature points, the system extracts the horizontal coordinates (x coordinates) of the two key pixels in their respective first images. Then, the difference between the two horizontal coordinates is calculated, which represents the horizontal displacement of the same physical point on the image plane from one camera perspective to another camera perspective.

[0134] For example, suppose there is a pair of matching feature points, where the horizontal coordinate of the key pixel P_A in camera A is x_A, and the horizontal coordinate of the corresponding key pixel P_B in camera B is x_B. Then, the horizontal coordinate difference of this pair of matching feature points is Δx = x_B - x_A.

[0135] Determine a set of target difference values ​​from the difference values, wherein the set of target difference values ​​minimizes the error between the expected image positions of all key pixels in the first image captured by the two target ordinary cameras and the actual key pixel positions calculated based on the set of target difference values ​​and the camera parameters of the target ordinary cameras;

[0136] In the disclosed embodiment, the horizontal coordinate difference calculated directly by matching the feature point pairs may not be optimal. Therefore, an optimization method is used to determine a set of target difference values ​​that can minimize the error between the expected image position and the actual image position of all key pixels calculated based on these difference values ​​and camera parameters. This usually involves minimizing an error function, such as the reprojection error.

[0137] For example, an iterative optimization algorithm (such as the least squares method, Levenberg-Marquardt algorithm, etc.) is used to find the optimal target difference set. In this process, the expected position of each key pixel in the image of the other camera is calculated based on the current difference estimate, combined with the intrinsic parameters (focal length, optical center, etc.) and extrinsic parameters (rotation matrix, translation vector, etc.) of cameras A and B. Then, the difference between these expected positions and the actual matching positions is compared, and the difference estimate is adjusted to minimize this difference.

[0138] A second distance between the stopped storage stacker and the target ordinary camera is determined according to the baseline distance of the baseline between the two target ordinary cameras and the target difference.

[0139] In the disclosed embodiment, these differences and the baseline distance between the cameras (i.e., the horizontal distance between the optical centers of the two cameras) can be used to estimate the second distance between the stacker and the camera. This is usually based on the principle of triangulation, that is, using the image position difference of the same physical point observed by the two cameras and the geometric relationship between the cameras to calculate the distance from the physical point to the camera.

[0140] For example, assume that the baseline distance between cameras A and B is b, and the target difference Δx is known. Based on the principle of triangulation, the distance d from the stacker to camera A (or B, since they are equidistant) can be calculated using the following formula:

[0141] d = b×f / Δx, where f is the focal length of the camera.

[0142] In a possible implementation, in step S11, determining the main direction and feature description corresponding to the key pixel corresponding to each of the target ordinary cameras in the first image according to the grayscale value of the pixel in the first image captured by each of the target ordinary cameras includes:

[0143] In step S111, according to the grayscale values ​​of the pixels in the first image captured by each of the target ordinary cameras, the gradient direction and gradient amplitude corresponding to the grayscale values ​​of the pixels in a first neighborhood with a first preset number of pixels centered on the key pixel corresponding to each of the target ordinary cameras are calculated;

[0144] In the disclosed embodiment, for each key pixel, a neighborhood (referred to as the first neighborhood) is first defined around the pixel and includes a first preset number of pixels. Then, the grayscale value gradient of each pixel in the neighborhood is calculated, including the gradient direction and the gradient amplitude. The gradient direction indicates the direction in which the grayscale value changes fastest, while the gradient amplitude indicates the intensity of the change.

[0145] For example, assume that there is a key pixel P in the first image captured by the target ordinary camera, and its grayscale value is I(P). With P as the center, define a 5x5 neighborhood as the first neighborhood. For each pixel q in the neighborhood, calculate its gradient direction and gradient magnitude relative to P. The gradient direction theta_q and gradient magnitude M_q can be calculated by the following formula:

[0146] theta_q =arctan[G_y(q) / G_x(q)]

[0147] M_q =sqrt[G_x(q)^2 + G_y(q)^2]

[0148] Among them, G_x(q) and G_y(q) are the Sobel gradient values ​​of pixel q in the x direction and y direction respectively, sqrt is the square root in mathematical calculation, and arctan is the inverse tangent function in mathematical calculation.

[0149] In step S112, a first histogram is constructed with the sum of the gradient magnitudes as the ordinate and the direction angle of the gradient direction as the abscissa according to the gradient directions and the gradient magnitudes corresponding to the pixels in the first neighborhood centered on each of the key pixels;

[0150] In the disclosed embodiment, first, the gradient direction is quantized into 36 intervals (10 degrees per interval). Then, all pixels q in the first neighborhood of P are traversed, and the interval to which they belong is determined according to the gradient direction theta_q of q, and the gradient magnitude M_q of q is accumulated to the ordinate of the interval. Finally, a histogram of 36 entries is obtained, and the value of each entry represents the sum of the gradient magnitudes in the corresponding direction interval.

[0151] In step S113, the peak value of the first histogram is used as the main direction corresponding to the key pixel;

[0152] In the first histogram, the peak value represents the main direction of the pixel gradient around the key pixel. Therefore, the direction corresponding to the peak value is selected as the main direction of the key pixel. If there are multiple peak values ​​and the peak values ​​are close, any one of them can be selected as the main direction.

[0153] In step S114, a second histogram of pixels in a second neighborhood with a second preset number of pixels centered on the key pixel and corresponding to each of the target ordinary cameras is calculated in multiple preset directions, and multiple histogram vectors of the key pixel in the second histogram are connected to form a feature description of the key pixel.

[0154] In the disclosed embodiment, after determining the main direction of the key pixel, the system needs to calculate the second histogram of pixels in a neighborhood (referred to as the second neighborhood) centered on the pixel and containing a second preset number of pixels in multiple preset directions. These preset directions are usually related to the main direction and may be a series of equally spaced directions around the main direction. Then, the system connects the histogram vectors in these directions to form a high-dimensional vector as a feature description of the key pixel.

[0155] The detailed implementation includes: Selecting preset directions: Assume that 8 equally spaced directions with the main direction as the center and extending 15 degrees to the left and right (or any other angle range) are selected as preset directions. If the main direction is 175 degrees, then these directions may be 160 degrees, 165 degrees, ..., 185 degrees, 190 degrees (note that the direction cycle is handled, that is, 0 degrees and 360 degrees are equal).

[0156] Calculate the second histogram: For each pixel in the second neighborhood, assign these pixels to the nearest preset direction according to their gradient direction, and accumulate the gradient magnitude in each direction. In this way, the value of a histogram entry can be calculated for each preset direction.

[0157] Construct feature description: Connect the histogram entry values ​​in 8 preset directions into an 8-dimensional vector. This vector is the feature description of the key pixel P.

[0158] In a possible implementation, the decision tree model performs the following steps to generate the binary classification result:

[0159] In step S101, a plurality of adjacent first images captured by each of the common cameras are input into a position recognition node, and the position change amount of the storage stacker in the plurality of first images is determined by the position recognition node to obtain a first decision result;

[0160] The location recognition node receives multiple adjacent first images collected by ordinary cameras and uses image processing technology (such as feature matching, optical flow method, etc.) to analyze the location information of the warehouse stacker in these images. By comparing the position coordinates of the stacker in adjacent images, the position change is calculated. The purpose of this step is to preliminarily determine whether the stacker is stationary or has only slight movement.

[0161] Assume that the location recognition node receives three consecutive images (image A, image B, image C), and the outline of the warehouse stacker can be clearly identified in each image. Through the image processing algorithm, the position coordinates (such as the center point coordinates) of the stacker in images A, B, and C are extracted respectively. Then, the difference between the position coordinates of the stacker between adjacent images is calculated to obtain the position change. If the position change between image B and image A, and between image C and image B are both less than the preset first preset change threshold (such as 5 cm), it is preliminarily judged that the stacker may be stationary or moving slowly.

[0162] In step S102, when the first decision result indicates that the position change of the storage stacker in the plurality of the first images is less than a first preset change threshold, the plurality of the first images are input into a speed recognition node, and the speed change of the storage stacker in the plurality of the first images is determined by the speed recognition node to obtain a second decision result;

[0163] After determining in the first step that the stacker position change is less than the first preset change threshold, the plurality of first images are input into the speed recognition node. The speed recognition node uses the timestamp information and the position change to calculate the stacker speed change through a simple physical formula (such as speed = displacement / time). The purpose of this step is to further confirm whether the stacker has actually stopped running, because even if the position change is small, it may still be moving slowly.

[0164] Continuing with the above example, if the position identification node determines that the stacker position change is less than the first preset change threshold, the images and their corresponding timestamp information are passed to the speed identification node. The speed identification node first confirms the time interval of image acquisition (such as one image per second), and then calculates the speed change of the stacker based on the position change and the time interval. If the calculated speed change is greater than or equal to the second preset change threshold (such as 0.1 m / s), it is considered that the stacker has not actually stopped running, but is in a slow moving state.

[0165] In step S103, when the second decision result indicates that the speed change of the storage stacker in the plurality of the first images is greater than or equal to the second preset change threshold, a first recognition result that the storage stacker has not stopped running is obtained, and the plurality of adjacent first images captured by the ordinary camera are returned to be reacquired;

[0166] In the implementation of the present disclosure, after the speed recognition node determines that the stacker speed change is greater than or equal to the second preset change threshold, a first recognition result is obtained that the warehouse stacker has not stopped running. In order to obtain more accurate stacker status information, return to re-acquire multiple adjacent first images captured by the ordinary camera, and repeat the above position recognition and speed recognition process until it is confirmed that the stacker has indeed stopped running.

[0167] If the speed recognition node determines that the stacker crane speed change is greater than or equal to the second preset change threshold, it can output a prompt "the warehouse stacker crane has not stopped running" and automatically trigger the process of re-acquiring images. The new image sequence will be processed by the position recognition node and the speed recognition node again until it is finally confirmed that the stacker crane has stopped running.

[0168] In step S104, when the second decision result indicates that the speed change of the storage stacker in the plurality of the first images is less than the second preset change threshold, the plurality of the first images are input into a signal recognition node, and the signal recognition node is used to determine whether the stop signal light of the storage stacker in the plurality of the first images is on, to obtain a third decision result;

[0169] In the implementation of the present disclosure, after the speed recognition node outputs the second decision result of "the speed change is less than the second preset change threshold", the plurality of first images are input into the signal recognition node. The signal recognition node uses image processing technology (such as color recognition, shape matching, etc.) to detect whether the stop signal light of the storage stacker in the image is on.

[0170] First, the image is preprocessed, such as grayscale and filtering, to improve the accuracy of signal recognition.

[0171] Color Recognition: If the stop light is red, the signal recognition node will focus on identifying red areas in the image and check whether these areas match the expected shape and position of a stop light.

[0172] Shape matching: Use template matching or other shape recognition algorithms to confirm that the detected red area is indeed a stop light.

[0173] If the signal recognition node detects a lit stop light in multiple images, it outputs the third decision result of "the stop light is on"; otherwise, it outputs the result of "the stop light is not on".

[0174] In step S105, when the third decision result indicates that the stop signal light of the storage stacker in the plurality of the first images is not on, a first recognition result that the storage stacker has not stopped running is obtained, and the plurality of adjacent first images captured by the ordinary camera are returned to be re-acquired;

[0175] If the third decision result indicates that the stop signal light is not on, it is considered that the storage stacker may continue to run soon for some reason. At this time, the first recognition result of "the storage stacker has not stopped running" is obtained, and the adjacent multiple first images captured by the ordinary camera are returned to re-acquire to perform the entire recognition process again.

[0176] In step S106, when the third decision result indicates that the stop light of the warehouse stacker in multiple first images is on, the multiple first images are input into a specific position recognition node, and the specific position recognition node is used to determine whether the warehouse stacker in the multiple first images is temporarily stopped at a specific position to obtain the first recognition result.

[0177] After confirming that the stop signal light of the storage stacker is on, the plurality of first images are input into the specific position recognition node. The specific position recognition node uses a more sophisticated image processing technology to determine whether the stacker is temporarily stopped at a predetermined specific position.

[0178] Position coordinate extraction: First, the precise position coordinates of the stacker (such as the coordinates of the center point or a specific reference point) are extracted from the image.

[0179] Position comparison: These coordinates are compared with the preset specific position coordinates, taking into account a certain tolerance range (such as a few centimeters or pixels) to allow for small deviations between the actual position and the theoretical position.

[0180] Result judgment: If the position coordinates of the stacker fall within the tolerance range of the specific position, the specific position identification node outputs the first identification result of "the stacker is temporarily stopped at a specific position"; otherwise, it outputs the result of "the stacker is not temporarily stopped at a specific position" (although in this case, further troubleshooting or operator intervention may be required because the stop signal light is on).

[0181] In a possible implementation, the decision tree model further performs the following steps to generate the binary classification result:

[0182] In step S107, when the first decision result indicates that the position change of the warehouse stacker in multiple first images is greater than or equal to the first preset change threshold, a first recognition result that the warehouse stacker has not stopped running is obtained, and the adjacent multiple first images captured by the ordinary camera are returned to be re-acquired.

[0183] In the disclosed embodiment, after receiving a plurality of adjacent first images, the position recognition node calculates the position change of the stacker by comparing the position information (such as the center point coordinates) of the stacker in these images.

[0184] If the calculated position change is greater than or equal to a preset first preset change threshold (for example, a set minimum detectable displacement, such as 10 cm), the position identification node outputs a decision result indicating that the stacker has moved significantly during this period of time.

[0185] If the decision result indicates that the position change of the stacker is greater than or equal to the first preset change threshold, the first recognition result of "the storage stacker has not stopped running" is immediately obtained. The image is acquired again.

[0186] In order to continuously monitor the status of the stacker or wait for it to stop running, return to reacquire the adjacent first multiple images captured by the ordinary camera. For example, set a short delay (such as a few seconds) and then start the image acquisition and position recognition process again.

[0187] In a possible implementation, the temporary stop event type includes at least one of the following: cargo falling, abnormal track occupancy, artificial temporary stop of the stacker, and fire accident.

[0188] In a possible implementation, the storage system includes a multi-layered shelf, the storage stacker, the track arranged between adjacent shelves, and a storage conveying module arranged at one end of the track;

[0189] Wherein, the multi-layered shelf is constructed by column pieces and corbels, and the corbels are fixed on the column pieces;

[0190] The storage transport module is used to perform storage operations on the warehouse goods to be stored, and report the storage information of the warehouse goods to be stored to the monitoring terminal;

[0191] The storage stacker is used to receive the warehousing tasks and outbound tasks assigned by the monitoring terminal based on the warehousing information, and to stack the warehouse goods to be stored on the target shelf layer of the shelf for the warehousing tasks and to perform outbound pickup and delivery of the warehouse goods stacked on the target shelf layer for the outbound tasks.

[0192] In the disclosed embodiment, the storage system is planned with multiple stacker channels, the channels are arranged between adjacent shelves, and tracks for storage stackers to travel are arranged in the channels, wherein the warehousing conveying module is located at one end of the channels and tracks; in order to fully improve the space utilization rate of the material box warehouse, the constructed storage system is a double-row material box shelf, and the double-row material box shelf is a multi-layer structure. Multiple material box storage positions are arranged in the storage system for storing material boxes filled with goods. It is constructed of steel structures or aluminum profile structures such as column pieces and corbels. During the construction process, the corbels are fixed on the columns, and the distance between the two corbels between the column pieces can be adjusted so that the space formed between the corbels can meet the storage needs of different material boxes.

[0193] The storage stacker is used to complete the task of automatically storing and retrieval of material boxes. During operation, the storage stacker runs in the channel of the storage system, and walks along the track to the designated shelf row through the walking system. The lifting system lifts the loading platform along the column to the designated shelf layer. After the loading platform is in place, the fork is extended and retracted to complete the material box storage and retrieval. The control unit communicates with the storage stacker and controls the storage stacker to complete the loading and unloading of material boxes on the shelves.

[0194] In one possible implementation, the warehousing conveying module includes a plurality of conveying line platforms, an appearance detection device, and a code scanning device;

[0195] Wherein, the plurality of conveyor line platforms are independent power sections, corresponding to the loading section, the buffer section and the warehousing section respectively;

[0196] The code scanning device is used to scan the code to store the goods to be stored in the loading section;

[0197] The appearance detection device is used to perform appearance detection on the warehoused goods to be stored;

[0198] Among them, the goods to be stored can be placed in Figure 2 In the material box 11 shown.

[0199] The storage stacker is used to fork the storage goods to be stored in the warehouse from the storage section, and stack the storage goods to be stored in the warehouse to the target shelf layer of the shelf;

[0200] The cache section is used to temporarily store the warehouse goods to be stored in the warehouse when there is a backlog of warehouse goods to be stored in the warehouse entry section.

[0201] Among them, see Figure 2 As shown, the storage conveying module 3 may include three conveying line platforms 31, an appearance detection device 32, and a code scanning device 33. The three conveying platforms are three independent power sections, corresponding to the loading section, the buffer section, and the storage section, respectively. The purpose of setting the buffer section is to provide the ability to temporarily store material boxes. The number of buffer sections is determined based on the comprehensive consideration of the buffer demand and the site space.

[0202] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the present disclosure. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0203] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An intelligent storage system access monitoring system based on IoT technology, characterized in that: include: A monitoring terminal, a plurality of high-definition cameras and a plurality of ordinary cameras in communication with the monitoring terminal, wherein the plurality of high-definition cameras and the plurality of ordinary cameras are arranged along a track for a storage stacker to travel in a storage system, the ordinary cameras are in a normally open state, and the high-definition cameras are in a normally dormant state; The monitoring terminal is used to receive the first images captured in real time by the plurality of ordinary cameras, and input the plurality of adjacent first images captured by each ordinary camera into a pre-trained decision tree model, to obtain a first recognition result output by the decision tree model as to whether the storage stacker traveling on the track has stopped running, wherein the first recognition result is a binary classification result obtained by the decision tree model; When the first recognition result indicates that any of the storage stacker cranes running on the track has stopped running, a second image of the stopped storage stacker crane is captured by waking up the target high-definition camera; Identify the type of the temporary stop event according to the second image to obtain a target temporary stop event type, and execute a monitoring alarm corresponding to the target temporary stop event type; When the first recognition result indicates that any of the storage stacker cranes running on the track has stopped running, waking up the target high-definition camera to collect a second image of the stopped storage stacker crane includes: In the case where the first recognition result indicates that any of the storage stacker cranes traveling on the track has stopped operating, determining a target high-definition camera for capturing a second image of the storage stacker crane that has stopped operating; In the case where the first recognition result indicates that any of the storage stacker cranes traveling on the track has stopped running, determining any two target ordinary cameras from ordinary cameras that capture the first image used to determine the first recognition result; Determine the shooting distance between the stopped storage stacker and the target high-definition camera according to the first distance between the target high-definition camera and the target ordinary camera, the baseline distance between the two target ordinary cameras, and the first image acquired; The performance parameters of the target high-definition camera are determined according to the shooting distance, and after the target high-definition camera is remotely woken up, the high-definition camera is configured with the performance parameters as the target, and after the configuration is completed, a second image of the stopped storage stacker is collected through the target high-definition camera.

2. The system according to claim 1, characterized in that The method of determining the shooting distance between the stopped storage stacker and the target high-definition camera according to the first distance between the target high-definition camera and the target ordinary camera, the baseline distance between the two target ordinary cameras, and the acquired first image includes: Determining the number of layers of the pyramid in a process of constructing a scale space pyramid for the first image according to a baseline distance of the baselines between the two target ordinary cameras; According to a preset initial standard deviation and a preset standard deviation step size, and according to the number of layers, Gaussian blur is performed multiple times on the first image collected by the target ordinary camera to obtain a scale space pyramid corresponding to the number of layers for each target ordinary camera; For each layer in the scale space pyramid corresponding to each target ordinary camera, determine the gray value difference between each pixel and the pixel adjacent to it, and determine whether the pixel is a key pixel in the corresponding first image according to the gray value difference; Determine a second distance between the stopped storage stacker and the target ordinary camera according to the key pixel corresponding to each of the target ordinary cameras and the grayscale value of the pixel in the first image acquired; The shooting distance between the stopped storage stacker and the target high-definition camera is determined according to the second distance and the first distance between the target ordinary camera and the target high-definition camera.

3. The system according to claim 2, characterized in that The determining, according to the key pixel corresponding to each of the target ordinary cameras and the grayscale value of the pixel in the collected first image, a second distance between the stopped storage stacker and the target ordinary camera comprises: Determine, according to the grayscale value of the pixel in the first image captured by each of the target ordinary cameras, the main direction and feature description corresponding to the key pixel corresponding to each of the target ordinary cameras in the first image; According to the main directions and feature descriptions of the key pixels corresponding to the two target ordinary cameras, feature point matching is performed on the first images captured by the two target ordinary cameras to obtain matching feature point pairs; For each of the matching feature point pairs, determining the horizontal coordinates of the two key pixels in the matching feature point pair in the corresponding first image, and calculating the difference between the horizontal coordinates of the two key pixels; Determine a set of target difference values ​​from the difference values, wherein the set of target difference values ​​minimizes the error between the expected image positions of all key pixels in the first image captured by the two target ordinary cameras and the actual key pixel positions calculated based on the set of target difference values ​​and the camera parameters of the target ordinary cameras; A second distance between the stopped storage stacker and the target ordinary camera is determined according to the baseline distance of the baseline between the two target ordinary cameras and the target difference.

4. The system according to claim 3, characterized in that The determining, according to the grayscale value of the pixel in the first image captured by each of the target ordinary cameras, the main direction and feature description corresponding to the key pixel corresponding to each of the target ordinary cameras in the first image includes: According to the grayscale values ​​of pixels in the first image captured by each of the target ordinary cameras, calculate the gradient direction and gradient amplitude corresponding to the grayscale values ​​of pixels in a first neighborhood with a first preset number of pixels centered on the key pixel corresponding to each of the target ordinary cameras; According to the gradient directions and gradient magnitudes corresponding to the pixels in the first neighborhood centered on each of the key pixels, construct a first histogram with the sum of the gradient magnitudes as the ordinate and the direction angle of the gradient direction as the abscissa; Taking the peak value of the first histogram as the main direction corresponding to the key pixel; A second histogram of pixels in a second neighborhood with a second preset number of pixels centered on the key pixel and corresponding to each of the target ordinary cameras in multiple preset directions is calculated, and multiple histogram vectors of the key pixel in the second histogram are connected to form a feature description of the key pixel.

5. The system according to claim 1, characterized in that The decision tree model performs the following steps to generate the binary classification result: Inputting a plurality of adjacent first images captured by each of the common cameras into a position recognition node, determining the position change of the storage stacker in the plurality of first images through the position recognition node, and obtaining a first decision result; In a case where the first decision result indicates that the position change of the storage stacker in the plurality of the first images is less than a first preset change threshold, the plurality of the first images are input into a speed recognition node, and the speed change of the storage stacker in the plurality of the first images is determined by the speed recognition node to obtain a second decision result; When the second decision result indicates that the speed change of the storage stacker in the plurality of the first images is greater than or equal to the second preset change threshold, a first recognition result that the storage stacker has not stopped running is obtained, and the plurality of adjacent first images captured by the ordinary camera are returned to be reacquired; When the second decision result indicates that the speed change of the storage stacker in the plurality of the first images is less than the second preset change threshold, the plurality of the first images are input into a signal recognition node, and the signal recognition node is used to determine whether the stop signal light of the storage stacker in the plurality of the first images is on, to obtain a third decision result; When the third decision result indicates that the stop signal light of the storage stacker in the plurality of the first images is not on, a first recognition result that the storage stacker has not stopped running is obtained, and the plurality of adjacent first images captured by the common camera are returned to be reacquired; When the third decision result indicates that the stop light of the warehouse stacker in multiple first images is on, the multiple first images are input into a specific position recognition node, and the specific position recognition node is used to determine whether the warehouse stacker in multiple first images is temporarily stopped at a specific position to obtain the first recognition result.

6. The system according to claim 5, characterized in that The decision tree model also performs the following steps to generate the binary classification result: When the first decision result indicates that the position change of the warehouse stacker in multiple first images is greater than or equal to the first preset change threshold, a first recognition result that the warehouse stacker has not stopped running is obtained, and the multiple adjacent first images captured by the ordinary camera are returned to be re-acquired.

7. The system according to claim 1, characterized in that The temporary stop event type includes at least one of the following: cargo falling, abnormal track occupation, artificial temporary stop of the stacker, and fire accident.

8. The system according to any one of claims 1 to 7, characterized in that: The storage system includes a multi-layered shelf, the storage stacker, the track arranged between adjacent shelves, and a storage conveying module arranged at one end of the track; Wherein, the multi-layered shelf is constructed by column pieces and corbels, and the corbels are fixed on the column pieces; The storage transport module is used to perform storage operations on the warehouse goods to be stored, and report the storage information of the warehouse goods to be stored to the monitoring terminal; The storage stacker is used to receive the warehousing tasks and outbound tasks assigned by the monitoring terminal based on the warehousing information, and to stack the warehouse goods to be stored on the target shelf layer of the shelf for the warehousing tasks and to perform outbound pickup and delivery of the warehouse goods stacked on the target shelf layer for the outbound tasks.

9. The system according to claim 8, characterized in that The inbound conveying module includes a plurality of conveying line platforms, an appearance detection device, and a code scanning device; Wherein, the plurality of conveyor line platforms are independent power sections, corresponding to the loading section, the buffer section and the warehousing section respectively; The code scanning device is used to scan the code to store the goods to be stored in the loading section; The appearance detection device is used to perform appearance detection on the warehoused goods to be stored; The storage stacker is used to fork the storage goods to be stored in the warehouse from the storage section, and stack the storage goods to be stored in the warehouse to the target shelf layer of the shelf; The cache section is used to temporarily store the warehouse goods to be stored in the warehouse when there is a backlog of warehouse goods to be stored in the warehouse entry section.

Images