Tipper system and method for transporting cargo, and detection apparatus and method for detecting residual cargo

Abstract

A tipper system and method for transporting cargo, and a detection apparatus and method for detecting residual cargo are disclosed. The tipper system for transporting cargo may comprise: a transport cart having a loading panel on which cargo is to be loaded; and a tipper device for unloading the cargo loaded on the transport cart by tilting the transport cart. The tipper device may comprise: a data collection unit using a camera sensor to acquire an image of the loading panel, on which cargo is loaded, and distance data indicating the distance to the cargo and / or the loading panel included in the image; a height estimation unit for estimating the height of the loaded cargo on the basis of image data and the distance data; and an inclination determination unit for determining, on the basis of the estimated height, the angle at which a cart accommodation part is to be inclined.

Description

Tipper system and method for cargo transport, detection device and method for detecting residual cargo

[0001] The following disclosure relates to a tipper system and method for cargo transport, and a detection device and method for detecting residual cargo.

[0002] Daytime, evening, and same-day delivery can be realized through logistics automation. To increase logistics delivery speed, delivery systems can operate using a hub-and-spoke model. The hub-and-spoke model is a concept where goods are transported to a hub terminal regardless of the delivery area, and then transported from the hub terminal to the delivery area. A hub terminal acts as a relay, consolidating cargo collected and dispatched from regional sub-terminals before guiding it toward its final destination. Since a hub terminal is the center of the logistics network, a problem occurring at a single hub terminal can affect the entire network. For example, if a hub terminal becomes overloaded, all logistics flows passing through that terminal may experience delays. To prevent delays in logistics flow, highly automated systems are used at hub terminals. For instance, the sorting of parcels and cargo is automated using barcode scanners and sorters.

[0003] A tipper system for cargo transport according to one embodiment includes a transport cart having a loading panel on which cargo is loaded, and a tipper device for unloading cargo loaded on the transport cart by tilting the transport cart. The tipper device may include a data collection unit that acquires an image of the loading panel on which cargo is loaded and distance data indicating the distance to at least one of the cargo or the loading panel included in the image using a camera sensor, a height estimation unit that estimates the height of the loaded cargo based on the image data and distance data, and a tilt determination unit that determines the angle at which a cart receiving unit that receives the transport cart is tilted based on the estimated height.

[0004] The height estimation unit divides an image into multiple regions, obtains multiple distance data corresponding to the divided regions using a camera sensor, and can estimate the height of the loaded cargo based on the multiple distance data.

[0005] The height estimation unit estimates multiple heights corresponding to divided areas based on multiple distance data, and can determine the maximum value among the multiple estimated heights as the height of the loaded cargo.

[0006] The tilt determination unit can determine the angle at which the cart receiving unit tilts when the estimated height is greater than or equal to the critical height.

[0007] When the tilt adjustment unit detects movement around the tipper device using a motion detection sensor, it stops tilting the cart receiving unit, and the tipper device emits a warning sound and can switch to an emergency mode to manually control the tilt of the cart receiving unit.

[0008] A detection device for detecting residual cargo within a tipper device according to one embodiment comprises: a cart receiving unit that receives a transport cart including a loading panel on which cargo is loaded; a data collection unit that acquires distance data indicating an image of the loading panel on which cargo is loaded and a distance to at least one of the cargo or the loading panel included in the image using a camera sensor; a preprocessing unit that performs preprocessing by masking a background image in the acquired image, binarizing the masked image, and removing noise from the binarized image; and can determine whether there is residual cargo based on the area of ​​an object detected in the preprocessed image.

[0009] The preprocessing unit can mask areas in the image where scaled distance data is greater than or equal to a threshold distance value.

[0010] The residual cargo determination unit can determine the contour of an object based on distance data in a preprocessed image, determine the area of ​​an object based on the determined contour, and determine whether there is residual cargo based on the determined area.

[0011] The remaining cargo determination unit may determine that there is no remaining cargo if the determined area is smaller than the critical area, and determine that there is remaining cargo if the determined area is greater than or equal to the critical area.

[0012] When the remaining cargo determination unit determines that there is remaining cargo, the remaining cargo determination unit can control the remaining cargo to be unloaded from the transport cart by tilting the cart receiving unit.

[0013] A cargo transport method performed by a tipper system for cargo transport according to one embodiment may include: an operation in which a transport cart transports cargo loaded on a loading panel where cargo is loaded to a tipper device; an operation in which the tipper device acquires distance data indicating an image of the loading panel where cargo is loaded and a distance to at least one of the cargo or the loading panel included in the image using a camera sensor; an operation in which the tipper device estimates the height of the loaded cargo based on the image and distance data; an operation in which the tipper device determines an angle at which a cart receiving part in which the transport cart is received is tilted based on the estimated height; and an operation in which the tipper device controls the loaded cargo to be unloaded from the transport cart to a conveyor device by causing the cart receiving part to tilt by the determined angle.

[0014] The operation of estimating height may include: the operation of dividing an image into multiple regions; the operation of acquiring multiple distance data corresponding to the divided regions using a camera sensor; and the operation of estimating the height of the loaded cargo based on the multiple distance data.

[0015] The operation of estimating height may include: the operation of estimating multiple heights corresponding to divided areas based on multiple distance data; and the operation of determining the maximum value among the multiple estimated heights as the height of the loaded cargo.

[0016] The operation of determining the angle may include the operation of determining the angle at which the cart receiving part is tilted when the estimated height is greater than or equal to the threshold height.

[0017] The operation of determining the angle includes an operation of stopping the cart receiving portion from tilting when movement around the tipper device is detected using a motion detection sensor, and the tipper device may further include an operation of emitting a warning sound; and an operation of switching to an emergency mode for manually controlling the tilt of the cart receiving portion.

[0018] A method for detecting remaining cargo using a detection device according to one embodiment may include: receiving a transport cart including a loading panel on which cargo is loaded; acquiring distance data indicating an image of the loading panel on which cargo is loaded and a distance to at least one of the cargo or the loading panel included in the image using a camera sensor; performing preprocessing by masking a background image in the acquired image, binarizing the masked image, and removing noise from the binarized image; and determining whether there is remaining cargo based on the area of ​​an object detected in the preprocessed image.

[0019] The operation of performing preprocessing may include masking areas in the image where scaled distance data is greater than or equal to a threshold distance value.

[0020] The operation of determining whether there is residual cargo may include the operation of determining the contour of an object in a preprocessed image based on distance data, the operation of determining the area of ​​an object based on the determined contour, and the operation of determining whether there is residual cargo based on the determined area.

[0021] The operation of determining whether there is residual cargo can determine that there is no residual cargo if the determined area is smaller than the critical area, and determine that there is residual cargo if the determined area is greater than or equal to the critical area.

[0022] If the detection device determines that there is residual cargo, it may further include an operation to control the residual cargo to be unloaded from the transport cart by tilting the cart receiving part that accommodates the transport cart.

[0023] FIG. 1 is a block diagram illustrating a tipper system for cargo transport according to one embodiment.

[0024] FIG. 2 is a block diagram illustrating a detection device for detecting residual cargo within a tipper device according to one embodiment.

[0025] FIG. 3 is a diagram illustrating how a height estimation unit according to one embodiment estimates the height of a loaded cargo.

[0026] FIG. 4 is a diagram illustrating a detection device according to one embodiment determining whether there is residual cargo.

[0027] FIG. 5 is a flowchart illustrating a cargo transport method according to one embodiment.

[0028] FIG. 6 is a flowchart illustrating a method for detecting residual cargo according to one embodiment.

[0029] Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be modified and implemented in various forms. Accordingly, actual implementations are not limited to the specific embodiments disclosed, and the scope of this specification includes modifications, equivalents, or substitutions included in the technical concept described by the embodiments.

[0030] Terms such as "first" or "second" may be used to describe various components, but these terms should be interpreted solely for the purpose of distinguishing one component from another. For example, the first component may be named the second component, and similarly, the second component may be named the first component.

[0031] When it is stated that a component is "connected" to another component, it should be understood that it may be directly connected to or coupled with that other component, or that there may be other components in between.

[0032] Singular expressions include plural expressions unless the context clearly indicates otherwise. In this document, phrases such as “A or B,” “at least one of A and B,” “at least one of A or B,” “A, B or C,” “at least one of A, B and C,” and “at least one of A, B, or C” may each include any one of the items listed together with the corresponding phrase, or all possible combinations thereof. In this specification, terms such as “comprising” or “having” are intended to designate the existence of the described feature, number, step, action, component, part, or combination thereof, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0033] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this specification.

[0034] As used in this document, the term "part" refers to software or hardware components, such as FPGAs or ASICs, and the "part" performs certain roles. However, the meaning of "part" is not limited to software or hardware. The "part" may be configured to reside in an addressable storage medium or configured to operate one or more processors. For example, the "part" may include components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and "parts" may be combined into a smaller number of components and "parts" or further separated into additional components and "parts." Furthermore, the components and "parts" may be implemented to operate one or more CPUs within a device or secure multimedia card. Additionally, '~part' may include one or more processors.

[0035] Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the attached drawings, identical components are given the same reference numeral regardless of the drawing number, and redundant descriptions thereof will be omitted.

[0036]

[0037] FIG. 1 is a block diagram illustrating a tipper system for cargo transport according to one embodiment.

[0038] Referring to FIG. 1, the tipper system (100) may include a transport cart (110), a tipper device (120), and a conveyor device (130).

[0039] The transport cart (110) can receive cargo and move the cargo to the tipper device (120). The transport cart (110) may be equipped with a loading panel on which the cargo is loaded. For example, the transport cart (110) may be a roll container equipped with wheels and a loading panel on which the cargo is loaded.

[0040] The tipper device (120) can unload cargo loaded on the transport cart (110). The tipper device (120) can unload cargo loaded on the transport cart (110) by tilting the transport cart (110). For example, the tipper device (120) can unload cargo loaded on the transport cart (110) by tilting the transport cart (110) receiving portion in which the transport cart (110) is received.

[0041] The tipper device (120) may include a cart receiving section (121), a data collection section (122), a height estimation section (123), a slope determination section (124), and a slope adjustment section (125).

[0042] The cart receiving section (121) can accommodate a transport cart (110). For example, the cart receiving section (121) can accommodate a roll container. The cart receiving section (121) may be a structure made to fit the size of the roll container.

[0043] The data collection unit (122) can acquire data regarding the transport cart (110) received in the cart receiving unit (121) and data regarding the cargo loaded on the transport cart (110). For example, the data collection unit (122) can acquire an image of a loading panel on which cargo is loaded and distance data indicating the distance to at least one of the cargo or loading panel included in the image using a camera sensor. The camera sensor may be an RGB-D camera. An RGB-D camera can acquire depth data and color data. Depth data may be referred to as distance data. For example, an RGB-D camera can acquire the color of an object and the distance from the camera to the object. An RGB-D camera may be installed on a support provided within the tipper device (120) and can acquire distance data and color data regarding the loading panel, which is the ground area within the received transport cart (110), and the loaded cargo.

[0044] The height estimation unit (123) can estimate the height of the loaded cargo based on image data and distance data. The height estimation unit (123) can estimate the height of the loaded cargo using distance data. The estimation of the height of the loaded cargo by the height estimation unit (123) will be explained in more detail in FIG. 3.

[0045] The tilt determining unit (124) can determine the angle at which the cart receiving unit (121) is tilted based on the estimated height. For example, the tilt determining unit (124) can determine the angle at which the cart receiving unit (121) is tilted when the estimated height is greater than or equal to a critical height.

[0046] A tilt determining unit (124) according to one embodiment can determine the tilting angle according to the maximum height of the cargo loaded on the transport cart (110). The tilting angle according to the maximum height of the loaded cargo can be predefined by the user. For example, the tilting angle determined according to the maximum height of the loaded cargo can be defined as shown in the table below.

[0047] [Table 1]

[0048]

[0049] In Table 1, the angle of inclination can be larger as the maximum height of the loaded cargo increases. When the angle of inclination is increased as the maximum height of the loaded cargo increases, damage caused by falling cargo can be minimized when the cargo inside the transport cart (110) is unloaded onto the conveyor belt.

[0050] When the tilt determining unit (124) determines the tilting angle of the cart receiving unit (121), the tilt determining unit (124) may transmit a control signal based on the determined tilt to the tilt adjustment unit (125) using wired or wireless communication. For example, the tilt determining unit (124) may transmit a control signal packet based on the determined tilt to the tilt adjustment device using wireless communication (e.g., Bluetooth communication).

[0051] According to one embodiment, the tilt determining unit (124) may be in an emergency mode state. The emergency mode is a mode in which, when the tipper device (120) detects a moving object (e.g., a person) in the surroundings through a motion detection sensor, the tilt of the cart receiving unit (121) must be manually adjusted for safety. When the tilt determining unit (124) is in the emergency mode, the tilt determining unit (124) stops controlling the tilt of the cart receiving unit (121), and the tilt of the cart receiving unit (121) can be controlled by manual control by the user. The emergency mode can be terminated by resetting the position of the cart receiving unit (121) by returning the cart receiving unit (121) to its initial state.

[0052] The tilt adjustment unit (125) can control the loading of cargo from the transport cart (110) to the conveyor device (130) by tilting the cart receiving unit (121) by a determined angle. The tilt adjustment unit (125) may include a structure that can tilt forward around a rotation axis. The tilt adjustment unit (125) can control the cart receiving unit (121) to rotate around a rotation axis, and the cargo loaded on the transport cart (110) can be unloaded to the conveyor device (130).

[0053] The tilt adjustment unit (125) may include a tilt adjustment button that can adjust the tilt of the cart receiving unit (121) by user control. The tilt adjustment unit (125) can adjust the tilt of the cart receiving unit (121) by user button operation without following the control of the tilt determination unit (124). For example, the cart receiving unit (121) can be rotated by the user pressing the tilt adjustment up button, and the cargo loaded in the cart receiving unit (121) can be unloaded to the conveyor device (130).

[0054] The tipper device (120) may further include a sensor to check whether the tilt adjustment of the cart receiving part (121) is completed by the tilt adjustment part (125). For example, it may include a microphotosensor to check whether the cart receiving part (121) has been rotated to 0 degrees, 90 degrees, 100 degrees, 105 degrees, or 110 degrees.

[0055] A tipper device (120) according to one embodiment may be switched to an emergency mode that emits a warning sound and manually controls the tilt of the cart receiving section (121). For example, the tilt adjustment section (125) of the tipper device (120) may stop the cart receiving section (121) from tilting when it detects movement around the tipper device (120) through a motion detection sensor. The motion detection sensor may include an ultrasonic sensor, an infrared sensor, a radar sensor, etc. When the motion detection sensor detects movement around the tipper device (120), the tilt adjustment section (125) may immediately stop the tilt adjustment of the cart receiving section (121). The motion detection sensor may be installed inside the tipper device (120) or outside the tipper device (120) to detect movement around the tipper device (120). For example, the motion detection sensor may be installed outside the tipper device (120) at a point where the outermost point that appears when the tipper device (120) rotates meets the floor vertically.

[0056] The conveyor device (130) can transport cargo unloaded from the tipper device (120). The conveyor device (130) may include a conveyor belt, a conveyor roller, a conveyor chain, etc. For example, the conveyor belt can transport cargo unloaded from the tipper device (120) to a transport vehicle.

[0057] The tipper system (100) may include a load detection, tilt control automation, and an emergency stop function through abnormal object detection. The tipper system (100) may determine the height of the cargo loaded inside the roll container using an RGB-D camera. The tipper system (100) may adjust the tilt of the tipper device (120) based on the determined height value and monitor the transport process of the load. For user safety, the tipper system (100) may further include a motion detection sensor that detects movement in the area surrounding the tipper device (120), and if movement in the area surrounding the tipper device (120) is detected, the tilt adjustment operation of the tipper device (120) may be stopped immediately. The tipper system (100) can increase the efficiency and safety of logistics processing with a small number of personnel. The tipper system (100) may be used for cargo unloading or loading operations and may help prevent general fatigue, muscle pain, and musculoskeletal disorders in workers performing unloading or loading operations.

[0058]

[0059] FIG. 2 is a block diagram illustrating a detection device for detecting residual cargo within a tipper device according to one embodiment.

[0060] Referring to FIG. 2, the detection device (200) may include a cart receiving unit (210), a data collection unit (220), a preprocessing unit (230), and a remaining cargo determination unit (240).

[0061] The cart receiving section (210) can accommodate a transport cart including a loading panel loaded with cargo. Since the cart receiving section (210) corresponds to the cart receiving section (121) of FIG. 1, redundant descriptions below will be omitted.

[0062] The data collection unit (220) can acquire first distance data indicating an image of a loading panel on which cargo is loaded and a distance to at least one of the cargo or loading panel included in the image using a camera sensor. For example, the data collection unit (220) can acquire first distance data indicating an image of a loading panel on which cargo is loaded and a distance to at least one of the cargo or loading panel included in the image using a camera sensor. The camera sensor may be an RGB-D camera. An RGB-D camera may acquire depth data and color data. The depth data may be referred to as first distance data. For example, an RGB-D camera may acquire the color of an object and the distance from the camera to the object. An RGB-D camera may be installed on a support provided within a tipper device and may acquire first distance data and color data regarding the loading panel and the loaded cargo, which is a ground area within the received transport cart.

[0063] The preprocessing unit (230) can preprocess an image of a loading panel loaded with cargo and a first distance data. For example, the preprocessing unit (230) can mask the background image in the image of the loading panel loaded with cargo and remove noise from the masked image. The preprocessing performed by the preprocessing unit (230) will be explained in more detail in FIG. 4.

[0064] The remaining cargo determination unit (240) can determine whether there is remaining cargo based on the area of ​​an object detected in a preprocessed image. The remaining cargo determination unit (240) can determine the contour of the detected object and determine the area of ​​the object based on the determined contour. The remaining cargo determination unit (240) can determine that there is remaining cargo if the determined area is greater than or equal to a threshold area. If the remaining cargo determination unit (240) determines that there is remaining cargo, it can control the remaining cargo to be unloaded from the transport cart by tilting the cart receiving unit (210). The determination of whether there is remaining cargo by the remaining cargo determination unit (240) will be explained in more detail in FIG. 4.

[0065] After the unloading operation is completed, the detection device (200) can detect whether there is any remaining cargo inside the transport cart (e.g., roll container) by using an image of the loading panel on which the cargo is loaded, first distance data, and image preprocessing technology. The detection device (200) can visually indicate whether there is any remaining cargo.

[0066]

[0067] FIG. 3 is a diagram illustrating how a height estimation unit according to one embodiment estimates the height of a loaded cargo.

[0068] Referring to FIG. 3, the image (310) includes an area in which the height estimation unit divides the image of a loading panel loaded with cargo into a plurality of regions. As in the image (310), the image of a loading panel loaded with cargo can be divided into 36 cells.

[0069] The height estimation unit can acquire multiple distance data corresponding to the divided area using a camera sensor. The distance data can be represented as an image. The height estimation unit can acquire distance data represented as an image using a camera sensor (e.g., an RGB-D camera). The image (320) is distance data represented as an image, and the distance can be represented by adjusting the color saturation and / or brightness of the load and the load panel.

[0070] The height estimation unit can estimate the height of the loaded cargo based on multiple distance data. For example, the height estimation unit can estimate multiple heights corresponding to the divided area based on multiple distance data, and determine the maximum value among the estimated multiple heights as the height of the loaded cargo. Image (331) shows the average height in each cell. In image (330), the cell marked with a red rectangle is a cell containing the loaded cargo and is higher than the average height of the other cells.

[0071] The image (340) represents information about a cell. In the image (340), ROI (region of interest; ROI) represents a divided cell area, ROI X represents the X-coordinate of the current location in the selected cell area, ROY represents the Y-coordinate of the current location in the selected cell area, ROI Width represents the width of the selected cell area, ROI Height represents the height of the selected cell area, Max Height represents the maximum value among the measured heights, and Max Height Cell represents the cell coordinates of the selected cell.

[0072] The height of the cargo can be determined by the following mathematical formulas 1 and 2.

[0073] The raw distance (or raw depth value) measured by the camera can be converted into an actual physical distance using mathematical formula 1.

[0074] [Mathematical Formula 1]

[0075]

[0076] In mathematical formula 1 is a conversion factor used in camera sensors (e.g., RGB-D cameras) and is a factor used to convert the raw distance measured by the camera sensor into an actual physical distance. represents the raw distance measured by the camera sensor. represents the distance of the cell, and the unit is m.

[0077] The height of the loaded cargo can be determined by mathematical formula 2. The height of the loaded cargo relative to the loading panel of the transport cart is the height of the camera sensor and the distance of the cell ( It can be determined based on ).

[0078] [Mathematical Formula 2]

[0079]

[0080] is the height of the loaded cargo based on the loading panel of the transport cart, is the height of the camera sensor, represents the distance of the cell obtained from mathematical formula 1.

[0081] The height estimation unit is determined in each cell Outliers can be removed. The height estimator can remove outliers using the interquartile range (IQR) method. The interquartile range (IQR) method is a non-parametric method that does not make assumptions about the data distribution and can be applied to various data distribution types. Distance data may contain noise and may generate extreme distance values ​​(or depth values) in highly reflective objects or unexpected environments. For example, highly reflective cells in the image (320) may correspond to outliers. The IQR method can effectively identify and remove such outliers and provide reliable height measurement results.

[0082] Outliers can be determined using the following mathematical formulas 3 to 6.

[0083] [Mathematical Formula 3]

[0084]

[0085] [Mathematical Formula 4]

[0086]

[0087] [Mathematical Formula 5]

[0088]

[0089] [Mathematical Formula 6]

[0090]

[0091] In mathematical formula 3 (or Q1) represents the value located at the bottom 25% when the data is sorted in ascending order. In Equation 4 (or Q3) represents the value located at the top 75% when the data is sorted in ascending order. IQR represents the difference between the value located at the bottom 25% and the value located at the top 75% when the data is sorted in ascending order. In Equation 6 represents the range of outliers determined as values ​​that are more than 1.5 times away from the IQR. represents the determined height.

[0092] The height estimation unit can determine the average heights corresponding to each cell after removing outliers and performing additional processing (e.g., filtering negative values), and determine the maximum value among the average heights as the height of the loaded cargo.

[0093] When the height estimation unit determines (or estimates) the height of the loaded cargo, the height of the cargo may be transmitted to the tilt determination unit. Based on the received height of the cargo, the tilt determination unit may determine the tilt of the cart receiving unit according to the height of the cargo. For example, if the received height of the cargo is greater than or equal to a critical height, the tilt determination unit may determine the angle at which the cart receiving unit tilts to 110 degrees. The determined angle is transmitted to the tilt adjustment unit, and the tilt adjustment unit may tilt the cart receiving unit by 110 degrees. According to one embodiment, the tilt adjustment unit may adjust the tilt of the cart receiving unit according to the determined angle when the auto tilt adjustment mode is on, and may adjust the tilt of the cart receiving unit by user control when the tilt adjustment mode is off.

[0094]

[0095] FIG. 4 is a diagram illustrating a detection device according to one embodiment determining whether there is residual cargo.

[0096] Referring to FIG. 4, a detection device (e.g., the detection device (200) of FIG. 2) can determine whether there is any remaining cargo in a transport cart. The detection device can detect cargo remaining inside a transport cart (e.g., a roll container) using a camera sensor (e.g., an RGB-D camera).

[0097] The detection device can detect remaining cargo using an image of a loading panel loaded with cargo and first distance data obtained through a camera sensor. The first distance data represents distance data obtained from an image of a loading panel loaded with cargo that is not divided into multiple cell regions. Image (410) represents an image of a loading panel loaded with cargo, and image (420) represents first distance data expressed in the image. Since obtaining an image of a loading panel loaded with cargo and obtaining distance data from an image of a loading panel loaded with cargo correspond to the description of FIG. 3, a redundant description will be omitted.

[0098] The detection device can use a preprocessing unit (e.g., the preprocessing unit (230) of FIG. 2) to mask areas in the image where scaled distance data is greater than or equal to a threshold distance value. Masking refers to removing background areas from acquired depth data. Masking can enable better identification of objects in the area of ​​interest. Masking can be performed based on a clipping distance threshold set by the user. Masking can be performed based on Equation 7 below.

[0099] [Mathematical Formula 7]

[0100]

[0101] In mathematical formula 7, is a background-removed image Coordinates and coordinate, is the image before the background was removed Coordinates and coordinate, is the coordinate value of the depth image depth scale in The value multiplied by, is the distance threshold (clipping distance) set by the user, is the color value of the masked area.

[0102] A detection device according to one embodiment can binarize a masked image. Binarization can be performed using a clipping distance threshold used in masking. Binarization can be performed through the following mathematical formula 8, and the detection device can estimate the presence of an object during the binarization process.

[0103] [Mathematical Formula 8]

[0104]

[0105] In mathematical formula 8 represents the distance threshold, and represents a binary value of 0 or 1.

[0106] The image (430) represents regions of the binarized image. In the resulting binarized image (430), the object region can be represented in white and the background region in black.

[0107] The detection device can determine the contour of an object based on distance data from a preprocessed image. The detection device can determine the area of ​​the object based on the detected contour and determine whether there is residual cargo based on the determined area. If the determined area is smaller than a critical area, the detection device determines that there is no residual cargo, and if the determined area is greater than or equal to the critical area, it determines that there is residual cargo.

[0108] The detection device can remove noise and improve the shape of objects within an image by performing morphological operations on a binarized image. Morphological operations may include erosion, which reduces the contours of objects detected in the binarized image; dilation, which expands the contours of objects; opening, which removes noise; closing, which smoothly connects contours and connects broken contours; and gradient, which emphasizes contours.

[0109] The detection device can extract the outline of an object using a contour detection algorithm and determine the contour area based on the extracted outline. If the contour area is greater than or equal to a critical area, the detection device can determine that there is residual cargo.

[0110] The detection device can determine whether there is residual cargo based on the contour area by the following mathematical formula 9.

[0111] [Mathematical Formula 9]

[0112]

[0113] Here is the contour area, represents the critical area.

[0114] If the contour area is smaller than the critical area, the detection device can repeat operations for detecting remaining cargo within the tipper device. By performing repeated cargo detection operations, the accuracy of detecting remaining cargo can be increased and false positives can be reduced. If the detection device determines that there is remaining cargo, it may display a window indicating that the remaining cargo has been detected. As shown in image (440), if the remaining cargo is detected, the phrase "Object Detected: Yes" may be displayed. Additionally, the detection device may provide an image showing the outline of the detected object. If it determines that there is remaining cargo, the detection device may control the remaining cargo to be unloaded from the transport cart (e.g., the transport cart (110) in Fig. 1) by tilting the cart receiving part (e.g., the cart receiving part (210) in Fig. 2).

[0115] The detection device can increase the efficiency of logistics processing and enhance worker safety by detecting whether there is residual cargo within the tipper device.

[0116]

[0117] FIG. 5 is a flowchart illustrating a cargo transport method according to one embodiment.

[0118] The operations of the cargo transport method can be performed by a tipper system (e.g., the tipper system (100) of FIG. 1).

[0119] In operation (510), a transport cart (e.g., the transport cart (110) of FIG. 1) can transport cargo to a tipper device (e.g., the tipper device (120) of FIG. 1). The transport cart can transport cargo loaded on a loading panel to the tipper device.

[0120] In operation (520), the tipper device can accommodate a transport cart.

[0121] In operation (530), the tipper device can acquire image and distance data for a loading panel. The tipper device can acquire an image of a loading panel loaded with cargo and distance data indicating the distance to at least one of the cargo or loading panel included in the image using a camera sensor.

[0122] In operation (540), the tipper device can estimate the height of the loaded cargo. The tipper device can estimate the height of the loaded cargo based on image data and distance data. The tipper device can divide the image data into multiple regions, obtain multiple distance data corresponding to the divided regions using a camera sensor, and estimate the height of the loaded cargo based on the multiple distance data.

[0123] A tipper device according to one embodiment can estimate a plurality of heights corresponding to a divided area based on a plurality of distance data, and determine the maximum value among the estimated plurality of heights as the height of the loaded cargo.

[0124] In operation (550), the tipper device can determine the angle at which the cart receiving portion is tilted. The tipper device can determine the angle at which the cart receiving portion, in which a transport cart is received, is tilted based on an estimated height. For example, the tipper device can determine the angle at which the cart receiving portion is tilted if the estimated height is greater than or equal to a threshold height. If the tipper device detects movement around the tipper device using a motion sensor, it may stop tilting the cart receiving portion, emit a warning sound, and switch to an emergency mode to manually control the tilt of the cart receiving portion.

[0125] In operation (560), the tipper device can control the cart receiving section so that the loaded cargo is unloaded. For example, the tipper device can control the loaded cargo to be unloaded from the transport cart to the conveyor device by tilting the cart receiving section by a determined angle.

[0126] In operation (570), the conveyor device (e.g., the conveyor device (130) of FIG. 1) can transport cargo unloaded from the tipper device.

[0127]

[0128] FIG. 6 is a flowchart illustrating a method for detecting residual cargo according to one embodiment.

[0129] The operations of the cargo detection method can be performed by a detection device (e.g., the detection device (200) of FIG. 2).

[0130] In operation (610), the detection device (e.g., the detection device (200) of FIG. 2) can accommodate a transport cart (e.g., the transport cart (110) of FIG. 1). The transport cart may include a loading panel on which cargo is loaded.

[0131] In operation (620), the detection device can acquire image and distance data for a loading panel. The detection device can use a camera sensor to acquire an image of a loading panel on which cargo is loaded and distance data indicating the distance to at least one of the cargo or loading panel included in the image.

[0132] In operation (630), the detection device may perform preprocessing by masking the background image, binarizing the masked image, and removing noise from the binarized image. For example, the detection device may mask areas in the image where scaled distance data is greater than or equal to a threshold distance value, and binarize the masked areas.

[0133] In operation (640), the detection device can determine whether there is residual cargo based on the area of ​​an object detected in a preprocessed image. For example, the detection device can determine the contour of an object based on distance data in the preprocessed image, determine the area of ​​an object based on the detected contour, and determine whether there is residual cargo based on the determined area. The detection device can determine that there is no residual cargo if the determined area is smaller than a threshold area, and determine that there is residual cargo if the determined area is greater than or equal to the threshold area.

[0134] If the detection device determines that there is a remaining cargo (e.g., YES ("Yes") in operation (650), the detection device can control the remaining cargo to be unloaded in operation (660). For example, if the detection device determines that there is a remaining cargo, it can control the remaining cargo to be unloaded from the transport cart by tilting the cart receiving part that receives the transport cart.

[0135] The embodiments described above may be implemented as hardware components, software components, and / or combinations of hardware and software components. For example, the devices, methods, and components described in the embodiments may be implemented using a general-purpose computer or a special-purpose computer, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. In addition, other processing configurations, such as parallel processors, are also possible.

[0136] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or instruct the processing unit independently or collectively. Software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium, or device so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and may be stored or executed in a distributed manner. Software and data may be stored on computer-readable recording media.

[0137] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may store program instructions, data files, data structures, etc., either individually or in combination, and the program instructions recorded on the medium may be those specifically designed and configured for the embodiment or those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

[0138] The hardware device described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

[0139] Although the embodiments have been described above with reference to the limited drawings, those skilled in the art can apply various technical modifications and variations based thereon. For example, suitable results may be achieved even if the described techniques are performed in a different order than described, and / or if the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

[0140] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims set forth below.

Claims

1. In a tipper system for cargo transport, A transport cart equipped with a loading panel on which cargo is loaded; and It includes a tipper device for unloading cargo loaded on the transport cart by tilting the transport cart, The above tipper device is, A data collection unit that acquires an image of a loading panel on which the cargo is loaded and distance data indicating the distance to at least one of the cargo or loading panel included in the image using a camera sensor; A height estimation unit that estimates the height of the loaded cargo based on image data and the distance data; and A tilt determining unit comprising a tilt determining unit that determines the angle at which a cart receiving part accommodating the transport cart is tilted based on the above-mentioned estimated height, Tipper system.

2. In Paragraph 1, The above height estimation unit is, Dividing the above image into multiple regions, acquiring multiple distance data corresponding to the divided regions using the camera sensor, and estimating the height of the loaded cargo based on the multiple distance data. Tipper system.

3. In Paragraph 2, The above height estimation unit is, Estimating multiple heights corresponding to the divided area based on the above multiple distance data, and determining the maximum value among the estimated multiple heights as the height of the loaded cargo, Tipper system.

4. In Paragraph 1, The above slope determining unit is, When the above estimated height is greater than or equal to a critical height, determining the angle at which the cart receiving part is tilted, Tipper system.

5. In Paragraph 1, A tilt adjustment unit that controls the loading of cargo from the transport cart to the conveyor device by tilting the cart receiving portion by the determined angle. Includes more, The above-mentioned tilt adjustment unit is, When movement around the tipper device is detected using a motion detection sensor, the process of tilting the cart receiving part is stopped, and The above tipper device is, Transmitting a warning sound and switching to an emergency mode that manually controls the inclination of the cart receiving section, Tipper system.

6. A detection device for detecting residual cargo within a tipper device, A cart receiving section for accommodating a transport cart including a loading panel loaded with cargo; A data collection unit that acquires an image of a loading panel on which the cargo is loaded and distance data indicating the distance to at least one of the cargo or loading panel included in the image using a camera sensor; A preprocessing unit that performs preprocessing by masking a background image from the acquired image, binarizing the masked image, and removing noise from the binarized image; and A residual cargo determination unit that determines whether there is residual cargo based on the area of ​​an object detected in the above-mentioned preprocessed image. Detection device.

7. In Paragraph 6, The above preprocessing unit is, Masking the area where the scaled distance data in the above image is greater than or equal to a threshold distance value, Detection device.

8. In Paragraph 6, The above-mentioned residual cargo determination unit is, Determining the contour of the object based on the distance data in the above preprocessed image, and Determine the area of ​​the object based on the above-determined contour, and Determining whether there is residual cargo based on the area determined above, Detection device.

9. In Paragraph 8, The above-mentioned residual cargo determination unit is, If the area determined above is smaller than the critical area, it is determined that there is no remaining cargo, and If the area determined above is greater than or equal to the critical area, determining that there is residual cargo, Detection device.

10. In Paragraph 9, If it is determined that the above-mentioned remaining cargo exists, The above-mentioned residual cargo determination unit is, Controlling the remaining cargo to be unloaded from the transport cart by tilting the cart receiving portion, Detection device.

11. A method of transporting cargo performed by a tipper system for transporting cargo, The operation of a transport cart transporting the cargo loaded on a loading panel to a tipper device; The operation of the tipper device acquiring distance data using a camera sensor, an image of a loading panel on which the cargo is loaded, and distance data indicating the distance to at least one of the cargo or loading panel included in the image; The operation of the above tipper device estimating the height of the loaded cargo based on the image and the distance data; The operation of the tipper device determining the angle at which the cart receiving portion receiving the transport cart is tilted based on the estimated height; and The operation of controlling the loaded cargo to be unloaded from the transport cart to the conveyor device by causing the above tipper device to tilt the cart receiving portion by the above determined angle. including, Cargo transportation methods.

12. In Paragraph 11, The operation of estimating the above height is, The operation of dividing the above image into a plurality of regions; the operation of acquiring a plurality of distance data corresponding to the divided regions using the camera sensor; and the operation of estimating the height of the loaded cargo based on the plurality of distance data, Cargo transportation methods.

13. In Paragraph 12, The operation of estimating the above height is, The operation of estimating a plurality of heights corresponding to the divided area based on the plurality of distance data; and the operation of determining the maximum value among the estimated plurality of heights as the height of the loaded cargo, Cargo transportation methods.

14. In Paragraph 12, The operation of determining the above angle is, If the above estimated height is greater than or equal to a critical height, the method includes determining the angle at which the cart receiving part is tilted. Cargo transportation methods.

15. In Paragraph 12, The operation of determining the above angle is, When movement around the tipper device is detected using a motion detection sensor, the operation of stopping the cart receiving portion from tilting is included. The above tipper device is, The operation of transmitting a warning sound; and further comprising the operation of switching to an emergency mode that manually controls the inclination of the cart receiving portion, Cargo transportation methods.

16. A method for detecting residual cargo using a detection device, The action of accommodating a transport cart including a loading panel loaded with cargo; The operation of acquiring an image of a loading panel on which the cargo is loaded and distance data indicating the distance to at least one of the cargo or loading panel included in the image using a camera sensor; An operation to perform preprocessing by masking a background image from the acquired image, binarizing the masked image, and removing noise from the binarized image; and Operation of determining whether there is residual cargo based on the area of ​​the object detected in the above-mentioned preprocessed image including, Detection method.

17. In Paragraph 16, The operation of performing the above preprocessing is, A method comprising an operation of masking an area in the above image where the scaled distance data is greater than or equal to a threshold distance value. Detection method.

18. In Paragraph 17, The operation of determining whether there is any remaining cargo is, An operation to determine the contour of the object in the preprocessed image based on the distance data; The operation of determining an area for the object based on the above-determined contour; and A process including determining whether there is residual cargo based on the area determined above, Detection method.

19. In Paragraph 18, Operation to determine whether there is any remaining cargo An operation to determine that there is no remaining cargo when the above-determined area is smaller than the critical area; and If the above-determined area is greater than or equal to the critical area, the operation of determining that there is residual cargo is included. Detection method.

20. In Paragraph 19, If it is determined that the above-mentioned remaining cargo exists, The operation further includes controlling the remaining cargo to be unloaded from the transport cart by tilting the cart receiving portion that accommodates the transport cart. Detection method.

Images