AI and image recognition-based unmanned crane control system for loading coils
Patent Information
- Application Number
- KR1020240076187
- Authority / Receiving Office
- KR · KR
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2024-06-12
- Publication Date
- 2026-09-29
- Estimated Expiration
- 2044-06-12
Smart Images

Figure 112024063267510-PAT00001_ABST
Abstract
Description
Technology Field
[0001] The present invention relates to an unmanned crane control system, and more specifically, to an AI and image recognition-based unmanned crane control system for stacking coils that utilizes an unmanned crane controlled by a deep learning algorithm to stack coils in an orderly manner at the appropriate location within a storage yard. Background Technology
[0002] Generally, unmanned cranes are used for the purpose of transporting relatively heavy materials, such as coiled coils, to a desired location without human intervention. These unmanned cranes perform lifting operations to lift materials vertically from a material loading skid, traversing operations to move materials laterally, driving operations to move materials longitudinally, and lowering operations to lower materials to the ground or on the vehicle.
[0003] The hoisting, traversing, or driving movements during the material transfer process are automatically performed by an unmanned crane, and the lifting operation to place or load the coiled material onto the work floor or the vehicle's cargo compartment is carried out through so-called collaboration by exchanging hand signals with workers on the ground or around the vehicle.
[0004] However, in the case of winding operations, due to variables caused by environmental factors such as disturbances, it is extremely difficult to predict the stacking location of the coil in advance or determine the location in real time. Furthermore, since it is impossible to assess the condition of surrounding objects when transporting the coil to the desired stacking location, a lack of smooth communication with nearby workers could potentially cause damage to the coil and surrounding objects.
[0005] Furthermore, there is a persistent problem that heavy materials, such as wound coils, have a significantly higher potential to cause major casualties than other lightweight materials. Prior art literature
[0006] Korean Registered Patent Publication No. 10-0825364 (Method for measuring coil size and loading in an automated warehouse, published April 28, 2008) Korean Registered Patent Publication No. 10-1447455 (Method and device for loading metal coils, published October 6, 2014) Korean Published Patent Publication No. 10-2021-0048284 (Low-profile trailer for transporting coils and method for loading and unloading coils using the same, published May 3, 2021) The problem to be solved
[0007] The technical problem and objective of the present invention is to provide an AI and image recognition-based unmanned crane control system for stacking coils that can stack coils in an orderly manner on the ground or on a vehicle using an unmanned crane controlled by a deep learning algorithm.
[0008] In addition, other technical problems and objectives of the present invention will be described below, and it should be noted that they will be encompassed in a broader scope by means and combinations within the scope that can be easily derived from the matters described in the claims of the present invention and the disclosure of the embodiments thereof. means of solving the problem
[0009] An AI and image recognition-based unmanned crane control system for stacking coils according to a first embodiment of the present invention for achieving the above technical problem and purpose comprises: an AI and image recognition-based unmanned crane (10) control system for stacking coils (20), comprising: an incoming product management unit (110) that manages a database of general information regarding coils (20) received into a storage yard; a storage yard map generation unit (120) that subdivides the location of the entire planar area within the storage yard into n columns and rows, forms a storage yard map by assigning unique address information to each subdivided location, and generates two-dimensional x, y coordinate values for the location where the unmanned crane (10) performs a lifting or unloading operation based on the address information of the storage yard map; and a barcode label (130) provided by the incoming product management unit (110) and attached to one side of the outer periphery of the coil (20). A trigger sensor (140) installed at the end of the arm of the unmanned crane (10) and outputting a trigger signal to the barcode scanner below when the barcode label (130) is located within the recognition range; a barcode scanner (150) that scans the general information of the winding coil (20) indicated on the barcode label (130) by the trigger signal output from the trigger sensor (140) and transmits it to the AI control unit below; and an image recognition unit (160) composed of a plurality of cameras installed on one side of a wall or one side of a column within the storage yard, which hoists the winding coil (20) and measures the physical distance to the current position of the unmanned crane (10) moving along a pre-installed rail and the x, y coordinate values on the storage yard map. The system comprises an AI control unit (170) that controls the operation of the unmanned crane (10) and automatically selects the location where the winding coil (20) is wound and stacked by applying a deep learning algorithm based on information measured by the image recognition unit (160) and comparing the position of the unmanned crane (10) with the information of the barcode label (130).
[0010] According to the AI and image recognition-based unmanned crane control system for stacking coils according to the second embodiment of the present invention, the stacking of coils (20) by the unmanned crane (10) by the unmanned crane (10) is carried out sequentially starting from the coil (20) whose release date is closest to the current date, under the judgment control of the AI control unit (170), and the coils (20) transported to the corresponding release point by the unmanned crane (10) are arranged in a regular arrangement on the ground or on a vehicle.
[0011] According to the AI and image recognition-based unmanned crane control system for stacking coils according to the third embodiment of the present invention, the unmanned crane (10) is configured to move along the shortest path from the point initially recognized by the image recognition unit (160) to a preset shipping point under the intervention of the AI control unit (170), based on the physical distance to the current location measured by the image recognition unit (160) and the x, y coordinate values on the storage yard map.
[0012] According to the AI and image recognition-based unmanned crane control system for stacking coils according to the fourth embodiment of the present invention, the receiving product management unit (110) may be equipped with a local-based or cloud-based database server that classifies coils (20) received into the storage yard by specifications, manufacturing date, receiving date, and manufacturing code items, and manages the classified items by creating and managing them in different information tables.
[0013] According to the AI and image recognition-based unmanned crane control system for stacking coils according to the fifth embodiment of the present invention, the AI control unit (170) performs deep learning-based learning using a CNN (Convolutional Neural Network) algorithm, and can apply an SSD (Single Shot MultiBox Detector)-based FPN (Feature Pyramid Network) algorithm to improve the accuracy of selecting the location where the coil (20) is to be wound and stacked by compensating for errors that occur during the learning process according to the CNN algorithm.
[0014] According to the AI and image recognition-based unmanned crane control system for stacking winding coils according to the 6th embodiment of the present invention, the image recognition unit (160) may include: at least one main camera (161) having a position and a viewpoint to enable shooting of the unmanned crane (10); at least one sub-camera (162) that, when the main camera (161) temporarily fails to shoot the unmanned crane (10), re-estimates the error by reflecting the time difference from the time when the unmanned crane (10) is re-shot and then acquires an object image; and a camera control unit (163) that controls the unmanned crane (10) to be shot for a certain period of time through the main camera (161) to acquire an object image and transmits the acquired object image to the AI control unit (170) via a wired or wireless communication network.
[0015] According to the AI and image recognition-based unmanned crane control system for stacking a coil according to the seventh embodiment of the present invention, the unmanned crane (10) may further include a pressure sensor provided inside the arm and wirelessly communicating with an AI control unit (170), and when the pressure sensor is included, the winding speed for stacking the coil (20) at a corresponding location is controlled by the judgment of the AI control unit (170) according to the magnitude of the reaction force measured by the pressure sensor when the coil (20) is wound.
[0016] Specific details of other embodiments are included in the description of the invention, etc. Effects of the invention
[0017] According to the present invention, which is based on a unique solution means to solve the technical problem described above, there is an advantage in that winding coils in a storage yard can be stacked in an orderly manner at appropriate locations on the ground or on a vehicle by utilizing an unmanned crane controlled by a deep learning algorithm.
[0018] In particular, the present invention is an AI-based unmanned automation process that not only shortens the movement time of winding coils but also has the advantage of providing work convenience and resolving inbound and outbound bottlenecks. Brief explanation of the drawing
[0019] FIG. 1 is a schematic diagram showing the configuration of an AI and image recognition-based unmanned crane control system for stacking winding coils according to an embodiment of the present invention. FIG. 2 is a diagram showing the overall configuration of an AI and image recognition-based unmanned crane control system for stacking winding coils according to an embodiment of the present invention. FIG. 3 is a flowchart exemplarily showing the control flow of an AI and image recognition-based unmanned crane control system for stacking coils according to an embodiment of the present invention. Specific details for implementing the invention
[0020] The configuration and effects of an embodiment of the present invention, which performs the task of achieving the objective of the present invention and resolving the drawbacks of the prior art, are described in detail below in conjunction with the attached drawings.
[0021] FIG. 1 is a schematic diagram showing the configuration of an AI and image recognition-based unmanned crane control system for stacking coils according to an embodiment of the present invention, FIG. 2 is a diagram showing the overall configuration of an AI and image recognition-based unmanned crane control system for stacking coils according to an embodiment of the present invention, and FIG. 3 is a flowchart exemplarily showing the control flow of an AI and image recognition-based unmanned crane control system for stacking coils according to an embodiment of the present invention.
[0022] Referring to FIGS. 1 to 3, the AI and image recognition-based unmanned crane control system for stacking coils according to the present invention utilizes an unmanned crane based on a deep learning algorithm to stack coils in an orderly manner at the appropriate location within a storage yard. To this end, the system is broadly configured to include an incoming product management unit (110), a storage yard map generation unit (120), a barcode label (130), a trigger sensor (140), a barcode scanner (150), an image recognition unit (160), and an AI control unit (170).
[0023] The receiving product management unit (110), which is a configuration according to an embodiment of the present invention, manages various information of the winding coil (20) received in the storage yard, such as specifications of the winding coil (weight, width, diameter, etc.), manufacturing date, receiving date, manufacturing code, and scheduled shipping date, by creating a database.
[0024] In addition, the above-mentioned incoming product management unit (110) may separately provide a local or cloud-based database server for classifying the winding coils (20) received in the storage yard by item, such as specifications, manufacturing date, receiving date, manufacturing code, and scheduled shipping date, and for creating and managing each classified item in a different information table.
[0025] Meanwhile, the storage yard map generation unit (120), which is configured according to an embodiment of the present invention, subdivides the locations of the entire planar area within the storage yard into n rows and columns, and forms a storage yard map by assigning unique address information to each subdivided location.
[0026] Then, based on the address information of the above storage yard map, two-dimensional x, y coordinate values are generated for the location where the lifting or lowering operation of the unmanned crane (10) is performed.
[0027] For example, if the planar shape of the entire storage yard is subdivided into a number of virtual columns and rows, multiple rectangular or square spaces are formed, much like a chessboard, and these spaces form one area or standard where the unmanned crane (10) will perform lifting operations.
[0028] The barcode label (130), which is a component according to an embodiment of the present invention, has information such as detailed specifications for the corresponding winding coil (20) printed on it. As an example, the barcode label (130) may be provided by the incoming product management unit (110) and attached to one side of the outer circumference facing upward of the winding coil (20), as shown in FIG. 1.
[0029] In a further embodiment of the present invention, instead of attaching the barcode label (130) to the winding coil (20), the barcode label (130) may be engraved on one side of the outer circumference of the winding coil (20) using a so-called marker method.
[0030] According to an embodiment of the present invention, at least one trigger sensor (140) is installed at the end of the arm of the unmanned crane (10), and when the above-mentioned barcode label (130) is located within a preset recognition range, it outputs a trigger signal to the barcode scanner (150) to be described later.
[0031] In addition, in an organic relationship with the trigger sensor (140) described above, the barcode scanner (150), which is a configuration according to an embodiment of the present invention, scans the information of the winding coil (20) displayed on the barcode label (130) by means of the trigger signal output from the trigger sensor (140) and transmits it to the AI control unit (170) to be described later.
[0032] Meanwhile, the image recognition unit (160), which is a configuration according to an embodiment of the present invention, is installed on one side of a wall or one side of a column structure within the storage yard and is provided in multiple units to measure the physical distance to the current position of the unmanned crane (10) moving along a pre-installed rail while lifting the winding coil (20) and the two-dimensional x, y coordinate values on the storage yard map.
[0033] As another embodiment of the image recognition unit (160) above, as shown in FIG. 2, the image recognition unit (160) may include a main camera (161), a sub-camera (162) having a correction function, and a camera control unit (163) that controls the main camera (161) and the sub-camera (162).
[0034] Here, the main camera (161) constituting the image recognition unit (160) may be provided with at least one unit and has a position and viewpoint so that the unmanned crane (10) can be photographed.
[0035] In addition, the sub-camera (162) constituting the image recognition unit (160) may be provided with at least one unit, just like the main camera (161), and when the main camera (161) temporarily fails to photograph the unmanned crane (10), it functions to re-estimate the error by reflecting the time difference from when the unmanned crane (10) is re-photographed and then acquire the object image.
[0036] And, the camera control unit (163) constituting the image recognition unit (160) controls the unmanned crane (10) to be photographed for a certain period of time through the main camera (161) to acquire an object image, and transmits the acquired object image to the AI control unit (170) to be described later via a wired / wireless communication network.
[0037] At this time, the wired and wireless communication network may have multiple modules that operate using different communication protocols, different communication standards, different frequencies, or combinations thereof.
[0038] For example, while one module can be achieved using wired communication, communication between each component within a defined operating area can be achieved using a wireless communication link or vice versa.
[0039] Alternatively, two different wireless communication standards on different frequencies may be used. In this case, separation of communication by multiple modules allows the configurations to continue operating in offline mode, which is when communication with an external instance is temporarily offline.
[0040] Meanwhile, the AI control unit (170), which is configured according to an embodiment of the present invention, controls the overall operation of the unmanned crane (10) and automatically selects the location where the winding coil (20) is to be wound and stacked by applying a deep learning algorithm based on information measured by the image recognition unit (160) and comparing the location of the unmanned crane (10) with the general information of the barcode label (130).
[0041] Here, the deep learning algorithm equipped in the AI control unit (170) performs optimization of the stacking location through stacking data analysis and learning, for example, predicts the quantity, weight, and required amount of the winding coil (20) to promote efficient autonomous operation of the unmanned crane (10).
[0042] For example, as one example of a deep learning learning result as described above, the unmanned crane (10) of the present invention, which operates under the judgment control of the AI control unit (170), identifies the optimal movement path based on a deep learning algorithm and, rather than sequential loading and stacking by pre-programmed logic, identifies the overall flow of the winding coil (20) and selects the optimal stacking location, thereby enabling the achievement of maximum efficiency.
[0043] As another example of a deep learning learning result, the unmanned crane (10) can maximize work efficiency by being configured to move along the shortest path from the point initially recognized by the image recognition unit (160) to a preset shipping point under the intervention of the AI control unit (170), based on the physical distance to the current location measured by the image recognition unit (160) and the x, y coordinate values on the storage yard map.
[0044] In addition, the stacking of the coils (20) by the lifting operation of the unmanned crane (10) can be carried out sequentially starting from the coil (20) whose release date is closest to the current date, under the judgment control of the AI control unit (170).
[0045] As described above, the travel time of the winding coil (20) can be shortened, and a bottleneck can be resolved during shipment.
[0046] In addition, the coiled coil (20) transported to the corresponding shipping point by the above-mentioned unmanned crane (10) is arranged in a regular arrangement on the ground or on the vehicle.
[0047] Meanwhile, the AI control unit (170) according to an embodiment of the present invention can perform deep learning-based learning using a CNN (Convolution Neural Network) algorithm, and can apply an SSD (Single Shot MultiBox Detector)-based FPN (Feature Pyramid Network) algorithm to further improve the accuracy of selecting the location where the coil (20) is wound and stacked by compensating for errors that occur during the learning process according to the CNN algorithm.
[0048] In a further aspect of the present invention, the deep learning algorithm may also be combined with a Genetic Algorithm (GA) that provides a new path through mutation to compensate for the disadvantage of getting stuck in a local optimum and the generally well-known ant colony optimization algorithm, thereby providing an opportunity to move to a global optimum.
[0049] For reference, the AI control unit (170) of the present invention may be composed of a personal PC or a business PC installed at the site, and the control application installed on the PC may operate in conjunction with a control application installed on a pre-registered smartphone terminal.
[0050] Therefore, the administrator can check the overall operating status of the system in real time through a control application installed on their smartphone.
[0051] In addition, the above control application can provide optimized information by managing the type of the winding coil (20), storage status, storage accumulation time, receiving time, and expected shipping time in a database format.
[0052] In an additional embodiment of the present invention, a separate operating means (not shown in the drawing) capable of manually controlling the entire system may be provided in case automatic control of the system by the AI control unit (170) is not possible.
[0053] For example, the above operating means may be configured to enable wired or wireless communication through a wired or wireless communication network with an AI control unit (170), and may consist of a receiver that transmits a command signal input by a transmitter to the AI control unit (170) and a transmitter that inputs a command signal to the receiver in a specific frequency band.
[0054] At this time, the PCB (Printed circuit board) of the AI control unit (170) that supports manual operation is a one-chip type PLD (Programmable Logic Device) type, and the one-chip type PLD may include a Basic program area and a Ladder Logic program area, and may support a multi-tasking structure that executes the Basic program and the Ladder Logic program simultaneously.
[0055] Here, the above-mentioned one-chip PLD is a type of driving program for controlling the AI control unit (170) to drive, and uses the above-mentioned ladder logic program, and the data communication area communicating via a wired / wireless communication network can use the above-mentioned BASIC program.
[0056] Furthermore, the aforementioned ladder logic is one of the relay logic programming languages used in programmable logic controllers (PLDs). Moreover, ladder logic is a programming language entirely different from programming languages such as Fortran or C; although it uses conditional statements, subroutines, and FOR NEXT statements, their meanings are very different.
[0057] For example, in general programming languages, instructions are executed sequentially, and higher-level instructions are executed before the final instruction until a loop terminates; however, in ladder logic, the instruction code for each level is scattered and executed simultaneously.
[0058] On the other hand, ladder logic programming unfolds in a ladder shape like a flowchart rather than a program, and each ladder logic uses two vertical lines on the left and right so that the condition on the left leads to the output on the right.
[0059] Meanwhile, the unmanned crane (10) according to an embodiment of the present invention may further include a pressure sensor (not shown in the drawing) that is provided inside the arm and communicates wirelessly with an AI control unit (170).
[0060] As described above, when a pressure sensor is provided on the inner side of the arm of the unmanned crane (10), the winding speed for stacking the winding coil (20) at the corresponding location can be increased or decreased by the judgment of the AI control unit (170) according to the magnitude of the reaction force measured by the pressure sensor when winding the winding coil (20).
[0061] Meanwhile, the embodiments described above may be implemented by hardware, firmware, software, or a combination thereof.
[0062] In the case of implementation by hardware, the method according to the embodiments of the present invention may be implemented by one or more ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), processors, controllers, microcontrollers, microprocessors, etc.
[0063] In the case of implementation by the above firmware or software, the method according to the embodiments of the present invention may be implemented in the form of a module, procedure, or function that performs the functions or operations described above.
[0064] The above software code can be stored in a memory unit and executed by a processor. The memory unit may be located inside or outside the processor and may exchange data with the processor by various known means.
[0065] Furthermore, the present invention can be implemented as computer-loadable code on a computer-loadable recording medium. A computer-loadable recording medium may include all types of recording devices in which data that can be loaded by a computer system is stored. For example, computer-loadable recording media include ROM, RAM, SSD, USB, SD card, etc., and may also include implementations in the form of transmission via the Internet. In addition, the computer-loadable recording medium may be distributed across networked computer systems, allowing computer-loadable code to be stored and executed in a distributed manner.
[0066] Preferred embodiments of the present invention have been described above. However, those skilled in the art will understand that the present invention, as described so far, may be implemented in modified forms without departing from the essential characteristics of the invention. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. Furthermore, the scope of the present invention is defined by the claims, not by the foregoing description, and all variations within the equivalent scope should be interpreted as being included in the present invention. Explanation of the symbols
[0067] 10 : Unmanned crane 20 : Winding coil 110: Incoming Product Management Department 120: Storage Yard Map Generation Department 130 : Barcode label 140 : Trigger sensor 150 : Barcode scanner 160 : Image recognition unit 161 : Main Camera 162 : Sub Camera 163 : Camera control unit 170 : AI control unit
Claims
Claim 1 As an AI and image recognition-based unmanned crane (10) control system for stacking coils (20), the system comprises: an incoming product management unit (110) that manages a database of all information regarding coils (20) received in the storage yard; a storage yard map generation unit (120) that subdivides the location of the entire flat area within the storage yard into n columns and rows, forms a storage yard map by assigning unique address information to each subdivided location, and generates two-dimensional x, y coordinate values for the location where the unmanned crane (10) performs a lifting or lowering operation based on the address information of the storage yard map; a barcode label (130) provided by the incoming product management unit (110) and attached to one side of the outer periphery of the coil (20); a trigger sensor (140) installed at the end of the arm of the unmanned crane (10) and outputs a trigger signal to the barcode scanner below when the barcode label (130) is located within the recognition range; A barcode scanner (150) that scans general information of a winding coil (20) displayed on a barcode label (130) by means of a trigger signal output from a trigger sensor (140) and transmits it to an AI control unit below; an image recognition unit (160) composed of a plurality of cameras that are installed on one side of a wall or one side of a column within a storage yard and measure the physical distance to the current position of an unmanned crane (10) that lifts the winding coil (20) and moves along a pre-installed rail, and the x, y coordinate values on the storage yard map; and an AI control unit (170) that controls the operation of the unmanned crane (10) and automatically selects a position where the winding coil (20) is to be lifted and stacked by applying a deep learning algorithm based on the information measured by the image recognition unit (160) and comparing the position of the unmanned crane (10) with the general information of the barcode label (130).An AI and image recognition-based unmanned crane control system for stacking coils, characterized in that the stacking of coils (20) by the hoisting operation of the unmanned crane (10) is carried out sequentially starting from the coil (20) whose release date is closest to the current date under the judgment control of the AI control unit (170), and the coils (20) transported to the corresponding release point by the unmanned crane (10) are arranged in a regular arrangement on the ground or on a vehicle. Claim 2 delete Claim 3 An AI and image recognition-based unmanned crane control system for stacking coils according to claim 1, wherein the unmanned crane (10) moves along the shortest path from the point initially recognized by the image recognition unit (160) to a preset shipping point under the intervention of the AI control unit (170), based on the physical distance to the current location measured by the image recognition unit (160) and the x, y coordinate values on the storage yard map. Claim 4 In claim 1, the AI and image recognition-based unmanned crane control system for stacking coils is characterized in that the receiving product management unit (110) classifies coils (20) received in the storage yard by items of specifications, manufacturing date, receiving date, and manufacturing code, and has a local-based or cloud-based database server that creates and manages the classified items in different information tables. Claim 5 In claim 1, the AI control unit (170) performs deep learning-based learning using a CNN (Convolution Neural Network) algorithm, and applies an SSD (Single Shot MultiBox Detector)-based FPN (Feature Pyramid Network) algorithm to improve the accuracy of selecting the location where the coil (20) is wound and stacked by compensating for errors occurring during the learning process according to the CNN algorithm, thereby creating an AI and image recognition-based unmanned crane control system for stacking coils. Claim 6 In claim 1, the image recognition unit (160) comprises: at least one main camera (161) having a position and a viewpoint to enable shooting of the unmanned crane (10); at least one sub camera (162) that, when the main camera (161) temporarily fails to shoot the unmanned crane (10), re-estimates an error by reflecting the time difference from when the unmanned crane (10) is re-shot and then acquires an object image; and a camera control unit (163) that controls the unmanned crane (10) to be shot for a certain period of time through the main camera (161) to acquire an object image and transmits the acquired object image to the AI control unit (170) via a wired or wireless communication network; characterized in that it is an AI and image recognition-based unmanned crane control system for stacking winding coils. Claim 7 An AI and image recognition-based unmanned crane control system for stacking coils, wherein, in claim 1, the unmanned crane (10) further includes a pressure sensor that is provided inside the arm and communicates wirelessly with an AI control unit (170), and when the coil (20) is hoisted, the unwinding speed for stacking the coil (20) at a corresponding location is controlled by the judgment of the AI control unit (170) according to the magnitude of the reaction force measured by the pressure sensor.
Citation Information
Patent Citations
Loading and unloading machine for container, containermanagement system and loading and unloading method ofcontainer
KR1020060059758A
Magnet Crane Automation Device And Method Using Image-based Deep Learning And Distance Sensor
KR1020240061090A
Unmanned crane control system for coil products in / out
KR102649251B1