Mobile robot control apparatus and method

Abstract

The present disclosure relates to a mobile robot control apparatus and method, and includes the mobile robot control apparatus comprising: at least one memory including computer program instructions; and at least one processor for executing the computer program instructions, wherein the at least one processor controls that a mobile robot moves while being supported by a transfer line provided along a belt conveyor, determines, on the basis of sensing data collected from at least one from among a thermal imaging sensor, an acoustic sensor and an image sensor provided in the mobile robot, whether the abnormality of the belt conveyor is detected, and outputs, on the basis of the result of determining whether the abnormality is detected, an abnormality response control signal for performing a preset abnormality response operation.

Description

Mobile robot control device and method

[0001] The present disclosure relates to a mobile robot control device and method, and more specifically, to a mobile robot control device and method for detecting abnormalities in a belt conveyor.

[0002] Belt conveyors are devices that transport logistics via belts, and components such as belts and bearings may wear out or deteriorate during continuous use. Considering this, belt conveyors require continuous inspection and maintenance; however, continuously inspecting the entire section where the belt conveyor is installed demands significant manpower and cost.

[0003] In particular, belt conveyors used to transport raw materials or raw materials often have the problem of being difficult to access because they are installed over long distances, sometimes reaching hundreds of kilometers, and are frequently positioned high above the ground.

[0004] The present disclosure aims to provide a mobile robot control device and method capable of detecting abnormalities in a belt conveyor.

[0005] In one aspect, the embodiments may provide a mobile robot control device comprising at least one memory including computer program instructions and at least one processor for executing computer program instructions, wherein the at least one processor controls the mobile robot to move while being supported by a transfer line installed along a belt conveyor, determines whether an abnormality is detected in the belt conveyor based on sensing data collected from at least one of a thermal imaging sensor, an acoustic sensor, and an image sensor installed in the mobile robot, and outputs an abnormality response control signal for performing a preset abnormality response operation based on the result of determining whether an abnormality is detected.

[0006] In another aspect, the present embodiments may provide a method for controlling a mobile robot, comprising: a robot movement step for controlling the mobile robot to move while being supported by a transfer line installed along a belt conveyor; a sensing data collection step for collecting sensing data from at least one of a thermal imaging sensor, an acoustic sensor, and an image sensor installed on the mobile robot; an anomaly detection step for determining whether an anomaly is detected in the belt conveyor based on the sensing data; and an anomaly response step for outputting a control signal to perform a preset anomaly response action based on the result of determining whether an anomaly is detected in the belt conveyor.

[0007] According to the present disclosure, a mobile robot control device and method capable of detecting abnormalities in a belt conveyor can be provided.

[0008] FIG. 1 is a diagram illustrating the configuration of a belt conveyor monitoring system according to the present disclosure in an exemplary manner.

[0009] FIG. 2 is a diagram showing an example of the configuration of a mobile robot according to one embodiment.

[0010] FIG. 3 is a block diagram of a computing system according to one embodiment.

[0011] FIG. 4 is a diagram illustrating a configuration for detecting whether a fire has occurred using a mobile robot control device according to one embodiment.

[0012] FIG. 5 is a diagram illustrating, in an exemplary manner, a configuration for responding to a fire using a mobile robot control device according to one embodiment.

[0013] FIG. 6 is a diagram illustrating a configuration for detecting whether a fault exists using a mobile robot control device according to one embodiment.

[0014] FIG. 7 is a flowchart relating to a mobile robot control method according to one embodiment.

[0015] FIG. 8 is a flowchart illustrating an embodiment for detecting and responding to whether a fire has occurred in a mobile robot control method according to one embodiment.

[0016] Hereinafter, some embodiments of the present disclosure will be described in detail with reference to the exemplary drawings. In assigning reference numerals to the components of each drawing, the same components may have the same reference numeral as much as possible, even if they are shown in different drawings. Furthermore, in describing the embodiments, if it is determined that a detailed description of related known components or functions may obscure the essence of the technical concept, such detailed description may be omitted. Where terms such as "comprising," "having," or "consisting of" are used in this specification, other parts may be added unless "only" is used. Where a component is expressed in the singular, it may include a plural unless otherwise specified.

[0017] Additionally, terms such as first, second, A, B, (a), (b), etc., may be used to describe the components of the present disclosure. These terms are used merely to distinguish the components from other components, and the nature, order, sequence, or number of the components are not limited by such terms.

[0018] In describing the positional relationship of components, where it is stated that two or more components are "connected," "combined," or "joined," it should be understood that while the two or more components may be directly "connected," "combined," or "joined," they may also be "connected," "combined," or "joined" with other components "intervened." Here, the other components may be included in one or more of the two or more components that are "connected," "combined," or "joined" with one another.

[0019] In describing the temporal flow relationship regarding components, methods of operation, or methods of production, for example, when the temporal or sequential relationship is described using "after," "following," "next," or "before," it may include cases where the relationship is not continuous unless "immediately" or "directly" is used.

[0020] Meanwhile, where numerical values ​​or corresponding information regarding a component (e.g., levels, etc.) are mentioned, even without separate explicit notation, the numerical values ​​or corresponding information may be interpreted as including a range of error that may occur due to various factors (e.g., process factors, internal or external shocks, noise, etc.).

[0021]

[0022] FIG. 1 is a diagram illustrating the configuration of a belt conveyor monitoring system according to the present disclosure in an exemplary manner.

[0023] Referring to FIG. 1, a belt conveyor monitoring system (100) according to the present disclosure may include a belt conveyor (110), a mobile robot (120), a transfer line (130), a docking station (140), and a control center (150). The belt conveyor (110), the mobile robot (120), the transfer line (130), the docking station (140), and the control center (150) may be connected to each other via a physical or communication network.

[0024] A belt conveyor (110) may include a belt that rotates by external power and transports goods, a drive unit that provides power to rotate the belt, and guide rollers that support the belt and guide it to move along a predetermined path.

[0025] In some cases, the guide roll may include a bearing to enable rotational movement. And, when the belt conveyor is used continuously, wear or deterioration may occur in the bearing, so continuous inspection and maintenance are required, and the mobile robot (120) can perform such inspection and maintenance.

[0026] The mobile robot (120) is supported by a transfer line (130) and can move, and can detect abnormalities such as overheating, fire, or malfunction of the belt conveyor, and can perform an abnormality response action based on the abnormality detection result. A more detailed configuration of the mobile robot (120) is described below in FIG. 2.

[0027] The transfer line (130) can be installed along the belt conveyor. For example, the transfer line (130) can be installed along the length of the belt conveyor from the side of the belt conveyor. As another example, the transfer line (130) can be installed on the top of the belt conveyor. In this case, a support can be installed to support or secure the transfer line (130).

[0028] For example, the transfer line (130) may be installed in a manner that allows the mobile robot (120) to be supported by the transfer line (130) and move. For example, the transfer line (130) may be combined with the mobile robot (120) and may provide a path for moving forward and backward along the length of the belt conveyor based on electrical power input to the mobile robot (120).

[0029] For example, the transfer line (130) may include a wire, and a part of the mobile robot (120) may be installed in a form that is coupled to the wire and can move along the wire. Alternatively, the transfer line (130) may be installed in a rail-shaped configuration, so that a part of the mobile robot (120) is mounted on the rail and can move along the rail.

[0030] For example, the transfer line (130) can enable the movement of the mobile robot (120) based on electrical power output from the drive motor of the mobile robot (120). For another example, the transfer line (130) can enable the movement of the mobile robot (120) based on an electromagnetic force resulting therefrom by outputting an induced current.

[0031] The docking station (140) may provide a docking interface capable of mounting the mobile robot (120). In some cases, the docking station (140) may include a charging module capable of charging the power of the mobile robot (120) via wired or wireless connection.

[0032] For example, the docking station (140) may be installed at the start and end points of the belt conveyor (110), or between the start and end points of the belt conveyor (110). And, if the docking station (140) is installed between the start and end points of the belt conveyor (110), the mobile robot (120) may stop at the docking station (140) to be mounted or charged while moving along the path of the belt conveyor (110).

[0033] The control center (150) can transmit and receive signals with the mobile robot (120). For example, the control center (150) can transmit a signal requesting location information, operating status information, and remaining power information of the mobile robot (120). As another example, the control center (150) can receive an overheating alert signal, a fire alert signal, and a malfunction alert signal from the mobile robot (120).

[0034] In some cases, the belt conveyor monitoring system (100) may transmit and receive signals with a manager terminal (not shown). For example, the manager terminal may transmit a signal requesting location information of the mobile robot (120) to a control center (150) or to the mobile robot (120). As another example, the mobile robot control device may transmit location information of the mobile robot (120) to the manager terminal.

[0035] Meanwhile, the mobile robot control device according to the present disclosure can determine the current position of the mobile robot (120) based on a signal transmitted through a motor encoder, and based on the determined position, can generate information regarding the location of the abnormality in the event that an abnormality such as a breakdown, overheating, or fire occurs in the belt conveyor.

[0036] FIG. 2 is a diagram showing an example of the configuration of a mobile robot according to one embodiment.

[0037] Referring to FIG. 2, a mobile robot (200) according to one embodiment may include a thermal imaging sensor (210), an acoustic sensor (220), an image sensor (230), a fire extinguisher (240), and a power button (250).

[0038] The thermal imaging sensor (210) may include any sensor capable of detecting infrared rays, converting them into electrical signals, and converting them into image signals to output an image showing a temperature distribution, and may be controlled by a mobile robot control device according to the present disclosure. For example, it may include a Long Wave Infrared (LWIR) imaging camera.

[0039] For example, a thermal imaging sensor (210) enables a mobile robot (200) to sense thermal imaging data while moving along a belt conveyor, and a mobile robot control device according to the present disclosure can determine the temperature distribution of the belt conveyor, the degree of overheating, and whether a fire has occurred based on the thermal imaging sensing data.

[0040] The acoustic sensor (220) may include any device capable of sensing ambient sound and may be controlled by a mobile robot control device according to the present disclosure.

[0041] For example, the acoustic sensor (220) may include a piezoelectric sensor, a capacitive sensor, a magnetostrictive sensor, a fiber optic sensor, a MEMS microphone, etc.

[0042] For example, an acoustic sensor (220) enables the mobile robot (200) to sense acoustic data while moving along a belt conveyor, and the mobile robot control device according to the present disclosure can determine whether the belt conveyor or a belt, guide roll, bearing, etc. included therein is faulty based on the acoustic sensing data.

[0043] The image sensor (230) may include any device capable of sensing an image and may be controlled by a mobile robot control device according to the present disclosure.

[0044] For example, the image sensor (230) may include at least one of a CCD (Charge-Coupled Device) sensor and a CMOS (Complementary Metal-Oxide-Semiconductor) sensor. Additionally, it may include at least one of an RGB camera capable of sensing color image data or a monochrome camera capable of generating monochrome image data.

[0045] For example, an image sensor (230) enables the mobile robot (200) to sense image data while moving along a belt conveyor, and the mobile robot control device according to the present disclosure can determine whether a fire or smoke is occurring on the belt conveyor and around it based on the image sensing data.

[0046] The fire extinguishing device (240) may include any device used to extinguish a fire, and its operation may be controlled by a mobile robot control device according to the present disclosure.

[0047] For example, the fire extinguisher (240) may include a fire extinguishing substance that can be used to extinguish a fire and a configuration capable of spraying a hydrated substance. Here, the fire extinguishing substance may include not only powder form but also liquid or gaseous forms of fire extinguishing substances.

[0048] As a specific example, the fire extinguisher (240) may include a CO2 fire extinguisher or a powder fire extinguisher. In this case, the fire can be extinguished using a CO2 fire extinguishing agent or a powder fire extinguishing agent suitable for extinguishing fires that occur in the rubber, which is the main material of the belt.

[0049] The power button (250) may include any configuration capable of performing the operation of driving or shutting off the power of the mobile robot (200), and its operation may be controlled by the mobile robot control device according to the present disclosure.

[0050] Meanwhile, the present disclosure may include a mobile robot control device for implementing the aforementioned device and / or method. For example, the mobile robot control device may be implemented as a computing system.

[0051]

[0052] FIG. 3 is a block diagram of an exemplary computing system. A computing system or computing device may be used to include or implement components such as a system or a data processing system. A computing system includes a bus or other communication component for transmitting information, and a processor or processing circuit connected to the bus to process information. A computing system may also include one or more processors or processing circuits connected to the bus to process information. A computing system also includes main memory, such as random access memory (RAM) or other dynamic storage devices connected to the bus to store information, and instructions (instructions) to be executed by the processor. The main memory may be or may include a data store. The main memory may also be used to store location information, temporary variables, or other intermediate information during the execution of instructions by the processor. A computing system may further include ROM or other static storage devices connected to the bus to store static information and instructions for the processor. Storage devices, such as solid-state devices, magnetic disks, or optical disks, may be coupled to the bus to continuously store information and instructions. A storage device may include or be part of a data store.

[0053] A computing system may be connected to a display, such as a liquid crystal display or an active matrix display, to display information to a user via a bus. Input devices, such as a keyboard containing alphanumeric and other keys, may be connected to the bus to transmit information and command selections to the processor. Input devices may include a touch screen display. Input devices may also include cursor controls, such as a mouse, trackball, or cursor direction keys, to transmit directional information and command selections to the processor and to control cursor movement on the display. The display may be part of a data processing system, a client computing device, or other components.

[0054] The processes, systems, and methods described herein may be implemented by a computing system in response to a processor executing an array of instructions contained in main memory. These instructions may be read into main memory from other computer-readable media, such as storage devices. The execution of the array of instructions contained in main memory causes the computing system to perform the exemplary processes described herein. In a multiprocessing array, one or more processors may also be used to execute instructions contained in main memory. Hardwired circuits may be used in place of or in conjunction with hardware instructions with the systems and methods described herein. The systems and methods described herein are not limited to any specific combination of hardware circuits and software.

[0055] Although exemplary computing systems have been described above, the essence including the operations described herein may be implemented in other types of digital electronic circuits, or in computer software, firmware, or hardware including structures disclosed herein and structural equivalents thereof or combinations of one or more of these.

[0056] "Data processing system," "computing device," "module," "engine," "component," or "computing device" includes various devices, devices, and machines for processing data, including, for example, a programmable processor, a computer, a system on a chip, or a number of such items or combinations thereof. The device may include special-purpose logic circuits, for example, a field programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). In addition to hardware, the device may also include code that creates an execution environment for the corresponding computer program, for example, processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or a combination of one or more of these. The device and execution environment may realize various different computing model infrastructures, such as web services, distributed computing, and grid computing infrastructures. A content request module, a content rendering module, or a rendered content delivery module may include or share one or more data processing devices, systems, computing devices, or processors. Components of the system may include or share one or more data processing devices, systems, computing devices, or processors.

[0057] A computer program (also known as a program, software, software application, app, script, or code) may be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and may be distributed as a standalone program or in any form including modules, components, subroutines, objects, or other units suitable for use in a computing environment. A computer program may or may not correspond to a file in a file system. A computer program may be stored in a file containing other programs or data (e.g., one or more scripts stored in a markup language document), a single file dedicated to that program, or a portion of a file containing multiple coordinated files (e.g., files storing one or more modules, subprograms, or parts of code). A computer program may be distributed to be executed on a single computer or a single site, or on multiple computers distributed across multiple sites and interconnected by a communication network.

[0058] The processes and logic flows described herein may be performed by one or more programmable processors that execute one or more computer programs (e.g., components of a data processing system) to perform actions by operating input data and generating outputs. The processes and logic flows may also be performed by special-purpose logic circuits, e.g., a field programmable gate array (FPGA) or an application-specific integrated circuit (ASIC), and devices may also be implemented by special-purpose logic circuits. Devices suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, such as semiconductor memory devices like EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; optomagnetic disks; and CD-ROM and DVD-ROM disks. Processors and memory may be complemented or integrated by special-purpose logic circuits.

[0059] For example, a mobile robot control device according to the present disclosure comprises at least one memory including computer program instructions and at least one processor for executing computer program instructions, wherein the at least one processor controls the mobile robot to move while being supported by a transfer line installed along a belt conveyor, determines whether an abnormality is detected in the belt conveyor based on sensing data collected from at least one of a thermal imaging sensor, an acoustic sensor, and an image sensor installed in the robot, and outputs an abnormality response control signal for performing a preset abnormality response operation based on the result of determining whether an abnormality is detected.

[0060] The processor can control the mobile robot to move while being supported by a transfer line installed along the belt conveyor. In this case, the transfer line may be installed along the longitudinal direction of the belt conveyor from the side or top of the belt conveyor, and a support member may be installed to support or fix the transfer line.

[0061] For example, a processor can control the mobile robot to move forward and backward along the length of a belt conveyor based on electrical power input to at least one of the mobile robot or the transfer line, where the mobile robot and the transfer line are combined.

[0062] For example, the processor may collect thermal imaging sensing data from a thermal imaging sensor. In this case, the thermal imaging sensor may include any sensor capable of detecting infrared radiation, converting it into an electrical signal, and converting it into an image signal to output it in the form of an image representing a temperature distribution, and may be controlled to sense thermal imaging data for a belt conveyor on a path moved by a mobile robot control device according to the present disclosure.

[0063] For example, the processor may collect acoustic sensing data from an acoustic sensor. In this case, the acoustic sensor may include any device capable of sensing ambient sound, and may be controlled to sense sound generated on a belt conveyor along a path moved by the mobile robot control device according to the present disclosure.

[0064] For example, the processor may collect image sensing data from an image sensor. In this case, the image sensor may include any device capable of sensing images, and may be controlled by the mobile robot control device according to the present disclosure to sense images of a belt conveyor along the path of movement of a mobile robot.

[0065] The processor can determine whether an abnormality is detected in the belt conveyor based on sensing data collected from at least one of a thermal imaging sensor, an acoustic sensor, and an image sensor installed on the mobile robot.

[0066] For example, the processor may determine that a fire has occurred on the belt conveyor if it satisfies a temperature condition in which the thermal imaging sensing data includes an area above a preset threshold temperature.

[0067] For example, the critical temperature may include a preset first critical temperature and a second critical temperature. In this case, the second critical temperature may be set to a temperature higher than the first critical temperature.

[0068] For example, the processor determines that the belt conveyor is overheated when it satisfies a first temperature condition in which the thermal image sensing data includes an area that is above a first threshold temperature and below a second threshold temperature, and determines that a fire has occurred in the belt conveyor when it satisfies a second temperature condition in which the thermal image sensing data includes an area that is above a second threshold temperature.

[0069] For example, the processor may determine that a fire has occurred on the belt conveyor if the area above a critical temperature in the thermal imaging sensing data further satisfies the condition that the area is above a preset critical area.

[0070] For example, the processor may determine that a fire has occurred on the belt conveyor when both the temperature area condition, in which the area above a critical temperature in the thermal imaging sensing data is greater than a critical area, and the smoke area condition, in which the area where smoke is detected in the image sensing data is greater than a critical area, are satisfied.

[0071] For example, the processor may set a critical temperature by further considering received seasonal information. In this case, the seasonal information may be received from a control center or may be set based on date information received from a timer built into the mobile robot.

[0072] For example, the season information can be set to 'Winter' when the date information is December, January, or February, to 'Spring / Autumn' when it is March, April, May, September, or October, and to 'Summer' when it is June, July, August, or September.

[0073] For example, the critical temperature can be set to 65 degrees when the season information is 'winter', 75 degrees when it is 'spring / autumn', and 85 degrees when it is 'summer'.

[0074] For example, the processor may set at least one of a critical temperature, critical area, critical time, and critical number of frames by further considering the received climate information. In this case, the climate information may be received from a control center using a communication device installed on a mobile robot, or may be received through a pre-configured climate information network connection.

[0075] For example, if the outside temperature determined from climate information is less than -10°C, the critical temperature and critical area can be set relatively smaller. For another example, if the outside temperature is 30°C or higher, the critical temperature and critical area can be set relatively larger.

[0076] For example, the processor may determine that a fire has occurred on the belt conveyor if the time condition is further satisfied in which an area above a critical temperature in the thermal imaging sensing data is sensed continuously for longer than a preset critical time.

[0077] For example, the processor can acquire speed sensing data that senses the movement speed of the robot from a speed sensor installed on the robot, and set a threshold time and a threshold number of frames based on the speed sensing data.

[0078] For example, if the speed sensing data is less than 6 km / h, the threshold time can be set to 5 seconds, and if it is 6 km / h or more, the threshold time can be set to 3 seconds. As another example, if the speed sensing data is less than 6 km / h, the threshold number of frames can be set to 15 frames, and if it is 6 km / h or more, the threshold time can be set to 10 frames.

[0079] For example, the processor collects image sensing data from an image sensor, and if the condition for recognizing an area where smoke is detected in the image sensing data is satisfied, it can determine that a fire has occurred in the area where smoke is detected.

[0080] For example, the processor may determine that a fire has occurred in the area where smoke is detected if there is a condition in the image sensing data where smoke is detected and the area where smoke is detected is greater than a critical area.

[0081] For example, the processor may determine that a fire has occurred on the belt conveyor if it satisfies the condition that there is an area above a critical temperature in the thermal imaging sensing data and an area where smoke is detected in the image sensing data. In some cases, the processor may determine that a fire has occurred if it further satisfies the condition that the thermal imaging sensing data and the image sensing data satisfying the above conditions are sensed at the same time or in adjacent time periods.

[0082] For example, the processor can determine a fault acoustic evaluation score of acoustic sensing data using a preset artificial intelligence model, and if the fault acoustic evaluation score is greater than or equal to a preset fault threshold score, it can determine that a fault has occurred in the belt conveyor.

[0083] For example, the failure threshold score may include a preset first failure threshold score and a second failure threshold score. In this case, the second failure threshold score may be set to a score higher than the first failure threshold score.

[0084] For example, a belt conveyor failure may include a failure caused by misalignment of the guide rolls. In this case, during the operation of the belt conveyor, a sound different from that of the normal state may be generated in the area where the failure occurred due to the misalignment of the guide rolls.

[0085] For example, the processor can sense fault acoustic data regarding acoustics occurring in a fault area due to misalignment of guide rolls. In some cases, the processor can perform pre-learning by using the collected fault acoustic data as training data for an artificial intelligence model, evaluating the degree of similarity between the input acoustic data and the acoustics of the fault acoustic data, and determining the fault acoustic evaluation score.

[0086] For example, the processor may determine a normal state if the fault acoustic evaluation score is less than a first fault threshold score, determine a fault warning state if it is greater than or equal to the first fault threshold score and less than a second fault threshold score, and determine a fault occurrence state if it is greater than or equal to the second fault threshold score.

[0087] For example, the artificial intelligence model may include a pre-trained model that uses failure acoustic data in the event of a failure in the guide roll of a belt conveyor as training data, evaluates the degree of similarity between the input acoustic data and the acoustic of the failure acoustic data, and determines the failure acoustic evaluation score.

[0088] For example, an artificial intelligence model based on a Convolutional Neural Network (CNN) may include an input layer that receives acoustic data, two or more layers, a convolution layer that extracts features of the input acoustic data in the form of a feature map based on convolution operations, a pooling layer that reduces the size of the feature map extracted from the convolution layer and extracts key features, and a fully connected layer that performs final classification or final prediction of the acoustic data based on the extracted features.

[0089] In some cases, the artificial intelligence model may include converting acoustic data into a frequency domain signal when the acoustic data is in the form of a time domain signal and using it as training data or input data for the artificial intelligence model.

[0090] The processor can output an abnormality response control signal to perform a preset abnormality response action based on the result of determining whether an abnormality is detected.

[0091] For example, the processor may output a notification operation control signal to transmit information regarding the detected abnormality when an abnormality is detected in the belt conveyor. In this case, the notification operation control signal may include information regarding at least one of an overheating notification, a fire notification, and a failure notification. Additionally, the notification operation control signal may include a signal to control the transmission of a notification to at least one of the control center, docking station, and manager terminal of the belt conveyor system.

[0092] For example, the anomaly response operation may include a fire notification operation for transmitting fire information regarding the belt conveyor. For another example, the anomaly response operation may include a fire extinguisher operation for activating a fire extinguisher in the area where the fire occurred. For yet another example, the anomaly response operation may include a fault notification operation for transmitting fault information regarding the belt conveyor.

[0093] For example, the processor may output an overheating notification control signal to transmit overheating information when it is determined that the belt conveyor is overheated. For example, the overheating information may include information regarding the location of the area where overheating occurred on the belt conveyor, the temperature detected in that area, the date and time when overheating was detected, etc.

[0094] For example, the processor can output a fire notification control signal to transmit fire information when it is determined that a fire has occurred on the belt conveyor.

[0095] For example, fire information may include information regarding the location of the area where a fire occurred on the belt conveyor, the temperature detected in that area, the date and time when overheating was detected, etc. In some cases, fire information may include information regarding whether the operation of a fire extinguisher was executed according to the fire extinguisher control signal described below, and if said operation was executed, the temperature change of the fire area accordingly.

[0096] For example, if the processor determines that a fire has occurred on the belt conveyor, it may output a control signal to perform the operation of a fire extinguisher. In this case, the processor may output a fire extinguisher control signal to operate the fire extinguisher in the area where the fire occurred.

[0097] For example, the operation of a fire extinguisher may include spraying fire extinguishing liquid onto a belt conveyor section where it is determined that a fire has occurred, from a fire extinguisher installed on a mobile robot.

[0098] For example, the processor can control the operation of a fire extinguisher when conditions are satisfied such that there is an area above a critical temperature in the thermal image sensing data, the area above the critical temperature is greater than a critical area, and an area where smoke is detected in the image sensing data exists.

[0099] For example, the processor can control the operation of a fire extinguisher when the condition is satisfied that the area above a critical temperature in the thermal imaging sensing data is greater than or equal to a critical area, and the condition that the area where smoke is detected in the image sensing data is greater than or equal to a critical area.

[0100] For example, the processor may output a control signal to perform a fault notification operation when it is determined that a fault has occurred in the belt conveyor. For example, the belt conveyor failure may include a failure caused by belt tearing, a failure caused by bearing damage in the guide roll, and a failure caused by misalignment of the guide roll.

[0101] In FIGS. 4 to 6 below, exemplary embodiments of detecting and responding to abnormalities using a mobile robot control device according to one embodiment are described.

[0102] FIG. 4 is a diagram illustrating a configuration for detecting whether a fire has occurred using a mobile robot control device according to one embodiment.

[0103] Referring to FIG. 4, a mobile robot control device according to one embodiment can sense a certain area of ​​a belt conveyor (430) using a thermal imaging sensor (412) installed on a mobile robot (410) that travels along a transfer line (420), and can determine whether a fire has occurred in the sensed area based on the thermal imaging sensing data collected therefrom.

[0104] For example, if an area (432) above a critical temperature is detected in the thermal imaging sensing data, it can be determined that a fire has occurred on the belt conveyor (430). In this case, the area where the fire occurred can be determined to be the area (432) above the critical temperature and its surroundings within the belt conveyor (430).

[0105] And in some cases, if the area (432) above the critical temperature further satisfies the condition of being above the critical area, it can be determined that a fire has occurred. For example, if the area (432) above the critical temperature is below the critical area, unlike in FIG. 5 which will be described later, it is not determined that a fire has occurred, but rather that it is in an overheated state, so that control of the fire extinguisher (416) is not performed, and only an overheating notification operation is performed to send an overheating warning notification to a control center, etc.

[0106] Additionally, by further using an image sensor (not shown) installed on the mobile robot (410), it may be determined that a fire has occurred when the condition for smoke to be detected in the image sensing data is further satisfied.

[0107] FIG. 5 is a diagram illustrating, in an exemplary manner, a configuration for responding to a fire using a mobile robot control device according to one embodiment.

[0108] Referring to FIG. 5, a mobile robot control device according to one embodiment can control a fire extinguisher (516) installed on a mobile robot (510) to operate when it is determined that a fire has occurred on a belt conveyor (530).

[0109] For example, if an area (532) above a critical temperature is detected in the thermal imaging sensing data collected from the thermal imaging sensor (512) and it is determined that a fire has occurred, a fire notification operation to transmit a fire occurrence notification to a control center, etc., and a fire extinguisher operation to control a fire extinguisher (516) installed on the mobile robot (510) to spray fire extinguishing liquid, etc., can be performed.

[0110] And in some cases, it can be determined that a fire has occurred when the area (532) above the critical temperature further satisfies the condition of being above the critical area. For example, in FIG. 4, the area (432) above the critical temperature is below the critical area and only an overheating notification operation is performed, but in FIG. 5, the area (532) above the critical temperature is above the critical area and can be controlled to perform both a fire notification operation and a fire extinguisher operation.

[0111] FIG. 6 is a diagram illustrating a configuration for detecting whether a fault exists using a mobile robot control device according to one embodiment.

[0112] Referring to FIG. 6, a mobile robot control device according to one embodiment can sense a certain area of ​​a belt conveyor (630) using an acoustic sensor (614) installed on a mobile robot (610) that travels along a transfer line (620), and can determine whether a fault has occurred in the sensed area based on acoustic sensing data collected therefrom.

[0113] For example, by using a pre-trained artificial intelligence model with failure acoustic data as training data, and inputting acoustic sensing data into the model to evaluate the degree of similarity of the acoustics of the failure acoustic data, a failure acoustic evaluation score can be determined, and the occurrence of a failure in the sensed area can be determined by comparing the failure acoustic evaluation score with the failure threshold score.

[0114] And, as a result of the above process, if an area (632) exceeding the fault threshold score among the areas sensed using the acoustic sensor is detected, it can be determined that a fault has occurred in the belt conveyor, and a fault notification operation can be performed to transmit a fault notification to a control center, etc.

[0115]

[0116] Below, a method for controlling a mobile robot using a mobile robot control device capable of performing all the aforementioned contents of the present disclosure is described. Content that overlaps with the above description may be omitted depending on the circumstances, but all of the following methods may also be applicable.

[0117] FIG. 7 is a flowchart relating to a mobile robot control method according to one embodiment.

[0118] Referring to FIG. 7, a control method for a mobile robot according to one embodiment may include a robot movement step (S710), a sensing data collection step (S720), an anomaly detection step (S730), and an anomaly response step (S740).

[0119] For example, a mobile robot control method may include a robot movement step for controlling the mobile robot to move while being supported by a transfer line installed along a belt conveyor, a sensing data collection step for collecting sensing data from at least one of a thermal imaging sensor, an acoustic sensor, and an image sensor installed on the mobile robot, an anomaly detection step for determining whether an anomaly is detected in the belt conveyor based on the sensing data, and an anomaly response step for outputting a control signal to perform a preset anomaly response action based on the result of determining whether an anomaly is detected in the belt conveyor.

[0120] The robot movement step (S710) may include controlling the mobile robot to move while being supported by a transfer line installed along a belt conveyor.

[0121] The sensing data collection step (S720) may include collecting sensing data from at least one of a thermal imaging sensor, an acoustic sensor, and an image sensor installed on a mobile robot.

[0122] For example, the sensing data collection step (S720) may include collecting thermal imaging sensing data from a thermal imaging sensor.

[0123] For example, the sensing data collection step (S720) may include collecting acoustic sensing data from an acoustic sensor.

[0124] The abnormality detection step (S730) may include determining whether an abnormality is detected in the belt conveyor based on the sensing data.

[0125] For example, the anomaly detection step (S730) may include determining that a fire has occurred on the belt conveyor when a temperature condition is satisfied that includes an area above a preset threshold temperature in the thermal image sensing data.

[0126] For example, the critical temperature may include a preset first critical temperature and a second critical temperature. In this case, the second critical temperature may be set to a temperature higher than the first critical temperature.

[0127] For example, the abnormal detection step (S730) may include determining that the belt conveyor is overheated when a first temperature condition is satisfied in which an area above a first threshold temperature and below a second threshold temperature is included in the thermal image sensing data, and determining that a fire has occurred in the belt conveyor when a second temperature condition is satisfied in which an area above the second threshold temperature is included in the thermal image sensing data.

[0128] For example, the abnormal detection step (S730) may include outputting a fire notification control signal to transmit fire information when it is determined that a fire has occurred on the belt conveyor.

[0129] For example, the abnormal detection step (S730) may include outputting a fire extinguisher control signal to operate a fire extinguisher in the area where the fire occurred.

[0130] For example, the anomaly detection step (S730) may include determining that a fire has occurred on the belt conveyor if the area exceeding a threshold temperature in the thermal image sensing data further satisfies the condition that the area exceeds a preset threshold area.

[0131] For example, the anomaly detection step (S730) may include determining that a fire has occurred on the belt conveyor if the time condition is further satisfied in which an area with a temperature above a threshold temperature in the thermal image sensing data is sensed continuously for a preset threshold time or longer.

[0132] For example, in the anomaly detection step (S730), a fault acoustic evaluation score of the acoustic sensing data can be determined using a preset artificial intelligence model, and if the fault acoustic evaluation score is greater than or equal to a preset fault threshold score, it may be determined that a fault has occurred in the belt conveyor.

[0133] For example, the failure threshold score may include a preset first failure threshold score and a second failure threshold score. In this case, the second failure threshold score may be set to a score higher than the first failure threshold score.

[0134] For example, the abnormal detection step (S730) may include determining a normal state if the fault acoustic evaluation score is less than a first fault threshold score, determining a fault warning state if it is greater than or equal to the first fault threshold score and less than a second fault threshold score, and determining a fault occurrence state if it is greater than or equal to the second fault threshold score.

[0135] For example, the artificial intelligence model may include pre-training in which failure acoustic data in the event of a failure in the guide roll of a belt conveyor is used as training data to evaluate the degree of similarity between the input acoustic data and the failure acoustic data to determine a failure acoustic evaluation score.

[0136] The abnormal response step (S740) may include outputting a control signal to perform a preset abnormal response operation based on the result of determining whether an abnormality is detected in the belt conveyor.

[0137] For example, the anomaly response operation may include a fire notification operation for transmitting fire information regarding the belt conveyor. For another example, the anomaly response operation may include a fire extinguisher operation for activating a fire extinguisher in the area where the fire occurred. For yet another example, the anomaly response operation may include a fault notification operation for transmitting fault information regarding the belt conveyor.

[0138] For example, in the abnormal response step (S740), if it is determined that the belt conveyor is overheated, it may include outputting an overheat notification control signal to transmit overheat information.

[0139] For example, in the abnormal response step (S740), if it is determined that a fire has occurred on the belt conveyor, it may include outputting a fire extinguisher control signal to perform fire extinguisher operation.

[0140] For example, in the abnormal response step (S740), if it is determined that a failure has occurred in the belt conveyor, it may include outputting a control signal to perform the failure notification operation. In this case, the abnormal response step (S740) may include outputting a fire extinguisher control signal to operate a fire extinguisher in the area where the fire occurred.

[0141] FIG. 8 is a flowchart illustrating an embodiment for detecting and responding to whether a fire has occurred in a mobile robot control method according to one embodiment.

[0142] Referring to FIG. 8, a mobile robot control method according to one embodiment may include a thermal image reception and temperature calculation step (S810), a region determination step (S820) that is above a threshold temperature, a warning requirement determination step (S830), an RGB image reception and smoke area calculation step (S840), a smoke threshold area determination step (S850) and a warning value transmission step (S852), a fire occurrence determination and fire emergency alarm step (S860), and a fire extinguishing liquid spraying step (S870).

[0143] The thermal image reception and temperature calculation step (S810) may include calculating the temperature of a sensed area based on a thermal image received from a thermal image sensor. In this case, the temperature of the sensed area may be calculated in conjunction with the temperature distribution of the thermal image.

[0144] For example, it may include generating information that displays a range of temperatures corresponding to colors on a temperature distribution shown in a thermal image in conjunction with the temperature calculation result.

[0145] The step of determining an area above a critical temperature (S820) may include determining whether the temperature calculation result is above a preset critical temperature. For example, it may include determining an area above the critical temperature in the temperature distribution of a thermal image based on the temperature calculation result.

[0146] For example, if no area above a critical temperature is detected in the thermal image, the method for controlling a mobile robot according to one embodiment can be continued by returning to the starting step, and if an area above a critical temperature is detected in the thermal image, the step for determining a situation requiring a warning (S830) can be performed.

[0147] Additionally, the step of determining the region above the threshold temperature (S820) may include additionally marking the region above the threshold temperature on the temperature distribution of the thermal image in conjunction with the temperature distribution of the thermal image.

[0148] The warning requirement situation determination step (S830) may include determining that a warning is required if it is determined that there is a region above a critical temperature.

[0149] In contrast, even if it is determined that an area above the critical temperature exists, if the area above the critical temperature in the thermal image temperature distribution is less than or equal to the critical area, it is determined that a warning is not necessary, and if it is greater than the critical area, it is determined that a warning is necessary.

[0150] The RGB image reception and smoke area calculation step (S840) may include calculating the area of ​​smoke appearing in the image based on the RGB image received from the RGB image sensor.

[0151] The step of determining whether the smoke area is greater than or equal to a critical area (S850) may include determining whether the smoke area calculation result is greater than or equal to a preset critical area.

[0152] For example, if it is calculated that the smoke appearing in the RGB image is below a critical area, it is determined that although an area above the critical temperature in the thermal image was detected, the smoke occurred only within the critical area, and thus the warning value transmission step (S852) can be performed to transmit a warning value to a control center, an administrator terminal, a driver's room, etc. In this case, the transmitted warning value may include information regarding the area above the critical temperature appearing in the thermal image.

[0153] As another example, if it is calculated that the smoke appearing in the RGB image exceeds a critical area, it can be determined that a fire has occurred based on the fact that an area exceeding a critical temperature in the thermal image is detected, as well as that smoke is occurring in the RGB image exceeding a critical area. In this case, the fire occurrence determination and fire emergency alarm step (S860) can be performed.

[0154] The fire occurrence determination and fire emergency alarm step (S860) may include determining that a fire has occurred in the corresponding sensing part of the belt conveyor by satisfying all conditions, such as detecting an area above a critical temperature in the thermal image and detecting smoke above a critical area in the RGB image.

[0155] And, depending on the case, it may include determining that a fire has occurred in an area when comparing a thermal image and an RGB image, and further satisfying conditions such as the sensed times of the two images matching within a preset error time, an area above a threshold temperature in the thermal image, and an area where smoke is detected in the RGB image matching within a preset error distance.

[0156] In addition, if it is determined through the above process that a fire has occurred in the corresponding sensing part of the belt conveyor, it may include transmitting a fire emergency alarm signal containing fire information regarding the time of fire occurrence, location of fire occurrence, etc., to a control center, manager terminal, or operator's room.

[0157] The fire extinguishing liquid spraying step (S870) may include controlling the operation of a fire extinguishing device to spray the fire extinguishing liquid. In this case, the target point where the fire extinguishing liquid is sprayed may be set to at least one of an area above a critical temperature in the thermal image and an area where smoke is detected in the RGB image.

[0158] As another example, the target point where the extinguishing agent is sprayed can be set to the area that is common to the region above the critical temperature in the thermal image and the region where smoke is detected in the RGB image.

[0159] As described above, according to the present disclosure, a mobile robot control device and method for detecting abnormalities in a belt conveyor can be provided.

[0160] The devices, methods, configurations, glyphs, and operations described herein may be implemented in digital electronic circuits, or computer software, firmware, or hardware comprising structures disclosed herein and structural equivalents, or combinations of one or more of these. The glyphs described herein may be implemented as one or more computer programs, for example, as one or more modules of computer program instructions encoded on a computer storage medium to control execution by a data processing device or operation by a data processing device. Program instructions may be encoded in artificially generated propagated signals, for example, mechanically generated electrical, optical, or electromagnetic signals generated to encode information for transmission to a suitable receiver device for execution by a data processing device. The computer storage medium may be or may include a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more of these. Although the computer storage medium is not a propagated signal, the computer storage medium may be a source or destination of computer program instructions encoded in an artificially generated propagated signal. Additionally, a computer storage medium may be one or more individual physical components or media (e.g., multiple CDs, disks, or other storage devices) or may include. The operations described herein may be implemented as operations performed by a data processing device on data stored in one or more computer-readable storage devices or data received from other sources.

[0161] The foregoing description is merely an illustrative explanation of the technical concept of the present disclosure, and those skilled in the art to which the present disclosure pertains may make various modifications and variations within the scope of the essential characteristics of the technical concept. Furthermore, since these embodiments are intended to explain rather than limit the technical concept of the present disclosure, the scope of the technical concept is not limited by these embodiments.

[0162]

[0163] CROSS-REFERENCE TO RELATED APPLICATION

[0164] This patent application claims priority pursuant to Section 119(a) of the U.S. Patent Act (35 USC § 119(a)) to Korean Patent Application No. 10-2024-0169839 filed on November 25, 2024, all of which are incorporated by reference into this patent application. Additionally, this patent application claims priority in countries other than the United States for the same reasons as above, all of which are incorporated by reference into this patent application.

Claims

1. In a mobile robot control device, At least one memory containing computer program instructions; and It includes at least one processor that executes the above computer program instructions, The above-mentioned at least one processor is, A mobile robot is controlled to move while being supported by a transfer line installed along a belt conveyor, and Determining whether an abnormality is detected in the belt conveyor based on sensing data collected from at least one of a thermal imaging sensor, an acoustic sensor, and a video sensor installed on the mobile robot, and A mobile robot control device that outputs an abnormality response control signal to perform a preset abnormality response action based on the result of determining whether an abnormality is detected.

2. In Paragraph 1, The above processor is, A mobile robot control device that collects thermal imaging sensing data from the above thermal imaging sensor and determines that a fire has occurred on the belt conveyor when a temperature condition is satisfied in which an area above a preset threshold temperature in the above thermal imaging sensing data is included.

3. In Paragraph 2, The above critical temperature is, It includes a preset first threshold temperature and a second threshold temperature set to a temperature higher than the first threshold temperature, The above processor is, If the above thermal imaging sensing data satisfies a first temperature condition in which an area above the first threshold temperature and below the second threshold temperature is included, it is determined that the belt conveyor is overheated. A mobile robot control device that determines that a fire has occurred on the belt conveyor when the second temperature condition is satisfied, wherein the thermal imaging sensing data includes an area above the second threshold temperature.

4. In Paragraph 3, The above processor is, If it is determined that the above belt conveyor is overheated, an overheat notification control signal is output to transmit overheat information, and A mobile robot control device that outputs a fire notification control signal for transmitting fire information and a fire extinguisher control signal for operating a fire extinguisher in the area where the fire occurred when it is determined that a fire has occurred on the belt conveyor.

5. In Paragraph 2, The above processor is, A mobile robot control device that determines that a fire has occurred on the belt conveyor when the area above the threshold temperature in the above thermal imaging sensing data further satisfies the condition that the area above the threshold area is larger than a preset threshold area.

6. In Paragraph 2, The above processor is, A mobile robot control device that determines that a fire has occurred on the belt conveyor when the thermal imaging sensing data further satisfies a time condition in which an area above the threshold temperature is sensed continuously for longer than a preset threshold time.

7. In Paragraph 1, The above processor is, Acoustic sensing data is collected from the above acoustic sensor, and A mobile robot control device that determines a fault acoustic evaluation score of the acoustic sensing data using a preset artificial intelligence model, and determines that a fault has occurred in the belt conveyor if the fault acoustic evaluation score is greater than or equal to a preset fault threshold score.

8. In Paragraph 7, The above failure threshold score is, It includes a preset first failure threshold score and a second failure threshold score set to a score higher than the first failure threshold score, and The above processor is, A mobile robot control device that determines a normal state when the above fault acoustic evaluation score is less than the above first fault threshold score, determines a fault warning state when it is greater than or equal to the above first fault threshold score and less than the above second fault threshold score, and determines a fault occurrence state when it is greater than or equal to the above second fault threshold score.

9. In Paragraph 7, The above artificial intelligence model is, A mobile robot control device characterized by performing prior learning in which, when a failure occurs in the guide roll of the belt conveyor, the failure acoustic data is used as learning data, and the degree of similarity between the input acoustic data and the acoustic of the failure acoustic data is evaluated to determine the failure acoustic evaluation score.

10. In Paragraph 1, The above abnormal response operation is, Includes fire extinguisher operation for activating the fire extinguisher in the area where the fire occurred, The above processor is, A mobile robot control device that outputs a fire extinguisher control signal to perform the operation of the fire extinguisher when it is determined that a fire has occurred on the belt conveyor.

11. In Paragraph 1, The above abnormal response operation is, It includes a fault notification operation for transmitting fault information for the above belt conveyor, and The above processor is, A mobile robot control device that outputs a control signal to perform the fault notification operation when it is determined that a fault has occurred in the belt conveyor.

12. A robot movement step that controls a mobile robot to move while being supported by a transfer line installed along a belt conveyor; A sensing data collection step of collecting sensing data from at least one of a thermal imaging sensor, an acoustic sensor, and an image sensor installed on the mobile robot; An anomaly detection step for determining whether an anomaly is detected in the belt conveyor based on the above sensing data; and A mobile robot control method comprising an abnormality response step that outputs a control signal to perform a preset abnormality response action based on the result of determining whether an abnormality is detected in the belt conveyor.

13. In Paragraph 12, The above sensing data collection step is, It includes collecting thermal imaging sensing data from the above thermal imaging sensor, and The above abnormality detection step is, A mobile robot control method comprising determining that a fire has occurred on the belt conveyor when a temperature condition is satisfied in which an area above a preset threshold temperature is included in the thermal imaging sensing data.

14. In Paragraph 13, The above critical temperature is, It includes a preset first threshold temperature and a second threshold temperature set to a temperature higher than the first threshold temperature, The above abnormality detection step is, If the above thermal imaging sensing data satisfies a first temperature condition in which an area above the first threshold temperature and below the second threshold temperature is included, it is determined that the belt conveyor is overheated, and A mobile robot control method comprising determining that a fire has occurred on the belt conveyor when the thermal imaging sensing data satisfies a second temperature condition in which an area above the second threshold temperature is included.

15. In Paragraph 14, The above abnormal response step is, It includes outputting an overheat notification control signal to transmit overheat information when it is determined that the belt conveyor is overheated, A mobile robot control method comprising outputting a fire notification control signal for transmitting fire information and a fire extinguisher control signal for operating a fire extinguisher in the area where the fire occurred when it is determined that a fire has occurred on the belt conveyor.

16. In Paragraph 13, The above abnormality detection step is, A mobile robot control method comprising determining that a fire has occurred on the belt conveyor when the area above the threshold temperature in the thermal imaging sensing data further satisfies the condition that the area above the threshold area is larger than a preset threshold area.

17. In Paragraph 13, The above abnormality detection step is, A mobile robot control method comprising determining that a fire has occurred on the belt conveyor when the time condition is further satisfied in which an area above the threshold temperature in the thermal imaging sensing data is sensed continuously for longer than a preset threshold time.

18. In Paragraph 12, The above sensing data collection step is, It includes collecting acoustic sensing data from the above acoustic sensor, and The above abnormality detection step is, A mobile robot control method comprising determining a fault acoustic evaluation score of the acoustic sensing data using a preset artificial intelligence model, and determining that a fault has occurred in the belt conveyor if the fault acoustic evaluation score is greater than or equal to a preset fault threshold score.

19. In Paragraph 18, The above failure threshold score is, It includes a preset first failure threshold score and a second failure threshold score set to a score higher than the first failure threshold score, and The above abnormality detection step is, A mobile robot control method comprising determining a normal state when the above fault acoustic evaluation score is less than the above first fault threshold score, determining a fault warning state when the above first fault threshold score is greater than or equal to the above second fault threshold score and less than the above second fault threshold score, and determining a fault occurrence state when the above second fault threshold score is greater than or equal to the above second fault threshold score.

20. In Paragraph 18, The above artificial intelligence model is, A mobile robot control method characterized by performing prior learning in which, when a failure occurs in the guide roll of the belt conveyor, the failure acoustic data is used as learning data, and the degree of similarity between the input acoustic data and the acoustic of the failure acoustic data is evaluated to determine the failure acoustic evaluation score.

21. In Paragraph 12, The above abnormal response operation is, Includes fire extinguisher operation for activating the fire extinguisher in the area where the fire occurred, The above abnormal response step is, A mobile robot control method comprising outputting a fire extinguisher control signal to perform the operation of the fire extinguisher when it is determined that a fire has occurred on the belt conveyor.

22. In Paragraph 12, The above abnormal response operation is, It includes a fault notification operation for transmitting fault information for the above belt conveyor, and The above abnormal response step is, A mobile robot control method comprising outputting a control signal to perform a fault notification operation when it is determined that a fault has occurred in the belt conveyor.

Images