Companion mobile apparatus and control method therefor

Abstract

A companion mobile apparatus and a control method therefor are disclosed. The companion mobile apparatus according to one aspect of the present invention comprises: a mobile apparatus main body; an imaging module disposed on the side surface of the mobile apparatus main body; a sensor module disposed on the side surface of the mobile apparatus main body; a driving module for moving the mobile apparatus main body; and a processor provided inside the mobile apparatus main body and connected to the sensor module, the imaging module, the sensor module and the driving module, wherein the processor recognizes a companion target on the basis of lateral image data obtained through the imaging module, and controls the driving module such that the companion target is accompanied while a preset certain distance therefrom is maintained in the lateral direction of the companion target by using distance data measured through the sensor module.

Description

Companion vehicle and its control method

[0001] The present invention relates to a companion vehicle and a control method thereof, and more particularly, to a companion vehicle and a control method thereof that can accompany a designated companion in the left / right direction of the companion.

[0002]

[0003] In recent years, tasks previously performed by humans are increasingly being replaced by robots or machines in various fields. While robots are currently being used in many fields, their adoption is lagging in some areas.

[0004] For example, the agricultural sector is one where robotization is progressing slowly, with most of the work in the farming process still being performed by humans, and work machines still being operated manually by people.

[0005] Accordingly, in the agricultural field, research is continuously being conducted on robots that automatically perform necessary tasks and drive in the fields.

[0006] However, despite these efforts, the field work is problematic because the terrain is not flat due to gravel or sand, and vibrations occur as the machine moves, which interferes with precise work.

[0007] Meanwhile, technology for robots that follow humans is disclosed in Korean Patent Publication No. 10-2011-0119438.

[0008] In this tracking robot technology, ultrasonic waves, infrared rays, vision, lasers, etc. are used to make the robot follow a person.

[0009] However, conventional tracking systems have the inconvenience of requiring users (workers, etc.) in the front to turn their heads or entire bodies to check whether the robot in the rear is following properly, and there is a risk of safety accidents occurring if tracking is induced by walking backwards.

[0010] Furthermore, tracking is highly vulnerable when obstacles exist between the user and the robot. For example, when using a tracking robot for transporting agricultural products, logistics, or other items, an obstacle between the user and the robot can cause the robot to be unable to easily follow the human operator.

[0011] The background technology of the present invention is disclosed in Korean Patent Publication No. 10-2011-0119438 (published on November 2, 2011) titled ‘Human worker following mobile robot.’

[0012]

[0013] The present invention has been devised to improve the aforementioned problems, and an object of the present invention is to provide a companion vehicle and a control method thereof that can accompany a designated companion in the left / right direction of the companion.

[0014] The problems to be solved by the present invention are not limited to the problem(s) mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

[0015]

[0016] According to one aspect of the present invention, a companion mobile device includes a photographing module arranged on a side of a mobile body, a sensor module arranged on a side of the mobile body, a driving module for moving the mobile body, and a processor provided inside the mobile body and connected to the photographing module, the sensor module, and the driving module, wherein the processor recognizes a companion subject based on lateral image data acquired through the photographing module, and controls the driving module to accompany the companion subject while maintaining a predetermined distance from the companion subject in a lateral direction using distance data measured through the sensor module.

[0017] In the present invention, the processor can continuously detect the bounding box or virtual marker of the companion subject in the lateral image data to detect the movement of the companion subject, and control the driving module to move together with the companion subject while maintaining a certain distance from the companion subject in the lateral direction using the distance data.

[0018] In the present invention, the processor can control the driving module to continuously move together with the companion so that the location of the companion is displayed in a pre-designated area in the lateral image data.

[0019] In the present invention, the processor can determine at least one of a straight mode and a rotation mode based on the distance to the companion subject measured through the sensor module and the movement direction of the companion subject, and control the driving module to operate according to the determined mode.

[0020] The present invention further includes an attachment that is detachably mounted on the mobile body and can be exchanged depending on the task to be performed, and the processor can control the attachment to perform the task while moving along the accompanying subject in the lateral direction of the accompanying subject.

[0021] In the present invention, the processor can recognize the posture of the companion based on the lateral image data, and, if the recognized posture indicates a hand signal command, control the driving module or attachment to perform an operation corresponding to the hand signal command.

[0022] The present invention further includes a microphone for receiving a voice command, and the processor can control the driving module or attachment to perform an operation corresponding to the voice command when the voice command input through the microphone is the voice of the companion.

[0023]

[0024] A method for controlling a companion vehicle according to another aspect of the present invention is characterized by including a step in which a processor recognizes a companion subject based on lateral image data acquired through a photographing module, and a step in which the processor controls a driving module to accompany the companion subject while maintaining a predetermined distance from the companion subject in the lateral direction using distance data measured through a sensor module.

[0025] In the present invention, in the recognizing step, the processor can continuously detect the bounding box or virtual marker of the companion subject in the lateral image data to detect the movement of the companion subject.

[0026] In the controlling step of the present invention, the processor can control the driving module to continuously move together with the companion so that the location of the companion is displayed in a pre-designated area in the lateral image data.

[0027] The present invention may further include, after the controlling step, a step of controlling the processor to operate an attached attachment while moving along the accompanying subject in the lateral direction of the accompanying subject to perform the corresponding task.

[0028] In addition, other methods for implementing the present invention, other systems and computer programs for executing the above methods may be further provided.

[0029]

[0030] A companion vehicle and a control method thereof according to one embodiment of the present invention recognize a companion subject based on data acquired through a photographing module and a sensor module provided on the left and right sides, and move along the movement of the companion subject in the left / right direction of the recognized companion subject, thereby enabling the companion vehicle to easily move along the companion subject even when an obstacle is located between the companion vehicle and the companion subject, and has the effect of preventing a safety accident of the companion subject.

[0031] Meanwhile, the effects of the present invention are not limited to the effects mentioned above, and various effects may be included within a range obvious to those skilled in the art from the contents described below.

[0032]

[0033] FIG. 1 is an exemplary diagram illustrating a companion moving object accompanying a user according to one embodiment of the present invention.

[0034] FIG. 2 is a block diagram schematically showing the configuration of a companion moving body according to one embodiment of the present invention.

[0035] Figure 3 is an exemplary diagram illustrating a companion moving body according to one embodiment of the present invention.

[0036] Figure 4 is a flowchart for explaining a control method of a companion moving object according to one embodiment of the present invention.

[0037]

[0038] Hereinafter, a companion moving object and its control method according to one embodiment of the present invention will be described with reference to the attached drawings. In this process, the thickness of lines and the sizes of components depicted in the drawings may be exaggerated for clarity and convenience of explanation.

[0039] Furthermore, the implementations described herein may be implemented as, for example, a method or process, an apparatus, a software program, a data stream, or a signal. Even if discussed only in the context of a single form of implementation (e.g., discussed only as a method), the implementation of the discussed features may also be implemented in other forms (e.g., as an apparatus or a program). The apparatus may be implemented using suitable hardware, software, firmware, and the like. The method may be implemented in an apparatus such as a processor, which generally refers to a processing device including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. A processor also includes a communication device such as a computer, a cell phone, a personal digital assistant ("PDA"), and other devices that facilitate communication of information between end-users.

[0040] In addition, the terminology used in this specification is only used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprise" or "have" are intended to specify the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but do not exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof. Terms such as first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another.

[0041] Hereinafter, embodiments according to the present invention will be described in detail with reference to the attached drawings. In describing with reference to the attached drawings, identical or corresponding components are assigned the same drawing numbers, and redundant descriptions thereof will be omitted.

[0042]

[0043] FIG. 1 is an exemplary diagram illustrating a companion moving object accompanying a user according to one embodiment of the present invention.

[0044] Referring to FIG. 1, the accompanying mobile device (100) is positioned to the left or right of the user (10), recognizes the user (10) located to the left or right, and can accompany the recognized user (10) while maintaining a certain distance laterally. That is, the accompanying mobile device (100) can move together with the user (10) in the same direction as the moving direction of the user (10) while maintaining a certain distance laterally from the user (10). At this time, the accompanying mobile device (100) can move in a rotational manner as well as in a straight line.

[0045] Such accompanying mobile devices (100) may include, for example, devices capable of autonomous driving, such as mobile carts, transport vehicles, robots, agricultural machinery, and vehicles.

[0046] Accordingly, the accompanying mobile device (100) can perform various tasks while accompanying the user (10) in the lateral direction of the user (10). For example, if the accompanying mobile device (100) is equipped with a field plowing machine, the accompanying mobile device (100) can perform the field plowing task while traveling in the same direction as the user's (10) movement direction in the lateral direction of the user (10).

[0047] In the following description, the user (10) will be referred to as the accompanying subject.

[0048]

[0049] FIG. 2 is a block diagram schematically showing the configuration of a companion moving body according to one embodiment of the present invention, and FIG. 3 is an exemplary diagram for explaining a companion moving body according to one embodiment of the present invention.

[0050] Referring to FIGS. 2 and 3, a companion moving body (100) according to one embodiment of the present invention may include a moving body body (101), a communication module (110), a memory (120), a photographing module (130), a sensor module (140), a driving module (160), a processor (170), an attachment (180), a loading unit (190), and a power supply unit (200).

[0051] The mobile body (101) is a centrally located base component among the components of the accompanying mobile body (100), and is provided as a frame structure in which a plurality of frames are connected for a useful coupling structure of the driving module (160). It may be desirable to provide the mobile body (101) in a form in which the ground clearance is increased through the support coupling of the driving module (160).

[0052] For example, the mobile body (101) may be formed in various shapes such as a rectangle, and a space may be provided therein in which a communication module (110), a memory (120), a driving module (160), a power supply (200), and a processor (170) may be mounted. At this time, the shape and material of the mobile body (101) may be modified in various ways. In addition, a shooting module (130) and a sensor module (140) may be mounted on the left and right sides of the mobile body (101), and wheels (150) may be mounted on the lower part of the mobile body (101).

[0053] The mobile body (101) can maintain a waterproof form so as to be able to respond to changes in the external environment, such as weather changes such as rain.

[0054] The communication module (110) is a device for performing communication with an external device. Here, the external device may include a management terminal (or remote control device), another mobile device, etc. The communication module (110) may include at least one of a transmitting antenna, a receiving antenna, an RF (Radio Frequency) circuit capable of implementing various communication protocols, and an RF element to perform communication. In addition, the communication module (110) may be a device including hardware and software necessary for transmitting and receiving signals such as control signals or data signals through a wired or wireless connection with another network device. The communication module (110) may be implemented in various forms such as a short-range communication module, a wireless communication module, a mobile communication module, a wired communication module, and a V2X communication module. The communication module (110) may include other components in addition to the components described, or may not include some of the components described.

[0055] The memory (120) is a configuration that stores data related to the operation of the companion vehicle (100). In particular, the memory (120) may store applications (programs or applets) that recognize a companion target (10) based on lateral image data and allow the companion target (10) to be accompanied while maintaining a preset distance from the companion target (10) in the lateral direction of the companion target (10), and the stored information may be selectively selected by the processor (170) as needed. That is, the memory (120) stores various types of data generated during the execution of the operating system or application (program or applet) for driving the companion vehicle (100). At this time, the memory (120) collectively refers to a non-volatile storage device that maintains stored information even when power is not supplied and a volatile storage device that requires power to maintain the stored information. In addition, the memory (120) may perform a function of temporarily or permanently storing data processed by the processor (170). Here, the memory (120) may include a magnetic storage media or a flash storage media in addition to a volatile storage device that requires power to maintain stored information, but the scope of the present invention is not limited thereto.

[0056] The shooting module (130) is provided on the left or right side of the mobile body (101) and can acquire side image data and transmit it to the processor (170). Here, the side image data can include left-side image data or right-side image data.

[0057] These shooting modules (130) may include a camera, an RGB camera, a deep learning camera, a depth camera, a vision camera, a stereo camera, etc.

[0058] The sensor module (140) is arranged on the left or right side of the mobile body (101) and can generate information about an object located in the lateral direction of the accompanying mobile body (100). The information about the object can include at least one of information about the presence or absence of the object, location information about the object, distance information between the accompanying mobile body and the object, and relative speed information between the accompanying mobile body (100) and the object. The sensor module (140) is configured to detect an object located in the lateral direction of the accompanying mobile body (100) and can include at least one sensor. The sensor module (140) can include a plurality of sensors to determine the location, distance, and obstacles, such as a lidar, a radar, a camera, a position sensor, a weight sensor, a distance measuring sensor, an ultrasonic sensor, and an infrared sensor.

[0059] In particular, the distance measuring sensor is installed on the left / right sides of the mobile body (101), and can easily measure the distance between the accompanying mobile body (100) and the accompanying target (10). The distance measuring sensor may be one of a lidar sensor, a laser sensor, and an ultrasonic sensor, but is not limited thereto. The distance measuring sensor is used to recognize the distance to the accompanying target (10), and the accompanying mobile body (100) can move while maintaining a certain distance from the accompanying target (10) by using the distance data measured by the distance measuring sensor. Here, the accompanying target (10) may include a person moving together with the accompanying mobile body (100) in the lateral direction of the accompanying mobile body (100) or a person to be moved.

[0060] The drive module (160) moves the accompanying moving body (100) using power supplied from the power supply unit (200), and may be a motor drive driver that drives the wheels (150) provided in the accompanying moving body (100). The drive module (160) includes a motor that provides driving force to the wheels (150), and can drive the accompanying moving body (100) according to the driving force generated by the motor.

[0061] The driving module (160) may include motor drivers, wheel motors, rotation motors, etc. The motor drivers may drive wheel motors and rotation motors for driving the companion vehicle (100), the wheel motors may drive multiple wheels (150) for driving the companion vehicle (100), and the rotation motors may be driven for left-right rotation or up-and-down rotation of the main body of the companion vehicle (100) or the head of the companion vehicle (100), or may be driven for changing the direction or rotation of the wheels (150) of the companion vehicle (100).

[0062] The drive module (160) is configured to take charge of the mobility of the accompanying moving body (100) through rotation by power transmission, and is provided to play a role in increasing the ground clearance of the moving body (101) by supporting and connecting to the lower part of the moving body body (101) through a connection using a frame or the like.

[0063] The wheel (150) is installed at the bottom of the mobile body (101) and can move the accompanying mobile body (100). The accompanying mobile body (100) is positioned to the left or right of the preset accompanying subject (10) and moves together with the accompanying subject (10) in the left / right direction of the accompanying subject (10). The wheel (150) can be configured to ensure smooth driving even in environments such as rain.

[0064] The power supply unit (200) can supply power for driving the companion vehicle (100). The power supply unit (200) can include a battery driver and a lithium-ion battery. The battery driver can manage charging and discharging of the lithium-ion battery, and the lithium-ion battery can supply power for driving the companion vehicle (100). The power supply unit (200) can also be configured with a solar cell.

[0065] The attachment (180) is a component that is detachably mounted on the mobile body (101) and can be replaced depending on the task to be performed. For example, the attachment (180) may be a device for performing various tasks such as agricultural tasks (e.g., watering, pesticide application, planting grains / seeds, plowing fields / rice paddies, harvesting crops, etc.), snow removal, watering, logistics transportation, and disaster prevention tasks.

[0066] For example, the attachment (180) may be a disaster prevention device that sprays water and chemicals. Furthermore, the attachment (180) may be a device for handling obstacles or a device for performing unrestricted transportation-related tasks. Furthermore, the attachment (180) may include a device for lifting and moving obstacles to another location, a device for leveling road surfaces, a device for cutting obstacles, and the like.

[0067] The loading section (190) can be loaded with goods to be transported using the accompanying moving object (100). For example, crops, logistics, etc. can be loaded in the loading section (190).

[0068] The loading unit (190) may be formed on the upper part of the mobile body (101) or may be formed inside the mobile body (101). In addition, the loading unit (190) may be composed of a storage space (not shown) for storing items and a door (not shown) installed on one side of the storage space. Installing a storage space that opens and closes by a door has the advantage of preventing theft of items or damage due to collision during transport. In addition, the loading unit (190) may further include a conveyor (not shown) that automatically loads and unloads items.

[0069] The processor (170) can recognize the accompanying subject (10) based on the lateral image data acquired through the shooting module (130), and control the driving module (160) to accompany the accompanying subject (10) while maintaining a preset distance from the accompanying subject (10) in the lateral direction of the accompanying subject (10) using the distance data measured through the sensor module (140). Here, the accompanying subject (10) can include a person moving or a person to move together with the accompanying subject (100) in the lateral direction of the accompanying moving object (100).

[0070] Hereinafter, the operation of the processor (170) will be described in detail.

[0071] The processor (170) can recognize the accompanying subject (10) by applying an object recognition model to the lateral image data acquired through the photographing module (130). The object recognition model is a deep learning model for detecting the accompanying subject (10) from the lateral image data, and is an image-based object detection model. For example, the processor (170) can detect the accompanying subject (10) from the lateral image data using an object recognition algorithm such as YOLO or Fast R-CNN.

[0072] The processor (170) may recognize the companion subject (10) by using at least one of the appearance information, color information, clothing information, and gait pattern of the companion subject (10). That is, the processor (170) may distinguish the companion subject (10) by including the appearance information regarding the physique, skeleton, and body type of the designated companion subject (10), the color information included in the head or body, the clothing information regarding the shape and color of the clothing being worn, and the gait pattern including the gait, walking posture, walking habits, and stride of the companion subject (10). At this time, the processor (170) may recognize the companion subject (10) by using a deep learning-based Matching Network algorithm. The Matching Network algorithm may extract various data components such as color, shape, texture, and edge of a person detected from side image data, and pass the extracted data components through a Matching Network to obtain a feature vector. The processor (170) can recognize the companion target (10) by comparing the acquired feature vector with the preset companion target (10) and calculating the similarity.

[0073] When the companion target (10) is recognized, the processor (170) can perform labeling by assigning an ID to each of the companion target (10) and other objects (obstacles) to distinguish between the companion target (10) and other objects.

[0074] The processor (170) recognizes the companion subject (10) by forming a bounding box around the outline of the companion subject (10) in the lateral image data taken of the companion subject (10), and may also form a virtual marker containing positioning coordinate information at the center of the bounding box.

[0075] When a companion target (10) is recognized in the lateral image data, the processor (170) forms a rectangular bounding box so that the companion target (10) is positioned at the center for object detection, thereby enabling continuous detection of the companion target (10). That is, after forming the bounding box, the processor (170) displays a virtual marker at the center of the bounding box, and can determine the distance and location information between the companion moving object (100) and the companion target (10) based on the positioning coordinates displayed on the virtual marker.

[0076] In addition, the processor (170) can form a bounding box and a virtual marker in the lateral image data, calculate the coordinates of the accompanying subject (10) through image processing based on the positioning coordinates formed in the virtual marker, and detect the accompanying subject (10) at a certain distance apart in the X, Y, and Z axes based on the positioning coordinates.

[0077] When the companion target (10) is recognized, the processor (170) can control the drive module (160) to accompany the companion target (10) while maintaining a preset distance from the companion target (10) in the lateral direction of the companion target (10) using the distance data measured through the sensor module (140).

[0078] The processor (170) can control the companion moving object (100) to move together with the companion target (10) in the lateral direction of the companion target (10). Accompanying the companion target (10) may be accomplished by capturing the companion target (10) with a shooting module (130) mounted on the companion moving object (100) to obtain image data, labeling the position (position and size of the bounding box) of the companion target (10) for each image data to produce deep learning training data, recognizing the companion target (10) using a CNN architecture based on object detection, which is an object recognition technology, and acquiring distance information between the companion target (10) and the companion moving object (100) through the result detected through deep learning using a depth camera, and configuring an algorithm that enables driving at a certain distance and angle from the companion target (10).

[0079] The processor (170) continuously detects the accompanying moving object (100) from the lateral image data captured by the shooting module (130), and based on the detection result, controls the driving module (160) to move together with the accompanying object (10) while maintaining a certain distance from the accompanying object (10) in the lateral direction.

[0080] That is, the processor (170) continuously detects the bounding box or virtual marker of the companion subject (10) from the lateral image data to detect the movement of the companion subject (10), and controls the driving module (160) to move together with the companion subject (10) while maintaining a certain distance from the companion subject (10) in the lateral direction using the distance data measured through the sensor module (140).

[0081] Additionally, the processor (170) can continuously move together with the companion (10) so that the location of the companion (10) is displayed in a pre-designated area in the lateral image data.

[0082] Meanwhile, when the accompanying moving object (100) moves with the accompanying object (10) in the lateral direction of the accompanying object (10), there may be an obstacle in front. In this case, the accompanying moving object (100) needs to move while avoiding the obstacle in front.

[0083] Accordingly, the processor (170) can enable accompanying or autonomous driving with the accompanying person (10) by using obstacle detection and avoidance technology for static / dynamic obstacle avoidance during driving of the accompanying vehicle (100). At this time, the processor (170) can calculate the location of the accompanying vehicle (100) based on the Visual SLAM (VSLAM) technique indoors, and can calculate the location of the accompanying vehicle (100) based on at least one of GPS and IMU outdoors.

[0084] For example, the processor (170) can calculate the location of the companion vehicle (100) indoors using the VSLAM technique. More specifically, the processor (170) can generate 3D point cloud data of an indoor space using an RGB-D camera to apply the VSLAM technique, and can generate an indoor map from the data to enable the companion vehicle (100) to recognize and avoid objects in the space. In addition, the processor (170) can calculate and determine the indoor location of the companion vehicle (100) using the generated indoor map. In addition, the processor (170) can calculate the location and direction of the companion vehicle (100) outdoors using a global positioning system (GPS) or an inertial measurement unit (IMU). That is, the processor (170) can receive the location and direction in real time from the GPS, or calculate the location and direction using a gyro sensor and an acceleration sensor included in the IMU.

[0085] The processor (170) can control the attachment (180) to perform the corresponding task while moving along the companion target (10) in the lateral direction of the companion target (10).

[0086] For example, if the attachment (180) is a machine for plowing a field, the processor (170) can control the attachment (180) to perform a plowing operation while driving in the same direction as the movement direction of the accompanying subject (10) in the lateral direction of the accompanying subject (10).

[0087] Meanwhile, in the case of a companion moving body (100) that performs tasks such as plowing a field, it must be able to rotate because it must move along the furrows of the field.

[0088] Accordingly, the processor (170) can determine at least one of a straight mode and a rotation mode based on the distance to the companion (10) and the moving direction of the companion (10), and control the driving module (160) to operate according to the determined mode.

[0089] For example, when the companion target (10) moves in a direction different from the current moving direction, the processor (170) can control the driving module (160) to rotate in the moving direction of the companion target (10) and move while maintaining a certain distance from the companion target (10).

[0090] In addition, if the companion target (10) moves in a direction different from the current moving direction and the distance from the companion target (10) exceeds a preset distance, the processor (170) determines that the companion target (10) is rotating, and controls the driving module (160) to rotate in the moving direction of the companion target (10), and can control the driving module (160) to move while maintaining a preset distance from the companion target (10).

[0091] In addition, the processor (170) can recognize the posture of the accompanying subject (10) based on the lateral image data, and if the recognized posture indicates a hand signal command, control to perform an action corresponding to the hand signal command. Here, the hand signal command may include a turn command, a stop command, etc.

[0092] The accompanying subject (10) may be at a certain distance from the accompanying vehicle (100), making it difficult to directly operate the accompanying vehicle (100). In addition, the accompanying subject (10) may be holding gloves or an object, making it difficult to operate the accompanying vehicle (100) using a switch or remote control. In addition, the accompanying subject (10) may want to control the accompanying vehicle (100) from a distance with a specific hand signal command. In this case, the processor (170) analyzes the lateral image data acquired through the photographing module (130) to recognize the posture of the accompanying subject (10), and if the recognized posture indicates a hand signal command, it can control to perform an action corresponding to the hand signal command. At this time, the memory (120) stores various postures (poses) and hand signal commands for each posture (pose).

[0093] In addition, the companion vehicle (100) can receive control commands not only via hand signal commands but also via voice commands. In this case, the companion vehicle (100) may further include a microphone (not shown) for receiving voice commands. When a voice is input through the microphone, the processor (170) can determine whether the voice command input through the microphone is the voice of a preset companion target (10). If the determination result is that the voice is that of the companion target (10), the processor (170) can control the execution of an action corresponding to the voice command.

[0094] When the accompanying subject (10) moves with a remote control device that remotely controls the accompanying moving object (100), the processor (170) can measure the intensity of the magnetic field generated by the remote control device and, based on the intensity of the magnetic field, control the direction or speed of the accompanying moving object (100) so that the accompanying moving object (100) maintains a preset distance from the accompanying subject (10). For example, when the magnetic field intensity is compared with a preset reference magnetic field value and the error increases, the processor (170) can increase the moving speed of the accompanying moving object (100). When the magnetic field intensity becomes lower than the set allowable error with the micro-reference magnetic field value, the processor (170) can continue to move the accompanying moving object (100) in the direction in which it was moving immediately before.

[0095] The processor (170) can autonomously drive along a work path, and when it arrives at a work space where the work is to be performed, it can control the attachment (180) to operate to perform the work, and can control the attachment (180) to return to the original position when the work is completed.

[0096] The processor (170) is a component that controls the overall operation of the companion moving object (100) and can be implemented as an integrated circuit, a system on chip, or a mobile AP.

[0097] The processor (170) may be operatively connected to at least one of a communication module (110), a memory (120), a photographing module (130), a sensor module (140), a driving module (160), and a power supply (200). The processor (170) may be implemented as at least one of a central processing unit (CPU), an application specific integrated circuit (ASIC), a digital signal processor (DSP), a programmable logic device (PLD), a field programmable gate array (FPGA), a micro controller unit (MCU), and a system on chip (SoC), and may control a plurality of hardware or software components connected to the processor (170) by driving an operating system or an application, perform various data processing and calculations, and may be configured to execute at least one command stored in a memory (120) and store execution result data in the memory (120).

[0098]

[0099] Figure 4 is a flowchart for explaining a control method of a companion moving object according to one embodiment of the present invention.

[0100] Referring to FIG. 4, the processor (170) acquires lateral image data through the photographing module (130) (S402) and recognizes the accompanying subject (10) based on the acquired lateral image data (S404). At this time, the processor (170) can recognize the accompanying subject (10) by applying an object recognition model. In addition, the processor (170) can recognize the accompanying subject (10) by using one or more of the accompanying subject's (10) appearance information, color information, clothing information, and gait pattern.

[0101] When a companion target (10) is recognized, the processor (170) can perform labeling by assigning an ID to each of the companion target (10) and other objects (obstacles) to distinguish the companion target (10) from other objects. The processor (170) can recognize the companion target (10) by forming a bounding box around the outline of the companion target (10) in the lateral image data that captured the companion target (10), and can also form a virtual marker containing positioning coordinate information at the center of the bounding box.

[0102] When step S404 is performed, the processor (170) controls the drive module (160) to accompany the person to be accompanied (10) while maintaining a preset distance from the person to be accompanied (10) in the lateral direction (S406). At this time, the processor (170) can control the drive module (160) to accompany the person to be accompanied (10) while maintaining a preset distance from the person to be accompanied (10) in the lateral direction using the distance data measured through the sensor module (140). That is, the processor (170) continuously detects the bounding box or virtual marker of the person to be accompanied (10) in the lateral image data to detect the movement of the person to be accompanied (10), and can control the drive module (160) to move together with the person to be accompanied (10) while maintaining a preset distance from the person to be accompanied (10) in the lateral direction using the distance data measured through the sensor module (140).

[0103] Additionally, the processor (170) can continuously move together with the companion (10) so that the location of the companion (10) is displayed in a pre-designated area in the lateral image data.

[0104] When a workspace is recognized while performing step S406 (S408), the processor (170) controls the attachment (180) to operate while moving along the accompaniment target (10) in the lateral direction of the accompaniment target (10) to perform the corresponding task (S410). That is, when the processor (170) moves along the movement of the accompaniment target (10) in the lateral direction of the accompaniment target (10) and arrives at the workspace where the corresponding task is to be performed, the processor (170) controls the attachment (180) to operate to perform the task, and can control it to return to the original position upon completion of the task.

[0105]

[0106] As described above, the accompanying mobile device and its control method according to one embodiment of the present invention recognize a subject to be accompanied based on data acquired through a photographing module and a sensor module provided on the left and right sides, and move along the movement of the subject to be accompanied in the left / right direction of the recognized subject, thereby enabling the accompanying mobile device to easily follow the subject to be accompanied even when an obstacle is located between the accompanying mobile device and the subject to be accompanied, and has the effect of preventing a safety accident of the subject to be accompanied.

[0107] While the present invention has been described with reference to the embodiments illustrated in the drawings, these are merely exemplary, and those skilled in the art will understand that various modifications and equivalent embodiments are possible. Therefore, the true technical protection scope of the present invention should be defined by the following claims.

Claims

1. A shooting module placed on the side of the mobile body; A sensor module arranged on the side of the above moving body; A driving module for moving the above-mentioned mobile body; and Including a processor connected to the above shooting module, sensor module and driving module, The above processor, A companion mobile device characterized in that it recognizes a companion subject based on lateral image data acquired through the above-mentioned shooting module, and controls the driving module to accompany the companion subject while maintaining a predetermined distance from the companion subject in the lateral direction using distance data measured through the above-mentioned sensor module.

2. In paragraph 1, The above processor, A companion moving object characterized in that the movement of the companion is detected by continuously detecting the bounding box or virtual marker of the companion object in the lateral image data, and the driving module is controlled to move together with the companion object while maintaining a certain distance from the companion object in the lateral direction using the distance data.

3. In paragraph 1, The above processor, A companion mobile device characterized in that the driving module is controlled to continuously move together with the companion so that the location of the companion is displayed in a pre-designated area in the lateral image data.

4. In paragraph 1, The above processor, A companion moving object characterized in that it determines at least one of a straight mode and a rotation mode based on the distance to the companion object measured through the sensor module and the movement direction of the companion object, and controls the drive module to operate according to the determined mode.

5. In paragraph 1, It further includes an attachment that is detachably mounted on the above-mentioned mobile body and can be exchanged depending on the task to be performed. A companion mobile device characterized in that the processor controls the attachment to perform the corresponding task while moving along the companion target in the lateral direction of the companion target.

6. In paragraph 1, The above processor, A companion mobile device characterized in that it recognizes the posture of the companion subject based on the lateral image data, and if the recognized posture indicates a hand signal command, it controls the driving module or attachment to perform an action corresponding to the hand signal command.

7. In paragraph 1, It further includes a microphone for receiving voice commands, The above processor, A companion mobile device characterized in that, when a voice command input through the microphone is the voice of the companion, the driving module or attachment is controlled to perform an action corresponding to the voice command.

8. A step in which the processor recognizes the accompanying subject based on the lateral image data acquired through the shooting module; and A step in which the processor controls the driving module to accompany the companion while maintaining a preset distance from the companion in the lateral direction of the companion using distance data measured through the sensor module; A method for controlling a companion moving body, characterized by including a .

9. In paragraph 8, At the above recognition stage, A method for controlling a companion vehicle, characterized in that the processor continuously detects the bounding box or virtual marker of the companion subject from the lateral image data to detect the movement of the companion subject.

10. In paragraph 8, In the above controlling step, A control method for a companion moving object, characterized in that the processor controls the driving module to continuously move together with the companion so that the location of the companion is displayed in a pre-designated area in the lateral image data.

11. In paragraph 8, After the above controlling step, A method for controlling a companion vehicle, characterized in that the processor further comprises a step of controlling an attached attachment to perform a task while following the companion target in a lateral direction of the companion target.

Images