Work robot

Abstract

To accomplish an autonomous running through a desired route without position measurement.SOLUTION: A work robot includes: imaging units 200 which are provided at least in the forward, right and left travelling directions, and which pick up respective pieces of image data; an image information detecting unit 110 that detects each kind of information from the picked-up image data; an image information processing unit 120 which executes image processing analysis on the detected image data, and determines the presence or absence of an object subjected to an assist for running, and the presence or absence of an obstacle in the travelling direction; a running route determination unit 130 that determines the travelling direction based on the processing result by the image information processing unit 120; and a running control unit 150 that executes a control so as to run through the running route determined by the running route determination unit 130.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a working robot. [Background technology]

[0002] In recent years, autonomous mobile robots (mobile bodies) that can move autonomously within a limited area inside a building or outdoors based on the surrounding environment have been developed. Such autonomous mobile robots are expected to be used in a variety of industries, including agriculture, forestry, and transportation, as a substitute for labor or work and to make up for labor shortages.

[0003] As an example of this type of autonomous mobile robot, there is disclosed an automatic driving device that performs predetermined tasks on crops, which are detection objects arranged in the direction of travel, with the aim of automatically traveling a desired route without performing positioning, and that is equipped with a camera that photographs the crops in the direction of travel and acquires image data, a recognition unit that places multiple judgment windows at predetermined positions in the image data and recognizes the crops, including the multiple judgment windows, and an adjustment unit that adjusts the direction of travel based on the recognition results by the recognition unit (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2022 / 215416 Summary of the Invention [Problem to be solved by the invention]

[0005] The technology described in Patent Document 1 uses a camera mounted on the front of the autonomous driving device to recognize crops planted on a farm, and adjusts the position of the tires while straddling the planted crops.

[0006] Therefore, there was a problem in that if the crops could not be recognized, such as when no crops had been planted, or when the crops had been planted but had not germinated, or when the crops had germinated but not grown enough to be recognized, the driving of the automatic driving device could not be controlled.

[0007] The present invention has been made in view of the above-mentioned problems, and has as its object to provide a working robot that can automatically travel along a desired route without performing positioning. [Means for solving the problem]

[0008] Form 1: One or more embodiments of the present invention propose a work robot that moves along travel assistance objects that are aligned or extend in the direction of travel and performs a predetermined task, the work robot comprising: an imaging unit that is provided at least in front, to the right, and to the left of the direction of travel and that captures image data; an image information detection unit that detects various information from the captured image data; an image information processing unit that performs image processing and analysis on the detected image data to determine the presence or absence of the travel assistance objects and the presence or absence of obstacles in the direction of travel; a travel route determination unit that determines the direction of travel based on the processing results by the image information processing unit; and a travel control unit that executes control to travel along the travel route determined by the travel route determination unit.

[0009] Mode 2: One or more embodiments of the present invention propose a working robot in which the image information detection unit acquires various information including depth information and Euler angle information from the image data.

[0010] Form 3: One or more embodiments of the present invention propose a work robot in which the image information detection unit includes a calculation unit that calculates the Euler angle information from inertial data and calculates three-dimensional information for each pixel from information including the depth information and Euler angle information, and a first image information processing unit corresponding to the imaging unit ahead in the direction of travel includes a first determination processing unit that determines the presence or absence of steps or obstacles around the travel route based on the three-dimensional information for each pixel.

[0011] Form 4: One or more embodiments of the present invention propose a work robot in which the image information detection unit includes a calculation unit that calculates three-dimensional information for each pixel from information including the depth information and Euler angle information, and a second image information processing unit corresponding to the imaging unit to the right or left of the traveling direction includes a second determination processing unit that determines the driving assistance object based on the three-dimensional information for each pixel.

[0012] Form 5: One or more embodiments of the present invention propose a work robot in which the first judgment processing unit includes a 3D information processing unit that performs an averaging process to average the 3D information for each pixel and a compression process of the averaged 3D information for each pixel, and an information extraction unit that extracts the 3D information that exceeds a predetermined threshold based on height information included in the processed 3D information within a judgment area, and determines whether there are any steps or obstacles around the travel route based on the height information included in the processed 3D information within a judgment area.

[0013] Form 6: In one or more embodiments of the present invention, the second determination processing unit includes a three-dimensional information processing unit that performs an averaging process to average the three-dimensional information for each pixel and a compression process of the averaged three-dimensional information for each pixel, and an information extraction unit that extracts the three-dimensional information that exceeds a predetermined threshold based on height information included in the processed three-dimensional information within a determination area, and the work robot is proposed to determine the driving assistance object based on depth information and width information corresponding to the extracted three-dimensional information.

[0014] Form 7: One or more embodiments of the present invention propose a work robot in which the travel route is a path along the ridges of farmland as the travel assistance object, or a path between adjacent ridges.

[0015] Form 8: One or more embodiments of the present invention propose a work robot in which, when the first determination processing unit determines that a step that was present around the travel route has disappeared, the travel route determination unit determines the travel route such that the robot turns in the direction of the step, and, when the second determination processing unit determines that the ridge exists in both the left and right directions, the robot further determines the travel route such that the robot turns in the direction in which the travel assistance object extends.

[0016] Mode 9: One or more embodiments of the present invention propose a working robot in which the travel route is a path along a curb of a sidewalk as the travel assistance object.

[0017] Mode 10: One or more embodiments of the present invention propose a working robot in which the travel route is a path along an indoor or outdoor installed or placed object that serves as the travel assistance object. [Effects of the Invention]

[0018] According to one or more embodiments of the present invention, it is possible to automatically travel a desired route without performing positioning, which has the effect of improving work efficiency at low cost. [Brief explanation of the drawings]

[0019] [Figure 1] 1A is a left side view, FIG. 1B is a front view, and FIG. 1C is a top view of a working robot according to an embodiment of the present invention. [Figure 2] FIG. 2 is a configuration diagram of a control unit in the working robot according to the embodiment of the present invention. [Figure 3] FIG. 2 is a configuration diagram of a first image information detection unit in the working robot according to the embodiment of the present invention. [Figure 4] FIG. 3 is a configuration diagram of a second image information detection unit in the working robot according to the embodiment of the present invention. [Figure 5] FIG. 3 is a configuration diagram of a second determination processing unit in the working robot according to the embodiment of the present invention. [Figure 6] FIG. 3 is a configuration diagram of a second determination processing unit in the working robot according to the embodiment of the present invention. [Figure 7] FIG. 1 is a schematic diagram illustrating a travel route of a working robot according to an embodiment of the present invention. [Figure 8] FIG. 4 is a processing flow diagram of an image information detection unit in the working robot according to the embodiment of the present invention. [Figure 9] FIG. 4 is a processing flow diagram of an image information detection unit in the working robot according to the embodiment of the present invention. [Figure 10] A diagram showing the determination area of ​​the ridge determination process using image data obtained from the left and right imaging units in the working robot according to an embodiment of the present invention. [Figure 11] FIG. 4 is a processing flow diagram of an image information detection unit in the working robot according to the embodiment of the present invention. [Figure 12] 10A and 10B are diagrams showing a determination area for a step determination process using image data obtained from a forward imaging unit in a working robot according to an embodiment of the present invention. [Figure 13] FIG. 10 is a diagram showing a determination region for an obstacle determination process using image data obtained from a forward imaging unit in a working robot according to an embodiment of the present invention. [Figure 14] FIG. 1 is a process flow diagram of a step (ridge) detection method for a working robot according to an embodiment of the present invention. [Figure 15] FIG. 1 is a process flow diagram of a step (ridge) detection method for a working robot according to an embodiment of the present invention. [Figure 16] FIG. 1 is a process flow diagram of a step (ridge) detection method for a working robot according to an embodiment of the present invention. [Figure 17] FIG. 1 is a process flow diagram of a step (ridge) detection method for a working robot according to an embodiment of the present invention. [Figure 18] FIG. 1 is a process flow diagram of a step (ridge) detection method for a working robot according to an embodiment of the present invention. [Figure 19] FIG. 1 is a process flow diagram of a step (ridge) detection method for a working robot according to an embodiment of the present invention. [Figure 20] FIG. 1 is a process flow diagram of a step (ridge) detection method for a working robot according to an embodiment of the present invention. [Figure 21] 10A and 10B are diagrams showing the detection results of steps (ridges) in a working robot according to an embodiment of the present invention. [Figure 22] 10A and 10B are diagrams showing the detection results of steps (ridges) in a working robot according to an embodiment of the present invention. [Figure 23] 4 is a process flow of a traveling control unit in a working robot according to an embodiment of the present invention. [Figure 24] FIG. 10 is a diagram illustrating the process of the travel control unit when the working robot according to the embodiment of the present invention travels between two furrows. [Figure 25] 10 is a diagram illustrating the process of the travel control unit when the working robot according to the embodiment of the present invention travels along one side of a ridge. FIG. [Figure 26] 10 is a diagram showing a schematic diagram of the processing of the travel control unit when the working robot according to the embodiment of the present invention travels from one furrow to the space between both furrows. FIG. [Figure 27] FIG. 10 is a diagram conceptually illustrating a travel mode when a working robot according to an embodiment of the present invention travels along ridges in farmland. DETAILED DESCRIPTION OF THE INVENTION

[0020] <Embodiment> A working robot 1 according to this embodiment will be described with reference to FIGS. 1 to 27.

[0021] <About Work Robot 1> The working robot 1 according to this embodiment is a robot that moves along travel assistance objects that are aligned or extend in the traveling direction and performs a predetermined task. As shown in FIG. 1, the work robot 1 according to this embodiment includes a control unit 100, a first imaging unit 200A, a second imaging unit 200B, a third imaging unit 200C, and a movement mechanism 300. In this embodiment, as an example, a working robot that travels along a path along the ridges of farmland as an object to be assisted in traveling, or a path between adjacent ridges, will be described.

[0022] As shown in FIG. 1, the control unit 100 is housed in a housing and controls the movement mechanism 300 and other components based on image data obtained from the first imaging unit 200A, second imaging unit 200B, and third imaging unit 200C described below, thereby controlling the movement of the work robot 1 in accordance with the purpose and surrounding environment. The detailed configuration of the control unit 100 will be described below with reference to the drawings.

[0023] The first imaging unit 200A is provided in front of the vehicle in the direction of travel and captures image data in that direction. The second imaging unit 200B is provided on the left side of the vehicle in the direction of travel and captures image data in that direction. The third imaging unit 200C is provided on the right side of the vehicle in the direction of travel and captures image data in that direction.

[0024] The first imaging unit 200A, the second imaging unit 200B, and the third imaging unit 200C are each composed of, for example, a CCD camera, which captures images of one or more objects present in the direction of travel, and acquires image data containing various information including depth information and Euler angle information. The first imaging unit 200A, the second imaging unit 200B, and the third imaging unit 200C may be fixed in a predetermined direction, or may be mounted on a mount having three degrees of freedom, for example. The first imaging unit 200A, the second imaging unit 200B, and the third imaging unit 200C may be of visible light, infrared, or may be of both visible light and infrared. Depending on the application, a distance sensor or the like may be provided in addition to the first imaging unit 200A, the second imaging unit 200B, and the third imaging unit 200C, and imaging units may be provided in other directions (for example, rearward in the direction of travel) in addition to the front, right, and left sides of the direction of travel.

[0025] The moving mechanism 300 is, for example, a plurality of drive wheels provided on the bottom and crawlers attached so as to cover the outer periphery of the drive wheels. The plurality of drive wheels are driven by a plurality of motors.

[0026] <Configuration of control unit 100> The configuration of the control unit 100 according to this embodiment will be described with reference to FIGS.

[0027] As shown in FIG. 2, the control unit 100 according to this embodiment includes an image information detection unit 110, an image information processing unit 120, a driving route determination unit 130, a memory unit 140, and a driving control unit 150.

[0028] The image information detection unit 110 detects various types of information from the image data captured by the imaging unit 200 . The image information detection unit 110 acquires various pieces of information including depth information and Euler angle information from the image data captured by the imaging unit 200 . The image information detection unit 110 calculates and acquires Euler angle information from, for example, inertial data. The image information detection unit 110 calculates and acquires three-dimensional information for each pixel from information including depth information and Euler angle information. In this embodiment, depending on the installation direction of the imaging unit 200, the image information detection unit 110 performs separate processes: detecting various information from image data captured by the first imaging unit 200A provided in front of the vehicle in the direction of travel; and detecting various information from image data captured by the second imaging unit 200B and the third imaging unit 200C provided on the right or left side of the vehicle in the direction of travel. The various pieces of information detected by the image information detection unit 110 are output to the image information processing unit 120 via the bus line BL and the driving control unit 150, which will be described later.

[0029] The image information processing unit 120 performs image processing and analysis on the image data detected by the image information detection unit 110 to determine whether there are ridges as objects of travel assistance and whether there are any obstacles in the traveling direction. The image information processing unit 120 determines whether or not there are any steps or obstacles around the travel route based on, for example, three-dimensional information for each pixel obtained by calculation processing. The image information processing unit 120 determines the driving assistance object based on, for example, three-dimensional information for each pixel obtained by calculation processing. The image information processing unit 120 processes differently depending on the installation direction of the imaging unit 200. In this embodiment, the image information processing unit 120 is composed of a first image information processing unit 120A that performs image processing and analysis of various information detected from image data captured by the first imaging unit 200A provided in front of the traveling direction by the image information detection unit 110, to determine whether or not there are ridges as objects of driving assistance and whether or not there are obstacles in the traveling direction, and a second image information processing unit 120A that performs image processing and analysis of various information detected from image data captured by the second imaging unit 200B and the third imaging unit 200C provided on the right or left side of the traveling direction, to determine whether or not there are ridges as objects of driving assistance and whether or not there are obstacles in the traveling direction. The configurations and detection methods of the first image information processing section 120A and the second image information processing section 120B will be described in detail below with reference to the drawings.

[0030] The travel route determination unit 130 determines the travel route of the working robot 1 according to this embodiment based on the processing results of the first image information processing unit 120A or the second image information processing unit 120B. The driving route information determined by the driving route determination unit 130 is output to a driving control unit 150, which will be described later, via a bus line BL.

[0031] The memory unit 140 is composed of a ROM (Read Only Memory) or a RAM (Random Access Memory), and stores a control program used for control by the driving control unit 150 described later, various information detected by the image information detection unit 110, and judgment condition data used in the image information processing unit 120.

[0032] The driving control unit 150 executes control so that the vehicle travels along the driving route determined by the driving route determination unit 130 . The driving control unit 150 controls the overall operation of the control unit 100 based on a control program stored in the storage unit 140 .

[0033] <Configuration of the first image information processing unit 120A> As shown in FIG. 3, the first image information processing unit 120A includes an information input unit 121 and a first determination processing unit 122.

[0034] The information input unit 121 inputs three-dimensional information for each pixel from information including depth information and Euler angle information acquired by calculation in the image information detection unit 110. The three-dimensional information for each pixel input to the information input unit 121 is output to a first determination processing unit 122, which will be described later, via the bus line BL and the driving control unit 150.

[0035] <Configuration of second image information processing section 120B> As shown in FIG. 4, the second image information processing unit 120B includes an information input unit 121 and a second determination processing unit 123. It should be noted that the information input unit 121 has the same functions as those described above, and therefore a detailed description thereof will be omitted.

[0036] The second determination processing unit 123 determines whether or not a ridge exists as an object for travel assistance, based on the three-dimensional information for each pixel. The determination result of the second determination processing unit 123 is output to the driving control unit 150 via the bus line BL. The configuration and determination processing of the second determination processing unit 123 will be described in detail below with reference to the drawings.

[0037] <Configuration of the first determination processing unit 122> As shown in FIG. 5, the first determination processing unit 122 includes a three-dimensional information processing unit 1221 and an information extraction unit 1222.

[0038] The three-dimensional information processing unit 1231 performs an averaging process for averaging the three-dimensional information for each pixel, and a compression process for the averaged three-dimensional information for each pixel. The first determination processing unit 122 determines whether or not there are any steps or obstacles around the travel route based on the height information included in the processed three-dimensional information within the determination area.

[0039] The information extraction unit 1222 extracts three-dimensional information exceeding a predetermined threshold based on height information included in the processed three-dimensional information within the determination region.

[0040] <Configuration of second determination processing unit 123> As shown in FIG. 6, the second determination processing unit 123 includes a three-dimensional information processing unit 1231 and an information extraction unit 1232. It should be noted that the three-dimensional information processing unit 1231 has the same functions as the three-dimensional information processing unit 1221, and therefore detailed description thereof will be omitted.

[0041] The information extraction unit 1232 extracts three-dimensional information exceeding a predetermined threshold based on height information included in the processed three-dimensional information within the determination region. The second determination processing unit 123 determines whether or not there is a ridge as a travel assistance object based on the depth information and width information corresponding to the extracted three-dimensional information.

[0042] <Processing of the control unit 100> The processing of the control unit 100 according to this embodiment will be described as processing for each component with reference to FIGS. The processing of the control unit 100 will be explained using as an example a case where the working robot 1 travels along a travel route as shown in FIG.

[0043] <Processing of the image information detection unit 110> As shown in FIG. 8, the image information detection unit 110 maintains the standby mode until the timing of data update (step S110).

[0044] When it is time to update the data, the image information detection unit 110 detects RGB data, depth data, and IMU data from the image data captured by the imaging units 200A, 200B, and 200C, and calculates Euler angles from the detected IMU data.

[0045] The image information detection unit 110 integrates RGB data and depth data from the image data captured by the imaging units 200A, 200B, and 200C with Euler angle data obtained by arithmetic processing. Then, the image information detection unit 110 calculates three-dimensional information for each pixel from the integrated data group (step S130).

[0046] The three-dimensional information for each pixel calculated by the image information detection unit 110 is stored in the memory unit 140 via the bus line BL and the driving control unit 150 in a manner separated for each of the imaging units 200A, 200B, and 200C (step S140).

[0047] The image information detection unit 110 checks the status of the processing operation, and if it determines that the operation has ended, it ends the operation ("YES" in step S150), and if it determines that the operation has not ended ("NO" in step S150), it transitions the processing step to step S110.

[0048] <Processing of the second image information processing unit 120B> As shown in FIG. 9, the second image information processing unit 120B maintains the standby mode until the timing for updating the three-dimensional information for each pixel based on the image data captured by the second imaging unit 200B and the third imaging unit 200C (step S210).

[0049] The second image information processing unit 120B executes the ridge determination process when the timing for updating data arrives (step S220). The ridge determination process uses three-dimensional information for each pixel to determine whether or not a ridge or an obstacle exists in a given area, as shown in FIG. 10, for example.

[0050] The second image information processing unit 120B stores the determination result of the ridge determination process in the storage unit 140 via the bus line BL and the travel control unit 150 (step S230).

[0051] The second image information processing unit 120B checks the status of the processing operation, and if it determines that the operation has been completed, it terminates the operation ("YES" in step S240), and if it determines that the operation has not been completed ("NO" in step S240), it transitions the processing step to step S210.

[0052] <Processing of the first image information processing unit 120A> As shown in FIG. 11, the first image information processing section 120A maintains the standby mode until the timing for updating the three-dimensional information for each pixel based on the image data captured by the first imaging section 200A (step S310).

[0053] The first image information processing unit 120A starts the obstacle determination process when the timing for data update arrives (step S320). In the obstacle determination process, as shown in FIG. 13, determination is made using three-dimensional information for each pixel. Specifically, for example, it is determined whether an object that can be determined as an obstacle (an object of a certain height and width) can be confirmed in the obstacle determination area shown in FIG.

[0054] Next, the first image information processing unit 120A executes a step determination process (step S330). In the step determination process, as shown in FIG. 12, determination is made using three-dimensional information for each pixel based on image data captured by the imaging units 200A, 200B, and 200C. The furrows are often higher than the spaces between them (hereafter referred to as furrow spaces), so there are steps between the furrows. Specifically, by utilizing the above relationship, if a step is detected in a certain area, the vehicle makes a gentle turn in the opposite direction (a gentle pivot turn) to avoid hitting the ridges. Ridge determination is performed by checking for steps in the left and right regions shown in FIG.

[0055] The first image information processing unit 120A stores the determination results of the obstacle determination process and the step determination process in the storage unit 140 via the bus line BL and the traveling control unit 150 (step S340).

[0056] The first image information processing unit 120A checks the status of the processing operation, and if it determines that the operation has been completed, it terminates the operation ("YES" in step S340), and if it determines that the operation has not been completed ("NO" in step S340), it transitions the processing step to step S310.

[0057] <Processing of the first determination processing unit 122> The processing of the first determination processing unit 122 according to this embodiment will be described with reference to FIGS.

[0058] As shown in FIG. 14, the first determination processing unit 122 performs an average value compression process on the three-dimensional information (step S331). Specifically, the three-dimensional information processing unit 1221 of the first determination processing unit 122 executes a process of compressing three-dimensional information of H480×W848 to H60×W106 as shown in FIG. The compressed three-dimensional information is stored in the storage unit 140 via the bus line BL and the driving control unit 150.

[0059] As shown in FIG. 15, the first determination processing unit 122 performs a process of fitting the compressed three-dimensional information into the determination range (step S332). Specifically, as shown in FIG. 15, when the determination range is, for example, 4×4, the three-dimensional information processing unit 1221 of the first determination processing unit 122 deletes two columns, one column on each side in the width direction, from the compressed data of H60×W106 to obtain compressed data of H60×W104.

[0060] As shown in FIG. 16, the first determination processing unit 122 performs a step determination process based on height information (step S333). As shown in Figure 16, for example, the three-dimensional information processing unit 1221 of the first judgment processing unit 122 judges that if an inter-furrow judgment value and an on-furrow judgment value exist within a 4x4 judgment range of the compressed data, the result is True, and otherwise the result is False. Specifically, as shown in FIG. 16, the three-dimensional information processing unit 1221 of the first determination processing unit 122 performs determination processing on compressed data of, for example, H60×W104 by sliding a 4×4 determination range by one pixel in the width and height directions. In the above processing method, when the judgment range is adjusted, one column on each side in the width direction is deleted, for a total of two columns, so the judgment result also decreases by one pixel. Therefore, as shown in FIG. 17, the insufficient part of the judgment range is filled with False, and the judgment result of FIG. 16 and the judgment result of FIG. 17 are superimposed to perform an OR judgment.

[0061] As shown in FIG. 18, the first determination processing unit 122 performs processing to narrow down the step determination range based on the depth and width information (step S334).

[0062] <Processing of the second determination processing unit 123> The processing of the second determination processing unit 123 according to this embodiment will be described with reference to FIGS.

[0063] As shown in FIGS. 19 and 20, the second determination processing unit 123 adds, to the same data, an on-ridge determination based on depth, height, and width information for the area within the dotted line (step S335).

[0064] The second judgment processing unit 123 performs a judgment process in which, for example, if a pixel to be judged as a step and a pixel to be judged as a ridge exist within a judgment range (for example, 4x4) in the same data, the result is True, and otherwise the result is False (step S336).

[0065] The second judgment processing unit 123 determines the amount of left and right ridge (step) detection within the ridge (step) judgment result as the grasp amount based on the width information, and performs processing to determine the left and right ridge flags according to the grasp amount (step 337).

[0066] Then, the second determination processing unit 123 performs a restoration process, that is, restores three-dimensional information by changing H60×W104 to H60×W106 and H60×W106 to H480×W848 (step S338).

[0067] That is, the second determination processing unit 123 determines the height of the ridge region in the step determination array obtained in the same manner as the first determination processing unit 122. The processing at this time is the same convolution determination processing as that used when determining a step. The judgment condition is that the step judgment and on-ridge judgment are true. The determination result in the second determination processing unit 123 is the ridge (step) determination result in the area surrounded by the dotted lines in FIGS. Depending on the amount of this determination result, it is determined whether or not there is a gentle turn to the left or right.

[0068] 21 and 22 show the state of the demonstration experiment conducted outdoors. As shown in Figures 21 and 22, the upper left ridge, the upper right ridge, the left step determination result, the right step determination result, the left ridge determination result, and the right ridge determination result are displayed in shades of gray.

[0069] <Processing of the driving control unit 150> The processing of the driving control unit 150 will be described with reference to FIGS.

[0070] As shown in FIG. 23, the traveling control unit 150 acquires status information from the image information processing unit 120 and a motor control unit (not shown), etc., and processes the status information of the working robot (step S410).

[0071] The traveling control unit 150 acquires data such as the obstacle determination result, the step determination result, and the ridge determination result from the result of processing the state information of the working robot in step S410 (step S420).

[0072] When it is determined from the obstacle determination result that an obstacle is present ("NO" in step S430), traveling control unit 150 transitions the process to step S480 to issue a forced stop instruction (step S430). On the other hand, if traveling control unit 150 determines from the obstacle determination result that there is no obstacle ("YES" in step S430), it transitions the process to step S450.

[0073] If the travel control unit 150 determines in step S450 that the ridge determination result indicates that the ridge is not between ridges ("YES" in step S450), the process proceeds to step S460. In step S460, as shown in FIG. (1) The traveling control unit 150 continues the inter-furrow mode until a ridge is confirmed in the inter-furrow determination area based on the image data obtained from the second imaging unit 200B and the third imaging unit 200C. Then, the travel control unit 150 stops the work robot 1 when no ridges are detected in the furrow spacing determination area based on the image data obtained from the second imaging unit 200B and the third imaging unit 200C. (2) If the previous mode was the inter-furrow mode (one furrow to the right), the travel control unit 150 controls the work robot 1 to make a 90-degree pivot turn to the right (hereinafter referred to as a right turn). If the previous mode was the furrow mode (one furrow left), the travel control unit 150 performs control to make a 90 degree pivot turn to the left (hereinafter referred to as a left turn). The travel control unit 150 performs control so that if the previous mode was the furrow mode (both furrows) and a turn was made the previous time, the travel control unit 150 makes a 90 degree pivot turn in the opposite direction. If the previous mode was the furrow mode (both furrows) and there was no turning the previous time, the traveling control unit 150 performs control so as to make a 90 degree pivot turn to the right. (3) When the controlled operation mode is a right turn, the traveling control unit 150 controls the work robot 1 to move straight in the furrow determination area based on the image data obtained from the imaging unit 200C until the traveling area is determined to be a furrow. When the traveling control unit 150 determines that the traveling area is between rows, it controls the working robot 1 to turn 90 degrees to the right. On the other hand, when the controlled operation mode is a left turn, the traveling control unit 150 controls the work robot 1 to move straight in the furrow determination area based on the image data obtained from the second imaging unit 200B until the traveling area is determined to be a furrow. When the traveling control unit 150 determines that the traveling area is between rows, it controls the working robot 1 to turn 90 degrees left. (4) During a pivot turn of the work robot 1, the travel control unit 150 determines the furrows in the furrow determination area based on the image data obtained from the first imaging unit 200A, transitions to furrow mode according to the determination result, and performs control.

[0074] On the other hand, if the travel control unit 150 determines in step S450 that the area is between furrows based on the ridge determination result ("NO" in step S450), it transitions the process to step S470. In step S460, as shown in Figures 24 and 25, in the case of double ridges, the travel control unit 150 determines that there are ridges on both sides, and if no ridges are found on the left or right in the step determination area, controls the work robot 1 to move straight ahead. When a step is detected on the right side in the step detection area, the travel control unit 150 controls the working robot 1 to make a gentle turn to the left until the step is no longer detected, and then to move straight ahead. When a step is detected on the left side in the step detection area, the travel control unit 150 controls the working robot 1 to make a gentle turn to the right until the step is no longer detected, and then to move straight ahead.

[0075] In the case of a single ridge, the travel control unit 150 determines that there are ridges on both sides and controls the working robot 1 to basically move in a straight line. When a step is detected on the right side in the step detection area, the travel control unit 150 controls the working robot 1 to make a gentle turn to the left until the step is no longer detected, and then to move straight ahead. When a step is detected on the left side in the step detection area, the travel control unit 150 controls the working robot 1 to make a gentle turn to the right until the step is no longer detected, and then to move straight ahead.

[0076] In step S480, the traveling control unit 150 transmits the traveling mode of the working robot 1 to the motor control unit.

[0077] The traveling control unit 150 determines whether the traveling operation of the working robot 1 has ended (step S490), and if it determines that the traveling operation of the working robot 1 has ended ("YES" in step S490), it ends the control process. On the other hand, if the traveling control unit 150 determines that the traveling operation of the working robot 1 has not ended ("NO" in step S490), it transitions the process to step S410. FIG. 27 shows the relationship between the travel locations and travel modes of the working robot 1.

[0078] <Actions and Effects> As described above, the working robot 1 according to this embodiment is a working robot that moves along driving assistance objects that are aligned or extend in the direction of travel to perform predetermined tasks, and is provided with an imaging unit 200 that is provided at least in the forward, rightward, and leftward directions of travel to capture image data, an image information detection unit 110 that detects various information from the captured image data, an image information processing unit 120 that performs image processing and analysis of the detected image data to determine the presence or absence of driving assistance objects and obstacles in the direction of travel, a driving route determination unit 130 that determines the direction of travel based on the processing results by the image information processing unit 120, and a driving control unit 150 that controls the robot to travel along the driving route determined by the driving route determination unit 130. In other words, in the work robot 1 according to this embodiment, the image information detection unit 110 detects various pieces of information from captured image data, the image information processing unit 120 processes and analyzes the detected image data to determine whether there are any driving assistance objects and whether there are any obstacles in the direction of travel, the driving route determination unit 130 determines the direction of travel based on the processing results, and the driving control unit 150 executes control to drive along the determined driving route. Therefore, the vehicle can automatically travel along a desired route without performing positioning. Furthermore, even if the crops cannot be recognized, the robot can automatically travel along the desired route.

[0079] The image information detection unit 110 of the working robot 1 according to this embodiment acquires various types of information, including depth information and Euler angle information, from the image data. In other words, control is performed based on various information including depth information and Euler angle information, so that objects requiring driving assistance can be accurately captured. Therefore, the vehicle can automatically travel along a desired route without performing positioning. Furthermore, even if the crops cannot be recognized, the robot can automatically travel along the desired route.

[0080] The image information detection unit 110 of the working robot 1 according to this embodiment includes a calculation unit that calculates Euler angle information from inertial data and calculates three-dimensional information for each pixel from information including depth information and Euler angle information, and the first image information processing unit 120A corresponding to the imaging unit 200A in front of the traveling direction includes a first determination processing unit 122 that determines the presence or absence of steps or obstacles around the traveling route based on the three-dimensional information for each pixel. In other words, the image information detection unit 110 calculates Euler angle information from the inertial data and calculates three-dimensional information (height, width, depth) for each pixel from information including depth information and Euler angle information, and the first judgment processing unit 122 corresponding to the imaging unit 200A ahead in the direction of travel judges whether there are any steps or obstacles around the traveling route based on the three-dimensional information for each pixel. Therefore, it is possible to accurately detect whether there are any steps or obstacles around the travel route. Furthermore, the vehicle can automatically travel along a desired route without performing positioning. Furthermore, even if the crops cannot be recognized, the robot can automatically travel along the desired route.

[0081] The image information detection unit 110 of the work robot 1 according to this embodiment includes a calculation unit that calculates three-dimensional information for each pixel from information including depth information and Euler angle information, and the second image information processing unit 120B corresponding to the second imaging unit 200B or third imaging unit 200C to the right or left of the traveling direction includes a second determination processing unit 123 that determines the object of driving assistance based on the three-dimensional information for each pixel. In other words, the image information detection unit 110 calculates three-dimensional information for each pixel from information including depth information and Euler angle information, and the second judgment processing unit 123 of the second image information processing unit 120B corresponding to the second imaging unit 200B or third imaging unit 200C to the right or left of the traveling direction judges the driving assistance object based on the three-dimensional information for each pixel. Therefore, the object of travel assistance can be accurately captured. Furthermore, the vehicle can automatically travel along a desired route without performing positioning. Furthermore, even if the crops cannot be recognized, the robot can automatically travel along the desired route.

[0082] The first judgment processing unit 122 of the working robot 1 according to this embodiment includes a 3D information processing unit 1221 that performs averaging processing to average the 3D information for each pixel and compressing the averaged 3D information for each pixel, and an information extraction unit 1222 that extracts 3D information that exceeds a predetermined threshold based on height information contained in the processed 3D information within the judgment area, and judges whether there are any steps or obstacles around the travel route based on the height information contained in the processed 3D information within the judgment area. In other words, the three-dimensional information processing unit 1221 performs an averaging process to average the three-dimensional information for each pixel and a compression process on the averaged three-dimensional information for each pixel, the information extraction unit 1222 extracts three-dimensional information that exceeds a predetermined threshold based on the height information contained in the processed three-dimensional information within the judgment area, and the first judgment processing unit 122 judges whether there are any steps or obstacles around the driving route based on the height information contained in the processed three-dimensional information within the judgment area. Therefore, it is possible to accurately detect whether there are any steps or obstacles around the travel route. Furthermore, the vehicle can automatically travel along a desired route without performing positioning. Furthermore, even if the crops cannot be recognized, the robot can automatically travel along the desired route.

[0083] The second determination processing unit 123 of the working robot 1 according to this embodiment includes a three-dimensional information processing unit 1231 that performs an averaging process to average the three-dimensional information for each pixel and a compression process of the averaged three-dimensional information for each pixel, and an information extraction unit 1232 that extracts the three-dimensional information that exceeds a predetermined threshold based on height information included in the processed three-dimensional information within the determination area, and determines the traveling assistance object based on depth information and width information corresponding to the extracted three-dimensional information. In other words, the three-dimensional information processing unit 1231 performs an averaging process to average the three-dimensional information for each pixel and a compression process on the averaged three-dimensional information for each pixel, the information extraction unit 1232 extracts three-dimensional information that exceeds a predetermined threshold based on height information contained in the processed three-dimensional information within the judgment area, and the second judgment processing unit 123 judges the driving assistance object based on depth information and width information corresponding to the extracted three-dimensional information. Therefore, the object of travel assistance can be accurately captured. Furthermore, the vehicle can automatically travel along a desired route without performing positioning. Furthermore, even if the crops cannot be recognized, the robot can automatically travel along the desired route.

[0084] When the first determination processing unit 122 of the work robot 1 according to this embodiment determines that the steps that were present around the travel route have disappeared, the travel route determination unit 130 determines a travel route that changes direction in the direction of the steps, and when the second determination processing unit 123 determines that the ridges exist in both the left and right directions, it further determines the travel route to be oriented in the direction in which the travel assistance object extends. Therefore, even in cases where rows of ridges are formed, such as in farmland, the robot can automatically travel accurately and smoothly along the shortest route.

[0085] <Variation 1> In this embodiment, the ridges of farmland are used as an example of the object of travel assistance, but even if the object of travel assistance is a path along the curb of a sidewalk, it is possible to control the work robot 1 by performing control similar to that described above.

[0086] <Variation 2> In this embodiment, the ridges of farmland are used as an example of the object of travel assistance, but even if the object of travel assistance is a path along an indoor or outdoor installation or placement, it is possible to control the work robot 1 by performing control similar to that described above.

[0087] <Variation 3> In this embodiment, an example has been given of detecting depth information and Euler angle information from image data from the first imaging unit 200A, the second imaging unit 200B, and the third imaging unit 200C, but in addition to this information, color information may also be detected. By detecting color information, it is possible to remotely monitor the image of the surrounding area in which the working robot 1 is traveling, for example.

[0088] The working robot 1 of the present invention can be realized by recording the processing of the driving control unit 150 on a recording medium readable by a computer system, and having the driving control unit 150 read and execute the program recorded on this recording medium. The computer system here includes hardware such as an OS and peripheral devices.

[0089] Furthermore, if a WWW (World Wide Web) system is used, the "computer system" also includes the homepage provision environment (or display environment). The above program may be transmitted from a computer system that stores the program in a storage device or the like to another computer system via a transmission medium or by transmission waves in the transmission medium. Here, the "transmission medium" that transmits the program refers to a medium that has the function of transmitting information, such as a network (communication network) like the Internet or a communication line (communication line) like a telephone line.

[0090] The program may also be a program for implementing some of the above-mentioned functions, or may be a so-called differential file (differential program) that can implement the above-mentioned functions in combination with a program already stored in the computer system.

[0091] The above describes in detail an embodiment of the present invention with reference to the drawings. However, all accident information collection devices that can be implemented by a person skilled in the art by appropriately modifying the design based on the accident information collection device described above as an embodiment of the present invention also fall within the technical scope of the present invention as long as they include the gist of the present invention. Within the scope of the concept of the present invention, a person skilled in the art may conceive of various modifications and alterations, and it is understood that these modifications and alterations also fall within the technical scope of the present invention. For example, any embodiment in which a person skilled in the art has appropriately added or deleted components or modified the design, or added or omitted steps or modified conditions, is included within the technical scope of the present invention as long as it contains the gist of the present invention.

[0092] Furthermore, other effects and advantages brought about by the aspects described in this embodiment that are clear from the description in this specification or that can be appropriately thought of by a person skilled in the art are naturally understood to be brought about by the present invention. Various inventions can be formed by appropriately combining the multiple components disclosed in the above embodiments. For example, some components may be omitted from all the components shown in the embodiment. Furthermore, components from different embodiments may be combined as appropriate. [Explanation of symbols]

[0093] 1. Work robot 100;Control unit 110: Image information detection unit 120; Image information processing section 120A: First image information processing section 120B: Second image information processing section 121; Information input section 122: First determination processing unit 1221;3D information processing section 1222;Information extraction part 123: Second determination processing unit 1231;3D information processing section 1232;Information extraction part 130: Driving route determination unit 140;Memory part 150: Travel control unit 200A: First imaging unit 200B: Second imaging unit 200C: Third imaging unit 300;Movement mechanism

Claims

1. A working robot that moves along a traveling assistance object that is aligned or extends in the traveling direction and performs a predetermined task, imaging units provided at least in front, right, and left directions in the traveling direction, for capturing image data; an image information detection unit that detects various pieces of information from the captured image data; an image information processing unit that performs image processing and analysis on the detected image data to determine whether or not the driving assistance object is present and whether or not there is an obstacle in the traveling direction; a travel route determination unit that determines the traveling direction based on a processing result by the image information processing unit; a travel control unit that executes control so that the vehicle travels along the travel route determined by the travel route determination unit; A working robot equipped with a

2. The working robot according to claim 1 , wherein the image information detection unit acquires the various pieces of information including depth information and Euler angle information from the image data.

3. The image information detection unit a calculation unit that calculates the Euler angle information from the inertial data and calculates three-dimensional information for each pixel from information including the depth information and the Euler angle information; a first image information processing unit corresponding to the imaging unit located ahead in the traveling direction, 3. The working robot according to claim 2, further comprising a first determination processing unit that determines the presence or absence of steps or obstacles around the travel route based on the three-dimensional information for each pixel.

4. The image information detection unit a calculation unit that calculates three-dimensional information for each pixel from information including the depth information and Euler angle information; The second image information processing unit corresponding to the imaging unit in the right or left direction of the traveling direction is The working robot according to claim 3 , further comprising a second determination processing unit that determines the driving assistance object based on the three-dimensional information for each pixel.

5. The first determination processing unit a three-dimensional information processing unit that performs an averaging process to average the three-dimensional information for each pixel and a compression process to the averaged three-dimensional information for each pixel; an information extraction unit that extracts the three-dimensional information exceeding a predetermined threshold based on height information included in the processed three-dimensional information within a determination region; Including, 4. The working robot according to claim 3, wherein the presence or absence of steps or obstacles around the travel route is determined based on height information contained in the processed three-dimensional information within the determination area.

6. The second determination processing unit a three-dimensional information processing unit that performs an averaging process to average the three-dimensional information for each pixel and a compression process to the averaged three-dimensional information for each pixel; an information extraction unit that extracts the three-dimensional information exceeding a predetermined threshold based on height information included in the processed three-dimensional information within a determination region; Including, The working robot according to claim 4, wherein the travel assistance object is determined based on depth information and width information corresponding to the extracted three-dimensional information.

7. 7. The working robot according to claim 6, wherein the travel route is a path along ridges of farmland as the travel assistance object, or a path between adjacent ridges.

8. 8. The working robot according to claim 7, wherein, when a first determination processing unit determines that a step that was present around the travel route has disappeared, the travel route determination unit determines the travel route such that the robot turns in the direction in which the step was present, and when a second determination processing unit determines that the ridge is present in both the left and right directions, the robot further determines that the travel route turns in the direction in which the travel assistance object extends.

9. 7. The working robot according to claim 6, wherein the travel route is a path along a curb of a sidewalk as the travel assistance object.

10. 7. The working robot according to claim 6, wherein the travel route is a path along an indoor or outdoor installed or placed object that serves as the travel assistance object.

Images