Robot, robot control device, robot system, and robot control method

Abstract

To achieve appropriate control over robots. [Solution] The robot 100 is equipped with an action device E1 that acts on an object, a drive device E2 that drives the action device E1, an environmental information acquisition unit 110 that acquires environmental information indicating the state of the operating environment of the action device E1 and the drive device E2, an object information acquisition unit E3 that acquires object information relating to the object, an area condition generation unit 116 that generates area condition data representing the operable area of ​​the action device E1 and the drive device E2 in the operating environment using the environmental information, an action position calculation unit E4 that calculates the action position where the action device E1 acts on the object using the area condition data and object information, and a recording position calculation unit E5 that calculates a recording position for recording the action result of the action device E1 on the object using the area condition data and the calculated action position.

Description

Technical Field

[0001] The present invention relates to a robot, a robot control device, a robot system, and a robot control method.

Background Art

[0002] In recent years, with the decline of the working population, the need for automation of tasks that were previously performed manually by humans has increased, and work substitution by robots has been progressing. As a work substitution technology by robots, there is Patent Document 1. This publication states that "the robot body 1 in FIG. 2 mounts a work monitoring camera 18 and a lighting fixture 19 on a moving cart 2 by a device that can pan and tilt on a pan-tilt mechanism, and the external sensor 16 is mounted in an arrangement suitable for grasping the external environment without obstacles. This external sensor 16 uses ultrasonic waves, light, etc. that can grasp obstacles in all directions without contact, and can recognize and distinguish the wall surface and the arm mechanism 3 from obstacles and perform exclusion processing."

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, Patent Document 1 does not consider changing the positions of the camera and the lighting fixture independently. Therefore, there is a possibility that a record suitable for the work performed by the robot cannot be obtained. In view of the above circumstances, an object of the present invention is to provide a robot, a robot control device, a robot system, and a robot control method that can cope with various situations.

Means for Solving the Problems

[0005] To solve the above problems, the robot of the present invention is characterized by comprising: an action device that acts on an object; a drive device that drives the action device; an environmental information acquisition unit that acquires environmental information indicating the state of the operating environment of the action device and the drive device; an object information acquisition unit that acquires object information relating to the object; an area condition generation unit that uses the environmental information to generate area condition data representing the operable area of ​​the action device and the drive device in the operating environment; an action position calculation unit that uses the area condition data and the object information to calculate the action position in which the action device acts on the object; a recording position calculation unit that uses the area condition data and the action position to calculate a recording position for recording the result of the action device's action on the object; an action trajectory generation unit that uses the area condition data, the action position and the recording position to generate an action trajectory of the drive device; and an action control unit that uses the action trajectory to control the drive device and causes the action device to store the result of its action on the object in an action recording device. [Effects of the Invention]

[0006] According to the present invention, it is possible to provide a robot, a robot control device, a robot system, and a robot control method that can respond to various situations. [Brief explanation of the drawing]

[0007] [Figure 1] This is a diagram showing the configuration of the robot system according to the first embodiment. [Figure 2] This is a schematic side view of a robot applied to the first embodiment. [Figure 3] This is a schematic diagram of the environment under a railway vehicle in the first embodiment. [Figure 4] This is a schematic perspective view of an input terminal. [Figure 5] This is a functional block diagram of the main components of the robot system according to the first embodiment. [Figure 6] This is another functional block diagram of the main parts of the robot system according to the first embodiment. [Figure 7]It is a flowchart of the appearance inspection process in the first embodiment. [Figure 8] It is a schematic diagram of the environment under other railway vehicles in the first embodiment. [Figure 9] It is a schematic side view of the robot applied to the second embodiment. [Figure 10] It is a schematic diagram of the environment under the railway vehicle in the second embodiment. [Figure 11] It is a functional block diagram of the main part of the robot system according to the second embodiment. [Figure 12] It is another functional block diagram of the main part of the robot system according to the second embodiment. [Figure 13] It is a flowchart of the knocking sound inspection process in the second embodiment. [Figure 14] It is a schematic side view of the robot applied to the third embodiment. [Figure 15] It is a schematic diagram of the environment under the railway vehicle in the third embodiment. [Figure 16] It is a functional block diagram of the main part of the robot system according to the third embodiment. [Figure 17] It is another functional block diagram of the main part of the robot system according to the third embodiment. [Figure 18] It is a flowchart of the tightening torque inspection process in the third embodiment. [Figure 19] It is a schematic side view of the robot applied to the fourth embodiment. [Figure 20] It is a schematic diagram of the environment under the railway vehicle in the fourth embodiment. [Figure 21] It is a functional block diagram of the main part of the robot system according to the fourth embodiment. [Figure 22] It is another functional block diagram of the main part of the robot system according to the fourth embodiment. [Figure 23] It is a flowchart of the meter inspection process in the fourth embodiment. [Figure 24] It is a block diagram of a computer.

Modes for Carrying Out the Invention

[0008] [Overview of Embodiment] As an example of work replacement by a robot, an appearance inspection work can be cited. The appearance inspection work is a work of checking an object from various angles and confirming whether there are abnormalities such as scratches and dirt on the appearance. In particular, for the automation of appearance inspection work in a dark place, it is preferable to perform an appropriate lighting operation in combination with the acquisition of image data from various positions and angles. When applying the technology of Patent Document 1 described above, it is considered that a lighting device and a camera can be attached to the tip of a multi-degree-of-freedom robotic arm as means for measuring image data in a dark place. Thereby, it is considered that images taken from various positions and angles can be collected.

[0009] By the way, in a narrow environment such as under a railway vehicle where structures are complexly arranged, an operator (human) may perform inspection work. In this case, the operator performs the inspection while changing the relative positional relationship between the position where the inspection object is observed and the lighting device. And it is preferable if such inspection work can be replaced by a robot.

[0010] However, in Patent Document 1, the positional relationship between the camera and the lighting device is not particularly described in detail. Therefore, in the technology applying Patent Document 1, depending on the positional relationship between the camera and the lighting device, it is considered that there may be cases where the inspection object cannot be inspected accurately. The embodiment described below was devised in view of the above actual situation, and replaces the work of an operator in a narrow environment such as under a railway vehicle with a robot. For this reason, when automating inspection work using tools such as a lighting device, the tool operation by the operator and the result thereof are confirmed. Then, in accordance with the inspection object and the inspection environment, based on the recorded operations of the operator, the tool operation and the position of the means for recording the inspection result are independently controlled.

[0011] [First Embodiment] <Configuration of the First Embodiment> (Overall Configuration) FIG. 1 is a configuration diagram of a robot system 500 according to the first embodiment. In Figure 1, the robot system 500 comprises a robot 100, an input terminal 200, a storage and external processing unit 300, and an inspection work measurement system 600. These components of the robot system 500 can communicate with each other via a network 400 and exchange various types of data.

[0012] The inspection work measurement system 600 is attached to the worker 800 and records the status of the worker 800 and the object being measured. "Object being measured" refers to the object being measured in tasks such as inspection, processing, manufacturing, cleaning, and maintenance. The storage and external processing unit 300 stores data generated by the robot 100, input terminal 200, or inspection work measurement system 600. In particular, by storing the visual inspection data of the object being measured acquired by the worker in the storage and external processing unit 300, it becomes possible to use this data as training data when building the robot 100's functions based on machine learning.

[0013] Furthermore, if the processing power of the robot control device 111 (see Figure 5), which is a computer mounted on the robot 100, is insufficient, various calculations can be performed using the storage / external processing unit 300. In this way, the storage / external processing unit 300 supplements the processing power or memory capacity of the robot 100, input terminal 200, or inspection work measurement system 600. Therefore, if the processing power and memory capacity of the robot 100, input terminal 200, and inspection work measurement system 600 are sufficient, the storage / external processing unit 300 may be omitted.

[0014] Figure 2 is a schematic side view of the robot 100 applied to the first embodiment. The robot 100 includes a camera device 101, a first arm 102, a light source device 103, a second arm 104, a movement mechanism 105, a main body 106, and a transceiver 107.

[0015] The mobile mechanism 105 is a four-legged mobile mechanism located at the bottom of the main body 106, and can move the robot 100 in all directions on a plane. The first arm 102 is a serial link arm located at the top of the main body 106 and equipped with a camera device 101 at its tip. The second arm 104 is also a serial link arm equipped with a light source device 103 at its tip. The light source device 103 allows for setting the brightness, wavelength, and illumination range of the light. The transceiver 107 is located at the top of the main body 106.

[0016] Furthermore, although not shown in Figure 2, the main body 106 is equipped with a robot control device 111 (see Figure 5) that controls the robot 100, and various sensors (details will be described later) that detect the position and orientation of the robot. Force sensors are provided at the joints of the first arm 102, the second arm 104, and the movement mechanism 105. The total degrees of freedom of the first arm 102 and the second arm 104 between the camera device 101 and the light source device 103 is 5 or more. With this configuration, the optical axis irradiated from the light source device 103 can be set to any orientation relative to the position and orientation of the camera device 101.

[0017] In the configuration described above, the light source device 103 illuminates the positions of objects 1, 2, and 3 (see Figure 8) that are to be illuminated. In other words, it "acts on the working position (the position to be illuminated) of the object (illuminates it with light)," so the light source device 103 is sometimes called the "working device E1." The camera device 101 collects image data of objects 1, 2, and 3, and "acquires object information (in this case, image data) about the object," so the camera device 101 is sometimes called the "object information acquisition unit E3." The first arm 102, the second arm 104, and the moving mechanism 105 drive the working device E1 or the object information acquisition unit E3 and set their positions, so these are sometimes called the "driving device E2."

[0018] Figure 3 is a schematic diagram of the railway vehicle under-vehicle environment 700 in the first embodiment. This Figure 3 shows the work environment and also shows how inspection work data D600 (details will be described later) is recorded through the visual inspection performed by the worker 800. In the railway vehicle under-vehicle environment 700, parallel rails 5a and 5b are laid, and a wheel 4 is placed on them. Objects 1, 2, and 3 are positioned opposite each other on either side of the wheel 4 from the perspective of the worker 800. The worker 800 performs a visual inspection of objects 1, 2, and 3 on either side of the wheel 4. At that time, the worker 800 inspects the inspection target position O1 using a light source device 604 capable of recording the position and orientation of the light source. The inspection work measurement system 600 is attached to the worker 800. The worker 800's actions during the inspection as described above are recorded as inspection work data D600 by the inspection work measurement system 600.

[0019] Figure 4 is a schematic perspective view of the input terminal 200. The input terminal 200 includes a display unit 201, an input unit 202, and a transceiver 203. The transceiver 203 and the transceiver 107 (see Figure 2) installed on the robot 100 communicate bidirectionally via the network 400 (see Figure 1). As a result, the input terminal 200 outputs operation commands to the robot 100 via the transceivers 203 and 107 and performs status checks on the robot 100. Furthermore, the input terminal 200 outputs operation commands to the inspection work measurement system 600 and performs status checks on the inspection work measurement system 600.

[0020] Figures 5 and 6 are functional block diagrams of the main parts of the robot system 500 according to the first embodiment. Specifically, Figure 5 mainly shows the functional blocks of the robot 100, and Figure 6 mainly shows the functional blocks of the input terminal 200 and the inspection work measurement system 600. In Figure 5, the robot 100 includes, in addition to the elements (101-107) described in Figure 2, a position / attitude sensor 108, a force sensor 109, an environmental information acquisition unit 110, and a robot control device 111.

[0021] The robot control device 111 comprises a sensor signal processor 112, a motion generator 113, a light source controller 119, a motion control unit 120, and a motion recording device 121. The motion generator 113 comprises a camera trajectory generator 114, a light source trajectory generator 115, a region condition generation unit 116, an avoidance trajectory generator 117, and a motion trajectory generation unit 118.

[0022] Furthermore, in Figure 6, the input terminal 200 includes a display unit 201, an input unit 202, a transceiver 203, and an inspection work data storage unit 204. The display unit 201 displays various information to the worker 800. The input unit 202 inputs various information based on the operations of the worker 800. The inspection work data storage unit 204 stores the inspection work data D600 (details will be described later) output from the inspection work measurement system 600.

[0023] The transceiver 203 communicates bidirectionally with other elements of the robot system 500 (see Figure 1) via the network 400. In particular, the transceiver 203 receives inspection work data D600 from the inspection work measurement system 600 and stores it in the inspection work data storage unit 204, and also supplies the inspection work data D600 to the robot 100 as needed.

[0024] Furthermore, the inspection work measurement system 600 includes a gaze measurement sensor 601, an inspection target measurement sensor 602, a head position measurement sensor 603, a light source device 604, a transceiver 605, and an inspection work measurement controller 610. The inspection work measurement controller 610 also includes a sensor signal processor 611, an inspection work extractor 612, and a measurement result saver 613. The inspection work extractor 612 also includes a gaze extractor 614, an inspection target extractor 615, an eye position calculator 616, and a light source position and orientation calculator 617.

[0025] (Inspection work measurement system 600) In Figure 6, the eye-tracking sensor 601 in the inspection work measurement system 600 measures the position and line of sight of the worker 800 during work by, for example, measuring the eye movements of the worker 800, and outputs the measurement results. The inspection object measurement sensor 602 measures the appearance of the object (for example, 1, 2, and 3 shown in Figure 3) during the visual inspection work and outputs the measurement results. Therefore, the inspection object measurement sensor 602 is equipped with, for example, a camera that takes images of various structures in the direction the worker is facing, and a sensor capable of measuring the three-dimensional shape of the structure.

[0026] The head position and measurement sensor 603 is equipped with an acceleration sensor, a gyroscope, etc. (not shown) and measures the position and orientation of the head of the worker 800 wearing the inspection work measurement system 600, and outputs the measurement results. The light source device 604 is operated while being held by the worker 800 and emits light of a specified wavelength. The light source device 604 is also equipped with an acceleration sensor, a gyroscope, etc. (not shown) and measures the position and orientation of the machine itself, and outputs the measurement results. The transceiver 605 inputs and outputs various information via the network 400.

[0027] The inspection work measurement controller 610 includes a sensor signal processor 611, an inspection work extractor 612, and a measurement result saver 613. The inspection work extractor 612 also includes a gaze extractor 614, an inspection target extractor 615, an eye position calculator 616, and a light source position and orientation calculator 617.

[0028] The sensor signal processor 611 converts the measurement results from each of the elements 601 to 604 described above into physical parameters while the worker 800 is working. The inspection work extractor 612 includes a gaze extractor 614, an inspection target extractor 615, an eye position calculator 616, and a light source position and orientation calculator 617.

[0029] The gaze extractor 614 calculates the gaze of the worker 800 during the visual inspection based on the measurement results of the gaze measurement sensor 601 and the head position measurement sensor 603. The inspection target extractor 615 calculates the external shape of the inspection target (for example, objects 1, 2, and 3 in Figure 3) and the inspection path based on the measurement results of the inspection target measurement sensor 602. Here, "inspection path" refers to, for example, the inspection order of objects 1, 2, and 3.

[0030] The eye position calculator 616 calculates the eye position of the worker 800 during the visual inspection based on the measurement results of the gaze measurement sensor 601 and the head position measurement sensor 603. The light source position and orientation calculator 617 calculates the position and orientation of the light source device 604 based on the measurement results of the light source device 604.

[0031] In this manner, the inspection work extractor 612 outputs various data calculated based on the measurement results of the sensor signal processor 611. The measurement result saver 613 supplies the various data supplied from the inspection work extractor 612 as time-series data, known as inspection work data D600, to the input terminal 200, etc., via the transceiver 605 and the network 400. That is, the inspection work data D600 is time-series data during the visual inspection of the items listed below. • External shape and inspection route of the object to be inspected • The gaze of 800 workers • Eye position of 800 workers • Position and orientation of the light source device 604

[0032] (Robot 100) In Figure 5, the camera device 101 installed on the robot 100 captures image data for visual inspection. As shown in Figure 2, the camera device 101 is attached to the tip of the first arm 102. The position and attitude sensor 108 is equipped with an acceleration sensor, a gyroscope, etc., and measures the position and attitude of the robot 100 and outputs the measurement results.

[0033] The position and orientation sensor 108 measures the position and orientation of the robot 100, thereby determining the positional relationship between the robot 100 and objects 1, 2, and 3 (see Figure 8). Therefore, the position and orientation sensor 108 "acquires object information about the objects (in this case, the positional relationship between the objects and the robot 100)." For this reason, the position and orientation sensor 108, together with the camera device 101 described above, is sometimes referred to as the "object information acquisition unit E3."

[0034] The force sensor 109 is equipped with a torque sensor or the like that measures the torque of each joint of the robot 100, and measures the load applied to each part of the robot 100. The environmental information acquisition unit 110 is equipped with a point cloud sensor such as a laser scanner that can measure the three-dimensional shape of the environment around the robot 100, and measures the environment around the robot 100.

[0035] The sensor signal processor 112 converts the image data from the camera device 101, the measurement results from the position / attitude sensor 108, the measurement results from the force sensor 109, and the measurement results from the environmental information acquisition unit 110 into physical parameters and outputs them as sensor data D112.

[0036] The transceiver 107 generates data that defines the operation of the robot 100 based on the stored inspection work data D600 (see Figure 6) stored in the inspection work data storage unit 204 (see Figure 6) and the sensor data D112. The contents of this data are described below.

[0037] First, the camera trajectory generator 114 within the motion generator 113 generates camera trajectory data based on the inspection work data D600 and the specifications of the camera device 101, such as resolution and field of view. This camera trajectory data corresponds to the "line of sight of worker 800 during visual inspection" and "position of worker 800's eyes during visual inspection" included in the inspection work data D600. In other words, the camera trajectory generator 114 represents a camera trajectory that captures an image similar to what worker 800 was visually observing while performing the visual inspection.

[0038] Here, the camera trajectory is a plot of the time change of the "camera position," and the camera records (photographs) the results (illumination results) of the action of the operating device E1 (light source device 103) on objects 1, 2, and 3. Therefore, the camera trajectory can be thought of as "the recording position where the results of the action of the operating device E1 on objects 1, 2, and 3 are recorded." For this reason, the camera trajectory generator 114 is sometimes called the "recording position calculation unit E5."

[0039] The light source trajectory generator 115 generates light source trajectory data based on the inspection work data D600 and the usage conditions such as the illumination range of the light source device 103. This light source trajectory data corresponds to the "position and orientation of the light source device 604" included in the inspection work data D600. In other words, the light source trajectory generator 115 represents the movement trajectory of the light source that reproduces the lighting conditions when the worker 800 was performing the visual inspection work.

[0040] Here, the light source trajectory generator 115 can be thought of as calculating the operating position (illumination position) where the operating device E1 (light source device 103) acts on objects 1, 2, and 3 (illuminating objects 1, 2, and 3). For this reason, the light source trajectory generator 115 is sometimes called the "operating position calculation unit E4".

[0041] The domain condition generation unit 116 generates domain condition data representing the area in which the robot 100 can operate, based on the measurement results from the environmental information acquisition unit 110. Specifically, the domain condition generation unit 116 identifies the location and shape of obstacles based on the measurement results from the environmental information acquisition unit 110. Obstacles are, for example, the wheels 4 shown in Figure 3. The domain condition generation unit 116 then generates domain condition data based on this obstacle information and the mechanical constraints of the robot 100.

[0042] The avoidance trajectory generator 117 compares the camera trajectory data and light source trajectory data with the area condition data to determine whether the trajectory created by the camera trajectory data or the light source trajectory data interferes with an obstacle. If it determines that interference occurs in either trajectory, the avoidance trajectory generator 117 generates camera avoidance trajectory data and light source avoidance trajectory data.

[0043] Camera avoidance trajectory data is trajectory data that has been modified from the original camera trajectory data to avoid obstacles in the portion that interferes with them. Similarly, light source avoidance trajectory data is trajectory data that has been modified from the original light source trajectory data to avoid obstacles in the portion that interferes with them.

[0044] However, if no interference occurs in the original camera trajectory data, the camera avoidance trajectory data will be identical to the original camera trajectory data. Similarly, if no interference occurs in the original light source trajectory data, the light source avoidance trajectory data will be identical to the original light source trajectory data.

[0045] Thus, the camera avoidance trajectory data generated by the avoidance trajectory generator 117 can be considered, like the camera trajectory data, as "recording positions that record the results of the action of the action device E1 on objects 1, 2, and 3," and therefore the avoidance trajectory generator 117 can also be considered as a "recording position calculation unit E5." Furthermore, the light source avoidance trajectory data generated by the avoidance trajectory generator 117 can be considered, like the light source trajectory data, as "action positions (illumination positions) where the action device E1 (light source device 103) acts on objects 1, 2, and 3 (illuminating objects 1, 2, and 3)," and therefore the avoidance trajectory generator 117 also functions as an "action position calculation unit E4."

[0046] The motion trajectory generation unit 118 generates motion trajectory data based on camera trajectory data (or camera avoidance trajectory data) and light source trajectory data (or light source avoidance trajectory data). The motion trajectory data shows the motion trajectories of each joint of the first arm 102, the second arm 104, and the movement mechanism 105.

[0047] The light source controller 119 selects and sets the intensity, wavelength, and irradiation range of the light to be emitted from the light source device 103. For example, if the purpose is to inspect scratches on the surfaces of objects 1, 2, and 3, blue light may be emitted to make the scratches more visible. In this way, the light source controller 119 specifies the wavelength of the light to be emitted according to the irradiation area of ​​objects 1, 2, and 3.

[0048] Furthermore, the light source controller 119 may adjust the position or light intensity of the light source device 103 based on the results of comparing the image captured by the camera device 101 during operation with the inspection image taken by the worker 800 stored in the inspection work data storage device 204. In other words, it is preferable to adjust the position or light intensity of the light source device 103 to reproduce the illuminance of the inspection target position O1 (see Figure 3) during the visual inspection work by the worker 800.

[0049] The motion control unit 120 controls the movement of each joint of the drive unit E2, namely the first arm 102, the second arm 104, and the moving mechanism 105, based on the contents of the motion trajectory data generated by the motion trajectory generation unit 118. The motion control unit 120 then stores the images captured by the camera device 101 in the motion recording device 121 while the drive unit E2 is operating.

[0050] Furthermore, the motion control unit 120 generates a group of inspection images from the images stored in the motion recording device 121 and stores these images in the motion recording device 121. The group of inspection images is, for example, a video generated by aggregating the captured images.

[0051] <Operation of the First Embodiment> Next, the operation of the first embodiment will be described. Figure 7 is a flowchart of the visual inspection process in the first embodiment. In Figure 7, when the process proceeds to step S1, the worker 800 performs a visual inspection. Here, the worker 800 first inputs a work start command via the input section 202 of the input terminal 200 (see Figure 6). This work start command is input to the inspection work extractor 612 in the inspection work measurement controller 610 via the transceivers 203, 605 and the network 400.

[0052] Next, worker 800 performs a visual inspection in the railway vehicle under-vehicle environment 700 (see Figure 3). As a result, the inspection work measurement system 600 acquires the status of the visual inspection performed by worker 800 as inspection work data D600 (see Figure 6) and outputs this inspection work data D600 via the network 400. The input terminal 200 also displays the received inspection work data D600 on the display unit 201 and saves it in the inspection work data storage unit 204.

[0053] Next, the robot 100 is installed in an environment similar to the railway vehicle under-vehicle environment 700 (see Figure 3). An example of this is shown in Figure 8. Figure 8 is a schematic diagram of another railway vehicle under-vehicle environment 720 in the first embodiment. In the railway vehicle under-vehicle environment 720, a robot 100 is installed in place of the worker 800 and the inspection work measurement system 600. The rest of the configuration of the railway vehicle under-vehicle environment 720 is the same as that of the railway vehicle under-vehicle environment 700 (see Figure 3).

[0054] Returning to Figure 7, when the process proceeds to step S2, the trajectory data calculation process is executed. Specifically, a calculation start command is input to the robot 100 via the input unit 202 shown in Figure 6, and is input to the motion generator 113 and light source controller 119 in the robot control device 111 via the transceivers 203 and 107. As a result, the camera trajectory generator 114 generates camera trajectory data, and the light source trajectory generator 115 generates light source trajectory data.

[0055] Next, when the process proceeds to step S3, the region condition generation unit 116 in the robot control device 111 generates region condition data representing the operational region of the robot 100 based on the measurement results from the environmental information acquisition unit 110.

[0056] Next, when the process proceeds to step S4, the avoidance trajectory generator 117 compares the camera trajectory data and light source trajectory data with the area condition data and determines whether the trajectory of the camera trajectory data or the light source trajectory data interferes with an obstacle.

[0057] If "No" (no interference) is determined in step S4, the process proceeds to step S5, where the motion trajectory generation unit 118 generates motion trajectory data that realizes the camera trajectory data and the light source trajectory data. As described above, the generated motion trajectory data is time-series data of the movements of each joint of the first arm 102, the second arm 104, and the movement mechanism 105.

[0058] On the other hand, if "Yes" (interference occurs) is determined in step S4, the process proceeds to step S22. Here, the avoidance trajectory generator 117 generates camera avoidance trajectory data and light source avoidance trajectory data. Next, when the process proceeds to step S23, the motion trajectory generation unit 118 generates motion trajectory data that realizes the camera avoidance trajectory data and light source avoidance trajectory data.

[0059] Once step S5 or step S23 described above is completed, the process proceeds to step S6. Here, the motion control unit 120 drives the robot 100 according to the generated motion trajectory data. That is, the motion control unit 120 controls the movement of the joints of the first arm 102, the second arm 104, and the moving mechanism 105. As a result, the robot 100 performs actions that correspond to the movements of the worker 800.

[0060] During the execution of step S6, the motion recording device 121 generates and stores inspection image data based on images captured by the camera device 101. This inspection image data is, for example, image data compiled from the moving images captured by the camera device 101.

[0061] Furthermore, in step S6, the light source controller 119 controls the intensity, wavelength, and irradiation range of the light emitted from the light source device 103. For example, if the purpose is to inspect for scratches on the surface of the object to be inspected, blue light may be emitted to make the scratches more visible. In addition, during the execution of the operation, the image captured by the camera device 101 may be compared with the inspection image by the worker 800 stored in the inspection work data saver 204, and the posture of the second arm 104 or the light emitted by the light source controller 119 may be adjusted so that the images are equivalent. Through these operations, the robot 100 can reproduce the visual inspection performed by the worker 800 as shown in Figure 3, as shown in Figure 8.

[0062] Next, when the process proceeds to step S7, the input terminal 200 displays the inspection image data. Specifically, the inspection image data is input to the input terminal 200 sequentially via the motion recording device 121 (see Figure 5), transceiver 107, network 400, and transceiver 203 (see Figure 6). As a result, the input terminal 200 displays the inspection image data on its display unit 201.

[0063] [Second Embodiment] <Configuration of the second embodiment> Next, a robot system according to the second embodiment will be described. The overall configuration of the robot system according to the second embodiment is the same as that of the first embodiment (see Figure 1). Furthermore, the configuration of each element is the same as that of the robot system according to the first embodiment, except for the points described below. In the description of each embodiment, parts corresponding to parts of other embodiments mentioned above are denoted by the same reference numerals, and their descriptions may be omitted.

[0064] Figure 9 is a schematic side view of the robot 130 applied to the second embodiment. The robot 130 of this embodiment is applicable to impact sound inspection work and comprises a first arm 102, a second arm 104, a moving mechanism 105, a main body 106, a transceiver 107, a camera device with a microphone 131, and an impact sound device 133.

[0065] The microphone-equipped camera device 131 comprises a camera device 131a and a sound collection device 131b, and is mounted on the tip of the first arm 102 of the serial link mechanism. The impact sound device 133 is mounted on the tip of the second arm 104 of the serial link mechanism. The impact sound device 133 is a device that can apply impact to the object being inspected with any force. The total degrees of freedom of the first arm 102 and the second arm 104 between the microphone-equipped camera device 131 and the impact sound device 133 is 6 degrees of freedom or more.

[0066] With this configuration, the sound-making device 133 can perform sound-making operations on an object from any direction, regardless of the position and orientation of the microphone-equipped camera device 131. The elements of the robot 130 other than those described above are the same as those of the robot 100 of the first embodiment (see Figure 2). Therefore, in this embodiment, the operating device E1 includes the sound-making device 133, and the object information acquisition unit E3 includes the microphone-equipped camera device 131. And, as in the first embodiment, the drive device E2 includes the first arm 102, the second arm 104, and the moving mechanism 105.

[0067] Figure 10 is a schematic diagram of the railway vehicle under-vehicle environment 730 in the second embodiment. In the railway vehicle under-vehicle environment 730, parallel rails 5a and 5b are laid, and a wheel 4 is placed on them. From the perspective of the worker 800, an object 6 is positioned opposite the wheel 4. The object 6 is provided with fixing parts 62, which are bolted fastening parts, on various parts. The worker 800 performs a tapping inspection of the fixing parts 62 on either side of the wheel 4. In the illustrated state, the worker 800 is inspecting one of the fixing parts 62, which is the inspection target position O2, using a tapping device 606 capable of recording position and orientation. The actions during this inspection are recorded by the inspection work measurement system 630.

[0068] Figures 11 and 12 are functional block diagrams of the main parts of the robot system 530 according to the second embodiment. The differences from the robot system 500 in the first embodiment (see Figures 5 and 6) will be explained below. In Figure 12, the inspection work measurement system 630 is applied in place of the inspection work measurement system 600 in the first embodiment in the second embodiment. The inspection work measurement system 630 is equipped with a sound-making device 606 in place of the light source device 604, an ear position calculator 636 in place of the eye position calculator 616, and a sound-making position calculator 637 in place of the light source position and posture calculator 617.

[0069] The impact sound device 606 has a hammer-like shape and is operated by the worker 800 to strike the inspection target position O2 (see Figure 10). The impact sound device 606 is also equipped with an acceleration sensor, a gyro sensor, a load cell, etc., to measure the position, orientation and impact load of the impact sound device 606 and output the measurement results.

[0070] The ear position calculator 636 calculates the position of the worker 800's ears during the visual inspection based on the measurement results from the gaze measurement sensor 601 and the head position measurement sensor 603. The impact sound position calculator 637 calculates the position and orientation of the impact sound device 606 based on the measurement results from the impact sound device 606.

[0071] The measurement result storage unit 613 receives various data from the inspection work extractor 612 and supplies it to the input terminal 200, etc., via the transceiver 605 and network 400 as time-series data, which is inspection work data D630. In other words, the inspection work data D630 is time-series data of the items listed below. • External shape and inspection route of the object to be inspected • The gaze of 800 workers during the tapping sound inspection work. • Ear position of 800 workers during tapping sound testing. • Position, orientation, and impact load of the sounding device 606

[0072] In Figure 11, robot 130 is used in place of robot 100 in the first embodiment in the second embodiment. Robot 130 is equipped with a microphone-equipped camera device 131 in place of camera device 101, a microphone-equipped camera trajectory generator 134 in place of camera trajectory generator 114, a sound-making device trajectory generator 135 in place of light source trajectory generator 115, a sound-making device controller 139 in place of light source controller 119, and a sound-making device 133 in place of light source device 103.

[0073] First, the microphone-equipped camera trajectory generator 134 within the motion generator 113 generates microphone-equipped camera trajectory data based on the inspection work data D630 and the specifications of the sound collection performance of the microphone-equipped camera device 131. This microphone-equipped camera trajectory data corresponds to the "line of sight of worker 800 during the impact sound inspection work" and the "position of worker 800's ears during the impact sound inspection work" included in the inspection work data D630. In other words, the microphone-equipped camera trajectory data represents a trajectory that can record sound similar to the sound that worker 800 heard while performing the impact sound inspection work.

[0074] The impact sound device trajectory generator 135 generates impact sound device trajectory data based on the inspection work data D630 and the operating conditions of the impact sound device 133. This impact sound device trajectory data represents the movement trajectory of the impact sound device 133 that reproduces the impact sound conditions when the worker 800 was performing the impact sound inspection work. The impact sound device controller 139 drives the impact sound device 133 based on the impact sound device trajectory data to perform an impact sound operation on the target object.

[0075] Furthermore, the avoidance trajectory generator 117 in the second embodiment compares the microphone-equipped camera trajectory data and the impact sound device trajectory data with the area condition data to determine whether the trajectory generated by the microphone-equipped camera trajectory data or the impact sound device trajectory data interferes with an obstacle.

[0076] If the avoidance trajectory generator 117 determines that interference will occur in any of the trajectories, it generates avoidance trajectory data for the microphone-equipped camera and avoidance trajectory data for the sound-emitting device. The method for generating this data is the same as the method used to generate camera avoidance trajectory data and light source avoidance trajectory data in the avoidance trajectory generator 117 of the first embodiment.

[0077] With the above configuration, in the second embodiment, the operation position calculation unit E4 includes a sound-making device trajectory generator 135 and an avoidance trajectory generator 117, and the recording position calculation unit E5 includes a microphone-equipped camera trajectory generator 134 and an avoidance trajectory generator 117.

[0078] <Operation of the second embodiment> Next, the operation of the second embodiment will be described. Figure 13 is a flowchart of the impact sound inspection process in the second embodiment. In Figure 13, when the process proceeds to step S31, the worker 800 performs a tapping inspection. Here, first, worker 800 inputs a work start command via the input unit 202 of the input terminal 200 (see Figure 12). This work start command is input to the inspection work extractor 612 in the inspection work measurement controller 610 via the transceivers 203, 605 and the network 400.

[0079] As a result, the inspection work measurement system 630 acquires inspection work data D630 and outputs this inspection work data D630 via the network 400. The input terminal 200 displays the received inspection work data D630 on the display unit 201 and also saves it in the inspection work data storage unit 204. Next, the robot 130 is installed in an environment similar to the railway vehicle under-vehicle environment 730 (see Figure 10).

[0080] Next, when the process proceeds to step S32, the trajectory data calculation process is executed. Specifically, a calculation start command is input to the robot 130 via the input unit 202 shown in Figure 12, and is input to the motion generator 113 and the sound-making device controller 139 in the robot control device 111 via the transceivers 203 and 107. As a result, the microphone-equipped camera trajectory generator 134 generates microphone-equipped camera trajectory data, and the sound-making device trajectory generator 135 generates sound-making device trajectory data.

[0081] Next, when the process proceeds to step S33, similar to step S3 in the first embodiment, the region condition generation unit 116 in the robot control device 111 generates region condition data representing the operable region of the robot 100 based on the measurement results of the environmental information acquisition unit 110.

[0082] Next, when the process proceeds to step S34, the avoidance trajectory generator 117 compares the microphone-equipped camera trajectory data and the impact sound device trajectory data with the area condition data to determine whether the trajectory generated by the microphone-equipped camera trajectory data or the impact sound device trajectory data interferes with an obstacle.

[0083] If "No" (no interference) is determined in step S34, the process proceeds to step S35, where the motion trajectory generation unit 118 generates motion trajectory data that realizes the microphone-equipped camera trajectory data and the sound-making device trajectory data. As described above, the generated motion trajectory data is time-series data of the movements of each joint of the first arm 102, the second arm 104, and the moving mechanism 105.

[0084] On the other hand, if "Yes" (interference occurs) is determined in step S34, the process proceeds to step S322. Here, the avoidance trajectory generator 117 generates avoidance trajectory data for the microphone-equipped camera and avoidance trajectory data for the sound-making device. Next, when the process proceeds to step S323, the motion trajectory generation unit 118 generates motion trajectory data that realizes the avoidance trajectory data for the microphone-equipped camera and the avoidance trajectory data for the sound-making device.

[0085] Once step S35 or step S323 described above is completed, the process proceeds to step S36. Here, the motion control unit 120 drives the robot 100 according to the generated motion trajectory data. That is, the motion control unit 120 controls the movement of each joint of the first arm 102, the second arm 104, and the moving mechanism 105. As a result, the robot 100 performs actions corresponding to the movements of the worker 800. Specifically, under the control of the sound-making device controller 139, the sound-making device 133 performs a sound-making operation on the object with a sound-making force equivalent to the sound-making force of the worker 800. At that time, for example, the sound-making force can be changed for each object based on the pattern of sound-making force by the worker 800 stored in the inspection work data storage unit 204.

[0086] During the execution of step S36, the operation recording device 121 stores the audio signal recorded by the microphone-equipped camera device 131, and generates and stores test audio data based on the stored audio signal. This test audio data is, for example, an audio pattern obtained by decomposing the stored audio signal into multiple frequency components.

[0087] Next, when the process proceeds to step S37, the input terminal 200 performs playback and display processing of the test audio data. That is, the test audio data is input to the input terminal 200 sequentially via the operation recording device 121 (see Figure 11), transceiver 107, network 400, and transceiver 203 (see Figure 12). As a result, the input terminal 200 displays the spectrum of the test audio data on its display unit 201 and plays the audio signal.

[0088] [Third Embodiment] <Configuration of the third embodiment> Next, a robot system according to the third embodiment will be described. The overall configuration of the robot system according to the third embodiment is the same as that of the first embodiment (see Figure 1). Furthermore, the configuration of each element is the same as that of the robot system according to the first embodiment, except for the points described below.

[0089] Figure 14 is a schematic side view of the robot 140 applied to the third embodiment. The robot 130 of this embodiment is applicable to impact sound inspection work and includes a camera device 101, a first arm 102, a second arm 104, a moving mechanism 105, a main body 106, a transceiver 107, and a torque wrench device 143.

[0090] The torque wrench device 143 is mounted on the tip of the second arm 104 of the serial link mechanism. The torque wrench device 143 is a device that can perform tightening work on the bolt to be inspected with a preset torque of any desired torque. The total degrees of freedom of the first arm 102 and the second arm 104 between the camera device 101 and the torque wrench device 143 is 6 degrees of freedom or more.

[0091] This configuration allows the torque wrench device 143 to perform tightening operations from any direction relative to the position and orientation of the camera device 101. The elements of the robot 130 other than those described above are the same as those of the robot 100 in the first embodiment (see Figure 2). Therefore, in this embodiment, the operating device E1 includes the torque wrench device 143. And, as in the first embodiment, the drive device E2 includes the first arm 102, the second arm 104 and the moving mechanism 105, and the object information acquisition unit E3 includes the camera device 101.

[0092] Figure 15 is a schematic diagram of the railway vehicle under-vehicle environment 740 in the third embodiment. In the railway vehicle under-vehicle environment 740, parallel rails 5a and 5b are laid, and a wheel 4 is placed on them. From the perspective of the worker 800, an object 7 is positioned opposite the wheel 4. Bolts 72, which are fastening members, are provided on various parts of the object 7. The worker 800 performs a tightening torque inspection of the bolts 72 on either side of the wheel 4. In the illustrated state, the worker 800 is inspecting one bolt 72, which is the inspection target position O3, using a torque wrench 607 capable of recording position, orientation, and torque. The actions during this inspection are recorded by the inspection work measurement system 640.

[0093] Figures 16 and 17 are functional block diagrams of the main components of the robot system 540 according to the third embodiment. The differences from the robot system 500 in the first embodiment (see Figures 5 and 6) will be explained below. In Figure 17, the inspection work measurement system 640 is applied in place of the inspection work measurement system 600 in the first embodiment. The inspection work measurement system 640 includes a torque wrench 607 in place of the light source device 604, an inspection target extractor 645 in place of the inspection target extractor 615, and a torque wrench position calculator 647 in place of the light source position and attitude calculator 617.

[0094] Worker 800 performs a tightening torque inspection at the inspection target position O3 (see Figure 15) by operating the torque wrench 607. The torque wrench 607 is equipped with an acceleration sensor, gyro sensor, load cell, etc., and measures the position, orientation, and tightening torque of the torque wrench 607, and outputs the measurement results.

[0095] The inspection target extractor 645 calculates the external shape of the inspection target (for example, object 7 in Figure 15) and the inspection path, similar to the inspection target extractor 615 in the first embodiment (see Figure 6). Furthermore, the inspection target extractor 645 in this embodiment calculates the tightening torque applied by the torque wrench 607.

[0096] The torque wrench position calculator 647 also calculates the position, orientation, and tightening torque of the torque wrench 607 based on the measurement results of the torque wrench 607.

[0097] The measurement result storage unit 613 receives various data from the inspection work extractor 612 and supplies it to the input terminal 200, etc., via the transceiver 605 and network 400 as time-series data, which is inspection work data D640. In other words, the inspection work data D640 is time-series data of the items listed below. • External shape and inspection route of the object to be inspected • The gaze of 800 workers during tightening torque inspection. • Head and eye position of 800 workers during tightening torque inspection. • Position and orientation of torque wrench 607 • Tightening torque of Torque Wrench 607

[0098] In Figure 16, robot 140 is used in place of robot 100 in the first embodiment in the third embodiment. Robot 140 is equipped with a torque wrench device trajectory generator 145 in place of the light source trajectory generator 115, a torque wrench device controller 149 in place of the light source controller 119, and a torque wrench device 143 in place of the light source device 103.

[0099] The camera trajectory generator 114 within the motion generator 113 generates camera trajectory data similar to that of the first embodiment. That is, this camera trajectory data represents a camera trajectory that captures images similar to what the worker 800 was visually observing while performing the tightening torque inspection work.

[0100] Furthermore, the torque wrench device trajectory generator 145 within the motion generator 113 generates torque wrench device trajectory data based on the inspection work data D640 and the operating conditions of the torque wrench device 143. This torque wrench device trajectory data corresponds to the "position and orientation of the torque wrench 607" included in the inspection work data D640. In other words, the torque wrench device trajectory data represents a trajectory close to the trajectory of the torque wrench 607 when the worker 800 was performing the tightening torque inspection work.

[0101] The torque wrench device controller 149 drives the torque wrench device 143 while specifying an upper limit for the tightening torque of the bolt 72. As a result, the torque wrench device controller 149 performs a tightening torque inspection while tightening the bolt 72 within a range below the specified upper limit for tightening torque.

[0102] Furthermore, the avoidance trajectory generator 117 in the third embodiment compares the torque wrench device trajectory data and camera trajectory data with the region condition data generated by the region condition generation unit 116, and determines whether the trajectory of the torque wrench device trajectory data or the camera trajectory data interferes with an obstacle.

[0103] If the avoidance trajectory generator 117 determines that interference will occur in any of the trajectories, it generates torque wrench device avoidance trajectory data and camera avoidance trajectory data. The method for generating this data is the same as the method used to generate camera avoidance trajectory data and light source avoidance trajectory data in the avoidance trajectory generator 117 of the first embodiment.

[0104] With the above configuration, in the third embodiment, the operating position calculation unit E4 includes a torque wrench device trajectory generator 145 and an avoidance trajectory generator 117, and the recording position calculation unit E5 includes a camera trajectory generator 114 and an avoidance trajectory generator 117.

[0105] <Operation of the third embodiment> Next, the operation of the third embodiment will be described. Figure 18 is a flowchart of the tightening torque inspection process in the third embodiment. In Figure 18, when the process proceeds to step S41, the tightening torque inspection work is performed by worker 800. Here, worker 800 first inputs a work start command via the input section 202 of the input terminal 200 (see Figure 17). This work start command is input to the inspection work extractor 612 in the inspection work measurement controller 610 via the transceivers 203, 605 and the network 400.

[0106] As a result, the inspection work measurement system 640 acquires inspection work data D640 and outputs this inspection work data D640 via the network 400. The input terminal 200 displays the received inspection work data D640 on the display unit 201 and also saves it in the inspection work data storage unit 204. Next, the robot 140 is installed in an environment similar to the railway vehicle under-vehicle environment 740 (see Figure 15).

[0107] Next, when the process proceeds to step S42, the trajectory data calculation process is executed. Specifically, a calculation start command is input to the robot 140 via the input unit 202 shown in Figure 17, and is input to the motion generator 113 and the torque wrench device controller 149 in the robot control device 111 via the transceivers 203 and 107. As a result, the camera trajectory generator 114 in the motion generator 113 generates camera trajectory data, and the torque wrench device trajectory generator 145 generates torque wrench device trajectory data.

[0108] Next, when the process proceeds to step S43, similar to step S3 in the first embodiment, the region condition generation unit 116 in the robot control device 111 generates region condition data representing the operable region of the robot 100 based on the measurement results of the environmental information acquisition unit 110.

[0109] Next, when the process proceeds to step S44, the avoidance trajectory generator 117 compares the camera trajectory data and the torque wrench device trajectory data with the area condition data and determines whether the trajectory of the camera trajectory data or the torque wrench device trajectory data interferes with an obstacle.

[0110] If "No" (no interference) is determined in step S44, the process proceeds to step S45, where the motion trajectory generation unit 118 generates motion trajectory data that realizes the camera trajectory data and the torque wrench device trajectory data. As described above, the generated motion trajectory data is time-series data of the movements of each joint of the first arm 102, the second arm 104, and the moving mechanism 105.

[0111] On the other hand, if "Yes" (interference occurs) is determined in step S44, the process proceeds to step S422. Here, the avoidance trajectory generator 117 generates torque wrench device avoidance trajectory data and camera avoidance trajectory data. Next, when the process proceeds to step S423, the motion trajectory generation unit 118 generates motion trajectory data that realizes the torque wrench device avoidance trajectory data and camera avoidance trajectory data.

[0112] Once step S45 or step S423 described above is completed, the process proceeds to step S46. Here, the motion control unit 120 drives the robot 100 according to the generated motion trajectory data. That is, the motion control unit 120 controls the movement of each joint of the first arm 102, the second arm 104, and the moving mechanism 105. As a result, the robot 100 performs actions that correspond to the movements of the worker 800. Specifically, under the control of the torque wrench device controller 149, the torque wrench device 143 performs a tightening torque inspection while tightening the bolt 72 within a range below the upper limit of the specified tightening torque.

[0113] During the execution of step S46, the motion recording device 121 generates and stores inspection image data based on images captured by the camera device 101. This inspection image data is, for example, image data compiled from the moving images captured by the camera device 101.

[0114] Next, when the process proceeds to step S47, the input terminal 200 performs playback and display processing of the inspection image data. That is, the inspection image data is input to the input terminal 200 sequentially via the motion recording device 121 (see Figure 16), transceiver 107, network 400, and transceiver 203 (see Figure 17). As a result, the input terminal 200 displays the inspection image data on its display unit 201.

[0115] [Fourth Embodiment] <Configuration of the fourth embodiment> Next, a robot system according to the fourth embodiment will be described. The overall configuration of the robot system according to the fourth embodiment is the same as that of the first embodiment (see Figure 1). Furthermore, the configuration of each element is the same as that of the robot system according to the first embodiment, except for the points described below.

[0116] Figure 19 is a schematic side view of the robot 150 applied to the fourth embodiment. The robot 150 of this embodiment is applicable to meter inspection work that involves cleaning, and comprises a camera device 101, a first arm 102, a second arm 104, a moving mechanism 105, a main body 106, a transceiver 107, and a cleaning device 153.

[0117] The cleaning device 153 is attached to the tip of the second arm 104 of the serial link mechanism. The cleaning device 153 is a device that can remove dirt adhering to the inspection target meter O4 shown in Figure 20. The total degrees of freedom of the first arm 102 and the second arm 104 between the camera device 101 and the cleaning device 153 is 6 degrees of freedom or more.

[0118] This configuration allows the cleaning device 153 to perform cleaning operations from any direction relative to the position and orientation of the camera device 101. The elements of the robot 150 other than those described above are the same as those of the robot 100 in the first embodiment (see Figure 2). Therefore, in this embodiment, the operating device E1 includes the cleaning device 153. And, as in the first embodiment, the drive device E2 includes the first arm 102, the second arm 104 and the moving mechanism 105, and the object information acquisition unit E3 includes the camera device 101.

[0119] Figure 20 is a schematic diagram of the railway vehicle under-vehicle environment 750 in the fourth embodiment. In the railway vehicle under-vehicle environment 750, parallel rails 5a and 5b are laid, and a wheel 4 is placed on them. From the perspective of the worker 800, the object 8 is positioned opposite the wheel 4. The worker 800 inspects the meter 82 installed on the object 8, which is on either side of the wheel 4. In the illustrated state, the worker 800 uses a cleaning device 608 capable of recording position and orientation to clean dirt attached to one of the meter 82, which is the meter to be inspected O4, and reads the meter to be inspected O4. This operation during the inspection is recorded by the inspection work measurement system 650.

[0120] Figures 21 and 22 are functional block diagrams of the main parts of the robot system 550 according to the fourth embodiment. The differences from the robot system 500 in the first embodiment (see Figures 5 and 6) will be explained below. In Figure 22, the inspection work measurement system 650 is applied in place of the inspection work measurement system 600 in the first embodiment. The inspection work measurement system 650 includes a cleaning device 608 in place of the light source device 604, and a cleaning position calculator 657 in place of the light source position and orientation calculator 617.

[0121] Worker 800 cleans the surface of the meter O4 (see Figure 20) to be inspected by operating the cleaning device 608 and reads the meter O4. The cleaning device 608 is equipped with an acceleration sensor, a gyro sensor, a load cell, etc., and measures the position and orientation of the cleaning device 608 and outputs the measurement results.

[0122] Furthermore, the cleaning position calculator 657 calculates the position and orientation of the cleaning device 608 based on the measurement results of the cleaning device 608.

[0123] The measurement result storage unit 613 supplies various data received from the inspection work extractor 612 as time-series data, known as inspection work data D650, to the input terminal 200, etc., via the transceiver 605 and the network 400. In other words, the inspection work data D650 is time-series data of the items listed below. • External shape and inspection route of the object to be inspected • The gaze of 800 workers during meter inspection. • Head and eye position of 800 workers during meter inspection. • Position and orientation of the cleaning device 608

[0124] In Figure 21, robot 150 is used in place of robot 100 in the first embodiment in the fourth embodiment. Robot 150 is equipped with a cleaning device trajectory generator 155 in place of the light source trajectory generator 115, a cleaning device controller 159 in place of the light source controller 119, and a cleaning device 153 in place of the light source device 103.

[0125] The camera trajectory generator 114 within the motion generator 113 generates camera trajectory data similar to that of the first embodiment. That is, this camera trajectory data represents a camera trajectory that captures images similar to what the worker 800 was visually observing while performing the meter inspection work.

[0126] Furthermore, the cleaning device trajectory generator 155 within the motion generator 113 generates cleaning device trajectory data based on the inspection work data D650 and the operating conditions of the cleaning device 153. This cleaning device trajectory data corresponds to the "position and orientation of the cleaning device 608" included in the inspection work data D650. In other words, the cleaning device trajectory data represents a trajectory that is close to the trajectory of the cleaning device 608 when the worker 800 was performing the meter inspection work.

[0127] In other words, the cleaning device trajectory data is data that reproduces the cleaning work performed by worker 800 during meter inspection work involving cleaning, based on the position, orientation and pressing force of the cleaning device 608 during the meter inspection work involving cleaning performed by worker 800, and the operating conditions of the cleaning device 153. The cleaning device controller 159 drives the cleaning device 153 based on the cleaning device trajectory data to perform a cleaning operation on the target object.

[0128] Furthermore, in the fourth embodiment, the avoidance trajectory generator 117 compares the cleaning device trajectory data and camera trajectory data with the area condition data generated by the area condition generation unit 116, and determines whether the trajectory of the cleaning device trajectory data or the camera trajectory data interferes with an obstacle.

[0129] If the avoidance trajectory generator 117 determines that interference will occur in any of the trajectories, it generates cleaning device avoidance trajectory data and camera avoidance trajectory data. The method for generating this data is the same as the method used to generate camera avoidance trajectory data and light source avoidance trajectory data in the avoidance trajectory generator 117 of the first embodiment.

[0130] With the above configuration, in the fourth embodiment, the operation position calculation unit E4 includes the cleaning device 153 and the avoidance trajectory generator 117, and the recording position calculation unit E5 includes the camera trajectory generator 114 and the avoidance trajectory generator 117.

[0131] <Operation of the fourth embodiment> Next, the operation of the fourth embodiment will be described. Figure 23 is a flowchart of the meter inspection process in the fourth embodiment. In Figure 23, when the process proceeds to step S51, the worker 800 performs cleaning and inspection work. Here, the worker 800 first inputs a work start command via the input section 202 of the input terminal 200 (see Figure 22). This work start command is input to the inspection work extractor 612 in the inspection work measurement controller 610 via the transceivers 203, 605 and the network 400.

[0132] As a result, the inspection work measurement system 650 acquires inspection work data D650 and outputs this inspection work data D650 via the network 400. The input terminal 200 displays the received inspection work data D650 on the display unit 201 and also saves it in the inspection work data storage unit 204. Next, the robot 150 is installed in an environment similar to the railway vehicle under-vehicle environment 750 (see Figure 20).

[0133] Next, when the process proceeds to step S52, the trajectory data calculation process is executed. Specifically, a calculation start command is input to the robot 150 via the input unit 202 shown in Figure 22, and is input to the motion generator 113 and the cleaning device controller 159 in the robot control device 111 via the transceivers 203 and 107. As a result, the camera trajectory generator 114 in the motion generator 113 generates camera trajectory data, and the cleaning device trajectory generator 155 generates cleaning device trajectory data.

[0134] Next, when the process proceeds to step S53, similar to step S3 in the first embodiment, the region condition generation unit 116 in the robot control device 111 generates region condition data representing the operable region of the robot 100 based on the measurement results of the environmental information acquisition unit 110.

[0135] Next, when the process proceeds to step S54, the avoidance trajectory generator 117 compares the camera trajectory data and the cleaning device trajectory data with the area condition data and determines whether the trajectory of the camera trajectory data or the cleaning device trajectory data interferes with an obstacle.

[0136] If "No" (no interference) is determined in step S54, the process proceeds to step S55, where the motion trajectory generation unit 118 generates motion trajectory data that realizes the camera trajectory data and the cleaning device trajectory data. As described above, the generated motion trajectory data is time-series data of the movements of each joint of the first arm 102, the second arm 104, and the moving mechanism 105.

[0137] On the other hand, if "Yes" (interference occurs) is determined in step S54, the process proceeds to step S522. Here, the avoidance trajectory generator 117 generates cleaning device avoidance trajectory data and camera avoidance trajectory data. Next, when the process proceeds to step S523, the motion trajectory generation unit 118 generates motion trajectory data that realizes the cleaning device avoidance trajectory data and camera avoidance trajectory data.

[0138] Once step S55 or step S523 described above is completed, the process proceeds to step S56. Here, the motion control unit 120 drives the robot 100 according to the generated motion trajectory data. That is, the motion control unit 120 controls the movement of the joints of the first arm 102, the second arm 104, and the moving mechanism 105. As a result, the robot 100 performs actions that correspond to the movements of the worker 800.

[0139] In other words, the cleaning device controller 159 reproduces the cleaning work performed by worker 800 during meter inspection work that involves cleaning by controlling the force applied during the cleaning operation by the cleaning device 153. The cleaning device controller 159 may also compare the image captured by the camera device 101 with the inspection image of worker 800 stored in the inspection work data saver 204 when the operation is executed. If the comparison result indicates that the dirt attached to the meter cannot be removed, the cleaning device controller 159 may change the cleaning conditions and perform the cleaning operation by the cleaning device 153 again.

[0140] During the execution of step S56, the motion recording device 121 generates and stores inspection image data based on images captured by the camera device 101. This inspection image data is, for example, image data compiled from the moving images captured by the camera device 101.

[0141] Next, when the process proceeds to step S57, the input terminal 200 performs playback and display processing of the inspection image data. That is, the inspection image data is input to the input terminal 200 sequentially via the motion recording device 121 (see Figure 21), transceiver 107, network 400, and transceiver 203 (see Figure 22). As a result, the input terminal 200 displays the inspection image data on its display unit 201.

[0142] [Computer Configuration] Figure 24 is a block diagram of the Computer 980. The robots 100, 130, 140, and 150 mentioned above, the input terminal 200, the storage and external processing unit 300, and the inspection work measurement controller 610 are all equipped with one or more computers 980 as shown in Figure 24.

[0143] In Figure 24, the computer 980 comprises a CPU 981, a memory unit 982, a communication port 983, an input / output port 984, and a media port 985. Here, the memory unit 982 comprises a RAM 982a, a ROM 982b, and an SSD (Solid State Drive) 982c.

[0144] The communication port 983 is connected to the communication circuit 986. The input / output port 984 is connected to the input / output device 987. The media port 985 reads and writes data to the recording medium 988. The ROM 982b stores the IPL (Initial Program Loader) and other programs executed by the CPU. The SSD 982c stores application programs and various data. The CPU 981 implements various functions by executing application programs and other data read from the SSD 982c into the RAM 982a.

[0145] The internals of the robot control device 111, input terminal 200, and inspection work measurement controller 610, as shown in Figures 5 to 22, primarily represent functions implemented by application programs and the like as building blocks.

[0146] [Differentiation] The present invention is not limited to the embodiments described above, and various modifications are possible. The embodiments described above are illustrative examples provided to facilitate understanding of the present invention, and are not necessarily limited to those comprising all the described configurations. Furthermore, it is possible to replace parts of the configuration of one embodiment with those of another embodiment, and to add configurations from other embodiments to the configuration of one embodiment. It is also possible to delete parts of the configuration of each embodiment, or to add or replace other configurations. In addition, the control lines and information lines shown in the figures are those considered necessary for explanation, and do not necessarily represent all control lines and information lines required in the product. In practice, it can be assumed that almost all configurations are interconnected. Possible modifications to the above embodiments are as follows, for example.

[0147] (1) In the configuration of the first embodiment described above, the camera device 101 was mounted on the first arm 102, and the light source device 103 was mounted on the second arm 104. However, the second arm 104 may be omitted, and multiple light source devices 103 may be mounted on the robot 100. In this case, illumination in any optical axis direction can be reproduced by changing the posture with the drive device E2 and selecting the optimal number of multiple light source devices 103.

[0148] In other words, the light source controller 119 has the function of switching between the state of the drive unit and the multiple light source devices 103 while the position of the camera device 101 is fixed. This makes it possible to select one of multiple optical axis directions for a single position of the camera device 101.

[0149] (2) In each of the above embodiments, the robot control device 111 was located inside the robots 100, 130, 140, and 150. However, the robot control device 111 may be located outside the robot, for example, in the storage and external processing device 300 (see Figure 1). In this case, the robot control device 111 and the storage and external processing device 300 communicate via the network 400.

[0150] (3) The hardware of the robots 100, 130, 140, 150, input terminal 200, storage and external processing unit 300, and inspection work measurement controller 610 in the above embodiment can be implemented using a general-purpose computer. Therefore, programs that execute the processes corresponding to each block diagram and flowchart described above, as well as other various processes described above, may be stored in a storage medium (a computer-readable recording medium on which the program is recorded) or distributed via a transmission line.

[0151] (4) Although the processes corresponding to each block diagram and flowchart described above, and other various processes described above, were explained as software processes using a program in the above embodiment, some or all of them may be replaced with hardware processes using an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array), etc.

[0152] [Effects of the Embodiment] As described above, according to each embodiment, the robots 100, 130, 140, and 150 include an action device E1 that acts on objects 1-3 and 6-8, a drive device E2 that drives the action device E1, an environmental information acquisition unit 110 that acquires environmental information indicating the state of the operating environment of the action device E1 and the drive device E2, an object information acquisition unit E3 that acquires object information relating to objects 1-3 and 6-8, an area condition generation unit 116 that uses the environmental information to generate area condition data representing the operable area of ​​the action device E1 and the drive device E2 in the operating environment, and an area condition data and object information that The system includes: an action position calculation unit E4 that calculates the action position in which the action device E1 acts on objects 1-3, 6-8; a recording position calculation unit E5 that uses area condition data and the calculated action position to calculate a recording position for recording the action result of the action device E1 on objects 1-3, 6-8; an action trajectory generation unit 118 that uses area condition data, the calculated action position, and the calculated recording position to generate an action trajectory of the drive device E2; and an action control unit 120 that uses the action trajectory to control the drive device E2 and stores the result of the action device E1 acting on objects 1-3, 6-8 in an action recording device 121.

[0153] As a result, the motion trajectory generation unit 118 generates the motion trajectory of the drive device E2 using the region condition data, the calculated operating position, and the calculated recording position, thereby enabling appropriate control of the robots 100, 130, 140, and 150.

[0154] Furthermore, as in the robot 100 of the first embodiment, it is even more preferable that the operating device E1 includes a light source device 103 that illuminates the operating positions of the objects 1 to 3, and the object information acquisition unit E3 includes a camera device 101. This allows for appropriate control of the positions of the light source device 103 and the camera device 101 in the robot 100.

[0155] Furthermore, as in the robot 130 of the second embodiment, it is even more preferable that the operating device E1 is equipped with a sound-generating device 133 that generates a sound by colliding with the operating position of the object 6, and that the object information acquisition unit E3 is equipped with a sound-collecting device 131b. This allows for appropriate control of the sound-generating device 133 in the robot 130.

[0156] Furthermore, as in the robot 140 of the third embodiment, it is even more preferable that the object 7 is equipped with a bolt 72, and the operating device E1 is equipped with a torque wrench device 143 capable of tightening the bolt 72 at the operating position with a tightening torque within a preset range. This allows for proper control of the torque wrench device 143 in the robot 140.

[0157] Furthermore, as in the robot 150 of the fourth embodiment, it is even more preferable if the operating device E1 is equipped with a cleaning device 153 that removes dirt adhering to the operating position of the object 8, and the object information acquisition unit E3 is equipped with a camera device 101. This allows for proper control of the cleaning device 153 in the robot 150.

[0158] Furthermore, in the robot 100 of the first embodiment, it is even more preferable to further include a light source controller 119 that adjusts the position or light intensity of the light source device 103 based on the result of comparing the image captured by the camera device 101 with other images recorded in advance when operating the drive device E2. This makes it possible to adjust the position or light intensity of the light source device 103 according to other images recorded in advance.

[0159] Furthermore, in the robot 100 of the first embodiment, it is even more preferable that the drive device E2 has 5 or more degrees of freedom between the camera device 101 and the light source device 103. This allows for more appropriate control of the positional relationship between the camera device 101 and the light source device 103.

[0160] Furthermore, as in the modified robot 100 of the first embodiment, the light source device 103 is equipped with multiple light sources, and by switching between the state of the drive device E2 and the multiple light sources while the position of the camera device 101 is fixed, one of multiple optical axis directions can be selected for a single position of the camera device 101. It is even more preferable to further include a light source controller 119. This makes it possible to select the position in the optical axis direction while omitting position control for the light source device 103.

[0161] Furthermore, in the robot 100 of the first embodiment, the light source device 103 is capable of selectively irradiating light of multiple wavelengths, and it is even more preferable to further include a light source controller 119 that specifies the wavelength of light to be irradiated according to the target objects 1 to 3. This makes it possible to specify the wavelength of light to be irradiated according to the target objects 1 to 3. [Explanation of symbols]

[0162] 1-3, 6-8 Objects 72 volts 100, 130, 140, 150 robots 101 Camera device 103 Light source device 110 Environmental Information Acquisition Department 111 Robot control device 116 Area condition generator 118 Motion trajectory generation section 119 Light source controller 120 Operation Control Unit 121 Operation Recording Device 131b Sound collection device 133 Percussion device 140 robots 143 Torque wrench device 150 robots 153 Cleaning equipment 200 input terminals 500, 530, 540, 550 Robot Systems 980 Computer E1 Acting device E2 Drive Unit E3 Object Information Acquisition Unit E4 Action position calculation section E5 Recording position calculation unit

Claims

1. A device that acts on an object, A drive device for driving the aforementioned operating device, An environmental information acquisition unit acquires environmental information indicating the operating environment status of the operating device and the drive device, A target information acquisition unit that acquires target information relating to the aforementioned target object, A region condition generation unit generates region condition data representing the operable regions of the operating device and the drive device in the operating environment using the aforementioned environmental information, An operating position calculation unit calculates the operating position in which the operating device acts on the object using the area condition data and the object information, A recording position calculation unit calculates a recording position for recording the result of the action of the action device on the object using the area condition data and the action position, An operation trajectory generation unit generates the operation trajectory of the drive device using the aforementioned region condition data, the operation position, and the recording position. The system includes an action control unit that controls the drive unit using the aforementioned action trajectory and stores the result of the action performed by the operating device on the object in an action recording device. A robot characterized by the following features.

2. A robot according to claim 1, The operating device includes a light source device that illuminates the operating position of the object, The object information acquisition unit includes a camera device. A robot characterized by the following features.

3. A robot according to claim 1, The operating device includes a sound-generating device that generates a sound by colliding with the operating position of the object, The aforementioned object information acquisition unit includes a sound collection device. A robot characterized by the following features.

4. A robot according to claim 1, The aforementioned object is equipped with a bolt, The operating device includes a torque wrench device capable of tightening the bolt at the operating position with a tightening torque within a preset range. A robot characterized by the following features.

5. A robot according to claim 1, The operating device includes a cleaning device that removes dirt adhering to the operating position of the object, The object information acquisition unit includes a camera device. A robot characterized by the following features.

6. The robot according to claim 2, The drive device is further provided with a light source controller that adjusts the position or light intensity of the light source device based on the results of comparing the image captured by the camera device with other images recorded in advance when the drive device is operated. A robot characterized by the following features.

7. The robot according to claim 2, The drive device has 5 or more degrees of freedom between the camera device and the light source device. A robot characterized by the following features.

8. The robot according to claim 2, The aforementioned light source device comprises multiple light sources, The system further includes a light source controller that, while the position of the camera device is fixed, switches the state of the drive unit and the multiple light sources to select one of multiple optical axis directions for a given position of the camera device. A robot characterized by the following features.

9. The robot according to claim 2, The aforementioned light source device is capable of selectively emitting light of multiple wavelengths. The system further includes a light source controller that specifies the wavelength of light to be irradiated according to the object to be irradiated. A robot characterized by the following features.

10. A device that acts on an object, A drive device for driving the aforementioned operating device, A robot control device for controlling a robot equipped with the following features: A region condition generation unit generates region condition data representing the operable region of the operating device and the drive device in the operating environment, using environmental information indicating the operating environment state of the operating device and the drive device. An operating position calculation unit calculates the operating position in which the operating device acts on the object using the aforementioned region condition data and object information relating to the object, A recording position calculation unit calculates a recording position for recording the result of the action of the action device on the object, using the aforementioned region condition data and the aforementioned action position. An operation trajectory generation unit generates the operation trajectory of the drive device using the aforementioned region condition data, the operation position, and the recording position. The system includes an action control unit that controls the drive unit using the aforementioned action trajectory and stores the result of the action performed by the operating device on the object in an action recording device. A robot control device characterized by the following features.

11. The robot control device according to claim 10, A robot comprising: an action device that acts on the object; and a drive device that drives the action device. A robot system characterized by having the following features.

12. A device that acts on an object, A drive device that drives the aforementioned operating device and A robot control method for controlling a robot equipped with a computer, A step of generating region condition data representing the operable region of the operating device and the drive device in the operating environment, using environmental information indicating the state of the operating environment of the operating device and the drive device, A step of calculating the operating position in which the operating device acts on the object using the aforementioned region condition data and object information relating to the object, A step of calculating a recording position for recording the result of the action of the action device on the object, using the aforementioned region condition data and the aforementioned action position, A step of generating the operating trajectory of the drive device using the aforementioned region condition data, the operating position, and the recording position, The process includes controlling the drive device using the aforementioned motion trajectory and storing the result of the action of the operating device on the object in the motion recording device. A robot control method characterized by the following:

Images