Tool systems, tool management methods and procedures

Abstract

The problem to be overcome by the present invention is to reduce the possibility of causing a decrease in the reliability or accuracy of processing of a captured image generated by a camera unit. A tool system (1) includes a portable tool (2), a camera unit (5), a posture detection unit (26) and a processing unit (34). The tool (2) includes a drive unit (24) activated by power supplied from a power source. The camera unit (5) is equipped with the tool (2) and generates a captured image. The posture detection unit (26) is equipped with the tool (2) and detects the posture of the tool (2). The processing unit (34) performs processing based on the captured image and the posture.

Description

Technical Field

[0001] The present invention generally relates to tool systems, tool management methods, and programs, and more particularly to tool systems, tool management methods, and programs that include portable tools. Background Art

[0002] Patent Document 1 discloses a tool system comprising a portable tool having a drive unit activated by power supplied from a battery pack, and a camera unit attached to the tool. The camera unit is arranged so that, for example, a socket attached to an output shaft of the tool is within its camera range. The camera unit captures an image of a work subject (e.g., an object or location on which the tool is being used) while the tool is being used to perform work.

[0003] According to Patent Document 1, an image captured by a camera unit is used to identify a work object in which a tool is set in an appropriate position (i.e., a work object arranged so that the tool is ready to start working on the work object). That is, the tool system of Patent Document 1 includes a recognition unit. The recognition unit compares the captured image generated by the camera unit with a plurality of reference images stored in an image storage unit, and recognizes the actual work object captured in the captured image as the work object. In addition, according to Patent Document 1, when it is found that the work object recognized by the recognition unit does not comply with the working procedure in terms of the working process step for the work object, the tool system performs a process of, for example, deactivating the drive unit.

[0004] However, tools are used in various postures (orientations). Therefore, depending on the posture of the tool, processing using the captured image generated by the camera unit equipped with the tool (for example, recognition of the work object in Patent Document 1) may be performed with reduced reliability or accuracy.

[0005] Prior art literature

[0006] Patent Literature

[0007] Patent Document 1: Japanese Patent Application Laid-Open No. 2018-108633 Summary of the Invention

[0008] In view of the above background, an object of the present invention is to provide a tool system, a tool management method, and a program, all of which are configured or designed to reduce the possibility of causing a decrease in reliability or accuracy in processing of captured images generated using an imaging unit.

[0009] A tool system according to one aspect of the present invention includes a portable tool, a camera unit, a gesture detection unit, and a processing unit. The tool includes a drive unit activated by power supplied from a power source. The camera unit is provided with the tool and generates a captured image. The gesture detection unit is provided with the tool and detects the tool's gesture. The processing unit performs processing based on the captured image and the gesture.

[0010] A tool management method according to another aspect of the present invention includes a first step, a second step, and a third step. The first step includes acquiring a captured image from a camera unit provided with a portable tool, the tool including a drive unit activated by power supplied from a power source. The second step includes acquiring posture information related to the posture of the tool from a posture detection unit provided with the tool. The third step includes performing processing based on the captured image acquired in the first step and the posture information acquired in the second step.

[0011] A program according to yet another aspect of the present invention is designed to cause one or more processors to perform the above-mentioned tool management method. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is a block diagram of a tool system according to a first embodiment;

[0013] Figure 2 A is a perspective view illustrating the appearance of the tools included in the tool system as seen from one angle. Figure 2 B is a perspective view illustrating the appearance of the tool included in the tool system as seen from another angle;

[0014] Figure 3 Shows the sequence of operations to be performed by the tool system in the registration mode;

[0015] Figure 4 Shows the sequence of operations to be performed by the tool system in working mode;

[0016] Figure 5 is a schematic diagram illustrating various postures of the tools included in the tool system;

[0017] Figure 6 A is a schematic diagram illustrating how the tool system captures images of a work object when the tool has a first posture and when the tool has a third posture, respectively. Figure 6 B is exemplified by Figure 6 A is a schematic diagram of an image captured by the tool system;

[0018] Figure 7 conceptually illustrating how the image correction unit of the tool system corrects the captured image; and

[0019] Figure 8 is a block diagram of a tool system according to a second embodiment. DETAILED DESCRIPTION

[0020] (First embodiment)

[0021] (1) Overview

[0022] First, refer to Figure 1 An overview of the tool system 1 according to the exemplary embodiment will be described.

[0023] The tool system 1 according to the present embodiment includes a portable tool 2. The tool 2 includes, for example, a drive unit 24 including a motor. The drive unit 24 is activated using motive power (such as electricity, etc.) supplied from a power source such as a battery pack 201. Examples of this type of tool 2 include impact wrenches, nut drivers, oil pulse wrenches, screwdrivers (including impact screwdrivers), drills, drill screwdrivers, and various other types of tools. Using this type of tool 2 enables the user to perform various machining operations, such as attaching a fastening member (such as a bolt or nut, etc.) to a workpiece (machining operation object) as an operation object, or opening a hole through a workpiece, etc.

[0024] In addition, the tool system 1 according to the present embodiment further includes a camera unit 5. The camera unit 5 is provided to the tool 2. The camera unit 5 generates a captured image. The camera unit 5, for example, captures the output shaft 241 (see FIG. 2 ) attached to the tool 2. Figure 2 A) socket 242 (see Figure 2 Therefore, when the user is using the tool 2 to perform work, the imaging unit 5 captures an image of the work object and generates a captured image.

[0025] Thus, the tool system 1 according to this embodiment enables identification of the work object based on, for example, the captured image generated by the imaging unit 5, thereby enabling, for example, determination of whether the work being performed by the user using the tool 2 complies with the work procedure. Furthermore, the tool system 1 also enables determination of whether the work performed on the work object is good or poor based on the captured image generated by the imaging unit 5, thereby notifying the user of work instructions based on the work object and storing the image as a log (i.e., work record). As can be seen, the use of the image (captured image) generated by the imaging unit 5 provided to the tool 2 enables, for example, support for the user in performing their work using the tool 2 or in managing their work.

[0026] like Figure 1As shown, the tool system 1 according to this embodiment includes not only the tool 2 and the imaging unit 5, but also a posture detection unit 26 and a processing unit 34. That is, the tool system 1 includes the portable tool 2, the imaging unit 5, the posture detection unit 26, and the processing unit 34. The tool 2 includes a drive unit 24 that is activated by power supplied from a power source. The imaging unit 5 is provided to the tool 2 and generates a captured image. The posture detection unit 26 is provided to the tool 2 and detects the posture of the tool 2. The processing unit 34 performs processing based on the captured image and the posture.

[0027] According to this configuration, processing unit 34 performs processing based not only on the captured image generated by imaging unit 5 provided to tool 2, but also on the posture of tool 2 detected by posture detection unit 26. This enables tool system 1 to improve the reliability and accuracy of the processing performed by processing unit 34, compared to a case where processing unit 34 performs the same processing based solely on the captured image. For example, if the captured image has symmetry, it may be difficult to identify the work object based solely on the captured image. In contrast, if the posture of tool 2 is known, the work object can be more easily identified based on the captured image, even if the captured image has symmetry. This achieves the advantage of reducing the possibility of causing a decrease in the reliability or accuracy of processing using the captured image generated by imaging unit 5.

[0028] (2) Detailed structure

[0029] Next, refer to Figures 1 to 2 B will now explain the detailed structure of the tool system 1 according to this embodiment.

[0030] (2.1) Prerequisites

[0031] The tool system 1 according to this embodiment can be used, for example, in an assembly line for assembling products at a factory. Specifically, in this embodiment, it is assumed that the tool 2 included in the tool system 1 is a fastening tool, such as an impact wrench, for tightening fastening members (such as bolts or nuts). Specifically, it is assumed that this embodiment is applied to a situation where a single product has multiple locations to be fastened, requiring the user to attach a fastening member to each of these locations using the tool 2 at a single work site.

[0032] As used herein, a "location to be fastened" refers to the portion of a workpiece (the subject of machining) to be attached to a fastening member. For example, if the fastening member is a bolt, the location to be fastened is the area surrounding and covering the screw hole to be attached to the fastening member. That is, in this embodiment, a single workpiece has multiple such locations to be fastened.

[0033] As used herein, "working object" refers to an object (such as a workpiece, etc.) or working area on which the work is assumed to be performed by using tool 2. In particular, the working object on which tool 2 is set in an appropriate position is sometimes referred to as the "current working object" below. As used herein, the phrase "tool 2 is set in an appropriate position" refers to a situation in which tool 2 is placed so as to be ready to perform work on the working object. In addition, as used herein, the phrase "placed so as to be ready to perform work" refers not only to a situation in which tool 2 has come into contact with the working object, but also to a situation in which tool 2 is about to come into contact with the working object. That is to say, in a state in which tool 2 is set in an appropriate position on the working object, tool 2 may have come into contact with the working object or may not have yet come into contact with the working object. In this embodiment, as an example, it is assumed that each of the multiple parts to be fastened of a single workpiece is a working object.

[0034] As used herein, a "captured image" refers to an image captured by the camera unit 5, and includes still photos (still images) and motion photos (animation). A "motion photo" also includes, for example, a group of still photos captured by stop-motion animation. The captured image is not necessarily the output data itself provided by the camera unit 5. For example, the captured image may have undergone data compression, conversion to another data format, cropping of an image portion from the image captured by the camera unit 5, focus adjustment, brightness adjustment, contrast adjustment, or any of various other types of image processing as needed. In the present embodiment, it is assumed that the captured image is a full-color motion photo, for example.

[0035] Furthermore, as used herein, if something is “equipped with” something else, the former may be built into the latter (e.g., inseparably integrated with the latter) or may be merely attached to the latter as an external component (e.g., removably fixed with, for example, a coupling). That is, the camera unit 5 equipped with the tool 2 may be built into the tool 2 or merely attached to the tool 2 as an external component, whichever is appropriate.

[0036] In addition, as used herein, a "work process" means the process of a work to be performed using the tool 2. For example, if a series of work processes to be performed on a single work object or multiple work objects are defined as a single work process, the work process represents the order in which the work processes should be performed on the single work object or multiple work objects in the entire work process. More specifically, if an instruction related to the work to be performed on a single work object is a "work instruction", the work process is information representing a single work instruction or multiple work instructions for a single work process and the order in which the work processes should be performed. In other words, the work process indicates which of the single or multiple work processes the work object corresponds to, and also indicates the sequential position of the corresponding work processes. In the following description of this embodiment, it is assumed that the work process defines the order in which the work (including multiple work processes) should be performed on multiple work objects in a single workpiece.

[0037] (2.2) Tool structure

[0038] First, refer to Figure 2 A and Figure 2 B is used to illustrate the structure of the tool 2 in the tool system 1 according to this embodiment.

[0039] Specifically, the tool system 1 according to the present embodiment includes the portable tool 2 as described above. In the present embodiment, the tool 2 is configured to activate the drive unit 24 (see FIG. 2 ) by using electrical energy. Figure 1 ) of an electric tool. In particular, in this embodiment, it is assumed that the tool 2 is an impact wrench. Such a tool 2 can be used to perform a fastening operation of attaching a fastening member to a work object. The tool 2 includes an impact mechanism 25 (see FIG. 2 ) in addition to a drive unit 24. Figure 1 ).

[0040] In this case, the tool 2 is designed to use the battery pack 201 as a power source, and to activate the drive unit 24 using the power (electrical energy) supplied from the battery pack 201. In this embodiment, it is assumed that the battery pack 201 is included in the components of the tool 2. However, the battery pack 201 does not necessarily have to be one of the components of the tool 2. In other words, the battery pack 201 does not have to be included in the components of the tool 2.

[0041] The tool 2 further includes a body 20. The body 20 houses a drive unit 24 and an impact mechanism 25. In addition, the tool system 1 includes an imaging unit 5, a posture detection unit 26, a control unit 3, a storage unit 4, and a notification unit 211 (see FIG. 1 ). Figure 1) is also housed in the main body 20. That is, in this embodiment, the imaging unit 5, the posture detection unit 26, the control unit 3, the storage unit 4, and the notification unit 211, which are constituent elements of the tool system 1, are housed in the main body 20 of the tool 2 and are thereby integrated with the tool 2.

[0042] The main body 20 of the tool 2 includes a barrel 21, a grip 22, and an attachment member 23. The barrel 21 is formed in a cylindrical shape (for example, a cylindrical shape in this embodiment). The grip 22 protrudes along a normal line relative to a portion of the circumference of the barrel 21 (that is, along the radius of the barrel 21). The battery pack 201 is removably attached to the attachment member 23. In this embodiment, the attachment member 23 is provided at the front end of the grip 22. In other words, the barrel 21 and the attachment member 23 are connected together via the grip 22.

[0043] At least the drive unit 24 is housed in the barrel 21. The drive unit 24 includes a motor. The drive unit 24 is configured to be activated using power supplied to the motor from the battery pack 201 as a power source. The output shaft 241 protrudes from one axial end surface of the barrel 21. When the drive unit 24 is activated, the output shaft 241 rotates around a rotation axis Ax1 aligned with the direction in which the output shaft 241 protrudes. In other words, the drive unit 24 drives the output shaft 241 to rotate around the rotation axis Ax1. In other words, when the drive unit 24 is activated, torque is applied to the output shaft 241, thereby causing the output shaft 241 to rotate.

[0044] A cylindrical socket 242 for rotating a fastening member (such as a bolt or nut) is removably attached to the output shaft 241. The socket 242 rotates around the output shaft 241 together with the output shaft 241. The user can appropriately select the size of the socket 242 attached to the output shaft 241 according to the size of the fastening member. According to this configuration, activating the drive unit 24 causes the output shaft 241 to rotate, thereby causing the socket 242 to rotate together with the output shaft 241. If the fastening member is assembled into the socket 242 at this time, the fastening member rotates together with the socket 242, thereby completing the operation of tightening or loosening the fastening member. In this way, the tool 2 can complete the operation of tightening or loosening the fastening member by activating the drive unit 24.

[0045] Alternatively, a socket anvil may be attached to the output shaft 241 instead of the socket 242. The socket anvil may also be removably attached to the output shaft 241. This enables a drill bit, such as a screwdriver bit or a drill bit, to be attached to the output shaft 241 via the socket anvil.

[0046] The tool 2 includes the impact mechanism 25 as described above. The impact mechanism 25 is configured to apply an impact force in the rotational direction of the output shaft 241 when the tightening torque (the working value) exceeds a predetermined level. This enables the tool 2 to apply a greater tightening torque to the tightening member.

[0047] The grip 22 is the part that the user holds while performing work. The grip 22 is equipped with a trigger switch 221 and a forward / reverse switch 222. The trigger switch 221 is a switch for controlling the on / off state of the operation performed by the drive unit 24, and enables the number of rotations of the output shaft 241 to be adjusted according to the depth to which the trigger switch 221 is pulled. The forward / reverse switch 222 is a switch for switching the rotation direction of the output shaft 241 from clockwise to counterclockwise and vice versa.

[0048] The attachment member 23 is formed in a flat rectangular parallelepiped shape. The battery pack 201 is removably attached to the side of the attachment member 23 opposite to the grip portion 22.

[0049] The battery pack 201 includes a housing 202 made of resin and formed in a rectangular parallelepiped shape. The housing 202 contains a rechargeable battery (such as a lithium-ion battery) inside. The battery pack 201 supplies power to the drive unit 24, the control unit 3, the camera unit 5, and other components.

[0050] The attachment member 23 is also equipped with a control panel 231. The control panel 231 may include, for example, a plurality of push-button switches 232 and a plurality of LEDs (light-emitting diodes) 233. The control panel 231 enables the user to input various settings for the tool 2 and to confirm the status of the tool 2. That is, the user is allowed to change the operating mode of the tool 2 or check the remaining capacity of the battery pack 201, for example, by operating the push-button switches 232 of the control panel 231.

[0051] The attachment member 23 also includes a light emitting unit 234. The light emitting unit 234 includes, for example, an LED. When the user is using the tool 2 to perform work, the light emitting unit 234 emits light toward the work object. The light emitting unit 234 can be turned on and off by operating the control panel 231. Alternatively, the light emitting unit 234 can be automatically illuminated when the trigger switch 221 is turned on.

[0052] (2.3) Overall structure of the tool system

[0053] Next, refer to Figure 1 The overall structure of the tool system 1 according to this embodiment will be described.

[0054] As described above, the tool system 1 includes the portable tool 2, the imaging unit 5, the posture detection unit 26, and the control unit 3 (including the processing unit 34). Furthermore, in this embodiment, the tool system 1 includes, in addition to the tool 2 (including the battery pack 201), the imaging unit 5, the posture detection unit 26, and the control unit 3, a storage unit 4 and a notification unit 211. Note that the storage unit 4 and the notification unit 211 are not essential components of the tool system 1, and at least one of the storage unit 4 and the notification unit 211 may be omitted as appropriate.

[0055] The imaging unit 5, the posture detection unit 26, the control unit 3, the storage unit 4, and the notification unit 211 are housed in the body 20 of the tool 2 (see FIG. Figure 2 In this embodiment, for example, the camera unit 5 and the notification unit 211 may be housed in the barrel 21. The posture detection unit 26, the control unit 3, and the storage unit 4 may be housed in the grip portion 22 or the attachment member 23.

[0056] The imaging unit 5 is provided with the tool 2 and generates data as captured images. For example, the imaging unit 5 can be implemented as a camera including an image sensor and a lens. In this embodiment, the imaging unit 5 can be housed in (the barrel 21 of) the body 20 of the tool 2 and thus integrated with the tool 2. The imaging unit 5 is positioned toward the front end of the output shaft 241 to capture images of the workpiece while the user is using the tool 2 to perform work.

[0057] Specifically, the imaging unit 5 is provided in the front end portion of the barrel 21 so as to face the front end of the output shaft 241 (ie, the socket 242) so that the socket 242 attached to the output shaft 241 falls within the imaging range (see FIG. Figure 2 A and Figure 2 B). The optical axis of the imaging unit 5 is arranged to align with the rotation axis Ax1 of the output shaft 241. In this embodiment, the imaging unit 5 is arranged so that its optical axis is within a predetermined range from the rotation axis Ax1 of the output shaft 241, and the rotation axis Ax1 and the optical axis are substantially parallel to each other. Furthermore, the imaging unit 5 is configured to continuously capture images when the control unit 3 is activated, generate a time-series of the captured images in the form of a moving picture, and output the moving picture to the processing unit 34 of the control unit 3.

[0058] The posture detection unit 26 is equipped with the tool 2 and detects the posture of the tool 2. The posture detection unit 26 may include, for example, sensors such as an acceleration sensor and a gyro sensor. In the present embodiment, the posture detection unit 26 is equipped with the tool 2 so as to be accommodated in the body 20 (the gripping portion 22 or the attachment member 23) of the tool 2 and thereby integrated with the tool 2. In the present embodiment, the posture detection unit 26 includes, for example, a three-axis acceleration sensor and a three-axis gyro sensor. The three-axis acceleration sensor detects the acceleration on each of the three axes perpendicular to each other and outputs an electrical signal representing the acceleration thus detected. The three-axis gyro sensor detects the angular velocity around each of the three axes perpendicular to each other and outputs an electrical signal representing the angular velocity thus detected.

[0059] The posture detection unit 26 can detect the direction of gravity based on the output of, for example, an acceleration sensor, and detect, for example, the posture of the tool 2 by referring to the direction of gravity. In addition, for example, the posture detection unit 26 can also detect the angular velocity of the tool 2 moving while rotating based on the output of a gyro sensor, and further detect the rotation angle of the tool 2 based on the integration result of the angular velocity. For example, the posture detection unit 26 can distinguish between the posture of the tool 2 in which the grip 22 protrudes from the barrel 21 in a downward direction (corresponding to the direction of gravity) and the posture of the tool 2 in which the grip 22 protrudes from the barrel 21 in an upward direction. The posture detection unit 26 detects the posture of the tool 2 based on the output of sensors (including the acceleration sensor and the gyro sensor), and provides the detection result as posture information related to the posture of the tool 2 to the image correction unit 33 of the control unit 3 as needed.

[0060] The notification unit 211 may be implemented as an LED, for example. The notification unit 211 may be provided to the other end of the cylinder 21 of the body 20 opposite to the output shaft 241 so as to be easily viewed by the user during operation (see FIG. Figure 2 B).

[0061] The control unit 3 may, for example, include a microcontroller comprising one or more processors and one or more memories as its primary components. The microcontroller performs the functions of the control unit 3 by causing the one or more processors to execute programs stored in the one or more memories. The programs may be pre-stored in the memories. Alternatively, the programs may be distributed by storing them on a non-transitory storage medium such as a memory card or downloaded via a remote communication link. In other words, the programs are designed to cause the one or more processors to function as the control unit 3.

[0062] The control unit 3 performs functions such as the driving control unit 31, the imaging control unit 32, the image correction unit 33, the processing unit 34, the notification control unit 35, the determination unit 36, and the registration unit 37. Note that if no signal is given to the trigger switch 221 or the control panel 231 (see Figure 2 If an operation command is input into A), the control unit 3 enters the sleep mode. When any operation command is input into the trigger switch 221 or the control panel 231 during the sleep mode, the control unit 3 is activated.

[0063] The drive control unit 31 controls the drive unit 24. Specifically, the drive control unit 31 activates the drive unit 24 so that the output shaft 241 rotates at a speed determined by the pressing depth of the trigger switch 221 and at a speed determined by the forward / reverse switch 222 (see Figure 2 The device rotates in the direction of rotation set by A).

[0064] The drive control unit 31 also controls the drive unit 24 so that the tightening torque becomes equal to the torque setting (operation setting). The drive control unit 31 has a torque estimation function for estimating the magnitude of the tightening torque. In the present embodiment, the drive control unit 31 estimates the magnitude of the tightening torque based on the number of revolutions of the drive unit 24 (motor) or any other parameter until the estimated value of the tightening torque reaches the seating judgment level. When the estimated value of the tightening torque reaches the seating judgment level, the drive control unit 31 estimates the magnitude of the tightening torque based on the number of strokes of the impact mechanism 25. When it is found that the number of strokes of the impact mechanism 25 has reached a threshold number of times based on the torque setting, the drive control unit 31 determines that the tightening torque should have reached the torque setting and stops running the drive unit 24 (i.e., the motor). This enables the tool 2 to tighten the fastening member with a tightening torque that accurately matches the torque setting. "Torque setting" will be described in detail later.

[0065] The imaging control unit 32 controls the imaging unit 5. The imaging control unit 32 controls the imaging unit 5 so that the imaging unit 5 captures an image of the work object at least during the work using the tool 2. In this way, the imaging control unit 32 causes the imaging unit 5 to capture an image of the work object at least during the work using the tool 2.

[0066] The image correction unit 33 corrects the captured image based on the posture of the tool 2. The posture of the tool 2 is detected by the posture detection unit 26, and posture information related to the posture of the tool 2 is provided to the image correction unit 33 as needed. In other words, the image correction unit 33 has the ability to correct the captured image generated by the imaging unit 5 equipped with the tool 2 based on the posture of the tool 2 when the captured image was generated. In this case, the image correction unit 33 performs various corrections on the captured image through image processing.

[0067] In the present embodiment, the image correction unit 33 performs at least rotation compensation on the captured image according to the posture. That is, the image correction unit 33 can correct the captured image (rotation compensation) by rotating the captured image to an arbitrary degree. For example, the image correction unit 33 can obtain a vertically inverted captured image by rotating the captured image 180 degrees. In addition, in the present embodiment, the image correction unit 33 also performs at least distortion correction on the captured image according to the posture. That is, the image correction unit 33 can correct the captured image (distortion correction) by partially enlarging or reducing the captured image to an arbitrary degree. For example, the image correction unit 33 can obtain a captured image in which a rectangular subject is captured in a rectangular shape by performing distortion correction on the captured image in which the rectangular subject is captured in a trapezoidal shape. The image correction unit 33 selectively applies an appropriate correction from these various corrections according to the posture detected by the posture detection unit 26. The image correction unit 33 can appropriately combine rotation compensation and distortion correction.

[0068] The processing unit 34 performs processing based on the captured image and the posture. The processing unit 34 basically performs predetermined processing based on the captured image. Furthermore, in this embodiment, the processing unit 34 also performs predetermined processing while taking into account the posture of the tool 2 detected by the posture detection unit 26. This allows both the captured image and the posture of the tool 2 to be reflected in the processing performed by the processing unit 34.

[0069] In this embodiment, the processing unit 34 performs processing by using the captured image corrected by the image correction unit 33. That is, the processing unit 34 performs predetermined processing based on the captured image generated by the imaging unit 5 and corrected according to the posture of the tool 2. Performing processing using the captured image corrected according to the posture of the tool 2 in this way ultimately enables the processing unit 34 to perform processing based on both the captured image and the posture.

[0070] In the present embodiment, the processing unit 34 performs processing for identifying a work object captured on-site in a captured image (hereinafter referred to as a "real-shot work object") among a plurality of work objects as a current work object. That is, the processing unit 34 has a function for identifying the current work object (i.e., a work object in which the tool 2 is set in an appropriate position). Specifically, the processing unit 34 uses a plurality of reference images corresponding to a plurality of work objects as template data to perform pattern matching processing on the captured image, thereby identifying the real-shot work object. That is, the processing unit 34 identifies the work object captured on-site in the captured image by comparing the captured image with a plurality of reference images corresponding to the same plurality of work objects. The processing unit 34 identifies the work object thus identified as the current work object.

[0071] The processing unit 34 identifies the work object by, for example, performing image processing (pattern matching) on ​​the data (i.e., captured images) output from the imaging unit 5 in a moving picture format, frame by frame. Therefore, if the work object falls within the imaging range of the imaging unit 5, the processing unit 34 can determine which of the multiple work objects the imaging unit 5 is currently capturing (i.e., which of the multiple work objects the actual captured work object is). Multiple reference images are stored in the storage unit 4 (image storage unit 41).

[0072] Furthermore, if the work object thus identified does not conform to the work instructions defined by the work procedure, the processing unit 34 at least one of restricts the operation of the drive unit 24 and issues a notification. In other words, the processing unit 34 determines whether the work object identified by the processing unit 34 (i.e., the actual work object) conforms to the work instructions defined by the preset work procedure. In other words, the processing unit 34 determines whether the work object identified by the processing unit 34 is consistent with the work object specified by the work instructions included in the work procedure.

[0073] Specifically, the processing unit 34 extracts data on the work process associated with the live-shot work object from the process storage unit 44 of the storage unit 4. The processing unit 34 then determines whether the work object undergoing the current work instruction defined by the work process extracted from the process storage unit 44 is consistent with the work object identified as the live-shot work object. If these work objects are consistent with each other, the processing unit 34 determines that the identified work object should comply with the work instruction defined by the work process. On the other hand, if these work objects are inconsistent with each other, the processing unit 34 determines that the identified work object should not comply with the work instruction defined by the work process.

[0074] If it is determined as a result of this judgment that the work object does not conform to the work instructions defined by the work process, the processing unit 34 at least one of imposes restrictions on the operation of the drive unit 24 and issues a notification. As used herein, "notification" includes notifying not only the user but also an external terminal (such as a mobile communication device, etc.).

[0075] Specifically, if it is determined that the work object does not conform to the work instructions defined by the work procedure, the processing unit 34 will not allow the drive unit 24 to be activated even if the trigger switch 221 is pulled. In other words, the drive unit 24 is only allowed to be activated if the processing unit 34 determines that the work object should conform to the work instructions defined by the work procedure. Therefore, even if the tool 2 is set in the appropriate position on the work object that does not conform to the work procedure, the drive unit 24 remains deactivated, thereby prohibiting the fastening operation. This can reduce the possibility of performing operations in the wrong order. Optionally, if it is determined that the work object does not conform to the work instructions defined by the work procedure, the processing unit 34 can lock the trigger switch 221 to prevent the user from pulling the trigger switch 221 in this situation.

[0076] Furthermore, if it is determined that the work object does not conform to the work instructions defined by the work procedure, the processing unit 34 causes the notification control unit 35 to activate the notification unit 211. Thus, the notification unit 211 functions as a user notification unit for notifying the user that the tool 2 is now set in the appropriate position on the work object that does not conform to the work procedure.

[0077] That is, the processing unit 34 at least performs object recognition processing for identifying the work object as a predetermined processing based on the captured image and the posture. In other words, the processing unit 34 at least performs recognition of the work object as a (predetermined) processing. In addition, the processing unit 34 also performs the following process judgment processing as a predetermined processing based on the captured image and the posture, and the process judgment processing is used to compare the work object thus identified with the work instructions defined by the work process and judge their correspondence. In other words, the processing unit 34 performs judgment of the correspondence between the work object thus identified and the work instructions defined by the work process as a (predetermined) processing. In addition, if the result of the process judgment processing indicates that the work object does not conform to the work instructions, the processing unit 34 restricts the operation of the drive unit 24 and / or notifies.

[0078] The notification control unit 35 controls the notification unit 211 equipped with the tool 2. The notification control unit 35 preferably causes the notification unit 211 to light up in different ways in a case where the determination made by the processing unit 34 is inconsistent (i.e., in a case where it is determined that the work object should not comply with the work instructions defined by the work process) and in a case where the processing unit 34 recognizes a real-shot work object. For example, if the determination made by the processing unit 34 is inconsistent, the notification control unit 35 may cause the notification unit 211 to light up in red. On the other hand, if the processing unit 34 recognizes a real-shot work object, the notification control unit 35 causes the notification unit 211 to light up in green. This enables the user to identify whether the work object complies with the work process by checking the lighting state of the notification unit 211 with his eyes. Alternatively, in a case where the trigger switch 221 is pulled in a state where the determination made by the processing unit 34 is inconsistent, the notification control unit 35 may cause the notification unit 211 to light up.

[0079] The determination unit 36 ​​is configured to determine whether the tightening torque is normal when the fastening member is attached to the part to be fastened. In this case, the determination unit 36 ​​preferably determines whether the tightening torque is normal based on the work instructions defined by the work procedure. Specifically, the work instructions defined by the work procedure include a target torque value associated with the work object. This allows the determination unit 36 ​​to determine whether the work is being performed with the tightening torque specified by the work instructions by comparing the target torque value included in the work instructions with the tightening torque.

[0080] If, for example, the drive control unit 31 deactivates the drive unit 24 upon detecting that the number of strokes of the impact mechanism 25 has reached a threshold number, the determination unit 36 ​​determines that the tightening torque is normal. On the other hand, if, for example, the drive control unit 31 deactivates the drive unit 24 by turning off the trigger switch 221 before the number of strokes of the impact mechanism 25 has reached the threshold number, the determination unit 36 ​​determines that the tightening torque is insufficient (abnormal). The determination unit 36 ​​also performs result storage processing to store the determination result in the result storage unit 43, in association with the part to be tightened.

[0081] In this case, the tool 2 according to the present embodiment has at least a work mode and a registration mode as its operation modes. As used herein, the "work mode" refers to an operation mode in which a user uses the tool 2 to perform a certain type of operation. The registration mode herein refers to an operation mode in which a plurality of reference images are stored in the image storage unit 41 and a plurality of target torque values ​​are stored in the torque storage unit 42. The operation mode can be switched by, for example, operating the control panel 231. Alternatively, the operation mode can also be switched by operating another component (such as a dip switch) provided separately from the control panel 231.

[0082] When the operation mode of the tool 2 is the registration mode, the registration unit 37 performs image registration processing to store a plurality of reference images in the image storage unit 41 of the storage unit 4 and torque registration processing to store a plurality of target torque values ​​in the torque storage unit 42 of the storage unit 4 .

[0083] The registration unit 37 causes the torque storage unit 42 to store the torque setting when the fastening member is attached to the work object as the target torque value during the torque registration process. Specifically, when the operation mode of the tool 2 is the registration mode, the user can input any arbitrary torque value by operating the control panel 231, and the drive control unit 31 sets the input torque value as the torque setting. When the fastening member is attached to the work object, the registration unit 37 causes the torque storage unit 42 to store the torque setting at that time as the target torque value.

[0084] Furthermore, during the image registration process, the registration unit 37 causes the image storage unit 41 to store a still image, generated by, for example, causing the imaging unit 5 to capture an image of the work object, as a reference image. Specifically, if the tool 2 is in the registration mode, the trigger switch 221 also functions as a shutter release button. That is, when the trigger switch 221 is turned on, the imaging unit 5 generates a still image. The registration unit 37 causes the image storage unit 41 to store the still image as the reference image.

[0085] That is, according to this embodiment, if the operation mode of the tool 2 is in the registration mode, the trigger switch 221 functions both as a switch for activating the drive unit 24 and as a shutter release button for generating a reference image. Therefore, performing a fastening operation while the operation mode of the tool 2 is in the registration mode enables the registration unit 37 to perform torque registration and image registration in parallel. Specifically, the registration unit 37 causes the torque storage unit 42 to store the torque setting during the fastening operation as the target torque value, and also causes the image storage unit 41 to store a still image of the workpiece during the fastening operation as the reference image.

[0086] The storage unit 4 can be implemented as a semiconductor memory, for example, and performs the functions of the image storage unit 41, the torque storage unit 42 (target value storage unit), the result storage unit 43, and the process storage unit 44. In the present embodiment, the image storage unit 41, the torque storage unit 42, the result storage unit 43, and the process storage unit 44 are implemented as a single memory. However, this is only an example and should not be interpreted as limiting. Alternatively, these storage units 41, 42, 43, and 44 can also be implemented as four different memories. Alternatively, the storage unit 4 can also be implemented as a storage medium such as a memory card that can be attached to the tool 2 and removed from the tool 2.

[0087] The image storage unit 41 stores multiple reference images in association with multiple work objects. Each reference image is a still image obtained by photographing the associated work object. Alternatively, a single work object can be associated with multiple reference images obtained by photographing the work object from various angles or at multiple different sizes.

[0088] The torque storage unit 42 stores a plurality of target torque values ​​(target values) in a one-to-one association with a plurality of work objects. As used herein, a "target torque value" refers to a target value of a fastening torque when a fastening member is attached to an associated work object.

[0089] The result storage unit 43 stores the determination results obtained by the determination unit 36 ​​for a plurality of fastening locations in association with a plurality of work targets. It is recommended that the result storage unit 43 store the determination results obtained by the determination unit 36 ​​with a timestamp indicating the time of the work. This allows the determination results for the work targets to be differentiated for each product on the assembly line.

[0090] The process storage unit 44 stores data related to a single work process or multiple work processes. As described above, the work process means a process to be performed using the tool 2 and may be data defining in what order multiple work objects of a single workpiece should be worked.

[0091] (3) Operation

[0092] Next, refer to Figures 3 to 7 An exemplary operation of the tool system 1 according to the present embodiment will be described.

[0093] In the following example, how the tool system 1 operates when a user assembles two products of the same type (hereinafter referred to as the "first product" and the "second product") on an assembly line will be described. It is assumed that each product has four work objects (hereinafter referred to as the "first work object," the "second work object," the "third work object," and the "fourth work object"), and that the user uses the tool 2 to attach a fastening member to each of these work objects.

[0094] (3.1) Registration mode

[0095] First, refer to Figure 3The following describes an exemplary operation of tool system 1 during assembly of a first product. In this scenario, tool 2 is assumed to be in an initial state, in which registration unit 37 has not yet performed both image registration and torque registration. Specifically, in tool 2 in this initial state, the first through fourth reference images and first through fourth target torque values ​​corresponding to the first, second, third, and fourth work objects, respectively, have not yet been stored in image storage unit 41 or torque storage unit 42.

[0096] The user sets the operating mode of tool 2 to registration mode (in S1). Next, the user operates control panel 231 to input a torque value for the tightening torque when attaching the fastening member to the first work object (in S2). Drive control unit 31 sets the input torque value as the torque setting for the first work object. The user then performs the tightening operation of attaching the fastening member to the first work object by pulling trigger switch 221 (in S3). At this point, the first work object is photographed, thereby generating a still image of the first work object.

[0097] Upon completion of the tightening operation, the registration unit 37 performs a registration process (including image registration and torque registration) (in S4). Specifically, the registration unit 37 performs an image registration process, which causes the image storage unit 41 to store a still image of the first work object generated during the tightening operation in step S3 as a first reference image associated with the first work object. Furthermore, the registration unit 37 performs a torque registration process, which causes the torque storage unit 42 to store the torque setting when the tightening member was attached to the first work object during the tightening operation in step S3 as a first target torque value associated with the first work object. In other words, the first target torque value is associated with the first reference image.

[0098] In particular, according to this embodiment, the processing unit 34 performs process judgment processing. Therefore, in the registration process, the target torque value is registered as included in the work instruction. In other words, in the registration process, the work process is registered. In this example, the registration unit 37 registers the work process so that the work instruction indicating the work to be performed on the first work object becomes the first work instruction in the work process. Specifically, the registration unit 37 registers the work instruction indicating the work to be performed on the first work object as the work process to be performed "first" according to the work process, and the work instruction includes the first target torque value.

[0099] The determination unit 36 ​​performs result storage processing (in S5 ) that causes the result storage unit 43 to store a first determination result indicating whether the fastening torque when the fastening member is attached to the first work object is a normal fastening torque in association with the first work object.

[0100] In addition, the user follows the same operating process as the first operating object to perform tightening operations on the second operating object to the fourth operating object in this order. Specifically, the user operates the control panel 231 to input the torque value of the tightening torque when the tightening member is attached to the second operating object (in S6), and then performs the tightening operation of attaching the tightening member to the second operating object (in S7). At this time, a still image of the second operating object is generated, and the registration unit 37 performs registration processing (including image registration processing and torque registration processing) (in S8). The registration unit 37 registers the operation instruction indicating the operation to be performed on the second operating object as the operating procedure to be performed "second" according to the operating process, and the operation instruction includes a second target torque value. The judgment unit 36 ​​performs result storage processing (in S9), which causes the result storage unit 43 to store a second judgment result indicating whether the tightening torque during the tightening operation of step S7 is a normal tightening torque.

[0101] When the tightening operation on the second operation object is completed, the user performs the tightening operation on the third operation object. Specifically, the user operates the control panel 231 to input the torque value of the tightening torque when the tightening member is attached to the third operation object (in S10), and then performs the tightening operation of attaching the tightening member to the third operation object (in S11). At this time, a still image of the third operation object is generated, and the registration unit 37 performs registration processing (including image registration processing and torque registration processing) (in S12). The registration unit 37 registers the operation instruction indicating the operation to be performed on the third operation object as the operation procedure to be performed "third" according to the operation process, and the operation instruction includes the third target torque value. The judgment unit 36 ​​performs result storage processing (in S13), which causes the result storage unit 43 to store the third judgment result indicating whether the tightening torque during the tightening operation of step S11 is a normal tightening torque.

[0102] Despite Figure 3 Although not shown, upon completion of the tightening operation on the third work object, the user performs tightening operation on the fourth work object. The same series of processing steps as steps S10 to S13 are also performed on the fourth work object. Therefore, the registration unit 37 registers the work instruction to perform the work on the fourth work object as the "fourth" work step to be performed according to the work process, and this work instruction includes the fourth target torque value.

[0103] (3.2) Operation mode

[0104] After the work of assembling the first product in the registration mode (i.e., the fastening work of the first to fourth work objects) is completed, the user performs the work of assembling the second product. Figure 4 An exemplary operation of the tool system 1 when a user performs a task of assembling a second product will be described.

[0105] The user operates the control panel 231 to switch the operation mode of the tool 2 from the registration mode to the work mode (in S21). Then, the user performs the work of assembling the second product in a state where the operation mode of the tool 2 is set to the work mode.

[0106] First, to attach the fastening member to the first work object, the user places tool 2 in an appropriate position on the first work object. At this point, the imaging unit 5 generates a captured image of the first work object of the second product. The image correction unit 33 corrects the captured image of the first work object generated by the imaging unit 5 based on the posture of tool 2 detected by the posture detection unit 26 (in S22). Details of how the image correction unit 33 corrects the captured image will be described later in the section "(3.3) Image Correction."

[0107] The processing unit 34 uses the captured image corrected by the image correction unit 33 to perform object recognition processing to identify the first work object as the actual work object (in S23). In addition, the processing unit 34 performs process judgment processing to determine the correspondence between the first work object thus identified and the work instruction defined by the work process (in S24). That is, the processing unit 34 performs predetermined processing (i.e., object recognition processing and process judgment processing) based on the captured image generated by the camera unit 5 and corrected according to the posture of the tool 2. Using the captured image corrected according to the posture of the tool 2 for processing in this way ultimately enables the processing unit 34 to perform processing based on both the captured image and the posture.

[0108] During the process judgment process, processing unit 34 determines whether the identified work object conforms to the work instructions defined by the work process, that is, whether the work object conforms to the work process. At this time, processing unit 34 compares the next work instruction in the work process read from process storage unit 44 with the identified real-shot work object. In this case, the next work instruction in the work process (that is, the work instruction for the work step to be performed "first") is the work to be performed on the first work object. Therefore, processing unit 34 finds that both the identified work object and the work object included in the work instruction are consistent with the first work object.

[0109] If the determination made in the process judgment processing indicates agreement (i.e., if the work object thus identified conforms to the work instructions defined by the work process), the processing unit 34 sets the target torque value included in the work instructions as the torque setting. In other words, if the identified work object is the first work object, the processing unit 34 sets the first target torque value associated with the first work object as the torque setting (in S25).

[0110] More specifically, upon identifying the live work object, the processing unit 34 retrieves the target torque value associated with the live work object from the torque storage unit 42. The processing unit 34 can then set the target torque value thus retrieved as the torque setting. Furthermore, once the processing unit 34 has identified the live work object, the processing unit 34 retains the target torque value associated with the live work object as the torque setting until another work object is captured in the captured image. This allows the fastening component to be tightened at the target torque value associated with the live work object, even if, for example, the processing unit 34 becomes unable to identify the live work object immediately before starting the tightening operation.

[0111] In this state, the user performs a fastening operation to attach the fastening member to the first work object (in S26). The determination unit 36 ​​performs a result storage process (in S27) that causes the result storage unit 43 to store a first determination result indicating whether the fastening torque when the fastening member is attached to the first work object is a normal fastening torque in association with the first work object.

[0112] On the other hand, if, for example, the user fails to follow the work procedure, the determination made in the process judgment process may indicate inconsistency, that is, the identified work object may not conform to the work instructions defined by the work procedure. If the determination made in the process judgment process indicates inconsistency, the operation of the drive unit 24 is restricted (suspended) and the notification unit 211 is illuminated. This allows the user to notice that he or she is not following the correct work procedure.

[0113] In addition, the user follows the same operation process as the first operation object and performs the fastening operation on the second operation object to the fourth operation object in this order. Specifically, in order to attach the fastening member to the second operation object, the user sets the tool 2 at an appropriate position on the second operation object. In response, the image correction unit 33 corrects the captured image of the second operation object generated by the camera unit 5 according to the posture of the tool 2 detected by the posture detection unit 26 (in S28). Then, the processing unit 34 uses the captured image corrected by the image correction unit 33 to perform object recognition processing to identify the second operation object as a real-shot operation object (in S29). In addition, the processing unit 34 performs a process judgment processing (in S30) to determine the correspondence between the second operation object thus identified and the operation instruction defined by the operation process. In this case, the next operation instruction in the operation process (i.e., the operation instruction of the operation process to be performed "second") is the operation to be performed on the second operation object. Therefore, the processing unit 34 finds that both the identified operation object and the operation object included in the operation instruction are consistent with the second operation object.

[0114] Therefore, the processing unit 34 sets the second target torque value associated with the second work object as the torque setting (in S31). In this state, the user performs a tightening operation to attach the fastening member to the second work object (in S32). The determination unit 36 ​​performs a result storage process (in S33), which causes the result storage unit 43 to store the second determination result indicating whether the tightening torque when the fastening member was attached to the second work object was a normal tightening torque, in association with the second work object.

[0115] When the fastening operation on the second operation object is completed, the user performs the fastening operation on the third operation object. Specifically, in order to attach the fastening member to the third operation object, the user sets the tool 2 at an appropriate position on the third operation object. In response, the image correction unit 33 corrects the captured image of the third operation object generated by the camera unit 5 according to the posture of the tool 2 detected by the posture detection unit 26 (in S34). Then, the processing unit 34 uses the captured image corrected by the image correction unit 33 to perform object recognition processing to identify the third operation object as a real-shot operation object (in S35). In addition, the processing unit 34 performs a process judgment processing (in S36) to judge the correspondence between the third operation object thus identified and the operation instruction defined by the operation process. In this case, the next operation instruction in the operation process (i.e., the operation instruction of the operation process to be performed "thirdly") is the operation to be performed on the third operation object. Therefore, the processing unit 34 finds that both the identified operation object and the operation object included in the operation instruction are consistent with the third operation object.

[0116] Therefore, the processing unit 34 sets the third target torque value associated with the third work object as the torque setting (in S37). In this state, the user performs a tightening operation to attach the fastening member to the third work object (in S38). The determination unit 36 ​​performs a result storage process (in S39), which causes the result storage unit 43 to store the third determination result indicating whether the tightening torque when the fastening member was attached to the third work object was a normal tightening torque, in association with the third work object.

[0117] Despite Figure 4 Although not shown, upon completion of the tightening operation on the third work object, the user performs a tightening operation on the fourth work object. The same series of processing steps as steps S34 to S39 are also performed on the fourth work object. Specifically, processing unit 34 uses the captured image of the fourth work object, corrected based on the posture of tool 2, to identify the fourth work object as a real-image work object. It then determines the correspondence between the thus-identified fourth work object and the work instructions defined by the work procedure. Processing unit 34 then sets the fourth target torque value associated with the fourth work object as the torque setting.

[0118] (3.3) Image Correction

[0119] Next, refer to Figures 5 to 7 How the image correction unit 33 corrects the captured image will be described.

[0120] In the present embodiment, as described above, the image correction unit 33 corrects the captured image according to the posture of the tool 2 detected by the posture detection unit 26 .

[0121] In this embodiment, if Figure 5 As shown, it is assumed that the posture of the tool 2 to be detected by the posture detection unit 26 includes at least a first posture P1 to a fourth posture P4. These first postures P1 to fourth postures P4 correspond to four different directions, respectively, wherein each direction is defined by rotating the main body 20 of the tool 2 around the rotation axis Ax1 of the output shaft 241 by 90 degrees in a front view in a state where the tool 2 is set as a work object on a vertical plane (for example, in a state where the rotation axis Ax1 is aligned with a horizontal plane).

[0122] Figure 5 The appearance of the tool 2 in various postures is schematically illustrated. Specifically, the first posture P1 is the following posture of the tool 2: Figure 5 As shown by the solid line, the grip portion 22 protrudes downward (ie, in the direction of gravity) from the cylinder 21. The second posture P2 is another posture of the tool 2 as follows: Figure 5As shown by the imaginary line (double-dot chain line), the grip portion 22 protrudes rightward (i.e., horizontally) from the cylinder 21. The third posture P3 is another posture of the tool 2: Figure 5 As shown by the imaginary line (double-dot chain line), the grip portion 22 protrudes upward from the cylinder 21. The fourth posture P4 is another posture of the tool 2: Figure 5 As shown by the imaginary line (double-dotted chain line), the gripping portion 22 protrudes leftward (ie, in the horizontal direction) from the cylindrical body 21.

[0123] Under the assumption that the tool 2 can have any of these various postures, for example, if the captured image has symmetry, it may be difficult to identify the work object based on the captured image alone. Figure 6 A is a situation in which there are first to fourth work objects T1 to T4 on a disk-shaped workpiece W1 arranged along a vertical plane. These work objects T1-T4 are arranged at regular intervals along the circumference of the disk-shaped workpiece W1. In this case, it is difficult to identify the work objects based on captured images alone. For example, a captured image Im1 generated by capturing an image of the first work object T1 when the tool 2 has a first posture P1 in which the gripping portion 22 protrudes downward from the cylinder 21 will be the same as a captured image Im1 generated by capturing an image of the third work object T3 when the tool 2 has a third posture P3 in which the gripping portion 22 protrudes upward from the cylinder 21. That is, if Figure 6 As shown in FIG. 1 , when the tool 2 has the first posture P1 and the tool 2 has the third posture P3, the first working object T1 and the third working object T3 are photographed. Then, the photographed images Im1 obtained in this way will be as follows: Figure 6 As shown in B. Therefore, if the posture of the tool 2 is unknown, it is difficult to distinguish the “first work object T1 ” and the “third work object T3 ” such as these based on only the captured image Im1 .

[0124] In contrast, according to the present embodiment, the posture of the tool 2 is detected by the posture detection unit 26, thereby making it possible to recognize the work object even in this case. Specifically, if Figure 6 In the case of the captured image Im1 shown in FIG. 1B , the tool 2 has the first posture P1, and the work object captured in the captured image is the “first work object T1”. On the other hand, if Figure 6 In the captured image Im1 shown in FIG. B, tool 2 has the third posture P3, and the work object captured in the captured image is the "third work object T3." In this way, considering the posture of tool 2 enables the "first work object T1" and "third work object T3" to be distinguished from each other based on the captured image.

[0125] Therefore, according to this embodiment, the image correction unit 33 corrects the posture of the tool 2 to be detected by the posture detection unit 26 by, for example, Figure 7 The rotation compensation shown is used to correct the captured image. Specifically, Figure 7 As shown, if the posture of tool 2 detected by posture detection unit 26 is the first posture P1, image correction unit 33 generates a corrected captured image Im11 without rotating captured image Im1. On the other hand, if the posture of tool 2 detected by posture detection unit 26 is the third posture P3, image correction unit 33 generates a corrected captured image Im13 by rotating captured image Im1 180 degrees. As can be seen, work objects captured in the captured images can be distinguished from each other based on captured images Im11 and Im13, which have been corrected according to the posture of tool 2.

[0126] Furthermore, even if the difference in the posture of tool 2 is not as great as Figure 5 As is evident in the example shown, the captured image generated by the imaging unit 5 may also slightly differ depending on the slight tilt of the tool 2. These subtle differences in the captured image can also be compensated for by having the image correction unit 33 correct the captured image based on the posture of the tool 2 detected by the posture detection unit 26. For example, even when the tool 2 is used in the same first posture P1, the tool 2 held by the user may be slightly tilted to the right or left relative to the normal to the front face of the workpiece, depending on whether the user is right-handed or left-handed. Specifically, if the user holding the tool 2 is left-handed, the captured image will tend to be slightly tilted to the left compared to a right-handed user holding the tool 2 with their right hand. Even in this situation, regardless of whether the captured image is captured by a right-handed or left-handed person, by having the image correction unit 33 correct the captured image based on the posture of the tool 2 detected by the posture detection unit 26, the captured image can be made identical by either person.

[0127] (4) Modification

[0128] Note that the above-mentioned first embodiment is merely a typical embodiment among the various embodiments of the present invention and should not be construed as limiting. On the contrary, the first embodiment can be easily modified in various ways according to design choices or any other factors without departing from the scope of the present invention. The figures to be referred to in this specification are all schematic diagrams. The ratios of the sizes (including thicknesses) of the various constituent elements illustrated in these figures do not always reflect the actual size ratios of these constituent elements. In addition, the functions of the tool system 1 according to the first embodiment can also be implemented as a tool management method, a (computer) program, or a non-transitory storage medium storing the program. The tool management method according to one aspect includes a first step, a second step, and a third step. The first step includes acquiring a captured image from a camera unit 5 equipped with a portable tool 2. The tool 2 includes a drive unit 24 activated by power supplied from a power source. The second step includes acquiring posture information related to the posture of the tool 2 from the posture detection unit 26. The posture detection unit 26 is equipped with the tool 2. The third step includes performing processing based on the captured image Im1 acquired in the first step and the posture information acquired in the second step (see Figure 4 According to another aspect, the program is designed to cause one or more processors to perform the tool management method described above.

[0129] Next, the modified examples of the first embodiment will be listed one by one. Note that the modified examples to be described below can be appropriately combined and adopted. In the following description, any constituent element having the same function as its counterpart in the first embodiment described above will be designated by the same reference numeral as the reference numeral of the counterpart, and its description will be appropriately omitted here.

[0130] The tool system 1 according to the present invention includes a computer system in its control unit 3, for example. The computer system may include a processor and a memory as main hardware components. The functions of the tool system 1 according to the present invention can be performed by causing the processor to execute a program stored in the memory of the computer system. The program can be pre-stored in the memory of the computer system. Alternatively, the program can also be downloaded via a remote communication line, or distributed after being recorded on some non-transitory storage medium such as a memory card, an optical disk, or a hard disk drive (any of which is readable by the computer system). The processor of the computer system can be composed of a single or multiple electronic circuits including a semiconductor integrated circuit (IC) or a large-scale integrated circuit (LSI). As used herein, "integrated circuits" such as IC or LSI are referred to by different names depending on the degree of their integration. Examples of integrated circuits include system LSI, very large-scale integrated circuit (VLSI), and ultra-large-scale integrated circuit (ULSI). Alternatively, a field programmable gate array (FPGA) to be programmed after the LSI is manufactured or a logic device that allows reconfiguration of connections or circuit segments within the LSI can also be used as the processor. These electronic circuits can be integrated together on a single chip or distributed on multiple chips, whichever is appropriate. These multiple chips can be aggregated together in a single device or distributed across multiple devices, without limitation. As used herein, a "computer system" includes a microcontroller comprising one or more processors and one or more memories. Therefore, a microcontroller can also be implemented as a single or multiple electronic circuits comprising semiconductor integrated circuits or large-scale integrated circuits.

[0131] Furthermore, in the above embodiment, at least some of the functions of tool system 1 are integrated into a single housing (main body 20). However, this is not a necessary configuration for tool system 1. Alternatively, these components of tool system 1 may be distributed across multiple different housings. For example, some functions of control unit 3 may be provided in a housing separate from main body 20 of tool 2. Furthermore, at least some functions of control unit 3 may be implemented, for example, as a server or cloud computing system.

[0132] Note that the tool system 1 does not have to be applied to an assembly line for assembling articles in a factory, but may find any other application as well.

[0133] In the first embodiment described above, tool 2 is an impact wrench. However, tool 2 does not have to be an impact wrench, but may be, for example, a nut runner or an oil pulse wrench. Alternatively, tool 2 may also be, for example, a screwdriver (including an impact screwdriver) for tightening a screw (as a fastening member). In this case, a drill bit (such as a screwdriver bit, etc.) is attached to tool 2 instead of socket 242. In addition, tool 2 does not have to be configured to be powered by battery pack 201, but may also be configured to be powered by an AC power supply (commercial power supply). In addition, tool 2 does not have to be an electric tool, but may also be a pneumatic tool that includes an air motor (drive unit) that operates using compressed air (power) supplied from an air compressor (power source).

[0134] Furthermore, in the first embodiment described above, the work object is assumed to be each of multiple locations to be fastened within a single workpiece. However, this is merely an example and should not be construed as limiting. Alternatively, the work object may be a module, part, or product having multiple locations to be fastened. In this case, the multiple locations to be fastened within a single work object may have the same target torque value or different target torque values.

[0135] Furthermore, the processing unit 34 only needs to perform processing based on the captured image and the tool's posture, and does not need to be configured to perform work object recognition as processing, as in the first embodiment. For example, the processing unit 34 can perform processing to output the image as a log (including writing the image to a memory). In this case, the processing unit 34 outputs the captured image corrected by the image correction unit 33 as a log. This enables the processing unit 34 to output an image suitable for user confirmation or computer analysis as a log.

[0136] Alternatively, the processing unit 34 may also, for example, perform processing to indicate to the user the job to be performed. In this case, the processing unit 34 indicates the job to be performed based on the captured image corrected by the image correction unit 33. This can improve the reliability of the job instructions given by the processing unit 34. Furthermore, the processing unit 34 may also perform processing to determine whether the completed job is good or poor. In this case, the processing unit 34 determines whether the completed job is good or poor based on the captured image corrected by the image correction unit 33. This can improve the accuracy of the pass / fail determination made by the processing unit 34 on the completed job.

[0137] In the first embodiment described above, the posture detection unit 26 includes an acceleration sensor and a gyro sensor. However, this configuration is merely an example and should not be construed as limiting. Alternatively, the posture detection unit 26 may include only an acceleration sensor or only a gyro sensor. In addition, the acceleration sensor or the gyro sensor does not have to be a three-axis sensor, but may also be a sensor for detecting uniaxial or biaxial acceleration or angular velocity. Alternatively, instead of, or in addition to, the acceleration sensor and the gyro sensor, the posture detection unit 26 may include a sensor for detecting physical quantities other than acceleration and angular velocity. The posture detection unit 26 may use the captured image generated by the camera unit 5 to detect the posture. In addition, the posture detection unit 26 may also use a global navigation satellite system such as a global positioning system (GPS).

[0138] Optionally, the tool 2 may include a torque sensor for measuring the tightening torque. In this case, the drive control unit 31 controls the drive unit 24 so that the tightening torque measured by the torque sensor becomes the torque setting. In addition, the determination unit 36 ​​can determine whether the tightening torque is normal by comparing the measurement result of the torque sensor with the target torque value. In the case where it is found that the measurement result of the torque sensor falls within a predetermined range based on the target torque value, the determination unit 36 ​​determines that the tightening torque should be a normal tightening torque. On the other hand, in the case where it is found that the measurement result of the torque sensor falls outside the predetermined range based on the target torque value, the determination unit 36 ​​determines that the tightening torque should be an insufficient (abnormal) tightening torque.

[0139] Note that the image registration process does not necessarily require the image storage unit 41 to store the still image generated by the imaging unit 5 as a reference image. Alternatively, the image registration process may be a process for registering a still image downloaded from a server as a reference image in the image storage unit 41, or a process for registering a still image acquired from an external memory such as a memory card as a reference image in the image storage unit 41.

[0140] In addition, the notification unit 211 does not have to be a light-emitting unit such as an LED, but can also be implemented as an image display device such as a liquid crystal display or an organic electroluminescent (EL) display. Optionally, the notification unit 211 can be notified (presented) by any means other than display. For example, the notification unit 211 can also be implemented as a speaker or buzzer that emits sound (including voice). In this case, the notification control unit 35 preferably causes the notification unit 211 to emit different sounds when the determination made by the processing unit 34 indicates inconsistency and when the processing unit 34 recognizes the actual operation object. Alternatively, the notification unit 211 can also be implemented as a vibrator that generates vibrations or a transmitter for sending a notification signal to an external terminal (such as a mobile communication device) provided outside the tool 2. Optionally, the notification unit 211 can also have two or more functions selected from display, emitting sound, generating vibrations and establishing communication in combination.

[0141] The storage unit 4 can store work process data representing a predetermined order of work procedures to be performed on multiple work objects. In this case, the processing unit 34 selects a reference image for use in image processing (pattern matching) from multiple reference images according to the work process. Specifically, the processing unit 34 preferentially selects a reference image corresponding to the upcoming work object to be processed in the upcoming work process from the multiple reference images. As used herein, the "upcoming work object" is a work object to be processed after the work object recognized last time. The processing unit 34 performs image processing to compare the reference image selected as template data with the captured image. That is, the processing unit 34 selects the reference image by predicting the work object to be captured in the captured image next time according to the work process. This enables the processing unit 34 to identify the work object captured in the captured image in a shorter time.

[0142] Optionally, the processing unit 34 may also be configured to determine the type of the socket 242 attached to the tool 2 by performing image processing on the image captured by the camera unit 5. As used herein, "type" is information used to distinguish different parts from each other, and includes at least one information related to size (dimensions or length), shape or material. In the present embodiment, the processing unit 34 is configured to determine the length of the socket 242 attached to the tool 2. The processing unit 34 corrects the target torque value according to the length of the socket 242, and sets the target torque value corrected in this way as the torque setting. For example, the processing unit 34 corrects the target torque value associated with the actual working object by multiplying the target torque value by a coefficient corresponding to the length of the socket 242, and sets the target torque value corrected in this way as the torque setting. That is, the processing unit 34 controls the drive unit 24 so that the tightening torque becomes equal to the corrected target torque value. This can reduce the deviation of the tightening torque according to the length of the socket 242.

[0143] Optionally, the processing unit 34 may be further configured to determine a torque setting based on the detected length (or type) of the socket 242. The storage unit 4 stores torque values ​​corresponding one-to-one to various lengths of the socket 242. The processing unit 34 retrieves the torque value corresponding to the determined length of the socket 242 from the storage unit 4 and sets the value based on the torque value thus retrieved as the torque setting. For example, the processing unit 34 may set the torque value retrieved from the storage unit 4 as the torque setting. This enables tightening operations to be performed at a torque value corresponding to the type of socket 242.

[0144] Furthermore, the captured image generated by the imaging unit 5 does not necessarily need to be a moving picture; for example, it can also be a still picture of the moment the tightening operation is completed. If the captured image is a still picture, the determination unit 36 ​​causes the result storage unit 43 to store the still picture generated by the image capturing unit 5 in association with the determination result. This allows the user to view a still picture of a workpiece for which the tightening torque has been found to be insufficient, for example.

[0145] The camera unit 5 is not necessarily provided to the barrel 21 of the body 20, but may be provided to the attachment member 23 of the body 20 or the battery pack 201, for example. Likewise, the arrangement of the control unit 3, storage unit 4, and other components may be changed appropriately.

[0146] Furthermore, during the image registration process in the registration mode, the posture of the tool 2 detected by the posture detection unit 26 can also be stored in the storage unit 4 (image storage unit 41) in association with the reference image generated by the imaging unit 5. This enables the reference image and the posture of the tool 2 to be registered in association with each other. Therefore, in the work mode, once the captured image and the posture of the tool 2 are detected, the processing unit 34 can compare the reference image and the captured image associated with the same posture.

[0147] (Second embodiment)

[0148] In the tool system 1A according to the second embodiment, as Figure 8 As shown, the image correction unit 33 is omitted from the control unit 3 (see Figure 1 ) function, which is the main difference from the tool system 1 according to the first embodiment. In the following description, any constituent element in this second embodiment having the same function as its counterpart in the above-mentioned first embodiment will be designated by the same reference numeral as that of the counterpart, and its description will be appropriately omitted here.

[0149] In this embodiment, the processing unit 34 receives two pieces of input information consisting of first information based on a captured image and second information based on a posture, and thereby performs processing based on both the first information and the second information. That is, in this embodiment, the image correction unit 33 is omitted, so the captured image (first information) generated by the camera unit 5 and the posture information (second information) related to the posture of the tool 2 detected by the posture detection unit 26 are provided to the processing unit 34 instead of the image correction unit 33. In this case, the captured image is generated by the camera unit 5 equipped with the tool 2 and is supplied to the processing unit 34 when necessary. On the other hand, the posture of the tool 2 is detected by the posture detection unit 26, and the posture information related to the posture of the tool 2 is provided to the processing unit 34 when necessary.

[0150] That is, the processing unit 34 receives two pieces of information: the (uncorrected) captured image and the posture information, rather than the corrected captured image corrected according to the posture of the tool 2. The processing unit 34 then performs predetermined processing based on these two pieces of information (i.e., the first information and the second information). Therefore, in this embodiment, the processing unit 34 performs predetermined processing directly using the captured image and the posture of the tool, rather than indirectly using them as in the first embodiment (i.e., using the captured image corrected according to the posture of the tool 2).

[0151] The configuration of this embodiment also enables the processing unit 34 to take into account the posture of the tool 2. Therefore, even if, for example, the captured image has symmetry, the processing unit 34 can recognize the work object based on the captured image and the posture of the tool 2. For example, assuming that Figure 6 As shown in FIG. 1A , when the tool system 1A captures the first work object T1 and the third work object T3 when the tool 2 has the first posture P1 and the tool 2 has the third posture P3, respectively, Figure 6 Even in this case, the processing unit 34 can distinguish the “first work object T1” and the “third work object T3” from each other by determining whether the posture of the tool 2 when the photographed image Im1 was obtained was the first posture P1 or the third posture P3.

[0152] In addition, if Figure 8 As shown, the tool system 1A according to the present embodiment further includes a posture presenting unit 38. The posture presenting unit 38 presents posture information based on the detection result obtained by the posture detection unit 26. In the present embodiment, for example, the posture presenting unit 38 presents posture information by using the notification unit 211. If the notification unit 211 is an LED, it is preferable to light up the notification unit 211 in different ways according to the posture of the tool 2 detected by the posture detection unit 26. For example, if the tool 2 has the first posture P1, the posture presenting unit 38 can light up the notification unit 211 in red. On the other hand, if the tool 2 has the third posture P3, the posture presenting unit 38 can light up the notification unit 211 in green.

[0153] Such a gesture presenting unit 38 can prompt the user to use the tool 2 in a specific gesture, for example, by presenting the gesture of the tool 2 to the user.

[0154] Furthermore, in this embodiment, the processing unit 34 can receive two types of information: first information based on the captured image and second information based on the posture of the tool 2. The first information can be the image captured by the imaging unit 5 itself, or information obtained by performing appropriate processing such as correction on the captured image, whichever is appropriate. Similarly, the second information can be information related to the posture of the tool 2 detected by the posture detection unit 26 itself, or information obtained by performing appropriate processing on the information related to the posture of the tool 2 (such as multiplying the posture information by a predetermined coefficient, etc.), whichever is appropriate.

[0155] Note that the gesture presentation unit 38 is not an essential component of the tool system 1A according to the present embodiment. That is, the gesture presentation unit 38 can be omitted as appropriate from the configuration of the second embodiment.

[0156] The gesture presentation unit 38 may also be applied to the tool system 1 according to the first embodiment as needed.

[0157] Alternatively, the configurations described for the second embodiment (including its modifications) may be adopted by appropriately combining the various configurations described for the first embodiment (including its modifications).

[0158] (Summary)

[0159] As can be seen from the above description, the tool system (1, 1A) according to the first aspect includes a portable tool (2), a camera unit (5), a posture detection unit (26), and a processing unit (34). The tool (2) includes a drive unit (24) activated by power supplied from a power source. The camera unit (5) is equipped with the tool (2) and generates a captured image. The posture detection unit (26) is equipped with the tool (2) and detects the posture of the tool (2). The processing unit (34) performs processing based on the captured image and the posture.

[0160] According to this aspect, the processing unit (34) performs processing not only based on the captured image generated by the camera unit (5) equipped with the tool (2), but also based on the posture of the tool (2) detected by the posture detection unit (26). This improves the reliability and accuracy of the processing performed by the processing unit (34) compared to a case where the processing unit (34) performs the same processing based only on the captured image. For example, if the captured image has symmetry, it may be difficult to identify the work object based only on the captured image. On the other hand, if the posture of the tool (2) is known, it is easier to identify the work object based on the captured image even in the case where the captured image has symmetry. This achieves the following advantage: the possibility of causing a decrease in the reliability or accuracy of the processing using the captured image generated by the camera unit (5) is reduced.

[0161] The tool system (1, 1A) according to the second aspect, which can be implemented in combination with the first aspect, further includes an image correction unit (33). The image correction unit (33) corrects the captured image according to the posture. The processing unit (34) performs processing by using the captured image corrected by the image correction unit (33).

[0162] This aspect enables the processing unit (34) to perform processing based on the captured image and the posture by performing processing using the captured image corrected by the image correction unit (33).

[0163] In the tool system (1, 1A) according to the third aspect, which can be implemented in combination with the second aspect, the image correction unit (33) performs at least rotation compensation on the captured image according to the posture.

[0164] This aspect enables compensation for changes in posture due to, for example, rotation of the tool (2) by correcting the captured image.

[0165] In the tool system (1, 1A) according to the fourth aspect, which can be implemented in combination with the second aspect or the third aspect, the image correction unit (33) performs at least distortion correction on the captured image according to the posture.

[0166] This aspect enables compensation for changes in posture due to, for example, tilt of the tool (2) by correcting the captured image.

[0167] In the tool system (1, 1A) according to the fifth aspect that can be implemented in combination with any one of the first to fourth aspects, the processing unit (34) receives two input information consisting of first information based on the captured image and second information based on the posture, and thereby performs processing based on both the first information and the second information.

[0168] This aspect eliminates the need to perform processing for correcting captured images, thereby contributing to speeding up the processing.

[0169] The tool system (1, 1A) according to the sixth aspect, which can be implemented in combination with any one of the first to fifth aspects, further includes a gesture presenting unit (38). The gesture presenting unit (38) presents gesture information based on the detection result obtained by the gesture detecting unit (26).

[0170] This aspect enables information related to the posture of the tool (2) to be presented to the user based on the detection result obtained by the posture detection unit (26).

[0171] In the tool system (1, 1A) according to the seventh aspect, which can be implemented in combination with any one of the first to sixth aspects, the processing unit (34) performs at least recognition of the work object as processing.

[0172] This aspect enables recognition of a work object based on a captured image and a posture.

[0173] In the tool system (1, 1A) according to the eighth aspect that can be implemented in combination with the seventh aspect, the processing unit (34) performs, as processing, a determination of a correspondence between the identified work object and a work instruction defined by the work process.

[0174] This aspect enables determination of the work process based on the captured image and the posture.

[0175] In the tool system (1, 1A) according to the ninth aspect, which can be implemented in combination with any one of the first to eighth aspects, the posture detection unit (26) includes at least one of an acceleration sensor and a gyro sensor.

[0176] This aspect enables the posture of the tool (2) to be detected by causing the sensor to detect at least one of the acceleration of the tool (2) and the angular velocity of the tool (2).

[0177] The tool management method according to the tenth aspect includes a first step, a second step, and a third step. The first step includes acquiring a captured image from a camera unit (5) equipped with a portable tool (2), the portable tool (2) including a drive unit (24) activated by power supplied from a power source. The second step includes acquiring posture information related to the posture of the tool (2) from a posture detection unit (26) equipped with the tool (2). The third step includes performing processing based on the captured image acquired in the first step and the posture information acquired in the second step.

[0178] This aspect achieves the advantage of reducing the possibility of causing a decrease in the reliability or accuracy of processing of a captured image generated using the image capture unit (5).

[0179] The program according to the eleventh aspect is designed to cause one or more processors to perform the tool management method according to the tenth aspect.

[0180] This aspect achieves the advantage of reducing the possibility of causing a decrease in the reliability or accuracy of processing of captured images generated using the image capture unit (5).

[0181] Note that these are not the only aspects of the present invention, but various configurations of the tool systems ( 1 , 1A) according to the first and second embodiments (including modifications thereof) can also be implemented as tool management methods or programs.

[0182] Note that the constituent elements according to the second aspect to the ninth aspect are not essential constituent elements of the tool system (1, 1A) but may be omitted as appropriate.

[0183] Description of Reference Numerals

[0184] 1,1A Tool System

[0185] 2 Tools

[0186] 5 Camera Unit

[0187] 24 drive units

[0188] 26 Posture Detection Unit

[0189] 33 Image Correction Unit

[0190] 34 processing units

[0191] 38 Posture Presentation Unit

Claims

1. A tool system, comprising: A portable tool comprising a drive unit configured to be activated by power supplied from a power source; a camera unit, which is provided to the tool and configured to generate a captured image; a posture detection unit, which is equipped to the tool and is configured to detect a posture of the tool; a processing unit configured to perform processing based on the captured image and the gesture; as well as an image correction unit configured to correct the captured image according to the posture, Herein, the processing unit is configured to perform the processing by using the captured image corrected by the image correction unit.

2. The tool system according to claim 1, wherein: The image correction unit is configured to perform at least rotation compensation on the captured image according to the posture.

3. The tool system according to claim 1, wherein: The image correction unit is configured to perform at least distortion correction on the captured image according to the posture.

4. The tool system according to claim 1, wherein: The processing unit is configured to perform at least recognition of a work object as the processing.

5. A tool system comprising: A portable tool comprising a drive unit configured to be activated by power supplied from a power source; a camera unit, which is provided to the tool and is configured to generate a captured image; a posture detection unit, which is equipped to the tool and configured to detect the posture of the tool; as well as a processing unit configured to perform processing based on the captured image and the gesture, wherein the processing unit is configured to at least perform, as the processing, recognition of a work object captured on-site in the captured image among a plurality of work objects, wherein it is assumed that the work object is operated by using the tool, and The processing unit is configured to perform, as the processing, determination of a correspondence relationship between the identified work object and a sequence of the plurality of work objects defined by a work procedure.

6. The tool system according to claim 1 or 5, wherein: The processing unit is configured to receive two pieces of input information, namely, first information based on the captured image and second information based on the gesture, and thereby perform the processing based on both the first information and the second information. 7 . The tool system according to claim 1 , further comprising a posture presenting unit configured to present posture information based on the detection result obtained by the posture detecting unit.

8. The tool system according to claim 1 or 5, wherein: The posture detection unit includes at least one of an acceleration sensor and a gyro sensor.

9. A tool management method, comprising: A first step is to obtain a captured image from a camera unit provided to a portable tool, the portable tool including a drive unit configured to be activated by power supplied from a power source; A second step is to obtain posture information related to the posture of the tool from a posture detection unit equipped with the tool; A third step is used to perform processing based on the captured image acquired in the first step and the posture information acquired in the second step; as well as The fourth step is to correct the captured image according to the posture, Here, the third step performs the processing by using the captured image corrected by the fourth step. 10 . A non-transitory storage medium having a program stored thereon, wherein the program is designed to cause one or more processors to perform the tool management method according to claim 9 .

Images