Tool system, tool, work object recognition system, work object recognition method, and program

Abstract

The problem to be overcome by the present invention is to reduce power consumption. The tool system (1) includes a tool (2), a camera unit (5), a processing unit (35), and a setting status detection unit (34). The tool (2) is a portable tool (2) including a drive unit (24) which is activated by power supplied from a power source. The camera unit (5) is provided to the tool (2) and generates captured images. The processing unit (35) intermittently performs recognition processing for identifying the work object based on the captured images. The setting status detection unit (34) detects the state in which the tool (2) is set in the appropriate position on the work object.

Description

Technical Field

[0001] This invention generally relates to tool systems, tools, work object identification systems, work object identification methods, and procedures. More particularly, this invention relates to tool systems, portable tools, work object identification systems, work object identification methods, and procedures for use in portable tools. Background Technology

[0002] Patent Document 1 discloses a tool system including a portable tool having a camera unit and a drive unit activated by power supplied from a battery pack. The camera unit is arranged such that, for example, a socket attached to the tool's output shaft is covered within its camera range. The camera unit captures images of the work object (which may be, for example, the object or place where the work is being done) during operation using the tool.

[0003] According to Patent Document 1, images captured by a camera unit are used to identify the work object that the tool is positioned in the appropriate location (i.e., the work object arranged to prepare the tool for starting work on the work object). The tool system compares the captured images generated by the camera unit with multiple reference images stored in an image storage unit, thereby identifying the work object captured in the captured images.

[0004] If the tool system in Patent Document 1 attempts to identify the work object captured in the captured image while the camera unit is operating, power consumption increases.

[0005] Existing technical documents

[0006] Patent documents

[0007] Patent Document 1: Japanese Patent Application Publication No. 2019-042860 Summary of the Invention

[0008] In view of the above background, the object of the present invention is to provide a tool system, tool, work object identification system, work object identification method and program, all of which are configured or designed to reduce power consumption.

[0009] To overcome this problem, a tool system according to one aspect of the present invention includes a tool, a camera unit, a processing unit, and a setup state detection unit. The tool is a portable tool including a drive unit activated by power supplied from a power source. The camera unit is equipped with the tool and generates captured images. The processing unit intermittently performs recognition processing for identifying the work object based on the captured images. The setup state detection unit detects the state in which the tool is positioned appropriately on the work object.

[0010] A tool according to another aspect of the invention is designed for use in the aforementioned tool system. The tool includes a drive unit and a camera unit.

[0011] According to another aspect of the present invention, a work object identification system includes a processing unit and a setting state detection unit. The processing unit intermittently performs identification processing for identifying the work object based on images captured by a camera unit. The camera unit is equipped with a tool, which is a portable tool including a drive unit activated by power supplied from a power source. The setting state detection unit detects the state in which the tool is positioned appropriately on the work object.

[0012] According to another aspect of the present invention, a method for identifying a work object includes an identification processing step and a setting state detection step. The identification processing step is used to intermittently perform identification processing for identifying the work object based on images captured by a camera unit. The camera unit is equipped with a tool, which is a portable tool including a drive unit activated by power supplied from a power source. The setting state detection unit detects the state in which the tool is set in an appropriate position on the work object.

[0013] According to another aspect of the invention, the program is designed to cause one or more processors to perform the above-described job object identification method. Attached Figure Description

[0014] Figure 1 This is a block diagram of a tool system according to a typical embodiment;

[0015] Figure 2 A is a perspective view illustrating the appearance of the tool system from one angle. Figure 2 B is a perspective view illustrating the appearance of the tool system from another angle;

[0016] Figure 3 This shows the sequence of operations that the tool system will perform;

[0017] Figure 4 This is a flowchart illustrating an exemplary process of the operations to be performed by the tool system;

[0018] Figure 5 This is a flowchart illustrating an exemplary process of the operations to be performed by the tool system;

[0019] Figure 6 This is a flowchart illustrating an exemplary process of the operations to be performed by the tool system; and

[0020] Figure 7 This is a perspective view showing the appearance of a tool system according to a variant example. Detailed Implementation

[0021] Preferred embodiments of the invention will now be described in detail with reference to the accompanying drawings. In the following description of the embodiments, any pair of constituent elements having the same function will be designated by the same reference numerals, and their description will be omitted herein to avoid redundancy.

[0022] (Example)

[0023] (1) Overview

[0024] First, refer to Figure 1 This section provides an overview of the tool system 1 according to a typical embodiment.

[0025] The tool system 1 according to this embodiment includes a portable tool 2. The tool 2 includes, for example, a drive unit 24 containing a motor. The drive unit 24 is activated by a prime mover (such as electricity) supplied from a power source such as a battery pack 201. Examples of this type of tool 2 include impact wrenches, nut wrenches, oil pulse wrenches, screwdrivers (including impact screwdrivers), drills, drilling screwdrivers, and various other types of tools. Using this type of tool 2 allows the user to perform various machining operations, such as attaching fastening components (such as bolts or nuts) to a workpiece (the object of the machining operation) or making openings through the workpiece.

[0026] Furthermore, in the tool system 1 according to this embodiment, a camera unit 5 is provided for the tool 2. The camera unit 5 generates captured images. The camera unit 5 is, for example, attached to the output shaft 241 of the tool 2 (see reference). Figure 2 A) Socket 242 (Reference) Figure 2 A) is covered within its camera range (field of view). Therefore, while the user is working with tool 2, camera unit 5 captures an image of the work object and generates the captured image.

[0027] Therefore, the tool system 1 according to this embodiment enables the identification of a work object based on, for example, an image captured by the camera unit 5, thereby enabling, for example, the determination of whether the work being performed by the user using the tool 2 follows a work process. Furthermore, the tool system 1 also enables the determination of whether the work performed on the work object is good or bad based on the image captured by the camera unit 5, notifying the user of work instructions according to the work object, and storing the image as a log (i.e., a work record). It can be seen that using the image (captured image) generated by the camera unit 5 equipped with the tool 2, for example, enables the user to use the tool 2 to perform or manage his / her work.

[0028] like Figure 1As shown, the tool system 1 according to this embodiment includes a setting state detection unit 34 and a processing unit 35. That is, the tool system 1 includes a tool 2, a camera unit 5, a setting state detection unit 34, and a processing unit 35. The tool 2 includes a drive unit 24 activated by power supplied from a power source. The camera unit 5 is equipped with the tool 2 and generates captured images. The setting state detection unit 34 detects the state in which the tool 2 is set in an appropriate position on the work object. The processing unit 35 intermittently performs recognition processing based on the captured images to identify the work object.

[0029] This configuration enables tool system 1 to identify the work object based on the captured image after detecting that tool 2 has been positioned appropriately on the work object. This reduces the number of times tool system 1 performs futile identification processing when camera unit 5 is not capturing the area around the work object (e.g., when tool 2 is carried by the user). Therefore, tool system 1 according to this embodiment can reduce power consumption.

[0030] (2) Detailed Structure

[0031] Next, refer to Figures 1 to 2 The detailed structure of the tool system 1 according to this embodiment will be explained by B.

[0032] (2.1) Prerequisites

[0033] The tool system 1 according to this embodiment can be used, for example, in an assembly line at a factory for assembling workpieces (workpieces). Specifically, in this embodiment, it is assumed that the tool 2 included in the tool system 1 is a fastening tool such as an impact wrench for tightening fastening components (such as bolts or nuts). Specifically, it is assumed that this embodiment is applied to a situation where a single workpiece has multiple fastening points, thus requiring a user to attach the fastening components to each of these fastening points in a single work area using the tool 2.

[0034] As used herein, the term "intermittently" refers not only to situations where events occur at regular intervals but also to situations where events occur at irregular intervals. Therefore, the phrase "intermittently performing identification processing" means not only that identification processing occurs at regular intervals but also that it occurs at irregular intervals. Furthermore, the phrase "intermittently performing identification processing" also means performing identification processing when the stability determination unit 33 determines that the captured image is stable. Moreover, the phrase "intermittently performing identification processing" also means performing identification processing when the setting state detection unit 34 detects that the tool 2 is positioned appropriately on the work object.

[0035] As used herein, "the part to be fastened" refers to the portion of a workpiece to be attached to a fastening member. For example, if the fastening member is a bolt, the part to be fastened is the area surrounding and covering the screw hole to which the fastening member is attached and tightened. That is, in this embodiment, a single workpiece has multiple such parts to be fastened.

[0036] As used herein, "work object" refers to the object or work area where the work is assumed to be performed using tool 2. Specifically, the work object where tool 2 is currently positioned is sometimes referred to as the "current work object" below. As used herein, the phrase "tool 2 is currently positioned" means that tool 2 is placed in a position ready to perform work on the work object. Furthermore, as used herein, the phrase "tool 2 is currently positioned" refers not only to the situation where tool 2 is already in contact with the work object, but also to the situation where tool 2 is about to come into contact with the work object (i.e., tool 2 is approaching the work object). That is, while tool 2 is positioned on the work object, tool 2 may have already come into contact with the work object or may not yet be in contact with it. In this embodiment, as an example, it is assumed that each of the multiple fastening parts of a single workpiece is a work object.

[0037] As used herein, "captured image" refers to an image captured by camera unit 5, and includes still images (still pictures) and moving images (animations). "Moving images" also include, for example, a set of still images (frames) captured by stop-motion animation. The captured image is not necessarily the output data provided by camera unit 5 itself. For example, the captured image may undergo any image processing as needed, such as data compression, conversion to another data format, cropping of image portions from images captured by camera unit 5, focus adjustment, brightness adjustment, contrast adjustment, or various other types of image processing. In this embodiment, for example, it is assumed that the captured image is a panchromatic moving image. Furthermore, as used herein, the phrase "captured image stable" can refer to a situation where the tool 2 is positioned appropriately on the work object and no shaking occurs in camera unit 5. Additionally, the phrase "captured image stable" can also refer to a situation where the camera control of camera unit 5, such as automatic exposure (AE) and automatic white balance (AWB), is stable.

[0038] Furthermore, as used herein, "reference image" refers to an image generated based on the captured image generated by camera unit 5. In this embodiment, as an example, it is assumed that the "reference image" is a panchromatic still image. As used herein, "multiple reference images corresponding to multiple work objects" refers not only to the case where multiple reference images correspond to multiple work objects in a one-to-one manner, but also to the case where multiple reference images correspond to multiple work objects in a one-to-many manner. Alternatively, each of the multiple work objects may be associated with multiple reference images captured by photographing the work object from various angles or at multiple different sizes.

[0039] Furthermore, as used herein, if something is “equipped” to something else, the former can be built into the latter (e.g., integrally integrated with the latter), or can be attached to the latter simply as an external component (e.g., removably secured with, for example, a connector). As used herein, “external attachment” means, for example, attaching an auxiliary device separately to a machine to extend its functionality. Specifically, the phrase “attaching the first device externally to the second device” means not only attaching the first device itself as an auxiliary device to the second device, but also attaching a third device, in which the first device is built, as an auxiliary device to the second device. That is, the camera unit 5 equipped to tool 2 can be built into tool 2 or attached to tool 2 simply as an external component, either way is appropriate. According to this embodiment, the camera unit 5 is built into tool 2.

[0040] Furthermore, as used herein, "work process" means the sequence of work to be performed using tool 2. For example, if a series of work steps to be performed on a single work object or multiple work objects is defined as a single work step, then the work process indicates the order in which the work steps to be performed on the single work object or multiple work objects within the entire work process. More specifically, if the instruction related to the work to be performed on a single work object is a "work instruction," then the work process is information indicating a single work instruction or multiple work instructions for a single work step and the order in which the work steps should be performed. In other words, the work process indicates which work object corresponds to in the single or multiple work steps, and also indicates the sequential position of the corresponding work steps. In the following description of this embodiment, it is assumed that the work process defines the order in which multiple work objects in a single workpiece should be worked on (including multiple work steps).

[0041] (2.2) Structure of the tool system

[0042] like Figure 1 As shown, the tool system 1 according to this embodiment includes a tool 2 and a work object identification system 10.

[0043] (2.2.1) Structure of the tool

[0044] First, refer to Figure 1 , Figure 2 A and Figure 2 The structure of tool 2 in tool system 1 according to this embodiment will be described using B. Tool 2 includes control unit 3a, drive unit 24, impact mechanism 25, notification unit 211, and battery pack 201 (see reference). Figure 1 ).

[0045] According to this embodiment, tool 2 is a power tool that uses electrical energy to activate drive unit 24. Specifically, in this embodiment, tool 2 is assumed to be an impact wrench. This tool 2 can be used for fastening operations that attach fastening components to the work object.

[0046] In this configuration, tool 2 is designed to use battery pack 201 as a power source, utilizing the electrical power (electrical energy) supplied from battery pack 201 to activate drive unit 24. In this embodiment, it is assumed that battery pack 201 is included as a constituent element of tool 2. However, battery pack 201 need not be one of the constituent elements of tool 2. In other words, battery pack 201 may not be included as a constituent element of tool 2.

[0047] Tool 2 also includes a body 20. The body 20 houses the drive unit 24 and the impact mechanism 25. Additionally, in tool 2 according to this embodiment, the control unit 3a and the notification unit 211 are also housed in the body 20.

[0048] The body 20 of tool 2 includes a cylindrical body 21, a gripping portion 22, and an attachment portion 23. The cylindrical body 21 is formed in a cylindrical shape (e.g., a cylindrical shape in this embodiment). The gripping portion 22 protrudes along a normal to a portion of the circumferential surface of the cylindrical body 21 (i.e., along the radius of the cylindrical body 21). The battery pack 201 is removably attached to the attachment portion 23. In other words, the cylindrical body 21 and the attachment portion 23 are connected together via the gripping portion 22.

[0049] At least drive unit 24 is housed within cylinder 21. Drive unit 24 includes a motor. Drive unit 24 is configured to be activated using power supplied to the motor from battery pack 201, which serves as a power source. Output shaft 241 protrudes from one axial end face of cylinder 21. When drive unit 24 is activated, output shaft 241 rotates about a rotation axis Ax1 aligned with the direction in which output shaft 241 protrudes. That is, drive unit 24 drives output shaft 241 to rotate about rotation axis Ax1. In other words, when drive unit 24 is activated, torque is applied to output shaft 241, thereby causing output shaft 241 to rotate.

[0050] A cylindrical socket 242 for rotating fastening components (such as bolts or nuts) is removably attached to the output shaft 241. The socket 242, together with the output shaft 241, rotates about the rotation axis Ax1. The user can appropriately select the size of the socket 242 attached to the output shaft 241 according to the size of the fastening component. With this configuration, activating the drive unit 24 causes the output shaft 241 to rotate, thereby causing the socket 242 to rotate together with the output shaft 241. If the socket 242 is then fitted onto the fastening component, the fastening component rotates together with the socket 242, thereby completing the operation of tightening or loosening the fastening component. In this way, the tool 2 can perform the operation of tightening or loosening the fastening component by activating the drive unit 24.

[0051] Alternatively, a socket anvil can be attached to the output shaft 241 in place of socket 242. The socket anvil can also be removably attached to the output shaft 241. This allows drill bits (such as screwdriver bits or drill bits) to be attached to the output shaft 241 via the socket anvil.

[0052] Tool 2 includes the impact mechanism 25 as described above. The impact mechanism 25 is configured to apply an impact force in the rotational direction to the output shaft 241 when the tightening torque (operating value) exceeds a predetermined level. This allows tool 2 to apply a larger tightening torque to the tightening member.

[0053] The grip 22 is the part that the user holds while performing the operation. The grip 22 is equipped with a trigger switch 221 (operating unit) and a forward / reverse switch 222. The trigger switch 221 is a switch used to control the ON / OFF state of the drive unit 24 to activate or deactivate it. The trigger switch 221 has an initial position and an ON position. When the trigger switch 221 is pressed or pulled to the ON position by the user, the drive unit 24 is activated. Additionally, the trigger switch 221 allows adjustment of the rotation speed of the output shaft 241 based on the depth to which the trigger switch 221 is pulled (i.e., based on the manipulation variable of the trigger switch 221). The forward / reverse switch 222 is a switch used to switch the rotation direction of the output shaft 241 from clockwise to counterclockwise and from counterclockwise to clockwise.

[0054] The attachment portion 23 is formed in a flat cuboid shape. The battery pack 201 is removably attached to the side of the attachment portion 23 opposite to the grip portion 22.

[0055] The battery pack 201 includes a housing 202 made of resin and formed in a cuboid shape. The housing 202 contains a rechargeable battery (such as a lithium-ion battery). The battery pack 201 supplies power to the drive unit 24, control unit 3a, notification unit 211, work object identification system 10, and other components.

[0056] The attachment 23 is also equipped with an operation panel 231. The operation panel 231 may include, for example, a plurality of push-button switches 232 and a plurality of LEDs (light-emitting diodes) 233. The operation panel 231 allows the user to input various settings of the tool 2 and check the status of the tool 2. That is, for example, by operating the push-button switches 232 of the operation panel 231, the user can change the operating mode of the tool 2 or check the remaining capacity of the battery pack 201.

[0057] The attachment 23 also includes a light-emitting unit 234 for taking pictures. The light-emitting unit 234 includes, for example, an LED. When the user is using the tool 2 to perform work, the light-emitting unit 234 emits light toward the work object. The light-emitting unit 234 can be turned on and off by operating the operation panel 231. Alternatively, the light-emitting unit 234 can also be automatically lit when the trigger switch 221 is turned on.

[0058] The notification unit 211 can be implemented as an LED, for example. The notification unit 211 can be located at the opposite end of the cylinder 21 of the body 20, opposite the output shaft 241, so that it can be easily seen by the user during operation (see reference). Figure 2 (B).

[0059] In this case, the tool 2 according to this embodiment has at least a work mode and a registration mode as its operating modes. As used herein, "work mode" refers to the operating mode in which the user uses the tool 2 to perform a work. Registration mode refers to the operating mode in which a reference image corresponding to the work object is generated. The operating mode can be switched, for example, by pressing the push button switch 232 and other components of the operation panel 231. Alternatively, the operating mode can also be switched by operating another component (such as the trigger switch 221 or the DIP switch) that is separate from the operation panel 231.

[0060] Control unit 3a may, for example, include a microcontroller comprising one or more processors and one or more memories as its main constituent element. The microcontroller performs the functions of control unit 3a by causing one or more processors to execute programs stored in one or more memories. The programs may be pre-stored in the memory. Alternatively, the programs may be distributed after being stored on a non-transitory storage medium such as a memory card or downloaded via a telecommunications line. In other words, the programs are designed to cause one or more processors to function as control unit 3a.

[0061] Control unit 3a performs functions such as driving control unit 31, notification control unit 36, and torque determination unit 37. Note that if no operation command is input to trigger switch 221 or operation panel 231 within a certain period of time, control unit 3a enters sleep mode. When any operation command is input to trigger switch 221 or operation panel 231 during sleep mode, control unit 3a is activated.

[0062] The drive control unit 31 controls the drive unit 24. Specifically, the drive control unit 31 activates the drive unit 24 to cause the output shaft 241 to rotate at a rotational speed determined by the pressing depth of the trigger switch 221 and in a rotational direction set by the forward / reverse switch 222.

[0063] The drive control unit 31 also controls the drive unit 24 such that the tightening torque becomes equal to the torque setting value. The drive control unit 31 has a torque estimation function for estimating the magnitude of the tightening torque. In this embodiment, the drive control unit 31 estimates the magnitude of the tightening torque based on the rotational speed of the drive unit 24 (motor) or any other parameter until the estimated value of the tightening torque reaches the positioning judgment level. When the estimated value of the tightening torque reaches the positioning judgment level, the drive control unit 31 estimates the magnitude of the tightening torque based on the number of strokes of the impact mechanism 25. When it is found that the number of strokes of the impact mechanism 25 has reached a threshold number based on the torque setting value, the drive control unit 31 determines that the tightening torque should have reached the torque setting value and stops operating the drive unit 24 (i.e., the motor). This allows the tool 2 to tighten the fastening member with a tightening torque that accurately matches the torque setting value.

[0064] Notification unit 36 ​​controls notification unit 211. Notification control unit 36 ​​preferably illuminates notification unit 211 in different ways depending on whether the determination made by processing unit 35 is inconsistent or consistent. For example, if the determination made by processing unit 35 is inconsistent, notification control unit 36 ​​can illuminate notification unit 211 in red. On the other hand, if the determination made by processing unit 35 is consistent, notification control unit 36 ​​can illuminate notification unit 211 in green. This allows the user to visually check the illumination status of notification unit 211 to identify whether the work object conforms to the work process. Optionally, when trigger switch 221 is pulled in a state where the determination made by processing unit 35 is inconsistent, notification control unit 36 ​​can illuminate notification unit 211.

[0065] The torque determination unit 37 is configured to determine whether the tightening torque is a normal tightening torque when the fastening member is attached to the part to be tightened. In this case, the torque determination unit 37 preferably determines whether the tightening torque is a normal tightening torque based on the work instruction defined by the work process. Specifically, the work instruction defined by the work process includes a target torque value associated with the work object. This allows the torque determination unit 37 to determine whether the work is being performed with the tightening torque specified by the work instruction by comparing the target torque value included in the work instruction with the tightening torque.

[0066] If, for example, the drive control unit 31 deactivates the drive unit 24 when the number of strokes of the impact mechanism 25 reaches a threshold number, the torque determination unit 37 determines that the tightening torque should be normal. On the other hand, if, for example, the drive control unit 31 deactivates the drive unit 24 by disconnecting the trigger switch 221 before the number of strokes of the impact mechanism 25 reaches the threshold number, the torque determination unit 37 determines that the tightening torque should be insufficient (abnormal). The torque determination unit 37 also performs result storage processing to associate the determination result with the part to be tightened and store it in the result storage unit 43.

[0067] (2.2.2) Structure of the work object identification system

[0068] Next, refer to Figure 1 , Figure 2 A and Figure 2 Let's use B to illustrate the structure of the work object recognition system 10. The work object recognition system 10 includes a camera unit 5, a control unit 3b, a storage unit 4, a posture detection unit 26, a distance measurement unit 27, and a pressure detection unit 28.

[0069] The control unit 3b, storage unit 4, camera unit 5, posture detection unit 26, distance measurement unit 27, and pressure detection unit 28 are housed in the body 20 of the tool 2. In this embodiment, the camera unit 5 and distance measurement unit 27 may be housed, for example, in the cylinder 21. The pressure detection unit 28 is housed in the portion of the grip 22 located closer to the rear (i.e., opposite to the side with the trigger switch 221). The control unit 3b, storage unit 4, and posture detection unit 26 are housed in the grip 22 or the attachment portion 23.

[0070] The camera unit 5 generates data as a captured image. The camera unit 5 may be, for example, a camera including an image sensor and a lens. In this embodiment, the camera unit 5 may be housed in the body 20 (cylinder 21) of the tool 2. The camera unit 5 is positioned facing the front end of the output shaft 241 to capture an image of the work object while the user is using the tool 2 to perform the work.

[0071] Specifically, the camera unit 5 is disposed in the front end of the cylinder 21 facing the front end of the output shaft 241 (i.e., facing the socket 242), such that the socket 242 attached to the output shaft 241 falls within the camera's field of view (see reference). Figure 2 A and Figure 2 (B) The optical axis of the imaging unit 5 is arranged to be aligned with the rotation axis Ax1 of the output shaft 241. In this embodiment, the imaging unit 5 is arranged such that its optical axis is within a predetermined distance from the rotation axis Ax1 of the output shaft 241, and the rotation axis Ax1 and the optical axis are substantially parallel to each other. Note that the imaging unit 5 does not need to generate an image such that the socket 242 attached to the output shaft 241 falls within its imaging range. Instead, the imaging unit 5 only needs to generate an image for identifying the current work object. As used herein, "image for identifying the current work object" refers to an image generated when the imaging unit 5 captures a workpiece in the state where the tool 2 is currently positioned on the work object. According to the invention, it is assumed that the work object in the position where the tool 2 is positioned is captured in the image. The image only needs to be one that allows the user to identify the current work object. Therefore, the work object in the position where the tool 2 is currently positioned does not need to be included within the imaging range of the image.

[0072] The posture detection unit 26 detects the posture of the tool 2. The posture detection unit 26 may include, for example, a motion sensor 261 such as an accelerometer or a gyroscope. In this embodiment, as described above, the posture detection unit 26 is housed within the body 20 (grip 22 or attachment 23) of the tool 2. In this embodiment, the posture detection unit 26 includes, for example, a three-axis accelerometer and a three-axis gyroscope as motion sensors 261. The three-axis accelerometer detects acceleration on each of the three mutually perpendicular axes and outputs an electrical signal representing the detected acceleration. The three-axis gyroscope detects angular velocity around each of the three mutually perpendicular axes and outputs an electrical signal representing the thus detected angular velocity.

[0073] The posture detection unit 26 can detect the direction of gravity based on, for example, the output of an accelerometer, and detect, for example, the posture of tool 2 by referencing the direction of gravity. Additionally, for example, the posture detection unit 26 can also detect the angular velocity of tool 2 moving during rotation based on the output of a gyroscope sensor, and can also detect the rotation angle of tool 2 based on the integral of the angular velocity. For example, the posture detection unit 26 can detect, in a mutually distinguishable manner, the posture of tool 2 with the grip 22 protruding from the cylinder 21 downwards (corresponding to the direction of gravity) and the posture of tool 2 with the grip 22 protruding from the cylinder 21 upwards. As used herein, "posture of tool 2" refers, for example, to the posture of tool 2 determined by various rotation angles (e.g., tilt, pitch, and yaw angles) around three axes referenced to the direction of gravity. The posture detection unit 26 detects the movement and posture of the tool 2 based on the output of the motion sensor 261 (including an accelerometer and a gyroscope sensor), and provides the detection results as posture information related to the movement and posture of the tool 2 to the setting state detection unit 34 of the control unit 3b as needed.

[0074] The distance measuring unit 27 measures the distance between the tool 2 and the work object. The distance measuring unit 27 includes, for example, a distance sensor 271, such as a radio detection and ranging (RADAR) sensor, a light detection and ranging (LiDAR) sensor, or an ultrasonic sensor. For example, the LiDAR sensor could be an infrared sensor. In this embodiment, the distance measuring unit 27 is housed within the body 20 (cylinder 21) of the tool 2 as described above. Specifically, the distance measuring unit 27 and the camera unit 5 are disposed in the front end of the cylinder 21, facing the front end of the output shaft 241 (i.e., facing the socket 242). In this embodiment, the distance measuring unit 27 includes, for example, an ultrasonic sensor. The ultrasonic sensor is a time-of-flight distance sensor used to measure the distance to an object (such as a workpiece or work object) by emitting ultrasonic waves and measuring the time taken to receive the ultrasonic waves reflected from the object. The ultrasonic sensor outputs an electrical signal representing the distance thus measured.

[0075] The distance measurement unit 27 detects the distance between the work object (or workpiece) and the tool 2 based on the output of the distance sensor 271. The distance measurement unit 27 outputs the detection result as distance information related to the distance between the tool 2 and the work object to the setting status detection unit 34 of the control unit 3b.

[0076] The pressure detection unit 28 detects that the tool 2 is pressed against the work object. In this embodiment, as described above, the pressure detection unit 28 is housed in the portion of the body 20 (grip 22) of the tool 2 closer to the back surface. The pressure detection unit 28 according to this embodiment includes a pressure sensor 281, for example, a metal strain gauge or a semiconductor strain gauge. The pressure sensor 281 detects the pressure applied to the back surface of the grip 22 and outputs an electrical signal representing the pressure thus detected.

[0077] The pressure detection unit 28 detects that the tool 2 is pressed against the work object based on the output of the pressure sensor 281. In this case, for example, the force applied to the back of the grip 22 when the user is pressing the tool 2 against the work object is greater than the force applied to the back of the grip 22 when the user is carrying the tool 2. Therefore, the pressure detection unit 28 detects that the tool 2 is pressed against the work object when it finds that the pressure detected by the pressure sensor 281 is equal to or greater than a threshold pressure. The pressure detection unit 28 outputs the detection result as information related to the pressing of the tool 2 to the setting state detection unit 34 of the control unit 3b.

[0078] Control unit 3b may, for example, include a microcontroller comprising one or more processors and one or more memories as its main constituent element. The microcontroller performs the functions of control unit 3b by causing one or more processors to execute programs stored in one or more memories. The programs may be pre-stored in the memory. Alternatively, the programs may be distributed after being stored on a non-transitory storage medium such as a memory card or downloaded via a telecommunications line. In other words, the programs are designed to cause one or more processors to function as control unit 3b.

[0079] Control unit 3b performs functions such as those of camera control unit 32, stability determination unit 33, setting status detection unit 34, processing unit 35, and registration unit 38. Note that if no operation command is input to trigger switch 221 or operation panel 231 within a certain time period, control unit 3b enters sleep mode. Control unit 3b is activated when any operation command is input to trigger switch 221 or operation panel 231 during sleep mode.

[0080] The camera control unit 32 is configured to control the camera unit 5. When the control unit 3b is activated, the camera control unit 32 according to this embodiment causes the camera unit 5 to begin camera operation.

[0081] The stability determination unit 33 determines whether the captured image generated by the camera unit 5 is stable. According to this embodiment, the stability determination unit 33 performs stability determination processing when the tool 2 is operating in the work mode. This stability determination processing is used to determine whether the captured image is stable based on multiple frames included in the captured image.

[0082] According to this embodiment, the stability determination unit 33 calculates the difference between multiple frames, and determines that the captured image is stable when the difference is equal to or less than a threshold. Specifically, the stability determination unit 33 calculates the difference between the latest frame (current frame) included in the captured image and the previous frame before the latest frame. In the following description, the latest frame included in the captured image will sometimes be referred to as the "first frame", and the frame before the latest frame will sometimes be referred to as the "second frame". The stability determination unit 33 calculates the difference as the difference between the brightness value (which may be a density value or a grayscale value) in a specific area of ​​the first frame and the brightness value (which may be a density value or a grayscale value) in the corresponding specific area of ​​the second frame. The stability determination unit 33 uses, for example, the sum of squared differences (SSD) or the sum of absolute differences (SAD) to calculate the difference. In this case, the specific area in the first frame and the second frame may be, for example, a region predefined by coordinates in the captured image. The specific area in the first frame and the specific area in the second frame have the same set of coordinates. In addition, the number of specific regions (one or more) defined in the first and second frames needs to be at least one, but preferably multiple, to improve the accuracy of stability determination processing.

[0083] The stability determination unit 33 compares the difference degree with a threshold, and determines that the captured image is stable when the difference degree is equal to or less than the threshold. When the captured image is determined to be stable, the stability determination unit 33 outputs stability information to the setting state detection unit 34 and the processing unit 35. On the other hand, when the difference degree is found to be greater than the threshold, the stability determination unit 33 does not determine that the captured image is stable. When the captured image is not determined to be stable, the stability determination unit 33 does not output stability information to the setting state detection unit 34 or the processing unit 35.

[0084] The setting status detection unit 34 detects whether the tool 2 is set in an appropriate position on the work object. According to this embodiment, the setting status detection unit 34 performs setting status detection processing to determine whether the tool 2 is set in an appropriate position on the work object when the tool 2 is operating in work mode.

[0085] According to this embodiment, the setting state detection unit 34 detects the state in which the tool 2 is set in an appropriate position on the work object based on the posture information provided by the posture detection unit 26, the distance information provided by the distance measurement unit 27, and the pressing information provided by the pressing detection unit 28.

[0086] The setting state detection unit 34 detects the state in which the tool 2 is positioned appropriately on the work object based on the posture information provided by the posture detection unit 26. If the posture of the tool 2 detected by the posture detection unit 26 is a predetermined posture, the setting state detection unit 34 detects that the tool 2 is positioned appropriately on the work object. As used herein, "predetermined posture" may refer to, for example, the posture of the tool 2 such that, when compared with a reference posture, the angle difference between the posture of the tool 2 and the reference posture is equal to or less than a threshold. Specifically, the predetermined posture refers to the posture of the tool 2 such that the sum or average of the differences between the rotation angles of the tool 2 detected by the posture detection unit 26 relative to its current posture around the three axes and the rotation angles of the tool 2 defined relative to the reference posture around the three axes are equal to or less than a threshold. Furthermore, as used herein, "reference posture" refers to the posture of the tool 2 in which the grip 22 protrudes downward from the cylinder 21 (corresponding to the direction of gravity). According to this embodiment, when the average value of the difference between the rotation angle of tool 2 detected by posture detection unit 26 relative to its current posture around three axes and the rotation angle of tool 2 defined by the reference posture around three axes is equal to or less than 5 degrees, the setting state detection unit 34 determines that tool 2 has a predetermined posture. Note that "the average value of the difference between the rotation angle of tool 2 detected by posture detection unit 26 relative to its current posture around three axes and the rotation angle of tool 2 defined by the reference posture around three axes" will sometimes be simply referred to as "the angular difference between the current posture of tool 2 and its reference posture" in the following text. That is, according to this embodiment, when the average value of the difference between the rotation angle of tool 2 detected by posture detection unit 26 relative to its current posture around three axes and the rotation angle of tool 2 defined by the reference posture around three axes is equal to or less than 5 degrees, the setting state detection unit 34 detects that tool 2 is set in an appropriate position on the work object. Alternatively, the state detection unit 34 may be configured to detect the state in which the tool 2 is positioned on the work object, under one of the following conditions: the average difference between the rotation angle of the tool 2 detected by the posture detection unit 26 relative to its current posture around the three axes and the rotation angle of the tool 2 defined relative to the reference posture around the three axes is equal to or less than 5 degrees.

[0087] Additionally, the setting state detection unit 34 also detects the state in which the tool 2 is positioned appropriately on the work object based on the distance information provided by the distance measurement unit 27. Specifically, the setting state detection unit 34 detects that the tool 2 is positioned appropriately on the work object when it finds that the distance between the tool 2 and the work object detected by the distance measurement unit 27 falls within a preset range. Alternatively, the setting state detection unit 34 may set one of the following conditions for detecting the state in which the tool 2 is positioned appropriately on the work object: the distance between the tool 2 and the work object falls within a preset range. As used herein, the case where "the distance between the tool 2 and the work object falls within a preset range" means that the absolute value of the difference calculated by subtracting the distance between the tool 2 and the work object detected by the distance measurement unit 27 from the reference distance is equal to or less than a threshold distance. As used herein, "reference distance" refers to a distance that is defined as a reference for the setting state detection unit 34 to detect the state in which the tool 2 is positioned appropriately on the work object. The reference distance may, for example, be the distance between the tool 2 and the work object detected by the distance measurement unit 27 when a reference image is captured, and is associated with the reference image. Furthermore, the reference distance can be slightly longer than the distance between the distance sensor 271 of the distance measuring unit 27 and the front end of the socket 242. Note that the absolute value of the difference calculated by subtracting the distance between the tool 2 and the work object detected by the distance measuring unit 27 from the reference distance will sometimes be simply referred to as the "distance difference" below.

[0088] Furthermore, the setting state detection unit 34 also detects the state in which the tool 2 is positioned appropriately on the work object based on the pressing information provided by the pressing detection unit 28. Specifically, the setting state detection unit 34 detects that the tool 2 is positioned appropriately on the work object when it finds that the value of the pressure applied to the back of the grip portion 22, detected by the pressing detection unit 28, is equal to or greater than a threshold pressure. Alternatively, the setting state detection unit 34 may set one of the following conditions for detecting that the tool 2 is positioned appropriately on the work object: the value of the pressure applied to the back of the grip portion 22 is equal to or greater than a threshold pressure.

[0089] Furthermore, according to this embodiment, the setting state detection unit 34 also detects the state in which the tool 2 is set in the appropriate position on the work object based on the pressing depth of the trigger switch 221. Specifically, the setting state detection unit 34 detects that the tool 2 is set in the appropriate position on the work object when it finds that the trigger switch 221 has been half-pressed by the user. As used herein, the phrase "half-pressed" refers to the state in which the trigger switch 221 is half-pressed between the initial position and the on position. Specifically, in this document, "half-pressed" means that the trigger switch 221 has been pressed to approximately the middle level between the initial position and the on position. The setting state detection unit 34 detects that the tool 2 is set in the appropriate position on the work object when it finds that the trigger switch 221 has been half-pressed between the initial position and the on position. Alternatively, the setting state detection unit 34 may set one of the following conditions for detecting that the tool 2 is set in the appropriate position on the work object: the trigger switch 221 has been half-pressed between the initial position and the on position.

[0090] Furthermore, according to this embodiment, the setting state detection unit 34 also detects that the tool 2 is positioned appropriately on the work object when it obtains stability information from the stability determination unit 33 (i.e., when the stability determination unit 33 determines that the captured image is stable). Alternatively, the setting state detection unit 34 may set one of the following conditions for detecting that the tool 2 is positioned appropriately on the work object: the stability determination unit 33 determines that the captured image is stable.

[0091] When the tool 2 is detected to be in the appropriate position on the work object, the setting status detection unit 34 outputs setting status detection information to the processing unit 35. On the other hand, unless the setting status detection unit 34 detects that the tool 2 is in the appropriate position on the work object, the setting status detection unit 34 does not output setting status detection information to the processing unit 35.

[0092] According to this embodiment, the setting state detection unit 34 detects that the tool 2 is positioned appropriately on the work object when it finds that the current posture of the tool 2 is a predetermined posture, the distance difference is equal to or less than a threshold distance, the pressure applied to the back of the grip 22 is equal to or greater than a threshold pressure, the stability determination unit 33 determines that the captured image is stable, and the trigger switch 221 is half-pressed. Note that, according to this embodiment, the setting state detection unit 34 does not detect that the tool 2 is positioned appropriately on the work object when it does not find that the current posture of the tool 2 is a predetermined posture. Furthermore, according to this embodiment, the setting state detection unit 34 does not detect that the tool 2 is positioned appropriately on the work object when it finds that the distance difference is greater than a threshold distance. Furthermore, according to this embodiment, the setting state detection unit 34 does not detect that the tool 2 is positioned appropriately on the work object when it finds that the pressure applied to the back of the grip 22 is less than a threshold pressure. Moreover, unless the stability determination unit 33 determines that the captured image is stable, the setting state detection unit 34 does not detect that the tool 2 is positioned appropriately on the work object. Furthermore, unless the trigger switch 221 is half-pressed, the setting state detection unit 34 according to this embodiment does not detect that the tool 2 is set in the appropriate position on the work object.

[0093] According to this embodiment, the processing unit 35 performs predetermined processing based on the captured image when it receives at least one of the stability information provided by the stability determination unit 33 and the setting state detection information provided by the setting state detection unit 34. In other words, the processing unit 35 performs recognition processing based on the captured image when the work object is highly likely to be successfully identified. If the processing unit 35 starts recognition processing too early, not only will the work object fail to be identified, but the recognition processing will also fail to start at the optimal time when the user is ready to start working with the tool 2 held in his or her hand. This is because the recognition processing by the processing unit 35 takes 0.5 to 1.0 seconds. Unless the recognition processing can start when the user is ready to start working with the tool 2 held in his or her hand, it will cause a significant delay when the recognition processing ends, which may interfere with the user's work rhythm. Conversely, according to this embodiment, the processing unit 35 can perform recognition processing when the work object is highly likely to be successfully identified (i.e., at the optimal time when the user is ready to start working with the tool 2 held in his or her hand), thereby reducing the possibility of a significant delay when the recognition processing ends. Furthermore, recognition processing can be performed based on images captured by the camera unit 5 that have undergone stable camera control, such as automatic exposure (AE) or automatic white balance (AWB), thereby helping to improve the accuracy of the recognition processing. Note that if the processing unit 35 according to this embodiment does not receive at least one of the stability information provided by the stability determination unit 33 and the setting state detection information provided by the setting state detection unit 34, the processing unit 35 will not perform the predetermined processing based on the captured image.

[0094] The processing unit 35 intermittently performs recognition processing to identify the current work object in which tool 2 is currently positioned among multiple work objects, as a predetermined process. That is, the processing unit 35 has the function of identifying the current work object captured in the captured image. Specifically, the processing unit 35 performs image processing to compare the captured image taken by the camera unit 5 with multiple reference images, thereby identifying the current work object captured in the captured image among multiple work objects. In this case, the multiple reference images are stored in the storage unit 4 (image storage unit 41).

[0095] Specifically, the processing unit 35 uses multiple reference images corresponding to multiple work objects as template data to perform pattern recognition processing on the captured image, thereby identifying the current work object. In other words, the processing unit 35 identifies the current work object captured in the captured image by comparing the captured image with the multiple reference images corresponding to the multiple work objects.

[0096] As used herein, "pattern recognition processing" refers to image processing used to identify what an object in an image is based on its shape. Examples of this type of pattern recognition processing include pattern matching processing and processing for identifying objects in an image using a learned model created through machine learning. Pattern matching processing, as used herein, refers to processing that compares template data with objects (such as captured images) using the template data described above. Furthermore, any suitable algorithm can be used in the methods used for machine learning. For example, deep learning algorithms can be employed.

[0097] Furthermore, if the identified work object does not conform to the work instructions defined by the work process, the processing unit 35 performs at least one of imposing restrictions on the operation of the drive unit 24 and issuing a notification. In other words, the processing unit 35 determines whether the work object identified by the processing unit 35 (i.e., the current work object) conforms to the work instructions defined by the preset work process. That is, the processing unit 35 determines whether the work object identified by the processing unit 35 is consistent with the work object specified by the work instructions included in the work process.

[0098] Specifically, processing unit 35 retrieves data of the job process associated with the current job object from process storage unit 44 of storage unit 4. Then, processing unit 35 determines whether the job object receiving the current job instruction defined by the job process retrieved from process storage unit 44 matches the identified job object. If these job objects match, processing unit 35 determines that the identified job object should conform to the job instruction defined by the job process. On the other hand, if these job objects do not match, processing unit 35 determines that the identified job object should not conform to the job instruction defined by the job process.

[0099] When, as a result of such judgment, it is determined that the work object identified thereby does not conform to the work instructions defined by the work process, the processing unit 35 performs at least one of imposing restrictions on the operation of the drive unit 24 and issuing a notification. As used herein, for example, "notification" refers not only to directly notifying the user through the notification unit 211 of the tool system 1, but also to indirectly notifying the user via an external terminal (such as a mobile communication device).

[0100] Specifically, when it is determined that the identified work object does not conform to the work instructions defined by the work process, the processing unit 35 will not allow the drive unit 24 to be activated even if the trigger switch 221 is pulled. That is, the drive unit 24 is only allowed to be activated when the processing unit 35 determines that the identified work object conforms to the work instructions defined by the work process. Therefore, even if the tool 2 is currently positioned appropriately on a work object that does not conform to the work process, the drive unit 24 remains inactive, thereby prohibiting tightening operations. This reduces the possibility of performing work during incorrect work processes. Optionally, when it is determined that the identified work object does not conform to the work instructions defined by the work process, the processing unit 35 may lock the trigger switch 221 to prevent the user from pulling the trigger switch 221 in this situation.

[0101] Furthermore, when it is determined that the identified work object should not conform to the work instructions defined by the work process, the processing unit 35 causes the notification control unit 36 ​​to activate the notification unit 211. Therefore, the notification unit 211 serves as a user notification unit, which is used to notify the user that the tool 2 is now positioned appropriately on the work object that does not conform to the work process.

[0102] In other words, when processing unit 35 receives at least one of the stability information provided by stability determination unit 33 and the setting status detection information provided by setting status detection unit 34, it performs at least one identification process to identify the current work object as a predetermined process. Additionally, processing unit 35 performs a process judgment process as a predetermined process to compare the thus identified work object with the work instructions defined by the work process and thereby determine their correspondence. If the result of the process judgment process indicates that the work object does not conform to the work instructions, processing unit 35 restricts and / or notifies the operation of drive unit 24.

[0103] If the operation mode of tool 2 is registration mode, then registration unit 38 performs image registration processing to store multiple reference images in image storage unit 41 of storage unit 4 and torque registration processing to store multiple target torque values ​​in torque storage unit 42 of storage unit 4.

[0104] Additionally, during image registration processing, for example, the registration unit 38 causes the image storage unit 41 to store a still image generated by the camera unit 5 capturing the work object as a reference image. Specifically, if the tool 2's operating mode is registration mode, the trigger switch 221 also functions as a shutter release button. When the trigger switch 221 is turned on (i.e., pressed to the on position), the camera unit 5 generates a still image. The registration unit 38 then causes the image storage unit 41 to store this still image as a reference image.

[0105] Storage unit 4 can be implemented, for example, as a semiconductor memory, and performs the functions of image storage unit 41, torque storage unit 42 (target value storage unit), result storage unit 43, and process storage unit 44. In this embodiment, image storage unit 41, torque storage unit 42, result storage unit 43, and process storage unit 44 are implemented as a single memory. However, this is merely an example and should not be construed as limiting. Alternatively, these storage units 41, 42, 43, and 44 can also be implemented as multiple memories. Alternatively, storage unit 4 can also be implemented as a storage medium such as a memory card that can be attached to and removed from tool 2.

[0106] Image storage unit 41 stores multiple reference images associated with multiple job objects.

[0107] Torque storage unit 42 stores multiple target torque values ​​(target values) in a one-to-one association with multiple work objects. As used herein, "target torque value" refers to the target value of the fastening torque when the fastening member is attached to the associated work object.

[0108] The result storage unit 43 stores the determination results obtained by the torque determination unit 37 for multiple parts to be tightened, associated with multiple work objects. It is recommended that the result storage unit 43 store the determination results obtained by the torque determination unit 37 with a timestamp indicating the operation time. This allows for differentiation of work object determination results for each workpiece on the assembly line.

[0109] Process storage unit 44 stores data related to a single job process or multiple job processes. As mentioned above, a job process means a process that assumes the use of tool 2 to perform a job, and may, for example, be data used to define the order in which multiple job objects of a single workpiece should be performed.

[0110] (3) Operation

[0111] Next, refer to Figures 3 to 6 To illustrate exemplary operation of the tool system 1 according to this embodiment.

[0112] The following example illustrates how tool system 1 operates when a user is assembling multiple workpieces A1 in an assembly line. Assume each workpiece A1 has two work objects (hereinafter referred to as "first work object" and "second work object"), and assume the user uses tool 2 to attach fastening components to each of these work objects.

[0113] (3.1) Registration Mode

[0114] First, refer to Figure 3To illustrate the exemplary operation of tool system 1 in registration mode, we assume that tool 2 is in an initial state in which registration unit 38 has not yet performed image registration processing and torque registration processing. That is, in tool 2 in the initial state, the first reference image and the second reference image corresponding to the first work object and the second work object, respectively, as well as the first target torque value and the second target torque value, are not stored in image storage unit 41 or torque storage unit 42.

[0115] The user sets the operation mode of tool 2 to registration mode (in S1). Next, when the fastening component is attached to the first work object, the user operates the operation panel 231 to input the torque value for fastening (in S2). The drive control unit 31 sets the input torque value to the torque setting value for the first work object. Then, the user performs the fastening operation by pulling the trigger switch 221 to attach the fastening component to the first work object (in S3). At this time, a still image of the first work object is captured.

[0116] Upon completion of the fastening operation, the registration unit 38 performs registration processing (including image registration processing and torque registration processing) (in S4). Specifically, the registration unit 38 performs image registration processing, causing the image storage unit 41 to store a still image of the first work object generated during the fastening operation in step S3 as a first reference image corresponding to the first work object. Additionally, the registration unit 38 performs torque registration processing, which causes the torque storage unit 42 to store the torque setting value when the fastening member is attached to the first work object during the fastening operation in step S3 as a first target torque value associated with the first work object. In other words, the first target torque value is associated with the first reference image.

[0117] Specifically, according to this embodiment, the processing unit 35 performs process determination processing. Therefore, in the registration process, the target torque value is registered as included in the work instruction. In other words, in the registration process, the work process is registered. In this example, the registration unit 38 registers the work process such that the work instruction indicating the work to be performed on the first work object becomes the first work instruction in the work process. Specifically, the registration unit 38 registers the work instruction indicating the work to be performed on the first work object and including the first target torque value as the work procedure step to be performed "first" according to the work process.

[0118] The torque determination unit 37 performs a result storage process, which enables the result storage unit 43 to store a first determination result indicating whether the fastening torque when the fastening member is attached to the first work object is a normal fastening torque in association with the first work object (in S5).

[0119] In addition, the user follows the same work process as the first work object to perform the fastening operation on the second work object. Specifically, when the fastening component is attached to the second work object, the user operates the operation panel 231 to input the torque value of the fastening torque (in S6), and then performs the fastening operation of attaching the fastening component to the second work object (in S7). At this time, a still image of the second work object is generated, and the registration unit 38 performs registration processing (including image registration processing and torque registration processing) (in S8). The registration unit 38 registers the operation instruction indicating the operation to be performed on the second work object and including the second target torque value as the operation step to be performed "secondarily" according to the work process. The torque determination unit 37 performs result storage processing (in S9), which is used to make the result storage unit 43 store a second determination result indicating whether the fastening torque during the fastening operation in step S7 is a normal fastening torque.

[0120] When registering each work object of workpiece A1, the user operates the operation panel 231 to switch the operation mode of tool 2 from registration mode to work mode (in S10). Switching the operation mode of tool 2 from registration mode to work mode ends the registration mode.

[0121] Notice, Figure 3 The sequence shown is merely an example. Therefore, Figure 3 The processing steps shown can be performed in different orders as appropriate, additional processing steps can be added as needed, or at least one of the processing steps can be omitted as appropriate.

[0122] (3.2) Work Mode

[0123] Next, refer to Figures 4 to 6 This illustrates an exemplary operation of tool system 1 in the work mode.

[0124] Whenever the frame of the captured image by camera unit 5 is refreshed, tool system 1 performs... Figures 4 to 6 The process is as shown. When refreshing the frame of the captured image, the setting state detection unit 34 obtains pose information from the pose detection unit 26 (in S21). Next, the setting state detection unit 34 checks whether the (operation) state of the processing unit 35 is state St0 (in S22). As used herein, "state St0" refers to the idle state of the processing unit 35 where recognition processing has not yet started. On the other hand, the state "St1" used here refers to the standby state of the processing unit 35 where recognition processing has not yet started but can begin. Furthermore, the state "St2" used here refers to the state where the processing unit 35 is performing recognition processing.

[0125] If the state of processing unit 35 is state St0 (if the answer is "yes" in S22), then setting state detection unit 34 compares the acceleration of tool 2 with acceleration threshold T1 based on posture information (in S23). In this case, the acceleration threshold T1 according to this embodiment is approximately equal to zero. That is, setting state detection unit 34 checks whether tool 2 is moving at least slightly. In other words, setting state detection unit 34 checks, for example, whether tool 2 is not placed on a table or floor. Obviously, the state of tool 2 being placed on a table or floor is different from the state of tool 2 being properly positioned on the work object. When it is found that the acceleration of tool 2 is greater than threshold T1 (if the answer is "yes" in S23), setting state detection unit 34 compares the acceleration of tool 2 with another acceleration threshold T3 based on posture information (in S24). In this case, the acceleration threshold T3 according to this embodiment is a value greater than T1, and is set to a value close to the acceleration of tool 2 when the user is moving while carrying tool 2 (i.e., when the user is shaking tool 2). Clearly, the state in which the user is moving while carrying tool 2 or the state in which the user is shaking tool 2 is not the state in which tool 2 is set in the appropriate position on the work object.

[0126] When the acceleration of tool 2 is found to be less than the threshold T3 (if the answer is "yes" in S24), the state detection unit 34 is set to compare the angle difference between the current posture and the reference posture of tool 2 with the angle difference threshold T5 based on the posture information (in S25). In this embodiment, the angle difference threshold T5 may be, for example, 10 degrees. If the angle difference between the current posture and the reference posture of tool 2 is less than the threshold T5 (if the answer is "yes" in S25), the LED of the light-emitting unit 234 used for shooting is lit (in S26). Then, the state of the processing unit 35 changes to state St1 (i.e., standby state) (S27). In this case, processing enters... Figure 6 The processing step S65 is shown. Note that if the acceleration of tool 2 is equal to or less than threshold T1 (if the answer in S23 is "No"), if the acceleration of tool 2 is equal to or greater than threshold T3 (if the answer in S24 is "No"), or if the angle difference between the current pose and the reference pose of tool 2 is equal to or greater than threshold T5 (if the answer in S25 is "No"), then processing proceeds to step S65. Figure 6 The processing step S65 is shown.

[0127] On the other hand, if it is proven in step S22 that the state of processing unit 35 is not state St0, but state St1 or state St2 (i.e., if the answer in S22 is "no"), then state detection unit 34 is set to compare the acceleration of tool 2 with acceleration threshold T2 based on posture information (in S31). If it is found that the acceleration of tool 2 is greater than threshold T2 (if the answer in S31 is "yes"), state detection unit 34 is set to compare the acceleration of tool 2 with another acceleration threshold T4 based on posture information (in S32). If it is found that the acceleration of tool 2 is less than threshold T4 (if the answer in S32 is "yes"), state detection unit 34 is set to compare the angle difference between the current posture and the reference posture of tool 2 with the angle difference threshold T6 based on posture information (in S33). If the angle difference between the current posture and the reference posture of tool 2 is less than the angle difference threshold T6 (if the answer in S33 is "yes"), then processing proceeds. Figure 5 The processing step S41 is shown. Note that if the acceleration of tool 2 is equal to or less than the threshold T2 (if the answer in S31 is "No"), if the acceleration of tool 2 is equal to or greater than the threshold T4 (if the answer in S32 is "No"), or if the angle difference between the current pose and the reference pose of tool 2 is equal to or greater than the threshold T6 (if the answer in S33 is "No"), then the processing proceeds to the processing step S34.

[0128] In this situation, if the state of processing unit 35 is state St1 or state St2, the LED of the light-emitting unit 234 used for shooting is in the ON state. When the LED in the ON state is turned off (in S34), the state of processing unit 35 changes to state St0 (i.e., idle state) (in S35). Then, the state of the motor included in drive unit 24 changes to state St3 (in S36). As used herein, "state St3" refers to a state in which the motor of drive unit 24 is prohibited from running even if the user pulls the trigger switch 221. After the state of the motor has changed to state St3, processing proceeds... Figure 6 The processing step S65 is shown.

[0129] Note that, according to this embodiment, acceleration thresholds T1 and T2 are each set to have hysteresis, and threshold T2 is less than threshold T1. Similarly, acceleration thresholds T3 and T4 are also set to have hysteresis, and threshold T4 is greater than threshold T3. Likewise, angle difference thresholds T5 and T6 are also set to have hysteresis, and threshold T6 is greater than threshold T5.

[0130] Next, refer to Figure 5The processing steps S41 to S56 will be explained. The stability determination unit 33 checks the latest frame (first frame) of the captured image (in S41). Meanwhile, the setting state detection unit 34 obtains distance information from the distance measurement unit 27 (in S42) and press information from the press detection unit 28 (in S43). Next, the setting state detection unit 34 checks whether the state of the processing unit 35 is state St1 (in S44). If the state of the processing unit 35 is state St1 (if the answer in S44 is "yes"), the setting state detection unit 34 compares the angle difference between the current posture of the tool 2 and the reference posture with the angle difference threshold T7 based on the posture information (in S45). In this case, the angle difference threshold T7 according to this embodiment may be, for example, 5 degrees. When it is found that the angle difference between the current posture of the tool 2 and the reference posture is less than the angle difference threshold T7 (if the answer in S45 is "yes"), the stability determination unit 33 calculates the degree of difference between the latest frame of the captured image and the previous frame (second frame) preceding the latest frame. Then, the stability determination unit 33 compares its calculated difference degree with the difference degree threshold T9 (in S46). If it finds that its calculated difference degree is less than the threshold T9 (if the answer in S46 is "yes"), the stability determination unit 33 outputs stability information to the setting state detection unit 34 and the processing unit 35. Next, the setting state detection unit 34 calculates the distance difference based on the distance information and compares the distance difference with the threshold distance T11 (in S47). If it finds that the distance difference is less than the threshold distance T11 (if the answer in S47 is "yes"), the setting state detection unit 34 compares the pressure applied to the back of the grip 22 with the threshold pressure T13 based on the pressing information (in S48). If it finds that the pressure applied to the back of the grip 22 is greater than the threshold pressure T13 (if the answer in S48 is "yes"), the setting state detection unit 34 checks whether the trigger switch 221 has been half-pressed (in S49). Upon detecting that the trigger switch 221 has been half-pressed (if the answer in S49 is "yes"), the setting state detection unit 34 outputs setting state detection information to the processing unit 35. Then, the state of the processing unit 35 changes to state St2 (i.e., the state where the processing unit 35 is performing identification processing) (in S50), and processing begins... Figure 6 The processing step S61 is shown.

[0131] Note that if the angle difference is equal to or greater than threshold T7 (if answered "No" in S45), if the difference is equal to or greater than threshold T9 (if answered "No" in S46), if the distance difference is equal to or greater than threshold distance T11 (if answered "No" in S47), if the applied pressure is equal to or less than threshold pressure T13 (if answered "No" in S48), or if the trigger switch 221 is not half-pressed (if answered "No" in S49), then processing proceeds. Figure 6 The processing step S65 is shown.

[0132] When it is found in S44 that the state of processing unit 35 is not state St1 but state St2 (if the answer in S44 is "no"), setting state detection unit 34 compares the angle difference between the current pose and reference pose of tool 2 with the angle difference threshold T8 based on pose information (in S51). When it is found that the angle difference between the current pose and reference pose of tool 2 is less than the threshold T8 (if the answer in S51 is "yes"), stability determination unit 33 calculates the difference between the first frame and the second frame. Then, stability determination unit 33 compares the difference it calculates with the difference threshold T10 (in S52). When it is found that the difference it calculates is less than the threshold T10 (if the answer in S52 is "yes"), stability determination unit 33 outputs stability information to setting state detection unit 34 and processing unit 35. Next, setting state detection unit 34 calculates the distance difference based on distance information and compares the distance difference with the threshold distance T12 (in S53). When the distance difference is found to be less than the threshold distance T12 (if the answer in S53 is "yes"), the setting state detection unit 34 compares the pressure applied to the back of the grip 22 with the threshold pressure T14 based on the pressing information (in S54). When the pressure applied to the back of the grip 22 is found to be greater than the threshold pressure T14 (if the answer in S54 is "yes"), the setting state detection unit 34 checks whether the trigger switch 221 has been half-pressed (in S55). When the trigger switch 221 is found to have been half-pressed (if the answer in S55 is "yes"), the setting state detection unit 34 outputs the setting state detection information to the processing unit 35. Then, processing begins. Figure 6 The processing step S61 is shown.

[0133] Note that if the angle difference is equal to or greater than the threshold T8 (if the answer in S51 is "No"), if the difference is equal to or greater than the threshold T10 (if the answer in S52 is "No"), if the distance difference is equal to or greater than the threshold distance T12 (if the answer in S53 is "No"), if the applied pressure is equal to or less than the threshold pressure T14 (if the answer in S54 is "No"), or if the trigger switch 221 is not half-pressed (if the answer in S55 is "No"), then the state of the processing unit 35 changes from state St2 to state St1 (in S56), and processing begins. Figure 6 The processing step S64 is shown.

[0134] Note that, according to this embodiment, the angle difference thresholds T7 and T8 are each set to have hysteresis, and threshold T8 is greater than threshold T7. Similarly, the difference thresholds T9 and T10 are also set to have hysteresis, and threshold T10 is greater than threshold T9. Likewise, the threshold distances T11 and T12 are also set to have hysteresis, and threshold distance T12 is greater than threshold distance T11. Similarly, the threshold pressures T13 and T14 are also set to have hysteresis, and threshold pressure T14 is less than threshold pressure T13.

[0135] Next, refer to Figure 6 The processing steps S61 to S68 will be explained below. Upon receiving at least one of the stability information provided by the stability determination unit 33 and the setting state detection information provided by the setting state detection unit 34, the processing unit 35 performs recognition processing based on the captured image (in S61). Then, the processing unit 35 checks whether the current work object has been successfully identified and whether the identified work object follows the work process (in S62). If the processing unit 35 successfully identifies the current work object and the identified work object follows the work process (if the answer in S62 is "yes"), the motor state changes to state St4 (in S63). As used herein, "state St4" refers to the state in which the motor included in the drive unit 24 starts running in response to the trigger switch 221 being turned on by the user. After the motor state changes to state St4, the processing proceeds to processing step S65.

[0136] On the other hand, if the processing unit 35 fails to identify the current work object in S62, or unless the identified work object follows the work process (if the answer in S62 is "no"), the motor's state changes to state St3 (in S64). After the motor's state changes to state St3, the process proceeds to processing step S65.

[0137] Next, processing unit 35 checks whether the user has pulled trigger switch 221 (in S65). If the user has pulled trigger switch 221 to activate it (if the answer in S65 is "yes"), processing unit 35 checks whether the motor's state is state St4 (in S66). If the motor's state is state St4 (if the answer in S66 is "yes"), processing unit 35 enables the tightening operation to be performed by running the motor included in drive unit 24. As a result, the tightening operation is performed by running the motor (in S67). In this processing step, drive control unit 31 of tool 2 controls drive unit 24 so that the target torque value associated with the identified work object becomes the torque setting value. When the tightening operation is completed, the processing ends.

[0138] On the other hand, if the motor's state in step S66 is not state St4 but state St3 (if the answer in S66 is "no"), then the processing unit 35 performs an alarm operation, such as causing the notification unit 211 to light up in red (in S68). Note that in this case, the motor included in the drive unit 24 does not operate.

[0139] Notice, Figures 4 to 6 The flowchart shown is merely an example. Therefore, Figures 4 to 6 The processing steps shown can be performed in a different order as appropriate, additional processing steps can be added as needed, or at least one of the processing steps can be omitted as appropriate.

[0140] (4) Variations

[0141] Note that the above-described typical embodiments are merely exemplary embodiments among various embodiments of the present invention and should not be construed as limiting. Instead, the typical embodiments can be readily modified in various ways, depending on design choices or any other factors, without departing from the scope of the invention. All figures to be referenced in this specification are schematic diagrams. Therefore, the scale of the dimensions (including thickness) of the various constituent elements illustrated in these figures does not always reflect the actual size scale of these constituent elements.

[0142] Furthermore, according to the present invention, when various parameters are compared with their respective thresholds, the choice of the threshold, such as whether the phrase "greater than" covers the case where two values ​​are equal to each other, can be arbitrarily changed. Therefore, from a technical point of view, there is no difference between the phrase "greater than" and the phrase "equal to or greater than". Similarly, according to the present invention, when various parameters are compared with their respective thresholds, from a technical point of view, there is no difference between the phrase "equal to or less than" and the phrase "less than".

[0143] Furthermore, the function of the tool system 1 according to the typical embodiment can also be implemented as a job object identification method, a (computer) program, or a non-transitory storage medium storing the program. The job object identification method according to one aspect includes an identification processing step, a stability determination step, and a setup state detection step. The identification processing step includes identifying the job object based on an image captured by the camera unit 5. The camera unit 5 is equipped with a portable tool 2, which includes a drive unit 24 activated by power supplied from a power source. The stability determination step includes determining whether the captured image is stable. The setup state detection step includes detecting the state in which the tool 2 is positioned appropriately on the job object. Note that the job object identification method only needs to include at least the identification processing step and the setup state detection step. The program according to another aspect is designed to cause one or more processors to perform the above-described job object identification method.

[0144] Next, variations of the typical embodiments will be listed one by one. Note that the variations described below can be appropriately combined.

[0145] The tool system 1 can measure the distance between the camera unit 5 and the work object based on the image captured by the camera unit 5, which is implemented as a stereo camera. Then, the setting state detection unit 34 can detect that the tool 2 is set in the appropriate position on the work object when it finds that the absolute value of the difference calculated by subtracting the distance between the camera unit 5 and the work object from the reference distance is equal to or less than a threshold.

[0146] Optionally, while processing unit 35 is performing recognition processing, it may perform spin compensation and / or distortion correction on at least one of the captured image and the reference image based on the pose information. As used herein, "distortion correction" means correcting the captured image by partially expanding or shrinking the captured image (or reference image) to an arbitrary degree. For example, processing unit 35 may obtain a captured image of a rectangular subject in a rectangular shape by performing distortion correction on a captured image of a rectangular subject captured in a trapezoidal shape.

[0147] "Predetermined posture" can be the posture of tool 2 when the angle difference between the rotation angle of tool 2 around any one of the three axes and the corresponding rotation angle in the rotation angle of the reference posture of tool 2 is equal to or less than a threshold.

[0148] Instead of the posture detection unit 26, the setting state detection unit 34 can be configured to detect the movement and posture of the tool 2 based on the output of the motion sensor 261. That is, the setting state detection unit 34 can perform the function of the posture detection unit 26.

[0149] Instead of the distance measurement unit 27, the setting status detection unit 34 can be configured to detect the distance between the work object (or workpiece) and the tool 2 based on the output of the distance sensor 271. In other words, the setting status detection unit 34 can perform the function of the distance measurement unit 27.

[0150] Instead of the pressure detection unit 28, the setting status detection unit 34 can be configured to sense that the tool 2 is pressed against the work object based on the output of the pressure sensor 281. In other words, the setting status detection unit 34 can perform the function of the pressure detection unit 28.

[0151] In the image registration process of the registration mode, the pose of tool 2 (i.e., pose information) detected by pose detection unit 26 can also be stored in storage unit 4 (image storage unit 41) in association with the reference image generated by camera unit 5. This makes it possible to register the reference image and the pose of tool 2 in association with each other. Therefore, once the captured image and the pose of tool 2 are identified in the operation mode, processing unit 35 can compare the captured image with the reference image associated with that pose. Alternatively, the pose of tool 2 associated with the reference image can be defined as the reference pose.

[0152] Furthermore, in the image registration processing of the registration mode, the distance (distance information) between the tool 2 and the work object detected by the distance measurement unit 27 can be defined as a reference distance and stored in the storage unit 4 (image storage unit 41) in association with the reference image generated by the camera unit 5. Additionally, in the image registration processing of the registration mode, the pressure (pressing information) applied to the grip 22 detected by the press detection unit 28 can also be stored in the storage unit 4 (image storage unit 41) in association with the reference image generated by the camera unit 5.

[0153] The stability determination unit 33 can calculate the degree of matching (similarity) between the first frame and the second frame to determine whether the captured image is stable. For example, the stability determination unit 33 can calculate the degree of matching between the first frame and the second frame using normalized cross-correlation (NCC).

[0154] Optionally, the stability determination unit 33 can also calculate the difference degree during stability determination processing by comparing the brightness value of a specific region in the first frame with the brightness value of the corresponding specific region in one or more previous frames preceding the second frame. In this case, the processing load on the stability determination unit 33 increases in terms of stability determination processing, but the accuracy of stability determination processing is improved, which is an advantage.

[0155] Optionally, the stability determination unit 33 can perform stability determination by using other frames, including the second frame, as template data to perform pattern recognition processing on the first frame.

[0156] The tool system 1 according to the invention includes, for example, a computer system in its control units 3a, 3b. The computer system may include a processor and memory as primary hardware components. The function of the tool system 1 according to the invention can be performed by causing the processor to execute a program stored in the memory of the computer system. The program may be pre-stored in the memory of the computer system. Alternatively, the program may also be downloaded via telecommunication lines or distributed after being recorded on some non-transitory storage medium such as a memory card, optical disc, or hard disk drive (any of which is readable by the computer system). The processor of the computer system may consist of one or more electronic circuits including semiconductor integrated circuits (ICs) or large-scale integrated circuits (LSIs). As used herein, "integrated circuit," such as IC or LSI, is referred to by different names depending on its degree of integration. Examples of integrated circuits include system LSIs, very large-scale integrated circuits (VLSIs), and ultra-large-scale integrated circuits (ULSIs). Alternatively, a field-programmable gate array (FPGA) that is programmed after the LSI is manufactured, or a reconfigurable logic device that allows reconfiguration of connections or circuit sections within the LSI, may also be used as the processor. These electronic circuits may be integrated together on a single chip or distributed across multiple chips, either way being suitable. These multiple chips can be integrated together in a single device or distributed across multiple devices without limitation. As used herein, "computer system" includes a microcontroller containing one or more processors and one or more memories. Therefore, a microcontroller can also be implemented as a single or multiple electronic circuits comprising semiconductor integrated circuits or large-scale integrated circuits.

[0157] Furthermore, in the above embodiments, at least some functions of the tool system 1 are integrated together in a single housing (body 20). However, this is not a necessary configuration for the tool system 1. Alternatively, these constituent elements of the tool system 1 can be distributed in multiple different housings.

[0158] For example, some functions of control units 3a and 3b can be housed in a separate housing from the main body 20 of tool 2. Alternatively, at least some functions of control units 3a and 3b, or any other processor, can be implemented as a server or cloud computing system, for example.

[0159] Furthermore, in the above embodiment, the image storage unit 41 of tool 2 stores multiple reference images corresponding to multiple work objects. However, tool 2 need not store such multiple reference images corresponding to multiple work objects. Alternatively, the setting terminal 60 or server device may include an image storage unit that stores multiple reference images corresponding to multiple work objects. In this case, the processing unit 35 of tool 2 can access the image storage unit of the setting terminal 60 or server device to perform processing that compares the first captured image taken by the camera unit 5 with the reference images stored in the image storage unit and thereby identifies the current work object. Furthermore, tool 2 need not include a processing unit 35. Alternatively, the setting terminal 60 or server device may perform the functions of the processing unit 35. When tool 2 outputs the first captured image taken by the camera unit 5 to the setting terminal 60 or server device, the processing unit of the setting terminal 60 or server device performs image processing that compares the first captured image with the reference images and outputs the identification result of the current work object to tool 2.

[0160] Furthermore, for example, the camera unit 5 need not be equipped with the cylinder 21 of the main body 20, but can be equipped with the attachment part 23 of the main body 20 or the battery pack 201. Similarly, the arrangement of the control units 3a, 3b, storage unit 4 and other units can also be appropriately changed. In addition, the tool 2 may include the camera unit 5.

[0161] Optionally, such as Figure 7 As shown, the work object identification system 10 can be connected to the tool 2 as an external device. In this case, the control unit 3a of the tool 2 and the control unit 3b of the work object identification system 10 can be directly electrically connected to each other or communicate with each other via a communication unit. In the latter case, the communication unit can adopt a design conforming to, for example, Or a standard wireless communication protocol such as a low-power radio standard that does not require a license (e.g., a designated low-power radio standard). Furthermore, for example, the object identification system 10 may include a power source different from the battery pack 201, and this power source can be used as the power source for the camera unit 5 and the control unit 3b. This object identification system 10 can determine whether the captured image is stable and identify the object based on a stable captured image. Additionally, the object identification system 10 can detect the state in which the tool 2 is positioned appropriately on the object, and then identify the object based on the captured image. This reduces the number of times the object identification system 10 performs identification processing futilely, thereby reducing power consumption. Figure 7 In the example shown, the object recognition system 10, which includes a built-in camera unit 5, is attached to the tool 2 as an auxiliary device. That is, the camera unit 5 of the object recognition system 10 is set as an external device of the tool 2.

[0162] The job object identification system 10 only needs to include at least a setup status detection unit 34 and a processing unit 35. Furthermore, in the above-described typical embodiment, the job object identification system 10 is implemented as a single system including the setup status detection unit 34 and the processing unit 35. Alternatively, the job object identification system 10 can also be implemented as two or more systems. For example, the functions of the setup status detection unit 34 and the processing unit 35 can be distributed across two or more systems. Furthermore, at least one function of the setup status detection unit 34 and the processing unit 35 can be distributed across two or more systems. For example, the function of the setup status detection unit 34 can be distributed across two or more devices. Optionally, at least some functions of the job object identification system 10 can also be implemented as a cloud computing system.

[0163] When the captured image is determined to be stable, the stability determination unit 33 can output the stability information only to the setting state detection unit 34.

[0164] Even if the processing unit 35 does not receive stability information provided by the stability determination unit 33, the processing unit 35 can still perform predetermined processing including identification processing, as long as the processing unit 35 receives at least the setting status detection information provided by the setting status detection unit 34.

[0165] Note that tool system 1 does not have to be applied to assembly lines where workpieces are assembled in a factory; it can also be applied to any other applications.

[0166] In the above embodiment, tool 2 is an impact wrench. However, tool 2 does not have to be an impact wrench, but could also be, for example, a nut wrench or a hydraulic pulse wrench. Alternatively, tool 2 could also be, for example, a screwdriver (including an impact screwdriver) for tightening screws (as a fastening member). In this case, a drill bit (such as a screwdriver drill bit, etc.) is attached to tool 2 instead of the socket 242. Furthermore, tool 2 does not have to be configured to be powered by battery pack 201, but could also be configured to be powered by AC power (commercial power). Furthermore, tool 2 does not have to be a power tool, but could also be a pneumatic tool, which includes a pneumatic motor (drive unit) operated using compressed air (power) supplied from an air compressor (power source).

[0167] Furthermore, in the above-described typical embodiments, it is assumed that the work object is each of the multiple fastening parts in a single workpiece. However, this is merely an example and should not be construed as limiting. Alternatively, the work object may also be a module, part, or article of manufacture having multiple fastening parts. For example, if the work object is a module, part, or article of manufacture having multiple fastening parts, the multiple fastening parts of a single work object may have the same target torque value or different target torque values.

[0168] Optionally, tool 2 may include a torque sensor for measuring the tightening torque. In this case, drive control unit 31 controls drive unit 24 such that the tightening torque measured by the torque sensor becomes the torque set value. Furthermore, torque determination unit 37 can determine whether the tightening torque is normal by comparing the measurement result of the torque sensor with the target torque value. If the measurement result of the torque sensor falls within a predetermined range based on the target torque value, torque determination unit 37 determines that the tightening torque is normal. On the other hand, if the measurement result of the torque sensor falls outside the predetermined range based on the target torque value, torque determination unit 37 determines that the tightening torque is insufficient (abnormal).

[0169] Furthermore, the notification unit 211 need not be a light-emitting unit such as an LED, but can also be implemented as an image display device such as a liquid crystal display or an organic electroluminescent (EL) display. Optionally, the notification unit 211 can notify (present) by any means other than display. For example, the notification unit 211 can also be implemented as a speaker or buzzer that emits sound (including voice). In this case, the notification control unit 36 ​​preferably causes the notification unit 211 to emit different sounds when the determination made by the processing unit 35 is inconsistent and when the processing unit 35 identifies the current work object. Alternatively, the notification unit 211 can also be implemented, for example, as a vibrator that generates vibration or a transmitter for sending notification signals to an external terminal (such as a mobile communication device) located outside the tool 2. Optionally, the notification unit 211 can also have two or more functions selected from display, emitting sound, generating vibration, and establishing communication.

[0170] Storage unit 4 can store work process data representing a predetermined order of work steps to be performed on multiple work objects. In this case, processing unit 35 selects a reference image from multiple reference images for use in the recognition process based on the work process. Specifically, processing unit 35 preferentially selects a reference image from the multiple reference images that corresponds to the upcoming work object to be processed in the upcoming work step. As used herein, the "upcoming work object" is the work object to be processed after the last recognized work object. Processing unit 35 performs image processing to compare the reference image selected as template data with the captured image. That is, processing unit 35 selects the reference image by predicting the current work object to be captured in the next captured image based on the work process. This allows processing unit 35 to recognize the current work object captured in the captured image in a shorter time.

[0171] Optionally, the processing unit 35 may also be configured to determine the type of the socket 242 attached to the tool 2 by performing image processing on the captured image. As used herein, "type" is information used to distinguish different parts from each other and includes at least one piece of information related to size (dimension or length), shape, or material. In this embodiment, the processing unit 35 is configured to determine the length of the socket 242 attached to the tool 2. The processing unit 35 corrects the target torque value based on the length of the socket 242 and sets the corrected target torque value as the torque setting value. For example, the processing unit 35 corrects the target torque value associated with the current work object by multiplying the target torque value by a coefficient corresponding to the length of the socket 242 and sets the corrected target torque value as the torque setting value. That is, the processing unit 35 controls the drive unit 24 such that the tightening torque becomes equal to the corrected target torque value. This can reduce the deviation of the tightening torque based on the length of the socket 242.

[0172] Optionally, the processing unit 35 can also be configured to determine a torque setting value based on the detected length (or type) of the socket 242. The storage unit 4 stores torque values ​​that correspond one-to-one with various lengths of the socket 242. The processing unit 35 retrieves the torque value corresponding to the determined length of the socket 242 from the storage unit 4 and sets the value based on the thus retrieved torque value as the torque setting value. For example, the processing unit 35 can set the torque value retrieved from the storage unit 4 as the torque setting value. This enables fastening operations to be performed with a torque value corresponding to the type of a given socket 242.

[0173] Tool system 1 only needs to include at least tool 2 and job object identification system 10. Furthermore, in the above typical embodiment, tool system 1 is implemented as a single system including tool 2 and job object identification system 10. Alternatively, tool system 1 can also be implemented as two or more systems. For example, the functions of tool 2 and job object identification system 10 can be distributed across two or more systems. Furthermore, at least one function of tool 2 or job object identification system 10 can be distributed across two or more systems. For example, the function of job object identification system 10 can be distributed across two or more devices. Optionally, at least some functions of tool system 1 can also be implemented as a cloud computing system.

[0174] Furthermore, if the captured image is a still image, the stability determination unit 33 determines whether the captured image is stable based on at least two captured images captured within a predetermined time period.

[0175] (Summary)

[0176] As can be seen from the above description, the tool system (1) according to the first aspect includes a tool (2), a camera unit (5), a processing unit (35), and a setting state detection unit (34). The tool (2) is a portable tool (2) including a drive unit (24) which is activated by power supplied from a power source (battery pack 201). The camera unit (5) is provided to the tool (2) and generates captured images. The processing unit (35) intermittently performs recognition processing for identifying the work object based on the captured images. The setting state detection unit (34) detects the state in which the tool (2) is set in the appropriate position on the work object.

[0177] This aspect enables the tool system (1) to identify the work object based on the captured image after detecting that the tool (2) is positioned appropriately on the work object. This reduces the number of times the tool system (1) performs identification processing in vain, thereby reducing power consumption.

[0178] In the tool system (1) based on the second aspect, which can be implemented in conjunction with the first aspect, the processing unit (35) performs identification processing in response to the setting state detection unit (34) detecting that the tool (2) is set in the appropriate position on the work object.

[0179] This aspect enables the tool system (1) to perform identification processing only when it detects that the tool (2) is positioned appropriately on the work object. This allows for a greater reliability reduction in the number of times the tool system (1) performs identification processing in vain.

[0180] In the tool system (1) according to the third aspect, which can be implemented in conjunction with the first or second aspect, the tool (2) further includes a drive control unit (31). The drive control unit (31) changes the settings of the tool (2) based on the working conditions associated with the work object identified by the processing unit (35).

[0181] This aspect enables the tool (2) settings to be automatically changed based on the work conditions associated with the work object identified by the processing unit (35), thereby increasing the convenience of the tool system (1).

[0182] In the tool system (1) according to the fourth aspect, which can be implemented by combining any one of the first to third aspects, a state detection unit (34) is set to detect that the tool (2) is set in an appropriate position on the work object when it finds that the distance between the tool (2) and the work object falls within a preset range.

[0183] This aspect enables the tool system (1) to detect when the distance between the tool (2) and the work object falls within a preset range, that the tool (2) is positioned appropriately on the work object, and to perform identification processing. This allows for a more reliable reduction in the number of times the tool system (1) performs identification processing in vain.

[0184] The tool system (1) according to the fifth aspect, which can be implemented in conjunction with the fourth aspect, also includes a distance sensor (271) that measures the distance between the tool (2) and the work object. The setting state detection unit (34) detects that the tool (2) is set in an appropriate position on the work object when it finds that the distance measured by the distance sensor (271) falls within a preset range.

[0185] This makes it easier to measure the distance between the tool (2) and the work object by using a distance sensor (271).

[0186] In the tool system (1) according to the sixth aspect, which can be implemented by combining any of the first to fifth aspects, a state detection unit (34) is set to detect the state that the tool (2) is set in the appropriate position on the work object when the tool (2) is found to be in a predetermined posture.

[0187] According to this aspect, when the tool (2) is found to be in a predetermined posture, the tool system (1) detects that the tool (2) is positioned appropriately on the work object and performs identification processing. This enables the tool system (1) to reduce the number of times it performs identification processing in vain with higher reliability.

[0188] The tool system (1) according to the seventh aspect, which can be implemented in conjunction with the sixth aspect, also includes a motion sensor (261) for detecting the posture of the tool (2). The setting state detection unit (34) detects that the tool (2) is set in an appropriate position on the work object when it finds that the posture of the tool (2) detected by the motion sensor (261) is a predetermined posture.

[0189] This aspect enables the motion sensor (261) to detect the posture of the tool (2), thereby making it easier to detect its posture.

[0190] The tool system (1) according to the eighth aspect, which can be implemented by combining any of the first to seventh aspects, also includes a stability determination unit (33) that determines whether the captured image is stable. The setting state detection unit (34) detects that the tool (2) is set in an appropriate position on the work object in response to the stability determination unit (33) determining that the captured image is stable.

[0191] This aspect enables the tool system (1) to detect and recognize the tool (2) being positioned appropriately on the work object when the stability determination unit (33) determines that the captured image is stable. This allows the tool system (1) to reduce the number of times it performs recognition processing in vain with higher reliability.

[0192] In the tool system (1) based on the ninth aspect, which can be implemented in conjunction with the eighth aspect, the captured image comprises multiple frames. The stability determination unit (33) calculates the degree of difference between the multiple frames, and determines that the captured image is stable when the degree of difference is equal to or less than the threshold (T9; T10).

[0193] This aspect makes it possible to determine whether a captured image is stable using a simple method.

[0194] In the tool system (1) according to the tenth aspect, which can be implemented by combining any of the first to ninth aspects, a state detection unit (34) is provided to detect the state in which the tool (2) is set in an appropriate position on the work object when it senses that the tool (2) is pressed against the work object.

[0195] According to this aspect, when the tool (2) is sensed to be pressed against the work object, the tool system (1) detects that the tool (2) is in the appropriate position on the work object and performs identification processing. This enables the tool system (1) to reduce the number of times it performs identification processing in vain with higher reliability.

[0196] The tool system (1) according to the eleventh aspect, which can be implemented in conjunction with the tenth aspect, also includes a pressure sensor (281) that detects the pressure applied to the tool (2). The setting state detection unit (34) detects that the tool (2) is set in the appropriate position on the work object when it finds that the pressure detected by the pressure sensor (281) is equal to or greater than the threshold pressure (T13; T14).

[0197] This aspect makes it easier for the tool system (1) to sense that the tool (2) is pressed against the work object by having the pressure sensor (281) detect the pressure.

[0198] In the tool system (1) according to the twelfth aspect, which can be implemented by combining any of the first to eleventh aspects, the tool (2) further includes an operation unit (221) for activating the drive unit (24). The operation unit (221) has an initial position and an on position, and activates the drive unit (24) when pressed to the on position. The setting state detection unit (34) detects that the tool (2) is set in the appropriate position on the work object when it finds that the operation unit (221) is half-pressed between the initial position and the on position.

[0199] According to this aspect, when the operating unit (221) of the detection tool (2) is half-pressed between the initial position and the on position, the tool system (1) detects that the tool (2) is set in the appropriate position on the work object and performs identification processing. This makes it possible to reduce the number of times the tool system (1) performs identification processing in vain with higher reliability.

[0200] Note that the constituent elements according to the second to twelfth aspects are not necessary constituent elements of the tool system (1), but can be appropriately omitted.

[0201] The tool (2) according to the thirteenth aspect is designed for use in the tool system (1) according to any one of the first to twelfth aspects. The tool (2) includes a drive unit (24) and a camera unit (5).

[0202] This aspect enables the tool system (1) to identify the work object based on the captured image after detecting that the tool (2) is positioned appropriately on the work object. This reduces the number of times the tool system (1) performs identification processing in vain.

[0203] The work object recognition system (10) according to the fourteenth aspect includes a processing unit (35) and a setting status detection unit (34). The processing unit (35) intermittently performs recognition processing for recognizing the work object based on the captured images generated by the camera unit (5). The camera unit (5) is equipped with a tool (2), which is a portable tool (2) including a drive unit (24) activated by power supplied from a power source. The setting status detection unit (34) detects the state in which the tool (2) is set in an appropriate position on the work object.

[0204] This aspect enables the object recognition system (10) to identify the object based on the captured image after detecting that the tool (2) is positioned appropriately on the object. This reduces the number of times the object recognition system (10) performs recognition processing in vain, thereby reducing power consumption.

[0205] The object identification method according to aspect fifteen includes an identification processing step and a setting state detection step. The identification processing step includes intermittently performing identification processing for identifying the object based on images captured by the camera unit (5). The camera unit (5) is equipped with a tool (2), which is a portable tool (2) including a drive unit (24) activated by power supplied from a power source. The setting state detection step includes detecting the state in which the tool (2) is set in an appropriate position on the object.

[0206] This aspect enables the identification of the work object based on the captured image after detecting that the tool (2) is positioned appropriately on the work object. This reduces the number of futile identification processes, thereby reducing power consumption.

[0207] The procedure according to the sixteenth aspect is designed to enable one or more processors to perform the job object identification method according to the fifteenth aspect.

[0208] This aspect enables one or more processors to identify the work object based on the captured image after detecting that the tool (2) is positioned appropriately on the work object. This reduces the number of times one or more processors perform identification processing in vain, thereby reducing power consumption.

[0209] Explanation of reference numerals in the attached figures

[0210] 1. Tool System

[0211] 2 tools

[0212] 201 Battery Pack (Power Source)

[0213] 221 Trigger Switch (Operating Unit)

[0214] 24 drive units

[0215] 261 Motion Sensor

[0216] 271 Distance Sensor

[0217] 281 Pressure Sensor

[0218] 31 Drive Control Unit

[0219] 33 Stability Detection Unit

[0220] 34. Set up a status detection unit

[0221] 35 processing units

[0222] 5 camera units

[0223] 10. Job Object Recognition System

Claims

1. A tool system, comprising: A tool, including a drive unit configured to be activated using power supplied from a power source, is a portable tool. A camera unit is provided to the tool and configured to generate captured images; The processing unit is configured to intermittently perform identification processing for recognizing the target object based on the captured images; as well as A status detection unit is configured to detect the state in which the tool is positioned appropriately on the work object. The processing unit is configured to perform the identification processing in response to the setting state detection unit detecting that the tool is set in an appropriate position on the work object.

2. The tool system according to claim 1, wherein, The tool also includes a drive control unit configured to change the tool's settings based on job conditions associated with the job object identified by the processing unit.

3. The tool system according to claim 1 or 2, wherein, The setting state detection unit is configured to detect that the tool is set in an appropriate position on the work object when it senses that the tool is pressed against the work object.

4. The tool system of claim 3 further includes a pressure sensor configured to detect pressure applied to the tool. in, The setting status detection unit is configured to detect that the tool is set in an appropriate position on the work object when the pressure detected by the pressure sensor is equal to or greater than a threshold pressure.

5. The tool system according to claim 1 or 2, wherein, The setting state detection unit is configured to detect that the tool is set in an appropriate position on the work object when the distance between the tool and the work object falls within a preset range.

6. The tool system of claim 5 further includes a distance sensor configured to measure the distance between the tool and the work object. in, The setting status detection unit is configured to detect that the tool is set in an appropriate position on the work object when the distance measured by the distance sensor falls within the preset range.

7. The tool system according to claim 1 or 2, wherein, The setting state detection unit is configured to detect that the tool is set in an appropriate position on the work object when the tool is found to be in a predetermined posture.

8. The tool system of claim 7 further includes a motion sensor configured to detect the posture of the tool. in, The setting state detection unit is configured to detect that the tool is set in an appropriate position on the work object when the motion sensor detects that the tool's posture is the predetermined posture.

9. A tool system, comprising: A tool, including a drive unit configured to be activated using power supplied from a power source, is a portable tool. A camera unit is provided to the tool and configured to generate captured images; The processing unit is configured to intermittently perform identification processing for recognizing the target object based on the captured images; A status detection unit is configured to detect the status of the tool being positioned appropriately on the work object; as well as A stability determination unit is configured to determine whether the captured image is stable. The setting state detection unit is configured to detect that the tool is set in an appropriate position on the work object in response to the stability determination unit determining that the captured image is stable.

10. The tool system according to claim 9, wherein, The captured images include multiple frames, and The stability determination unit is configured to calculate the difference between the plurality of frames, and determine that the captured image is stable if the difference is equal to or less than a threshold.

11. A tool system, comprising: A tool, including a drive unit configured to be activated using power supplied from a power source, is a portable tool. A camera unit is provided to the tool and configured to generate captured images; The processing unit is configured to intermittently perform identification processing for recognizing the target object based on the captured images; as well as A status detection unit is configured to detect the state in which the tool is positioned appropriately on the work object. The tool also includes an operation unit configured to activate the drive unit. The operating unit has an initial position and an on position, and is configured to activate the drive unit when pressed to the on position. The setting state detection unit is configured to detect that the tool is set in the appropriate position on the work object when the operation unit is half-pressed between the initial position and the on position.

12. A tool used in a tool system according to any one of claims 1 to 11, said tool comprising: The driving unit; The camera unit; The processing unit; as well as The setting status detection unit.

13. A job object identification system, comprising: A processing unit is configured to intermittently perform identification processing for recognizing a work object based on images captured by a camera unit, the camera unit being provided to a tool, the tool being a portable tool including a drive unit configured to be activated using power supplied from a power source; as well as A status detection unit is configured to detect the state in which the tool is positioned appropriately on the work object. The processing unit is configured to perform the identification processing in response to the setting state detection unit detecting that the tool is set in an appropriate position on the work object.

14. A method for identifying work objects, comprising: The identification processing step is used to intermittently perform identification processing for identifying the work object based on the captured images generated by the camera unit, the camera unit being equipped with a tool, the tool being a portable tool including a drive unit configured to be activated using power supplied from a power source; as well as A status detection step is set up to detect the status of the tool being positioned appropriately on the work object. The identification processing step is performed in response to the setting state detection step detecting that the tool is set in an appropriate position on the work object.

15. A non-transitory storage medium storing a program designed to cause one or more processors to perform the job object identification method according to claim 14.

16. A computer program product comprising a program designed to cause one or more processors to perform the job object identification method according to claim 14.

Images