Tool system, tool, reference image generation method, and program

Abstract

The problem to be overcome by the present invention is to use an image suitable for identifying the current work object as a reference image. A tool system (1) is used for a plurality of tools (2a, 2b). Each of the plurality of tools (2a, 2b) is a portable tool and includes: a drive unit (24) that is activated by power supplied from a power source; and an imaging unit (5). The tool system (1) includes a reference image generation unit (67) and an output unit (66). The reference image generation unit (67) generates a reference image to be compared with a first captured image based on a second captured image. The first captured image is captured by the imaging unit (5) when the current work object is being identified. The second captured image is also captured by the imaging unit (5). The output unit (66) causes the display unit (62) to display the second captured image when the reference image generation unit (67) is generating the reference image.

Description

Technical Field

[0001] The present invention generally relates to a tool system, a tool, a method for generating a reference image, and a program. More specifically, the present invention relates to a tool system for use in a portable tool, a portable tool, a method for generating a reference image for use in the tool system, and a program. Background Art

[0002] Patent Document 1 discloses a tool system including a portable tool having an imaging unit and a drive unit activated by power supplied from a battery pack. The imaging unit is arranged to cover, for example, a socket attached to an output shaft of the tool within its imaging range. The imaging unit captures an image of an operation object (which may be, for example, an object or a place on which an operation is performed using the tool) during an operation using the tool.

[0003] According to Patent Document 1, the image captured by the imaging unit is used to identify the current operation object at which the tool is set in an appropriate position (i.e., the operation object arranged so that the tool is ready to start an operation on the operation object). The tool system of Patent Document 1 includes an identification unit. The identification unit compares the captured image generated by the imaging unit with a plurality of reference images stored in the image storage unit and thereby identifies the current operation object. Further, according to Patent Document 1, in a case where the current operation object identified by the identification unit does not conform to a reference working procedure in terms of a working process step for the operation object, the tool system, for example, performs a process of deactivating the drive unit.

[0004] Prior Art Documents

[0005] Patent Documents

[0006] Patent Document 1: Japanese Unexamined Patent Application Publication No. 2018-108633 Summary of the Invention

[0007] Preferably, the tool system of Patent Document 1 uses an image suitable for identifying the current operation object as a reference image.

[0008] In view of the above background, an object of the present invention is to provide a tool system, a tool, a method for generating a reference image, and a program, all of which are configured or designed to use an image suitable for identifying the current operation object as a reference image.

[0009] To overcome this problem, a tool system according to one aspect of the present invention is for a plurality of tools. Each tool among the plurality of tools is a portable tool and includes: a drive unit that is activated using power supplied from a power source; and an imaging unit. The tool system includes a reference image generation unit. The reference image generation unit generates a reference image to be compared with a first captured image. The first captured image is captured by the imaging unit of the first tool currently set in a proper position among the plurality of tools when identifying a current work object on which the first tool, which is one of the plurality of tools, is currently set in the proper position. The reference image generation unit generates the reference image based on a second captured image. The second captured image is captured by the imaging unit of a second tool, which is one of the plurality of tools. The tool system further includes an output unit. The output unit causes the display unit to display the second captured image when the reference image generation unit is generating the reference image.

[0010] A tool according to another aspect of the present invention is used in the above tool system and includes a drive unit and an imaging unit.

[0011] A reference image generation method according to still another aspect of the present invention includes a reference image generation step. The reference image generation step includes generating a reference image to be compared with a first captured image based on a second captured image. The first captured image is captured by the imaging unit of the first tool currently set in a proper position among the plurality of tools when identifying a current work object on which the first tool, which is one of the plurality of tools, is currently set in the proper position. Each tool among the plurality of tools is a portable tool and includes: a drive unit that is activated using power supplied from a power source; and an imaging unit. The second captured image is captured by the imaging unit of a second tool, which is one of the plurality of tools. The reference image generation step further includes a display step. The display step includes causing the display unit to display the second captured image when the reference image is being generated.

[0012] A program according to still another aspect of the present invention is designed such that one or more processors perform the above reference image generation method.

[0013] A tool system according to another aspect of the present invention includes a plurality of tools, a reference image generation unit, and an output unit. Each of the plurality of tools is a portable tool and includes: a drive unit that is activated using power supplied from a power source; and an imaging unit. The reference image generation unit generates a reference image to be compared with a first captured image. The first captured image is captured by the imaging unit of the first tool currently set at an appropriate position among the plurality of tools when identifying, from among a plurality of work objects, the current work object where the first tool, which is one of the plurality of tools, is set at the appropriate position. The reference image generation unit generates the reference image based on a second captured image. The second captured image is captured by the imaging unit of a second tool, which is one of the plurality of tools. The output unit causes the display unit to display the second captured image while the reference image generation unit is generating the reference image. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a block diagram of a tool system according to an exemplary embodiment;

[0015] Figure 2 A is a perspective view showing the appearance of a tool included in the tool system as seen from one angle, Figure 2 B is a perspective view showing the appearance of the tool included in the tool system as seen from another angle;

[0016] Figure 3 shows a sequence of operations to be performed by the tool system;

[0017] Figure 4 is a schematic diagram illustrating an exemplary screen image displayed by the tool system;

[0018] Figure 5 is a schematic diagram illustrating an exemplary situation where a cropping area is set on the screen image displayed by the tool system;

[0019] Figure 6 is a schematic diagram illustrating an exemplary situation where a plurality of cropping areas are set on the screen image displayed by the tool system; and

[0020] Figure 7 is a schematic diagram illustrating an exemplary situation where a masking area is set on the screen image displayed by the tool system. DETAILED DESCRIPTION

[0021] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings. In the following description of the embodiments, any pair of constituent elements having the same function will be designated by the same reference numeral, and the description thereof will be omitted herein to avoid redundancy.

[0022] (Embodiment)

[0023] (1) Overview

[0024] First, reference Figure 1 will be used to describe an overview of the tool system 1 according to a typical embodiment.

[0025] The tool system 1 according to the present embodiment includes a plurality of (for example, two in the Figure 1 illustrated example) portable tools 2a, 2b. In the following description, when it is not necessary to distinguish between the two tools 2a, 2b from each other, the tools 2a, 2b will be collectively referred to as "tool 2" hereinafter. Each tool 2 includes, for example, a drive unit 24 including a motor. The drive unit 24 is activated by motive power (such as electric power etc.) supplied from a power source such as a battery pack 201 etc. Examples of this type of tool 2 include impact wrenches, nut runners, oil impulse wrenches, screwdrivers (including impact screwdrivers), drills, drill screwdrivers, and various other types of tools. Using this type of tool 2 enables a user to perform various processing operations, such as attaching fastening members (such as bolts or nuts etc.) to a workpiece (processing operation object) as the object of the operation, or making holes through the workpiece etc.

[0026] In addition, in the present embodiment, a camera unit 5 is provided for each of the two tools 2a, 2b. The camera unit 5 generates a captured image. The camera unit 5 covers, for example, a socket 242 (refer to Figure 2 A) attached to the output shaft 241 of the tool 2 (refer to Figure 2 A) within its imaging range (field of view). Therefore, when the user is using the tool 2 for an operation, the camera unit 5 captures an image of the operation object and generates a captured image.

[0027] When the tool system 1 according to the present embodiment operates in an operation mode (to be described later), it identifies the current operation object, which is the operation object among the plurality of operation objects where the tool 2 is currently set in an appropriate position, based on the captured image (first captured image) captured by the camera unit 5. For example, the tool system 1 can determine whether the operation being performed by the user using the tool 2 follows a predetermined operation process by identifying the current operation object. Using the first captured image obtained by the camera unit 5 provided in the tool 2 enables, for example, providing support for the operation of the user using the tool 2 or managing the operation of the user using the tool 2.

[0028] As Figure 1 shown, the tool system 1 according to the present embodiment includes not only the tool 2, but also a display unit 62, a reference image generation unit 67, and an output unit 66.

[0029] In the registration mode (to be described later), the reference image generation unit 67 generates a reference image corresponding to the work object based on the captured image (second captured image) captured by the imaging unit 5 of the tool 2.

[0030] When the reference image generation unit 67 is generating a reference image, the output unit 66 causes the display unit 62 to display the second captured image captured by the imaging unit 5.

[0031] According to this configuration, when the reference image generation unit 67 is generating a reference image based on the second captured image, the output unit 66 causes the display unit 62 to display the second captured image. This enables a user of the tool system 1, for example, to check the second captured image displayed on the display unit 62 while the reference image generation unit 67 is generating the reference image. Therefore, the tool system 1 according to the present embodiment can use an image more suitable for recognizing the current work object as the reference image.

[0032] (2) Detailed Structure

[0033] Next, reference will be made to Figures 1 to 2 B to describe the detailed structure of the tool system 1 according to the present embodiment.

[0034] (2.1) Premise

[0035] The tool system 1 according to the present embodiment can be used, for example, in an assembly line for assembling products at a factory. In particular, in the present embodiment, it is assumed that each of the tools 2a and 2b included in the tool system 1 is a fastening tool such as an impact wrench for tightening fastening members (such as bolts or nuts). Specifically, it is assumed that the present embodiment is applied to a case where a single workpiece has a plurality of fastening portions to be fastened, and thus it is necessary for the user to attach the fastening members to each of these fastening portions to be fastened at a single work site by using at least one of the two tools 2a and 2b.

[0036] As used herein, a "fastening portion to be fastened" refers to a portion of a workpiece (processing work object) to which a fastening member is to be attached. For example, if the fastening member is a bolt, the fastening portion to be fastened is an area that surrounds and covers the screw hole to which the fastening member is to be attached. That is, in the present embodiment, a single workpiece has a plurality of such fastening portions to be fastened.

[0037] As used herein, "operation object" refers to an object or an operation area for which an operation is assumed to be performed by using tool 2. In particular, the operation object at which tool 2 is currently set in an appropriate position is sometimes referred to as the "current operation object" hereinafter. As used herein, the phrase "tool 2 is currently set in an appropriate position" refers to a situation where tool 2 is placed and ready to perform an operation on the operation object. In addition, as used herein, the phrase "placed and ready to perform an operation" refers not only to a situation where tool 2 has come into contact with the operation object, but also to a situation where tool 2 is about to come into contact with the operation object. That is, in a state where tool 2 is currently set in an appropriate position on the operation object, tool 2 may have come into contact with the operation object or may not have come into contact with the operation object yet. In the present embodiment, by way of example, it is assumed that each fastening part to be fastened among a plurality of fastening parts of a single workpiece is an operation object.

[0038] As used herein, "captured image" refers to an image captured by imaging unit 5 and includes still pictures (still images) and moving pictures (videos). "Moving pictures" also include, for example, a set of still pictures captured by stop-motion shooting. The captured image does not have to be the output data itself provided by imaging unit 5. For example, the captured image may have been subjected to data compression, conversion to another data format, cropping of an image portion from the image captured by imaging unit 5, focus adjustment, brightness adjustment, contrast adjustment, or any image processing among various other types of image processing as needed. In the present embodiment, it is assumed that the captured image is, for example, an image generated by imaging the workpiece by using imaging unit 5 provided in tool 2 in a state where tool 2 is currently set in an appropriate position on the operation object. In the present embodiment, for example, it is assumed that the captured image is a full-color moving picture.

[0039] In addition, as used herein, "first captured image" refers to a captured image captured by imaging unit 5 while the user is performing an operation by using tool 2. On the other hand, "second captured image" refers to a captured image captured by imaging unit 5 while a reference image corresponding to the operation object is being generated.

[0040] In addition, as used herein, "reference image" refers to an image generated based on the second captured image captured by imaging unit 5. The "reference image" may include a plurality of frames corresponding to a single operation object or a plurality of operation objects. In the present embodiment, by way of example, it is assumed that the frames are full-color still pictures. As used herein, the "plurality of frames corresponding to a single operation object or a plurality of operation objects" refers not only to a situation where the plurality of frames correspond one-to-one to the plurality of operation objects, but also to a situation where the plurality of frames correspond one-to-many to a single operation object or a plurality of operation objects.

[0041] In addition, as used herein, if something is "equipped to" something else, the former can be built into the latter (e.g., integrally and inseparably with the latter), or can be attached to the latter only as an external component (e.g., removably fixed with a coupler, for example).

[0042] In addition, as used herein, "operation process" means the process of the operation to be performed using the tool 2. For example, if a series of operation process steps to be performed on a single work object or multiple work objects are defined as a single operation process, the operation process represents the order of the operation process steps to be performed on the single work object or multiple work objects throughout the operation process. More specifically, if an instruction related to the operation to be performed on a single work object is an "operation instruction", the operation process is information representing a single operation instruction or multiple operation instructions for a single operation process and the order in which the operation process steps should be performed. In other words, the operation process represents which work object corresponds to which of the single or multiple operation processes, and also represents the sequential position of the corresponding operation process. In the following description of this embodiment, it is assumed that the operation process defines the order in which operations (including multiple operation process steps) should be performed on multiple work objects in a single workpiece.

[0043] (2.2) Structure of the tool

[0044] First, reference will be made to Figure 1 、 Figure 2 A of Figure 2 and B of

[0045] to describe the structure of the tool 2 (2a, 2b) in the tool system 1 according to this embodiment. Each of the two tools 2a, 2b according to this embodiment has the same structure.

[0046] The tool 2 according to this embodiment is used in the tool system 1 and includes a drive unit 24 and an imaging unit 5. In particular, in this embodiment, it is assumed that the tool 2 is an impact wrench that activates the drive unit 24 by using electric energy. Such a tool 2 can be used to perform a fastening operation of attaching a fastening member to a work object. The tool 2 further includes an impact mechanism 25 in addition to the drive unit 24.

[0047] The tool 2 further includes a main body 20. A drive unit 24 and an impact mechanism 25 are accommodated in the main body 20. In addition, the imaging unit 5, the control unit 3, the storage unit 4, the notification unit 211, and the communication unit 26 included in the tool system 1 are also accommodated in the main body 20.

[0048] The main body 20 of the tool 2 includes a cylinder 21, a grip portion 22, and an attachment portion 23. The cylinder 21 is formed in a cylindrical shape (e.g., a circular cylindrical shape in the present embodiment). The grip portion 22 protrudes along a normal line with respect to a part of the circumferential surface of the cylinder 21 (i.e., along the radius of the cylinder 21). The battery pack 201 is removably attached to the attachment portion 23. In the present embodiment, the attachment portion 23 is provided at the front end of the grip portion 22. In other words, the cylinder 21 and the attachment portion 23 are coupled together via the grip portion 22.

[0049] At least the drive unit 24 is accommodated in the cylinder 21. The drive unit 24 includes a motor. The drive unit 24 is configured to be activated by the power supplied from the battery pack 201 as a power source to the motor. An output shaft 241 protrudes from one axial end surface of the cylinder 21. When the drive unit 24 is activated, the output shaft 241 rotates about a rotation axis Ax1 aligned with the direction in which the output shaft 241 protrudes. That is, the drive unit 24 drives the output shaft 241 to rotate about the rotation axis Ax1. In other words, when the drive unit 24 is activated, a torque is applied to the output shaft 241, thereby causing the output shaft 241 to rotate.

[0050] A cylindrical socket 242 for rotating a fastening member (such as a bolt or a nut) is removably attached to the output shaft 241. The socket 242 rotates about the rotation axis Ax1 together with the output shaft 241. The user can appropriately select the size of the socket 242 attached to the output shaft 241 according to the size of the fastening member. According to this configuration, activating the drive unit 24 causes the output shaft 241 to rotate, thereby causing the socket 242 to rotate together with the output shaft 241. If the socket 242 is assembled to the fastening member at this time, the fastening member rotates together with the socket 242, thereby completing the operation of tightening or loosening the fastening member. In this way, the tool 2 can complete the operation of tightening or loosening the fastening member by activating the drive unit 24.

[0051] Optionally, a socket anvil can be attached to the output shaft 241 instead of the socket 242. The socket anvil is also removably attached to the output shaft 241. This enables a drill bit (such as a screwdriver bit or a drill bit) to be attached to the output shaft 241 via the socket anvil.

[0052] The tool 2 includes the impact mechanism 25 as described above. The impact mechanism 25 is configured to apply an impact force in the rotational direction of the output shaft 241 when the fastening torque (operating value) exceeds a predetermined level. This enables the tool 2 to apply a greater fastening torque to the fastening member.

[0053] The grip portion 22 is the part that the user holds when he or she is performing an operation. The grip portion 22 is equipped with a trigger switch 221 and a forward / reverse switch 222. The trigger switch 221 is a switch for controlling the on / off state of the operation performed by the drive unit 24, and enables adjustment of the rotational speed of the output shaft 241 according to the depth to which the trigger switch 221 is pulled. The forward / reverse switch 222 is a switch for switching the rotational direction of the output shaft 241 from the clockwise direction to the counterclockwise direction and from the counterclockwise direction to the clockwise direction.

[0054] The attachment portion 23 is formed in a flat rectangular parallelepiped shape. The battery pack 201 is removably attached to the side of the attachment portion 23 opposite to the grip portion 22.

[0055] The battery pack 201 includes a housing 202 made of resin and formed in a rectangular parallelepiped shape. A rechargeable battery (such as a lithium-ion battery, etc.) is housed inside the housing 202. The battery pack 201 supplies power to the drive unit 24, the control unit 3, the imaging unit 5, the communication unit 26, the notification unit 211, and other constituent members.

[0056] The attachment portion 23 is also equipped with an operation panel 231. The operation panel 231 can include, for example, a plurality of push button switches 232 and a plurality of LEDs (light-emitting diodes) 233. The operation panel 231 enables the user to input various settings of the tool 2 and confirm the state of the tool 2. That is, for example, by operating the push button switches 232 of the operation panel 231, the user is allowed to change the operation mode of the tool 2 or check the remaining capacity of the battery pack 201.

[0057] The attachment portion 23 also includes a light-emitting unit 234. The light-emitting unit 234 includes an LED, for example. When the user is using the tool 2 to perform an operation, the light-emitting unit 234 emits light toward the operation object. The light-emitting unit 234 can be turned on (ON) and off (OFF) by operating the operation panel 231. Alternatively, the light-emitting unit 234 can also be automatically lit when the trigger switch 221 is turned on.

[0058] (2.3) Overall Structure of the Tool System

[0059] Next, the overall structure of the tool system 1 according to the present embodiment will be described with reference to Figure 1 to illustrate.

[0060] As described above, the tool system 1 includes the portable tool 2. Additionally, in the present embodiment, the tool system 1 further includes the setting terminal 60.

[0061] (2.3.1) Tool

[0062] First, reference will be made to Figure 1 and Figure 2 A of Figure 2 and B of

[0063] to describe the structure of the tool 2. In this case, each tool 2 according to the present embodiment has at least an operation mode and a registration mode as its operation modes. As used herein, the "operation mode" refers to the operation mode in which the user uses the tool 2 to perform an operation. The registration mode herein refers to the operation mode of generating a reference image corresponding to the operation object. The operation mode can be switched by, for example, operating the operation panel 231. Alternatively, the operation mode can also be switched by operating another member (such as a thumbwheel switch, etc.) separately provided from the operation panel 231.

[0064] The imaging unit 5, the control unit 3, the storage unit 4, the notification unit 211, and the communication unit 26 are accommodated in the main body 20 of the tool 2. In the present embodiment, for example, the imaging unit 5 and the notification unit 211 can be accommodated in the cylinder 21. The control unit 3, the storage unit 4, and the communication unit 26 can be accommodated in the grip portion 22 or the attachment portion 23.

[0065] Specifically, the imaging unit 5 is provided in the front end portion of the cylinder 21 to face the front end of the output shaft 241 (i.e., to face the socket 242), so that the socket 242 attached to the output shaft 241 falls within the imaging range (refer to Figure 2 A of Figure 2B). The optical axis of the imaging unit 5 is arranged to be aligned with the rotation axis Ax1 of the output shaft 241. In the present embodiment, the imaging unit 5 is arranged such that its optical axis is within a predetermined distance from the rotation axis Ax1 of the output shaft 241 and the rotation axis Ax1 and the optical axis are substantially parallel to each other. Note that the imaging unit 5 does not have to generate a captured image such that the socket 242 attached to the output shaft 241 falls within its imaging range. Instead, the imaging unit 5 only needs to generate a captured image for identifying the current work object. As used herein, a "captured image for identifying the current work object" refers to an image generated when the workpiece is captured by the imaging unit 5 while the tool 2 is set at an appropriate position on the work object in the current state. The captured image only has to be an image that enables the user to identify the current work object. Therefore, the work object on which the tool 2 is currently set at an appropriate position may or may not be covered within the imaging range of the captured image.

[0066] When the tool 2 is operating in the work mode, the imaging unit 5 generates a first captured image. The first captured image according to the present embodiment is an image generated when the workpiece is captured by the imaging unit 5 equipped on the tool 2 while the tool 2 operating in the work mode is currently set at an appropriate position on the work object. The work object on which the tool 2 is currently set at an appropriate position may or may not be covered within the imaging range of the first captured image. When the tool 2 is operating in the work mode, the imaging unit 5 outputs the first captured image as time-series data to the recognition processing unit 33 of the control unit 3.

[0067] On the other hand, when the tool 2 is operating in the registration mode, the imaging unit 5 generates a second captured image. The second captured image according to the present embodiment is an image generated when the workpiece is captured by the imaging unit 5 equipped on the tool 2 while the tool 2 operating in the registration mode is currently set at an appropriate position on the work object. The work object on which the tool 2 is currently set at an appropriate position may or may not be covered within the imaging range of the second captured image. When the tool 2 is operating in the registration mode, the imaging unit 5 outputs the second captured image as time-series data to the communication unit 26 so that the communication unit 26 sends the second captured image to the setting terminal 60.

[0068] The notification unit 211 can be implemented as an LED, for example. The notification unit 211 can be provided at the other end of the cylinder 21 of the main body 20 opposite to the output shaft 241 for easy viewing by the user during work (refer to Figure 2 B).

[0069] The communication unit 26 adopts compliance with, such as a standard wireless communication protocol such as a low-power radio standard that does not require a license (e.g., specifying a low-power radio standard). In this embodiment, the communication unit 26 communicates wirelessly with the setting terminal 60. Alternatively, the communication unit 26 may also communicate with the setting terminal 60 by a wired communication method.

[0070] The control unit 3 may include, for example, a microcontroller including one or more processors and one or more memories as its main components. The microcontroller controls the functions of the control unit 3 by causing one or more processors to execute programs stored in one or more memories. The programs may be pre-stored in the memory. Alternatively, the programs may also be distributed after being stored in a non-transitory storage medium such as a memory card or downloaded via a telecommunication line. In other words, the programs are designed such that one or more processors function as the control unit 3.

[0071] The control unit 3 performs functions such as the drive control unit 31, the imaging control unit 32, the recognition processing unit 33, the notification control unit 34, the determination unit 35, the registration unit 36, and the reflection processing unit 37. Note that if no operation command is input to the trigger switch 221 or the operation panel 231 within a certain period of time, the control unit 3 enters the sleep mode. When any operation command is input to the trigger switch 221 or the operation panel 231 during the sleep mode, the control unit 3 is activated.

[0072] The drive control unit 31 controls the drive unit 24. Specifically, the drive control unit 31 activates the drive unit 24 so that the output shaft 241 rotates at a rotational speed determined by the pressing depth of the trigger switch 221 and in the rotational direction set by the forward / reverse switch 222.

[0073] The drive control unit 31 also controls the drive unit 24 such that the tightening torque becomes equal to the torque setting (operation setting). The drive control unit 31 has a torque estimation function for estimating the magnitude of the tightening torque. In this embodiment, the drive control unit 31 estimates the magnitude of the tightening torque based on the rotational speed of the drive unit 24 (motor) or any other parameter until the estimated value of the tightening torque reaches the in-position judgment level. When the estimated value of the tightening torque reaches the in-position judgment level, the drive control unit 31 estimates the magnitude of the tightening torque based on the number of strokes of the impact mechanism 25. When it is found that the number of strokes of the impact mechanism 25 has reached the threshold number based on the torque setting, the drive control unit 31 determines that the tightening torque should have reached the torque setting and stops operating the drive unit 24 (i.e., the motor). This enables the tool 2 to fasten the fastening member with a tightening torque that exactly matches the torque setting.

[0074] The imaging control unit 32 is a processing unit configured to control the imaging unit 5 such that the imaging unit 5 generates a first captured image and a second captured image.

[0075] The recognition processing unit 33 performs image processing that compares the first captured image generated by the imaging unit 5 with a reference image, and recognizes the current work object captured in the first captured image from among a plurality of work objects. In the present embodiment, the reference image is stored in the storage unit 4 (image storage unit 41).

[0076] Specifically, the recognition processing unit 33 performs pattern recognition processing on the first captured image using the reference image including a plurality of frames corresponding to a plurality of work objects as template data, thereby recognizing the current work object. That is, the recognition processing unit 33 recognizes the current work object captured in the first captured image by comparing the first captured image with the reference image including a plurality of frames corresponding to a plurality of work objects.

[0077] The recognition processing unit 33 recognizes the current work object, for example, by performing image processing (pattern recognition processing) on the data output from the imaging unit 5 in a moving picture format (i.e., the first captured image) on a frame-by-frame basis.

[0078] As used herein, "pattern recognition processing" refers to image processing for identifying what object is captured in an image based on the shape of the object captured in the image. Examples of this type of pattern recognition processing include pattern matching processing and processing for identifying an object captured in an image by using a learned model created through machine learning. Pattern matching as used herein refers to processing for comparing template data with an object (such as the first captured image, etc.) using the above-described template data. In addition, any suitable algorithm can be used in the method of machine learning. For example, a deep learning algorithm can be employed. Further, if the work object recognized in this way does not conform to the work instruction defined by the work process, the recognition processing unit 33 performs at least one of restricting the operation of the drive unit 24 and issuing a notification. In other words, the recognition processing unit 33 determines whether the work object recognized by the recognition processing unit 33 (i.e., the current work object) conforms to the work instruction defined by a preset work process. That is, the recognition processing unit 33 determines whether the work object recognized by the recognition processing unit 33 is consistent with the work object specified by the work instruction included in the work process.

[0079] Specifically, the recognition processing unit 33 extracts data of the job process associated with the current job object from the process storage unit 44 of the storage unit 4. Then, the recognition processing unit 33 determines whether the job object subjected to the current job instruction defined by the job process extracted from the process storage unit 44 is consistent with the recognized job object. If these job objects are consistent with each other, the recognition processing unit 33 determines that the recognized job object should comply with the job instruction defined by the job process. On the other hand, if these job objects are not consistent with each other, the recognition processing unit 33 determines that the recognized job object should not comply with the job instruction defined by the job process.

[0080] In the case where it is determined as a result of such a judgment that the job object recognized in this way should not comply with the job instruction defined by the job process, the recognition processing unit 33 performs at least one of restricting the operation of the drive unit 24 and issuing a notification. As used herein, for example, "notification" includes not only notifying the user but also notifying an external terminal (such as a mobile communication device, etc.).

[0081] Specifically, in the case where it is determined that the job object recognized in this way should not comply with the job instruction defined by the job process, even if the trigger switch 221 is pulled, the recognition processing unit 33 does not allow the drive unit 24 to be activated. That is, only when the recognition processing unit 33 determines that the job object recognized in this way should comply with the job instruction defined by the job process, is the activation of the drive unit 24 allowed. Therefore, even if the tool 2 is currently set at an appropriate position on a job object that does not conform to the job process, the drive unit 24 remains deactivated, thereby prohibiting the fastening operation. This can reduce the possibility of operating according to the wrong job process. Optionally, in the case where it is determined that the job object recognized in this way should not comply with the job instruction defined by the job process, the recognition processing unit 33 can lock the trigger switch 221 to prevent the user from pulling the trigger switch 221 in such a situation.

[0082] In addition, in the case where it is determined that the job object recognized in this way should not comply with the job instruction defined by the job process, the recognition processing unit 33 activates the notification unit 211 of the notification control unit 34. Therefore, the notification unit 211 serves as a user notification unit for notifying the user that the tool 2 is now set at an appropriate position on a job object that does not conform to the job process.

[0083] That is to say, the recognition processing unit 33 performs at least object recognition processing for recognizing the current work object as a predetermined process based on the first captured image. In other words, the recognition processing unit 33 performs at least the recognition of the current work object as a (predetermined) process. In addition, the recognition processing unit 33 also performs the following process determination processing as a predetermined process based on the first captured image. This process determination processing is used to compare the work object recognized thereby with the work instruction defined by the work process and thereby determine their correspondence. In other words, the recognition processing unit 33 determines the correspondence between the work object recognized thereby and the work instruction defined by the work process as a (predetermined) process. If the result of the process determination processing indicates that the work object does not conform to the work instruction, the recognition processing unit 33 restricts the operation of the drive unit 24 and / or issues a notification.

[0084] The notification control unit 34 controls the notification unit 211 provided for the tool 2. The notification control unit 34 preferably causes the notification unit 211 to light up in different ways in the case where the determination made by the recognition processing unit 33 is inconsistent and in the case where the determination made by the recognition processing unit 33 is consistent. For example, if the determination made by the recognition processing unit 33 is inconsistent, the notification control unit 34 may cause the notification unit 211 to light up in red. On the other hand, if the determination made by the recognition processing unit 33 is consistent, the notification control unit 34 may cause the notification unit 211 to light up in green. This enables the user to identify whether the work object conforms to the work process by visually checking the lighting state of the notification unit 211. Optionally, in the case where the trigger switch 221 is pulled in a state where the determination made by the recognition processing unit 33 is inconsistent, the notification control unit 34 may cause the notification unit 211 to light up.

[0085] The determination unit 35 is configured to determine whether the fastening torque is a normal fastening torque when the fastening member is attached to the part to be fastened. In this case, the determination unit 35 preferably determines whether the fastening torque is a normal fastening torque according to the work instruction defined by the work process. Specifically, the work instruction defined by the work process includes a target torque value associated with the work object. This enables the determination unit 35 to determine whether the work is being performed with the fastening torque specified by the work instruction by comparing the target torque value included in the work instruction with the fastening torque.

[0086] If, for example, the drive control unit 31 deactivates the drive unit 24 when the number of strokes of the impact mechanism 25 detected reaches a threshold number of times, the determination unit 35 determines that the fastening torque should be normal. On the other hand, if, for example, the drive control unit 31 deactivates the drive unit 24 by turning off the trigger switch 221 before the number of strokes of the impact mechanism 25 reaches the threshold number of times, the determination unit 35 determines that the fastening torque is insufficient (abnormal). The determination unit 35 also performs a result storage process for storing the determination result in association with the part to be fastened in the result storage unit 43.

[0087] The registration unit 36 performs an image registration process, a torque registration process, and an information registration process. The image registration process is a process of causing the image storage unit 41 of the storage unit 4 to store a reference image including a plurality of frames corresponding to a plurality of work objects. The torque registration process is a process of causing the torque storage unit 42 of the storage unit 4 to store a plurality of target torque values. The information registration process is a process of storing information related to the content of predetermined image processing performed by the reference image generation unit 67 when generating the reference image in the information storage unit 45 of the storage unit 4.

[0088] When performing the image registration process, the registration unit 36 causes the image storage unit 41 to store, for example, the reference image of each work object provided by the setting terminal 60 in association with the work object. In this case, the registration unit 36 can cause the image storage unit 41 to store a single reference image or two or more reference images for a single work object.

[0089] When performing the torque registration process, the registration unit 36 regards, for example, the torque value provided by the setting terminal 60 for each work object as the target torque value, and causes the torque storage unit 42 to store the target torque value in association with the work object.

[0090] When performing the information registration process, the registration unit 36 causes the information storage unit 45 to store, for example, the following information input through the setting terminal 60 in association with the reference image and the work process. This information is related to the content of predetermined image processing performed by the reference image generation unit 67 when generating the reference image. In the following description, such information related to the content of predetermined image processing will be hereinafter referred to as "processing information". Examples of the processing information include information related to the brightness adjustment performed by the reference image generation unit 67 when generating the reference image. The "information related to the brightness adjustment" includes, for example, information indicating which area in the reference image has undergone brightness adjustment and information indicating how much the brightness value of the second captured image has been adjusted.

[0091] The reflection processing unit 37 reflects information (i.e., processing information) related to predetermined image processing performed by the reference image generation unit 67 when generating a reference image onto the control of the imaging unit 5. The reflection processing unit 37 adjusts the exposure time, f-number, and other parameters when the imaging unit 5 generates a captured image according to the processing information. For example, if the reference image generation unit 67 performs brightness adjustment to reduce the brightness of the overall reference image when generating the reference image, the reflection processing unit 37 performs adjustment to shorten the exposure time to reduce the brightness of the first captured image captured by the imaging unit 5.

[0092] The storage unit 4 can be implemented as a semiconductor memory, for example, and performs the functions of the image storage unit 41, the torque storage unit 42 (target value storage unit), the result storage unit 43, the process storage unit 44, and the information storage unit 45. In the present embodiment, the image storage unit 41, the torque storage unit 42, the result storage unit 43, the process storage unit 44, and the information storage unit 45 are implemented as a single memory. However, this is merely an example and should not be construed as restrictive. Alternatively, these storage units 41, 42, 43, 44, and 45 can also be implemented as multiple memories. Alternatively, the storage unit 4 can also be implemented as a storage medium such as a memory card that can be attached to and removed from the tool 2.

[0093] The image storage unit 41 stores a reference image including a plurality of frames in association with a plurality of work objects. In the image storage unit 41, a single reference image frame can be stored in association with a single work object. Alternatively, in the image storage unit 41, a plurality of reference image frames obtained by photographing a single work object from various angles or in multiple different sizes can be stored in association with a single work object.

[0094] The torque storage unit 42 stores a plurality of target torque values (target values) in a one-to-one association with a plurality of work objects. As used herein, the "target torque value" refers to the target value of the fastening torque when the fastening member is attached to the associated work object.

[0095] The result storage unit 43 stores the determination results obtained by the determination unit 35 for a plurality of fastening portions in association with a plurality of work objects. It is recommended that: the result storage unit 43 stores the determination results obtained by the determination unit 35 in a state with a time stamp indicating the work time added. This enables the discrimination of the work object determination results for each workpiece on the assembly line.

[0096] The process storage unit 44 stores data related to a single job process or multiple job processes. As described above, a job process means a process of assuming work using the tool 2, and can be, for example, data for defining in what order to perform work on multiple job objects of a single workpiece.

[0097] The information storage unit 45 stores the processing information in association with the reference image and the job process.

[0098] (2.3.2) Setting terminal

[0099] Next, the structure of the setting terminal 60 will be described.

[0100] The setting terminal 60 can be, for example, a mobile telecommunications device such as a smart phone or a tablet computer, and includes a communication unit 61, a display unit 62, an operation unit 63, a control unit 64, and a storage unit 65.

[0101] The communication unit 61 adopts a wireless communication protocol conforming to a standard such as or a low-power radio standard that does not require a license (such as a specified low-power radio standard, etc.). In this embodiment, the communication unit 61 performs wireless communication with the communication unit 26 of the tool 2. However, this is only an example and should not be construed as restrictive. Alternatively, instead of communication by a wireless communication method, the communication unit 61 can also communicate with the communication unit 26 of the tool 2 by a wired communication method.

[0102] The display unit 62 and the operation unit 63 can be integrally implemented as a touch screen panel display, for example.

[0103] The control unit 64 can include, for example, a microcontroller including one or more processors and one or more memories as its main constituent elements. The microcontroller performs the functions of the control unit 64 by causing one or more processors to execute programs stored in one or more memories. The programs can be pre-stored in the memory. Alternatively, the programs can also be distributed after being stored in a non-transitory storage medium such as a memory card or downloaded via a telecommunications line. In other words, the programs are designed such that one or more processors serve as the control unit 64.

[0104] The control unit 64 is configured to control the communication unit 61, the display unit 62, and the operation unit 63. In addition, the control unit 64 also performs the functions of the output unit 66 and the reference image generation unit 67.

[0105] The reference image generation unit 67 generates a reference image including a plurality of frames corresponding to a single work object or a plurality of work objects. The reference image generation unit 67 generates a reference image including a plurality of frames based on the second captured image captured by the imaging unit 5 of the tool 2. More specifically, the reference image generation unit 67 generates a reference image by performing predetermined image processing on the second captured image. Optionally, when generating a reference image including a plurality of frames, the reference image generation unit 67 may change the content of the predetermined image processing frame by frame.

[0106] As used herein, "predetermined image processing" refers to image processing including, for example, cropping, masking, and brightness adjustment. The "cropping" process as used herein refers to designating a part of the reference image (second captured image) as a cropping area. The "cropping area" is an area to be compared with an area forming a part of the first captured image when the recognition processing unit 33 of the tool 2 is recognizing the current work object. In this case, once the cropping area is designated for the reference image, the remaining part of the reference image other than the part falling within the cropping area is not compared with the first captured image. The "masking" process as used herein refers to designating a part of the reference image (second captured image) as a masking area. The "masking area" is an area not compared with the first captured image when the recognition processing unit 33 of the tool 2 is recognizing the current work object. In this case, once the masking area is designated for the reference image, the remaining part of the reference image other than the part falling within the masking area is compared with the first captured image. The "brightness adjustment" as used herein refers to an adjustment for increasing or decreasing the brightness of at least a part of the reference image (second captured image). The brightness adjustment is appropriately performed according to the environment around the work object when the second captured image is captured. For example, in a case where a workpiece on which a user is performing an assembly operation is irradiated with light from, for example, a lighting fixture, resulting in a so-called "highlight overflow" phenomenon in which at least a part of the reference image has a saturated brightness value, brightness adjustment can be performed. Performing image processing including cropping, masking, and brightness adjustment enables the use of an image more suitable for recognizing the current work object as the reference image.

[0107] Note that the reference image generation unit 67 may generate a single reference image frame or a plurality of reference image frames in association with a single work object. The reference image generation process to be performed by the reference image generation unit 67 will be described in detail in the section of "(3.3) Reference Image Generation Process".

[0108] When the reference image generation unit 67 is generating a reference image, the output unit 66 causes the display unit 62 to display the second captured image. The output unit 66 according to the present embodiment causes the display unit 62 to display the second captured image received by the communication unit 61. In the tool system 1 according to the present embodiment, the tool 2a and the display unit 62 are provided separately from each other. This enables the tool system 1 according to the present embodiment to display the second captured image while reducing the size of the tool 2a.

[0109] The storage unit 65 can be implemented as a semiconductor memory such as a flash memory. The storage unit 65 stores the reference image generated by the reference image generation unit 67.

[0110] (3) Operation

[0111] Next, reference Figures 3 to 7 will be made to describe an exemplary operation of the tool system 1 according to the present embodiment.

[0112] In the following description, an example will be described of how the tool system 1 operates when a user is performing an assembly operation on a workpiece in an assembly line. Further, in the example to be described below, the workpiece on which the user performs the assembly operation has two work objects, and the user performs an operation of attaching a fastening member to the two work objects by using the tool 2. Further, in the following description, the tool system 1 generates a reference image based on the second captured image captured by the imaging unit 5 provided in the tool 2a. Then, the user performs an operation of attaching a fastening member to the two work objects by using, for example, a tool 2b different from the tool 2a.

[0113] (3.1) Registration mode

[0114] First, reference Figure 3 will be made to describe an exemplary operation of the tool system 1 when assembling a workpiece. In this case, it is assumed that the two tools 2a and 2b are in an initial state in which the registration unit 36 has not performed any of the image registration process, the torque registration process, the information registration process, and other processes. That is, in the tools 2a and 2b in the initial state, none of the reference image, the target torque value, and the processing information corresponding to the first work object TG1 and the second work object TG2 have been stored in the image storage unit 41, the torque storage unit 42, or the information storage unit 45.

[0115] First, in response to an operation performed by the user, the tool 2a sets the operation mode to the registration mode (in S1). Next, the user sets the front end portion of the tool 2a (i.e., the socket 242) at an appropriate position on the first work object TG1 of the workpiece, and can turn on the trigger switch 221 once. In response, the imaging unit 5 of the tool 2a captures the first work object TG1 (in S2). Accordingly, the tool 2a generates a second captured image corresponding to the first work object TG1 and transmits the second captured image to the setting terminal 60 (in S3). Next, the imaging unit 5 captures the second work object TG2 in the same manner as in the first work object TG1 (in S4). Accordingly, the tool 2a generates a second captured image corresponding to the second work object TG2 and transmits the second captured image to the setting terminal 60 (in S5).

[0116] When the communication unit 61 of the setting terminal 60 receives the second captured image, the output unit 66 of the setting terminal 60 causes the display unit 62 to display a screen image D1 including any one of the second captured images (in S6).

[0117] Figure 4 An exemplary screen image D1 displayed on the display unit 62 when the reference image generation unit 67 is generating a reference image based on the second captured image is illustrated. As used herein, a "screen image" such as the screen image D1 refers to an image (including text, graphics, and icons) displayed on the display unit 62 of the setting terminal 60. Alternatively, the "screen image" may also be, for example, an image projected onto a screen by a display device such as a projector.

[0118] The screen image D1 is an image including the second captured image displayed when the reference image generation unit 67 is generating a reference image based on the second captured image. As Figure 4 shown, the screen image D1 includes a first region R1, a second region R2, and a third region R3. The first region R1 is a region for displaying the second captured image. The second region R2 is a region indicating the position of the work object corresponding to the reference image generated by the reference image generation unit 67 in a certain work process. The third region R3 is a region for displaying one or more objects B1 to B4.

[0119] Specifically, objects B1 and B2 are used to change the second captured image to be displayed in the first region R1 or the operation process associated with the reference image. Object B3 is used to switch the screen image D1 to a screen image that enables the user to input the target torque value of the operation object corresponding to the reference image (i.e., the second captured image displayed in the first region R1). Object B4 is used to register the reference image generated based on the second captured image in association with a certain operation object. The user is allowed to advance the reference image generation process performed by the reference image generation unit 67 by manipulating these objects B1 to B4.

[0120] The user is allowed to determine whether the second captured image captured by the imaging unit 5 is suitable for the reference image by checking the second captured image included in the screen image D1 displayed on the display unit 62.

[0121] The reference image generation unit 67 generates a reference image by performing predetermined image processing on the second captured image displayed on the display unit 62. The reference image generation process will be described in detail in the section of "(3.3) Reference image generation process".

[0122] When the user performs a predetermined operation such as manipulating object B4, the reference image generation unit 67 ends the reference image generation process (in S7). As a result of the operations performed so far, the reference image including two frames corresponding to two operation objects of the workpiece on which the user is performing the assembly operation is stored in the storage unit 65 of the setting terminal 60. In addition, the processing information and the target torque value associated with the thus generated reference image are also stored in the storage unit 65 of the setting terminal 60. Furthermore, the reference image, the processing information, and the target torque value are also stored in the storage unit 65 in association with the operation process of the operation step.

[0123] (3.2) Operation mode

[0124] Next, in response to a predetermined operation performed by the user on the operation panel 231 of the tool 2b, the tool 2b sets the operation mode to the operation mode (in S8). In this case, the tool 2b is different from the tool 2a equipped with the imaging unit 5 that captures the second captured image.

[0125] At this time point, a reference image including a plurality of frames respectively corresponding to two work objects of the workpiece on which the user is performing an assembly operation has not been registered with the tool 2b. In response to a predetermined operation performed by the user on the operation panel 231 of the tool 2b, the tool 2b causes the communication unit 26 to send a request to the setting terminal 60 for sending the reference image, the processing information associated with the reference image, and the target torque value (in S9). The communication unit 61 of the setting terminal 60 receives the sending request from the tool 2b. When receiving the sending request from the tool 2b, the control unit 64 of the setting terminal 60 causes the communication unit 61 to send the reference image of each work object stored in the storage unit 65, the processing information associated with the reference image, and the target torque value data to the tool 2b (in S10). Next, the communication unit 26 of the tool 2b receives the reference image, the processing information, and the target torque value data of each work object from the setting terminal 60. Then, the registration unit 36 of the control unit 3 performs the process of registering the reference image, the processing information, and the target torque value of each work object (in S11). When this registration process ends, the user is ready to perform an assembly operation on the workpiece using the tool 2b.

[0126] Next, the user follows the operation process for the assembly operation of the workpiece and sequentially performs operations on each workpiece object (i.e., the first work object TG1 and the second work object TG2) using the tool 2b. First, the user sets the front end portion (i.e., the socket 242) of the tool 2b at an appropriate position on the first work object TG1 of the workpiece and can turn on the trigger switch 221 once. In response, the imaging unit 5 of the tool 2b generates a first captured image corresponding to the first work object TG1. At this time, the reflection processing unit 37 of the tool 2b reflects the processing information to the control of the imaging unit 5 according to the operation process. For example, if it is assumed that the operation to be performed by the user next is a fastening operation on the first work object TG1, the reflection processing unit 37 reflects the processing information associated with the first work object TG1 to the control of the imaging unit 5. Reflecting information related to image processing such as the brightness adjustment performed on the reference image to the control of the imaging unit 5 enables the imaging unit 5 to generate a first captured image suitable for comparison with the reference image.

[0127] The recognition processing unit 33 performs image processing for comparing the first captured image generated by the imaging unit 5 with the reference image stored in the image storage unit 41, thereby recognizing the current work object as the first work object TG1 (in S12). Accordingly, the drive control unit 31 sets the target torque value associated with the first work object TG1 as the torque setting value (in S13). Further, the notification control unit 34 controls the notification unit 211 to cause the notification unit 211 to make the following notification: the current work object at the proper position where the tool 2b is currently set has been recognized as the first work object TG1.

[0128] When it is confirmed based on the content of the notification made by the notification unit 211 that the current work object has been recognized as the first work object TG1, the user performs a fastening operation of attaching a fastening member to the first work object TG1 (in S14).

[0129] The determination unit 35 determines whether the fastening torque when attaching the fastening member to the first work object TG1 is a normal fastening torque. Then, the determination unit 35 transmits the determination result (the first determination result) to the setting terminal 60 via the communication unit 26 (in S15). When the communication unit 61 of the setting terminal 60 receives the first determination result from the tool 2b, the control unit 64 causes the storage unit 65 to store the first determination result in association with the first work object TG1 (in S16).

[0130] In addition, the user also performs a fastening operation on the second work object TG2 following the same operation procedure as that for the first work object TG1.

[0131] Specifically, the user sets the front end of the tool 2b (i.e., the socket 242) at an appropriate position on the second work object TG2, and can turn on the trigger switch 221 once. In response, the imaging unit 5 captures a first captured image corresponding to the second work object TG2. At this time, the reflection processing unit 37 of the tool 2b reflects the processing information on the control of the imaging unit 5 according to the operation process. Then, the recognition processing unit 33 performs image processing, thereby recognizing the current work object as the second work object TG2 (in S17). Therefore, the drive control unit 31 sets the target torque value associated with the second work object TG2 as the torque setting value (in S18). In addition, the notification control unit 34 controls the notification unit 211 to cause the notification unit 211 to make the following notification: the current work object at the appropriate position where the tool 2b is currently set has been recognized as the second work object TG2. When it is confirmed based on the content of the notification made by the notification unit 211 that the current work object has been recognized as the second work object TG2, the user performs a fastening operation of attaching a fastening member to the second work object TG2 (in S19). The determination unit 35 determines whether the fastening torque used in the fastening operation in step S19 is a normal torque. Then, the determination unit 35 sends the determination result (the second determination result) to the setting terminal 60 via the communication unit 26 (in S20). When the communication unit 61 of the setting terminal 60 receives the second determination result from the tool 2b, the control unit 64 causes the storage unit 65 to store the second determination result in association with the second work object TG2 (in S21).

[0132] After the user has performed an assembly operation on the first work object TG1 and the second work object TG2 of one workpiece in this way using the tool 2b, the user will perform an assembly operation on other workpieces in the same way with the operation mode of the tool 2b set to the work mode.

[0133] Note that Figure 3 The flowchart shown only shows an exemplary operation of the tool system 1. Therefore, Figure 3 The processing steps shown can be appropriately carried out in a different order, additional processing steps can be performed as needed, or at least one of these processing steps can be appropriately omitted.

[0134] (3.3) Reference Image Generation Processing

[0135] Next, the reference Figures 4 to 7 will be used to further elaborate on how the reference image generation unit 67 performs the process of generating a reference image.

[0136] As described above, when the reference image generation unit 67 is generating a reference image based on the second captured image, the output unit 66 causes the display unit 62 to display the screen image D1. At Figure 4In the example shown, a second captured image corresponding to the first work object TG1 is displayed in the first region R1 of the screen image D1. The image Im1 of the second captured image is an image of the socket 242 currently set at an appropriate position on the first work object TG1. The image Im2 of the second captured image is an image of the second work object TG2.

[0137] The reference image generation unit 67 performs predetermined image processing according to an operation command input by the user by operating the operation panel. That is, the user is allowed to operate the screen image D1 while the second captured image displayed on the inspection display unit 62 to cause the reference image generation unit 67 to perform predetermined image processing. For example, if the user finds that the second captured image (reference image) displayed on the display unit 62 is relatively dark as a whole (that is, if he or she finds that its brightness is low), then he or she can generate a reference image that is brighter than the original second captured image by causing the reference image generation unit 67 to perform brightness adjustment to increase the brightness.

[0138] Optionally, the user can also cause the reference image generation unit 67 to perform predetermined image processing involving at least one of cropping and masking by performing a predetermined operation such as dragging or pinching on the screen image D1.

[0139] Figure 5 An exemplary situation is illustrated in which a part of the reference image has been defined as a cropping area (i.e., the area surrounded by the rectangle Tr1) in response to a predetermined operation performed by the user on the screen image D1. Once the reference image is generated by setting the cropping area, the remaining part of the reference image other than the cropping area is not compared with the first captured image. In Figure 5 In the example shown, only the cropping area of the reference image displayed in the first region R1 and surrounded by the rectangle Tr1 is compared with the first captured image.

[0140] The reference image with the cropping area set is stored in the storage unit 65 together with additional information including information related to the coordinates of the cropping area in the entire reference image. Such additional information is used when the recognition processing unit 33 compares the first captured image with the cropped reference image. That is, when recognizing the current work object, the recognition processing unit 33 regards a part of the first captured image as the area to be compared with the cropping area according to the coordinate information included in the additional information. In this case, each of the first captured image and the second captured image (reference image) is a captured image captured by the imaging unit 5 in a state where the tool 2 is set at an appropriate position on the work object. Therefore, the coordinates of this part of the first captured image are substantially the same as the coordinates of the cropping area in the reference image. Therefore, the recognition processing unit 33 defines a specific area of the first captured image as the area to be compared with the cropping area according to the coordinate information of the cropping area. As used herein, the "specific area" refers to the area of the first captured image to be subjected to the pattern recognition process. In the present embodiment, it is assumed that the area of the first captured image having the same coordinate set as the cropping area is the specific area. Then, the recognition processing unit 33 recognizes the current work object by comparing the cropping area with the specific area of the first captured image.

[0141] When the result of the pattern recognition process indicates that the similarity (i.e., the degree of coincidence) between the cropping area of the reference image and the specific area of the first captured image is equal to or greater than a predetermined value (threshold), the recognition processing unit 33 determines that the reference image and the first captured image are identical. On the other hand, if the similarity between the cropping area of the reference image and the specific area of the first captured image is less than the predetermined value (threshold), the recognition processing unit 33 determines that the reference image and the first captured image are not identical.

[0142] In addition, the additional information is information that enables the cropping area to be moved and / or rotated relative to the first captured image when the recognition processing unit 33 compares the cropping area and the first captured image with each other. The additional information also includes coordinate information related to the area around (or near) the cropping area. This enables the recognition processing unit 33 to define the specific area of the first captured image as an area wider than the cropping area according to the coordinate information of the area around the cropping area. In the present embodiment, the area with the coordinate set identical to the coordinate set of the cropping area and the coordinate set of the area around (near) the cropping area (i.e., the coordinate set of the coordinate information included in the additional information) is defined as the specific area. Then, the recognition processing unit 33 can perform the pattern recognition process of moving and / or rotating the cropping area relative to the first captured image within the specific area of the first captured image. This enables, for example, accurate recognition of the current work object even when the first captured image and the reference image are captured from slightly different positions.

[0143] In addition, the additional information defines at least one of a range in which the cropping area is movable relative to the first captured image and a range in which the cropping area is rotatable relative to the first captured image. That is, the range in which the cropping area can be moved and / or rotated relative to the first captured image is limited to a specific area of the first captured image defined by the coordinate information of the additional information. Limiting the movable and rotatable ranges of the cropping area can reduce the burden on the pattern recognition process applied to the recognition processing unit 33.

[0144] Optionally, the reference image generation unit 67 may set a plurality of cropping areas with respect to the second captured image (reference image) displayed on the display unit 62. Figure 6 An example is shown in which a plurality of cropping areas (i.e., the area surrounded by the rectangle Tr2 and the area surrounded by the rectangle Tr3) are set on the reference image in response to a predetermined operation performed by the user on the screen image D1.

[0145] In the case of setting a plurality of cropping areas with respect to the reference image, the reference image generation unit 67 may weight each of the plurality of cropping areas. In Figure 6 , the image Im3 represents the weight of the cropping area surrounded by the rectangle Tr2, and the image Im4 represents the weight of the cropping area surrounded by the rectangle Tr3. As used herein, "weight" represents the degree of contribution of each cropping area to the recognition of the current job object by the recognition processing unit 33. In the present embodiment, when the recognition processing unit 33 is recognizing the current job object, the cropping area with a larger weight contributes significantly more to the recognition of the current job object than the cropping area with a smaller weight. In the present embodiment, the recognition processing unit 33 calculates a score for each cropping area by multiplying the similarity between each cropping area and the first captured image by the weight of the cropping area. In addition, the recognition processing unit 33 adds up the scores of the plurality of cropping areas. When it is found that the total score is equal to or greater than a predetermined score, the recognition processing unit 33 determines that the first captured image and the reference image match each other.

[0146] Figure 7 An exemplary case is shown in which a part of the reference image is defined as a shielding area (i.e., the shaded area surrounded by the rectangle Ma1) in response to a predetermined operation performed by the user on the screen image D1. In the case of generating a reference image by setting a shielding area, the remaining part of the reference image other than the shielding area is compared with the first captured image. In Figure 7 In the example shown, the remaining part of the reference image displayed on the first area R1 other than the shielding area surrounded by the rectangle Ma1 is to be compared with the first captured image.

[0147] A shielding area is set so that unwanted image areas can be removed from the images to be compared, in which some body parts of the user, light projected from lighting fixtures, or any other unwanted things are reflected on the surface of the workpiece on which the user is performing an assembly operation. In particular, the tool system 1 according to the present embodiment causes the imaging unit 5 provided in the tool 2 to generate a second captured image (reference image), so that, for example, the front end portion (i.e., the socket 242) of the tool 2 is generally captured as part of the reference image. In this case, if a certain part (such as a screwdriver bit) that looks different depending on the stop angle is attached to the front end of the tool 2, the image of the screwdriver bit captured as part of the reference image may be different from the image of the screwdriver bit captured as part of the first captured image. In this case, setting the front end portion of the tool 2 as the shielding area enables the recognition processing unit 33 to more accurately recognize the current operation object. That is, setting the shielding area enables the reference image generation unit 67 to generate a reference image more suitable for recognizing the current operation object.

[0148] The reference image with the shielding area set and the reference image with the cropping area set are also stored in the storage unit 65 together with additional information that is coordinate information related to the shielding area in the entire reference image.

[0149] (4) Variant

[0150] Note that the above-described exemplary embodiments are merely exemplary embodiments among the various embodiments of the present invention and should not be construed as restrictive. On the contrary, the exemplary embodiments can be easily modified in various ways according to design choices or any other factors without departing from the scope of the present invention. All the figures to be referred to in this specification are schematic diagrams. Therefore, the ratios of the dimensions (including thicknesses) of the respective components illustrated in these figures do not always reflect the actual dimensional ratios of these components. In addition, the functions of the tool system 1 according to the exemplary embodiment can also be implemented as a reference image generation method, a (computer) program, or a non-transitory storage medium storing the program. The reference image generation method according to one aspect includes a reference image generation step. The reference image generation step includes: generating a reference image to be compared with the first captured image when currently identifying a current work object in which a first tool 2b among a plurality of tools 2a, 2b is currently set at an appropriate position, based on a second captured image. Each of the plurality of tools 2a, 2b is a portable tool and includes: a drive unit 24 that is activated using power supplied from a power source; and an imaging unit 5. The first captured image is captured by the imaging unit 5 of the first tool 2b that is currently set at an appropriate position among the plurality of tools 2a, 2b when currently identifying the current work object. The second captured image is captured by the imaging unit 5 of the second tool 2a among the plurality of tools 2a, 2b. The reference image generation step further includes a display step. The display step includes: causing the display unit 62 to display the second captured image when the reference image is being generated. The program according to another aspect is designed such that one or more processors perform the above-described reference image generation method.

[0151] Next, modification examples of the exemplary embodiments will be listed one by one. Note that the modification examples to be described below can be combined as appropriate. In the following description, any component having the same function as the counterpart in the above-described exemplary embodiment will be designated by the same reference numeral as that of the counterpart, and the description thereof will be appropriately omitted here.

[0152] In the above embodiment, when comparing the cropped area with the first captured image, the recognition processing unit 33 compares a specific area of the first captured image with the cropped area based on the coordinate information of the cropped area. However, this is merely an example and should not be construed as restrictive. Alternatively, the recognition processing unit 33 can perform pattern recognition processing using the cropped area on the entire first captured image without using the coordinate information of the cropped area.

[0153] In addition, in the above-described embodiment, the recognition processing unit 33 adds up the scores of the plurality of cropped regions, and determines that the first captured image and the reference image match each other when it is found that the total score is equal to or greater than a predetermined score. However, this is merely an example and should not be construed as restrictive. Alternatively, the recognition processing unit 33 may also determine that the first captured image and the reference image match each other when it is found that the score of any one of the plurality of cropped regions is equal to or greater than a predetermined score.

[0154] In addition, in the above-described embodiment, the recognition processing unit 33 calculates the score for each cropped region by multiplying the similarity between each cropped region and the first captured image by the weight of the cropped region. However, this is merely an example and should not be construed as restrictive. Alternatively, the recognition processing unit 33 may also calculate the score for each cropped region by performing some other type of arithmetic operation that correlates the weight with the similarity (for example, by adding the weight to the similarity between each cropped region and the first captured image).

[0155] In addition, in the above-described embodiment, the reflection processing unit 37 reflects the processing information on the control of the imaging unit 5 according to the operation process in the operation mode. However, this is merely an example and should not be construed as restrictive. Alternatively, regardless of the operation process, the reflection processing unit 37 may also reflect the processing information on the control of the imaging unit 5. For example, if a reference image is generated by adjusting the brightness by the reference image generation unit 67, the reflection processing unit 37 may reflect the information related to the brightness adjustment on the control of the imaging unit 5 regardless of the operation process.

[0156] The tool system 1 according to the present invention includes a computer system, for example, in its control unit 3 and setting terminal 60. The computer system may include a processor and a memory as main hardware components. The functions of the tool system 1 according to the present invention can be performed by causing the processor to execute a program stored in the memory of the computer system. The program may be pre-stored in the memory of the computer system. Alternatively, the program may also be downloaded via a telecommunication line or distributed after being recorded on some non-transitory storage medium such as a memory card, optical disc, or hard disk drive (any of which is readable by the computer system). The processor of the computer system may be constituted by a single or multiple electronic circuits including semiconductor integrated circuits (ICs) or large-scale integrated circuits (LSIs). As used herein, an "integrated circuit" such as an IC or LSI is called by different names according to its degree of integration. Examples of integrated circuits such as ICs and LSIs include system LSIs, very large-scale integrated circuits (VLSIs), and ultra large-scale integrated circuits (ULSIs). Optionally, a field-programmable gate array (FPGA) to be programmed after manufacturing the LSI or a logic device allowing reconfiguration of connections or circuit sections inside the LSI may also be employed as the processor. These electronic circuits may be integrated together on a single chip or distributed on multiple chips, either of which is appropriate. These multiple chips may be aggregated together in a single device or distributed in multiple devices without limitation. As used herein, a "computer system" includes a microcontroller including one or more than one processor and one or more than one memory. Thus, the microcontroller may also be implemented as a single or multiple electronic circuits including semiconductor integrated circuits or large-scale integrated circuits.

[0157] Furthermore, in the above embodiment, at least some functions of the tool system 1 are aggregated together in a single housing (body 20). However, this is not a necessary configuration of the tool system 1. Alternatively, these constituent elements of the tool system 1 may be distributed in multiple different housings. For example, some functions of the control unit 3 may be provided in a housing separately provided from the body 20 of the tool 2. Alternatively, at least some functions of the control unit 3 may also be implemented, for example, as a server or a cloud computing system.

[0158] Note that the tool system 1 does not have to be applied to an assembly line for assembling workpieces in a factory, but may also find any other applications.

[0159] In the above first embodiment, the tool 2 is an impact wrench. However, the tool 2 does not have to be an impact wrench, but can also be, for example, a nut runner or an oil pulse wrench. Alternatively, the tool 2 can also be, for example, a screwdriver (including an impact screwdriver) for tightening a screw (as a fastening member). In this case, a drill bit (such as a screwdriver bit, etc.) instead of the socket 242 is attached to the tool 2. In addition, the tool 2 does not have to be configured to be powered by the battery pack 201, but can also be configured to be powered by an AC power supply (commercial power supply). Further, the tool 2 does not have to be an electric tool, but can also be a pneumatic tool, which includes a pneumatic motor (drive unit) that operates using compressed air (power) supplied from an air compressor (power source).

[0160] In addition, in the above typical embodiment, it is assumed that the work object is each fastening part among a plurality of fastening parts in a single workpiece. However, this is only an example and should not be construed as restrictive. Alternatively, the work object can also be a module, part, or product having a plurality of fastening parts. In this case, the plurality of fastening parts of a single work object can have the same target torque value or different target torque values from each other.

[0161] Alternatively, there can be only one work object. In addition, the reference image can be only one frame corresponding to a single work object.

[0162] In addition, in the above embodiment, the image storage unit 41 of the tool 2 stores a reference image including a plurality of frames corresponding to a plurality of work objects respectively. However, the tool 2 does not have to store such a reference image including a plurality of frames corresponding to a plurality of work objects respectively. Alternatively, the setting terminal 60 or the server device can include an image storage unit for storing such a reference image including a plurality of frames corresponding to a plurality of work objects respectively. In this case, the recognition processing unit 33 of the tool 2 can access the image storage unit of the setting terminal 60 or the server device to perform the process of comparing the first captured image captured by the imaging unit 5 with the reference image stored in the image storage unit and thereby recognizing the current work object. In addition, the tool 2 does not have to include the recognition processing unit 33. Alternatively, the setting terminal 60 or the server device can perform the function of the recognition processing unit 33. When the tool 2 outputs the first captured image captured by the imaging unit 5 to the setting terminal 60 or the server device, the processing unit of the setting terminal 60 or the server device performs the image processing of comparing the first captured image with the reference image and outputting the recognition result of the current work object to the tool 2.

[0163] Optionally, the tool 2 may include a torque sensor for measuring the fastening torque. In this case, the drive control unit 31 controls the drive unit 24 such that the fastening torque measured by the torque sensor becomes the torque setting. Further, the determination unit 35 may determine whether the fastening torque is normal by comparing the measurement result of the torque sensor with the target torque value. When it is found that the measurement result of the torque sensor falls within a predetermined range based on the target torque value, the determination unit 35 determines that the fastening torque should be a normal fastening torque. On the other hand, when it is found that the measurement result of the torque sensor falls outside the predetermined range based on the target torque value, the determination unit 35 determines that the fastening torque should be an insufficient (abnormal) fastening torque.

[0164] In addition, the notification unit 211 does not have to be a light-emitting unit such as an LED, but may also be implemented as an image display device such as a liquid crystal display or an organic electroluminescence (EL) display. Optionally, the notification unit 211 may notify (present) by any means other than display. For example, the notification unit 211 may also be implemented as a speaker or buzzer that emits sound (including voice). In this case, the notification control unit 34 preferably causes the notification unit 211 to emit different sounds in the case where the determination made by the recognition processing unit 33 indicates inconsistency and in the case where the recognition processing unit 33 recognizes the current work object. Alternatively, the notification unit 211 may also be implemented as a vibrator that generates vibration or a transmitter that sends a notification signal to an external terminal (such as a mobile communication device, etc.) provided outside the tool 2. Optionally, the notification unit 211 may also have two or more functions selected from display, emitting sound, generating vibration, and establishing communication in combination.

[0165] The storage unit 4 may store operation process data representing a predetermined order of operation process steps to be performed on a plurality of work objects. In this case, the recognition processing unit 33 selects a reference image frame for use in image processing (pattern recognition processing) from the reference images including a plurality of frames according to the operation process. Specifically, the recognition processing unit 33 preferentially selects one reference image frame corresponding to the upcoming work object to be processed in the upcoming operation process step from the plurality of reference image frames. As used herein, the "upcoming work object" is the work object to be processed after the last recognized work object. The recognition processing unit 33 performs image processing for comparing the reference image selected as the template data with the first captured image. That is, the recognition processing unit 33 selects the reference image by predicting the work object to be captured in the first captured image next according to the operation process. This enables the recognition processing unit 33 to recognize the work object captured in the first captured image in a shorter time.

[0166] Optionally, the recognition processing unit 33 may also be configured to determine the type of the socket 242 attached to the tool 2 by performing image processing on the first captured image. As used herein, "type" is information for distinguishing different parts from each other and includes at least one piece of information related to size (dimension or length), shape, or material. In the present embodiment, the recognition processing unit 33 is configured to determine the length of the socket 242 attached to the tool 2. The recognition processing unit 33 corrects the target torque value according to the length of the socket 242 and sets the thus corrected target torque value as the torque setting. For example, the recognition processing unit 33 corrects the target torque value associated with the current work object by multiplying the target torque value by a coefficient corresponding to the length of the socket 242 and sets the thus corrected target torque value as the torque setting. That is, the recognition processing unit 33 controls the drive unit 24 such that the fastening torque becomes equal to the corrected target torque value. This can reduce the deviation of the fastening torque according to the length of the socket 242.

[0167] Optionally, the recognition processing unit 33 may also be configured to determine the torque setting according to the detected length (or type) of the socket 242. In the storage unit 4, torque values corresponding one-to-one to various lengths of the socket 242 are stored. The recognition processing unit 33 obtains the torque value corresponding to the determined length of the socket 242 from the storage unit 4 and sets the value based on the thus obtained torque value as the torque setting. For example, the recognition processing unit 33 may set the torque value obtained from the storage unit 4 as the torque setting. This enables the fastening operation to be performed with a torque value corresponding to the type of the given socket 242.

[0168] In the above-described exemplary embodiment, the recognition processing unit 33 recognizes the current work object by performing pattern recognition processing. However, the pattern recognition processing is merely an exemplary processing that enables the recognition processing unit 33 to recognize the current work object. Alternatively, instead of the pattern recognition processing, the recognition processing unit 33 may also recognize the current work object by determining the correspondence between the first captured image and a reference image or performing any one of various types of image processing for searching for a reference image corresponding to the first captured image.

[0169] In addition, the imaging unit 5 does not have to be provided to the cylinder body 21 of the main body 20, but may be provided to, for example, the attachment portion 23 of the main body 20 or the battery pack 201. Similarly, the arrangements of the control unit 3, the storage unit 4, and other units may also be appropriately changed.

[0170] Note that the setting terminal 60 does not have to be a mobile communication device that the user can carry around. Alternatively, the setting terminal 60 may also be, for example, a server that can communicate with the communication unit 26 of the tool 2.

[0171] The tool system 1 only needs to include at least a reference image generation unit 67 and an output unit 66. That is to say, the tool 2 is not an essential component of the tool system 1. In addition, in the above typical embodiment, the tool system 1 is implemented as a single system including a reference image generation unit 67 and an output unit 66. Alternatively, the tool system 1 can also be implemented as two or more systems. For example, the functions of the reference image generation unit 67 and the output unit 66 can be distributed in two or more systems. In addition, at least one function of the reference image generation unit 67 or the output unit 66 can be distributed in two or more systems. For example, the function of the reference image generation unit 67 can be distributed in two or more devices. Optionally, at least some functions of the tool system 1 can also be implemented as a cloud computing system.

[0172] (Generalization)

[0173] As can be seen from the above description, the tool system (1) according to the first aspect is for a plurality of tools (2a, 2b). Each tool among the plurality of tools (2a, 2b) is a portable tool and includes: a drive unit (24) that is activated by using power supplied from a power source; and an imaging unit (5). The tool system (1) includes a reference image generation unit (67). The reference image generation unit (67) generates a reference image to be compared with a first captured image. The first captured image is captured by the imaging unit (5) of the first tool (2a; 2b) currently set in place among the plurality of tools (2a, 2b) when identifying a current work object among a plurality of work objects where the first tool (2a; 2b) is currently set in place. The reference image generation unit (67) generates a reference image based on a second captured image. The second captured image will be captured by the imaging unit (5) of the second tool (2a; 2b) among the plurality of tools (2a, 2b). The tool system (1) further includes an output unit (66). The output unit (66) causes the display unit (62) to display the second captured image when the reference image generation unit (67) is generating the reference image.

[0174] This aspect enables the user to check the second captured image while the reference image is being generated, thereby enabling a more suitable image for identifying the current work object to be used as the reference image.

[0175] In the tool system (1) according to the second aspect that can be implemented in combination with the first aspect, the display unit (62) is provided separately from the plurality of tools (2a, 2b).

[0176] According to this aspect, none of the plurality of tools (2a, 2b) includes a display unit (62), thereby making it possible to reduce the size of the plurality of tools (2a, 2b) and thereby improve the workability of the plurality of tools (2a, 2b).

[0177] The tool system (1) according to the third aspect, which can be implemented in combination with the second aspect, further includes a communication unit (61) that is ready to communicate with each of the plurality of tools (2a, 2b) via at least one of wired communication and wireless communication. The communication unit (61) receives a second captured image from the second tool (2a; 2b). The output unit (66) causes the display unit (62) to display the second captured image received by the communication unit (61).

[0178] According to this aspect, the output unit (66) causes the display unit (62) to display the second captured image received by the communication unit (61). Therefore, it is not necessary to transfer the second captured image that the output unit 66 causes the display unit 62 to display via a storage medium such as a USB memory.

[0179] In the tool system (1) according to the fourth aspect, which can be implemented in combination with any one of the first aspect to the third aspect, the reference image generation unit (67) generates a reference image by performing predetermined image processing on the second captured image displayed on the display unit (62).

[0180] According to this aspect, the image generated by performing predetermined image processing on the second captured image displayed on the display unit (62) is used as the reference image, thereby making it possible to use an image more suitable for recognizing the current work object as the reference image.

[0181] In the tool system (1) according to the fifth aspect, which can be implemented in combination with the fourth aspect, the reference image generation unit (67) performs predetermined image processing according to an operation command input by the user.

[0182] According to this aspect, the image generated by performing predetermined image processing on the second captured image displayed on the display unit (62) according to the operation command input by the user is used as the reference image, so that it is possible to use an image more suitable for recognizing the current work object as the reference image.

[0183] In the tool system (1) according to the sixth aspect, which can be implemented in combination with the fourth aspect or the fifth aspect, the reference image includes a plurality of frames corresponding to a single work object or a plurality of work objects. The reference image generation unit (67) can change the content of the predetermined image processing frame by frame when generating the reference image including a plurality of frames.

[0184] According to this aspect, when generating a reference image including a plurality of frames, the reference image generation unit (67) changes the content of predetermined image processing frame by frame, thereby enabling appropriate predetermined image processing to be performed frame by frame.

[0185] In the tool system (1) according to the seventh aspect, which can be implemented in combination with any one of the fourth aspect to the sixth aspect, the predetermined image processing involves at least one of a cropping process and a masking process. The cropping process includes: setting a cropping area to be compared when currently identifying a job object. The masking process includes: setting a masking area that is not compared when currently identifying a job object.

[0186] According to this aspect, predetermined image processing involving at least one of a cropping process and a masking process is performed on the second captured image displayed on the display unit (62). This enables the tool system (1) to compare the image area in the second captured image that is suitable for identifying the current job object with the first captured image.

[0187] In the tool system (1) according to the eighth aspect, which can be implemented in combination with the seventh aspect, additional information is provided to the reference image generated by the cropping process. The additional information is information that enables the cropping area to be moved and / or rotated relative to a part of the first captured image when the cropping area is being compared with the part of the first captured image.

[0188] According to this aspect, when a part of the first captured image is being compared with the cropping area, the tool system (1) moves and / or rotates the cropping area relative to the first captured image. Therefore, the tool system (1) enables the accuracy of identifying the current job object to be further improved.

[0189] In the tool system (1) according to the ninth aspect, which can be implemented in combination with the eighth aspect, the additional information defines at least one of the range in which the cropping area can move relative to the first captured image and the range in which the cropping area can rotate relative to the first captured image.

[0190] According to this aspect, the cropping area is moved and / or rotated within the range defined by the additional information, thereby reducing the processing load when the tool system (1) is identifying the current job object.

[0191] In the tool system (1) according to the tenth aspect, which can be implemented in combination with any one of the seventh aspect to the ninth aspect, the predetermined image processing includes a cropping process. The reference image generation unit (67) can set a plurality of cropping areas for the second captured image displayed on the display unit (62). The reference image generation unit (67) can weight each of the plurality of cropping areas.

[0192] According to this aspect, the reference image generation unit (67) can set a plurality of cropping regions and weight each of the plurality of cropping regions, thereby enabling the generation of a reference image that is more suitable for recognizing the current work object.

[0193] The tool system (1) according to the eleventh aspect, which can be implemented in combination with any one of the fourth aspect to the tenth aspect, further includes a reflection processing unit (37). The reflection processing unit (37) reflects information related to predetermined image processing onto the control of the imaging unit (5).

[0194] According to this aspect, information related to predetermined image processing performed when generating the reference image is reflected onto the control of the imaging unit (5), thereby enabling the imaging unit (5) to capture a first captured image suitable for comparison with the reference image.

[0195] Note that the constituent elements according to the second aspect to the eleventh aspect are not essential constituent elements of the tool system (1), but can be appropriately omitted.

[0196] The tool (2a, 2b) according to the twelfth aspect is used in the tool system (1) according to any one of the first aspect to the eleventh aspect, and includes a drive unit (24) and an imaging unit (5).

[0197] According to this aspect, when generating the reference image, a second captured image is displayed on the display unit (62). This enables the user to check the second captured image while the reference image is being generated.

[0198] The reference image generation method according to the thirteenth aspect includes a reference image generation step. The reference image generation step includes: generating, based on the second captured image, a reference image to be compared with the first captured image when recognizing the current work object for which the first tool, which is one of the plurality of tools (2a, 2b), is currently set in the appropriate position. The first captured image is captured by the imaging unit (5) of the first tool (2a; 2b) that is currently set in the appropriate position among the plurality of tools (2a, 2b) when recognizing the current work object for which the first tool (2a; 2b) is currently set in the appropriate position. Each of the plurality of tools (2a, 2b) is a portable tool and includes: a drive unit (24) that is activated using power supplied from a power source; and an imaging unit (5). The second captured image is captured by the imaging unit (5) of the second tool, which is one of the plurality of tools (2a, 2b). The reference image generation step further includes a display step. The display step includes: causing the display unit (62) to display the second captured image while the reference image is being generated.

[0199] According to this aspect, when the reference image is being generated, the second captured image is displayed on the display unit (62). This enables the user to check the second captured image while the reference image is being generated.

[0200] The program according to the fourteenth aspect is designed such that one or more processors perform the method according to the thirteenth aspect.

[0201] According to this aspect, when the reference image is being generated, the second captured image is displayed on the display unit (62). This enables the user to check the second captured image while the reference image is being generated.

[0202] The tool system (1) according to the fifteenth aspect includes a plurality of tools (2a, 2b), a reference image generation unit (67), and an output unit (66). Each of the plurality of tools (2a, 2b) is a portable tool and includes: a drive unit (24) that is activated using power supplied from a power source; and an imaging unit (5). The reference image generation unit (67) generates a reference image to be compared with the first captured image. The first captured image is captured by the imaging unit (5) of the first tool (2a; 2b) currently set in the appropriate position among the plurality of tools (2a, 2b) when currently identifying the current work object as one of the plurality of work objects on which the first tool (2a; 2b) is set in the appropriate position. The reference image generation unit (67) generates the reference image based on the second captured image. The second captured image is captured by the imaging unit (5) of the second tool (2a; 2b) which is one of the plurality of tools (2a, 2b). The output unit (66) causes the display unit (62) to display the second captured image while the reference image generation unit (67) is generating the reference image.

[0203] According to this aspect, when the reference image is being generated, the user is allowed to check the second captured image, thereby enabling an image more suitable for identifying the current work object to be used as the reference image.

[0204] Description of Reference Numerals

[0205] 1 Tool system

[0206] 2, 2a, 2b Tools

[0207] 24 Drive unit

[0208] 37 Reflection processing unit

[0209] 5 Imaging unit

[0210] 60 Setting terminal

[0211] 61 Communication unit

[0212] 62 Display unit

[0213] 66 Output unit

[0214] 67 Reference image generation unit

Claims

1. A tool system for a plurality of tools, each of the plurality of tools being a portable tool and comprising: A drive unit configured to be activated by power supplied from a power source; and a camera unit, the tool system comprising: A reference image generation unit configured to generate a reference image to be compared with a first captured image, the first captured image being an image of a current work object captured by the camera unit of the first tool when the first tool, which is one of the plurality of tools, is currently set at an appropriate position while identifying the current work object from among the plurality of work objects, The reference image generation unit is configured to generate the reference image based on a second captured image, the second captured image being an image of a work object in which the second tool, which is one of the plurality of tools, is currently set at an appropriate position, captured by the camera unit of the second tool. The tool system further includes an output unit configured to cause a display unit to display the second captured image while the reference image generation unit is generating the reference image.

2. The tool system according to claim 1, wherein The display unit is provided separately from the plurality of tools.

3. The tool system according to claim 2, further comprising a communication unit configured to be ready to communicate with each of the plurality of tools via at least one of wired communication and wireless communication, Among them, The communication unit is configured to receive the second captured image from the second tool, which is one of the plurality of tools, and The output unit is configured to cause the display unit to display the second captured image received by the communication unit.

4. The tool system according to any one of claims 1 to 3, wherein The reference image generation unit is configured to generate the reference image by performing predetermined image processing on the second captured image displayed on the display unit.

5. The tool system according to claim 4, wherein The reference image generation unit is configured to perform the predetermined image processing according to an operation command input by a user.

6. The tool system according to claim 4, wherein The reference image includes a plurality of frames corresponding to a single work object or a plurality of work objects, and When generating the reference image including the plurality of frames, the reference image generation unit can change the content of the predetermined image processing frame by frame.

7. The tool system according to claim 4, wherein The predetermined image processing involves at least one of a cropping process and a masking process. The cropping process is used to set a cropping area to be compared when identifying the current work object, and the masking process is used to set a masking area that will not be compared when identifying the current work object.

8. The tool system according to claim 7, wherein The reference image generated by the cropping process is provided with additional information, and The additional information is information that enables the cropping area to be moved and / or rotated relative to the first captured image when the cropping area is compared with a part of the first captured image.

9. The tool system according to claim 8, wherein The additional information defines at least one of a range in which the cropping area can move relative to the first captured image and a range in which the cropping area can rotate relative to the first captured image.

10. The tool system according to claim 7, wherein the predetermined image processing includes the cropping processing, the reference image generation unit is capable of setting a plurality of cropping areas for a second captured image displayed on the display unit, and the reference image generation unit is capable of weighting each of the plurality of cropping areas.

11. The tool system according to claim 4, further comprising a reflection processing unit configured to reflect information related to the predetermined image processing on control of the imaging unit.

12. A tool used in the tool system according to any one of claims 1 to 11, the tool comprising: the drive unit; and the imaging unit.

13. A method for generating a reference image, comprising: a reference image generation step of generating a reference image to be compared with a first captured image based on a second captured image, the first captured image being an image of a current work object captured by an imaging unit of the first tool when the first tool, which is one of a plurality of tools, is currently set in a proper position and being recognized as the current work object, each of the plurality of tools being a portable tool and including a drive unit and the imaging unit, the drive unit being configured to be activated by power supplied from a power source, the second captured image being an image of a work object of the second tool, which is one of the plurality of tools, currently set in a proper position and captured by the imaging unit of the second tool, the reference image generation step further includes a display step of causing a display unit to display the second captured image when the reference image is being generated.

14. A non-transitory storage medium having stored thereon a program designed to cause one or more processors to perform the reference image generation method according to claim 13.

15. A tool system, comprising: a plurality of tools, each of the plurality of tools being a portable tool and including a drive unit and an imaging unit, the drive unit being configured to be activated by power supplied from a power source; and a reference image generation unit configured to generate a reference image to be compared with a first captured image, the first captured image being an image of a current work object captured by an imaging unit of the first tool when the first tool, which is one of the plurality of tools, is currently set in a proper position and being recognized as the current work object from among a plurality of work objects, the reference image generation unit being configured to generate the reference image based on a second captured image, the second captured image being an image of a work object of the second tool, which is one of the plurality of tools, currently set in a proper position among a plurality of work objects and captured by the imaging unit of the second tool, The tool system further includes an output unit configured to cause a display unit to display the second captured image while the reference image generation unit is generating the reference image.

Images