Bolt fastening work management system

Abstract

To provide a bolt fastening work management system capable of preventing an occurrence of an incompatible event such as fastening omission and a fastening order error at a work site at which fastened states of an enormous number of bolts have to be managed.SOLUTION: A bolt fastening work management system is provided with a control part for performing self position detection on a map generated by error correction by using an environmental map created on the basis of a VSLAM executed through imaging means and position information acquired through position information acquisition means for detecting position information of a system terminal owned by a worker, propriety determination of a work position on the basis of a comparison between a predetermined work instruction position and a self position, the designation and propriety determination of a bolt fastening position based on the VSLAM in the case that the work instruction position is appropriate, and propriety determination of a fastening torque detected at bolt fastening, and can output a result obtained by appropriately performing each determination as work records.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a system that assists in creating a work record, and in particular to a system that is suitable for determining whether or not the tightening work of a bolt or the like has been performed appropriately. [Background technology]

[0002] A known technology for automatically creating work records at a work site, including information such as the location of workers, is a work management support device such as that disclosed in Patent Document 1. The work management support device disclosed in Patent Document 1 projects location information obtained via a data collection device such as a beacon onto a map of the work site, which serves as work site information stored in the memory unit of a server, and records information such as where in the work site a worker was working at a given time, thereby improving work eligibility.

[0003] However, the work management support device disclosed in Patent Document 1 may have difficulty determining whether the worker's working position is appropriate unless it stores in advance map information of the work site and information on the location of the work object.As a result, when the shape or installation location of the work object changes, it is difficult to accurately grasp the work status, and there is a problem that it is difficult to manage the tightening status and the appropriateness of the tightening order for a huge number of bolts with only this type of system. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-199514 Summary of the Invention [Problem to be solved by the invention]

[0005] Therefore, the present invention aims to provide a bolt tightening work management system that makes it possible to accurately record work positions and work conditions in order to prevent non-conformance events such as forgetting to tighten or tightening in the wrong order at work sites where the tightening status of a huge number of bolts needs to be managed. [Means for solving the problem]

[0006] The bolt tightening work management system of the present invention, which is intended to achieve the above-mentioned object, is characterized by having a control unit that detects a self-position on a map generated by error correction using an environmental map created based on VisualSLAM (VSLAM) implemented via an imaging means and location information obtained via a location information acquisition means for detecting the location information of a system terminal held by the worker, determines whether the work position is appropriate based on a comparison between a predetermined work instruction position and the self-position, and, if the work instruction position is appropriate, specifies a bolt tightening position based on VSLAM and determines whether it is appropriate, and determines whether the tightening torque detected during bolt tightening is appropriate, and is capable of outputting the results of each appropriate determination as a work record.

[0007] In addition, in a bolt tightening operation management system having the above-mentioned features, the location information acquisition means may be configured to include a plurality of beacons with predetermined installation positions, calculate the distance between each beacon and the system terminal based on the strength of the radio waves output from each beacon and reaching the system terminal, and detect the self-position of the system terminal on the environmental map based on the distances between at least three beacons and the system terminal. With this configuration, it is possible to acquire self-position information with high accuracy even in an indoor space.

[0008] In addition, in a bolt tightening work management system having the above-mentioned features, the suitability of the bolt tightening position may be determined by detecting variations in the image of the tool used for tightening and deriving the coordinates of the rotation center of the tool based on a difference area obtained by contour extraction. With such features, it becomes possible to determine whether the bolt being worked on is a specified bolt based on position information of the tool rotation center and the bolt.

[0009] Furthermore, in a bolt tightening work management system having the above-mentioned features, it is desirable that the determination of the appropriateness of the tightening torque is made by detecting vibrations when a preset torque wrench swings and converting these vibrations into torque. With such features, it is possible to digitize the tightening torque of an analog torque wrench. [Effects of the Invention]

[0010] With a bolt tightening work management system having the above-described features, when creating work records, it is possible to accurately record work positions and work conditions, even at work sites where it is necessary to manage the tightening status of a huge number of bolts. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a block diagram showing the configuration of a bolt tightening operation management system according to an embodiment. FIG. [Figure 2] FIG. 10 is a flow diagram for explaining the linking of each image with position information, which is carried out to reduce errors caused by mobile imaging while creating an environmental map in a bolt tightening work management system according to an embodiment. [Figure 3] FIG. 10 is a flow chart for explaining the work location determination, work position determination, and torque suitability determination during work execution by the bolt tightening work management system according to the embodiment. [Figure 4] FIG. 10 is a diagram showing an example of a system display screen in the bolt tightening operation management system according to the embodiment. [Figure 5] FIG. 10 is a diagram showing an example of a system display screen that specifies the positional relationship between an environmental map, a 3D model, and a bolt regarding the output of a bolt position in a bolt tightening work management system according to an embodiment. [Figure 6] FIG. 10 is a flowchart showing a process for identifying the position of a bolt to be worked on in the bolt tightening work management system according to the embodiment. [Figure 7] FIG. 1 is a flow diagram for arranging a 3D model on an environmental map. [Figure 8] FIG. 10 is a flow diagram illustrating a process for determining whether a bolt tightening position is appropriate in a bolt tightening operation management system according to an embodiment. [Figure 9] 9A and 9B are diagrams showing an image of successive images, difference processing, thinning, and calculation of the coordinates of the rotation center of the torque wrench when determining the tightening position of the bolt in FIG. 8. [Figure 10] FIG. 10 is a flow diagram for explaining a process for determining whether a bolt tightening torque is appropriate in a bolt tightening operation management system according to an embodiment. [Figure 11] FIG. 1 is a diagram showing a waveform of vibrations that occur when a preset torque wrench swings and reaches a set torque. [Figure 12] 12 is a graph showing the relationship between the area of ​​a range surrounded by the waveform shown in FIG. 11 and 0 as a base point, and torque. DETAILED DESCRIPTION OF THE INVENTION

[0012] The following describes in detail an embodiment of the bolt tightening operation management system of the present invention with reference to the drawings. Note that the embodiment shown below is a part of a preferred form for carrying out the present invention, and it goes without saying that even if some of the configurations or means are changed, the change still constitutes part of the present invention as long as the effect is achieved.

[0013] [composition] First, the configuration of a bolt tightening work management system according to this embodiment will be described with reference to Figure 1. A bolt tightening work management system 10 according to this embodiment includes at least an imaging means (camera) 30, a position information acquisition means (position sensor) 40, a tool acceleration acquisition means (tool sensor) 60, a system terminal 12, and a database 50. The camera 30 is a camera capable of capturing images, and is an element that performs VisualSLAM (VSLAM), which will be described in detail later, and acquires images for creating an environmental map. Note that other imaging devices such as Lidar, a depth camera, or a stereo camera may also be used.

[0014] The position sensor 40 is an element that links each image with position information in order to reduce errors caused by movement during image capture when creating an environmental map. Specifically, it may be a sensing device such as a two-dimensional code or a beacon. In this embodiment, from the perspective of obtaining detailed position information in an indoor space such as a factory, multiple beacons are used as the position sensors 40, and distances are calculated according to the strength of the emitted radio waves to obtain position information between points in a point cloud that contains three-dimensional information generated when creating an environmental map.

[0015] The tool sensor 60 detects vibrations that occur when the torque wrench is rotated, and activates the camera 30 to determine whether the tightening position is appropriate based on the captured image. It is also an element that detects vibrations that occur when the torque wrench (e.g., a preset torque wrench) swings when the set torque is achieved, and determines whether the tightening torque is appropriate. Specifically, the former is configured to detect vibrations that occur when the torque wrench is rotated, capture an image of the torque wrench, and calculate the position of the tightening bolt (the center of rotation of the torque wrench) from the processed image. The latter is configured to detect vibrations that occur when the torque wrench swings when the set torque is achieved, and calculate the tightening torque from the processed vibration information.

[0016] The system terminal 12 is an element for creating an environmental map, inserting the user's own position into the environmental map, determining whether the work instruction position is appropriate, and outputting the determination results, based on the images and position information acquired via the above-mentioned camera 30 and position sensor 40. The system terminal 12 according to this embodiment is only required to include at least a recording unit 14, a processing unit 16, an input unit 18, a display unit 20, a control unit 22, and a temporary storage unit (memory) 24.

[0017] The recording unit 14 is an element for temporarily storing data obtained via the aforementioned camera 30 and position sensor 40. For example, if a beacon is used as the position sensor 40, the recording unit 14 acquires and temporarily stores radio waves from multiple beacons, and also temporarily stores data obtained via the camera 30.

[0018] The processing unit 16 is an element that performs processing based on data input via the recording unit described above, and programs started via the input unit (details of which will be described later) and data obtained. For example, in the case of a position sensor 40 that uses a beacon, multiple radio waves of different strengths are input to the recording unit 14, and the processing unit 16 calculates the distance between the system terminal 12 and each beacon (position sensor 40) based on the radio wave conditions, etc., and performs processing to calculate the position information of the system terminal 12.

[0019] The input unit 18 is an element for inputting information necessary for various processes, selecting a program to be started, etc. In this embodiment, input may be made via a touch panel that constitutes the display unit 20. The display unit 20 is a display screen of the system terminal 12, and is an element for visibly displaying various information such as selection information for a program, processing results by the processing unit 16, and input contents by the input unit 18.

[0020] The control unit 22 is a so-called calculation means that performs processing to execute data and programs input via the input unit 18 or stored in the temporary storage unit 24 from the processing unit 16. For example, when saving video as a work record, the processing unit 16 detects vibrations via the tool sensor 60, determines whether the data output from the input unit 18 is appropriate, acquires the video data via the camera 30, and stores the video data in the temporary storage unit 24.

[0021] The temporary storage unit 24 is an element for temporarily storing data read from the database 50, which will be described in detail later, programs executed via the processing unit 16 and the control unit 22, and data obtained by various processes, and writing the data to the database 50.

[0022] The database 50 is an element for storing the environmental map obtained via the system terminal 12, integrated data in which the user's own position is embedded on the environmental map, and the like.

[0023] The bolt tightening operation management system 10 according to this embodiment is configured to have a tool sensor 60 attached to the work tool, making it possible to determine whether the tightening operation is being performed properly. Although there are no limitations on the tool sensor 60, it is desirable that the detected value can be converted into electronic data.

[0024] In this embodiment, a lotion sensor is used. The lotion sensor is an element for detecting vibrations (a clicking swing) that occur when the torque wrench reaches the set tightening torque. In this embodiment, this vibration is analyzed by the system terminal 12 or the like, converted into torque, and a determination can be made as to whether the torque value in the tightening operation has been set appropriately.

[0025] [Actions and Effects] In the bolt tightening work management system 10 configured as described above, the worker's own position is first estimated as shown in FIG. 2. An environmental map is then created for managing the positions of the tightening bolts. Specifically, the system terminal 12 detects changes in the image captured by the camera 30 (S10) and activates SLAM from a recorded program (S20). In this embodiment, SLAM based on the image captured by the camera 30, known as VSLAM, is activated. After SLAM is activated, an image is acquired by the camera 30 attached to the system terminal 12 (S30), and an environmental map is created based on this image (S40).

[0026] On the other hand, the created environmental map is the result of overlaying multiple images from various angles while moving, and lens aberrations of the camera 30, etc., will result in discrepancies between the environmental map and actual coordinates, in other words, movement errors.

[0027] As the worker moves, the system terminal 12 receives radio waves from each of the multiple position sensors 40 (S50). Next, the system terminal 12 calculates its own position (the position of the system terminal 12) (S60) and associates it with the video captured by the camera 30 (S70). Next, the system terminal 12 reflects the self-position information associated with the video on the existing environmental map to create a corrected environmental map (S80). The created environmental map is then stored in the temporary storage unit 24 as an environmental map for estimating the worker's own position and managing the positions of the tightening bolts (S90). Specifically, the system terminal 12 detects the number of the beacon from which it is receiving radio waves and detects the strength of the radio waves (radio wave intensity) of the beacon having that number. This is because the strength of the radio wave intensity can be used to determine the distance between a specific beacon and the system terminal 12. The system terminal 12 calculates the distance from the beacon based on the radio wave intensity, and then associates it with the video captured at that position. Radio waves from a single beacon can only determine the radial distance from a specific beacon (the beacon from which the number was detected). Therefore, by repeating the identification of the beacon (detection of the beacon number) and detection of the radio wave intensity at least three times, it is possible to identify the intersection of the range circles, i.e., the position information of the system terminal 12. Next, by linking it to the video, the position information of each feature point on the video is calculated based on the position information of the center of the video. After that, based on the calculated position information of each feature point on the video, the positions between points in the point cloud on the environmental map created by SLAM are corrected, and a new environmental map is output. This allows for accurate estimation of the worker's own position and proper management of the tightening bolt positions.

[0028] Next, the process of determining work eligibility will be described with reference to Figure 3. Work eligibility is determined by determining whether the worker is performing the work at the designated position and whether the work itself has been performed properly. Whether the work itself has been performed properly is determined by detecting the torque caused by the tightening force of the bolt.

[0029] First, the environmental map saved as shown in FIG. 2 is loaded (S110). Next, SLAM is activated, and the worker moves to the work site. Here, newly captured images are compared with the image of the loaded environmental map by the operation of SLAM, and the environmental map with the worker's position loaded is output (S120). Thereafter, while the worker is moving, the work location for which a work instruction has been issued is detected, and the detected work location is compared with the worker's position information, and a determination is made as to whether the work location is appropriate (S130). Here, if the determination in S130 shows that the specified work location does not match the worker's position information, i.e., if the work location is determined to be inappropriate, a message indicating that the work location is inappropriate is output to the worker, for example, on the display unit 20. Upon receiving this message, the worker returns to S120 and moves on.

[0030] When determining the suitability of an actual work, the work location is first identified. Identifying the work location means selecting the equipment information (equipment name, site name, device name, etc.) to be used for the work (S140), as shown in an example of display on the display unit 20 of the system terminal 12 in Fig. 4.

[0031] After identifying the work location, the model number of the torque wrench for the work (for example, tightening a bolt) is input (S150) and the set torque is input (S160). Here, the torque wrench model number is an input item for "torque wrench" shown on the display unit 20. Also, the set torque is an input item for a detailed set value in the "torque reference range" shown on the display unit 20. After inputting the torque wrench model number and set torque, the display unit 20 of the system terminal 12 displays guidance to the bolt position that is the target of the tightening work (S170).

[0032] Guidance to the bolt position refers to visually displaying the position and number of the bolt to be tightened, as shown in the example display in Figure 4. Details of this display process (guidance to the bolt position) are shown in the flow chart in Figure 6. This process is performed by reading the bolt position information for the bolt to be tightened for the work location detected in the environmental map saved in Figure 2 (S170a). As shown in Figure 7, the environmental map information is read (S170a1), and a 3D model of the equipment, etc. is read (S170a2). The center of the 3D model is then identified, and the 3D model of the equipment, etc. to be tightened is placed on the environmental map in the same position as the actual equipment (S170a3). Editing operations such as rotating and moving the 3D model are performed by inputting coordinate information such as X, Y, and Z of the placement position, which are input items in the "Properties" and "Operation Panel" in Figure 5. After placing the 3D model in the appropriate space, it is saved together with the environmental map, and its position information in the environmental map is output (S170a4). Then, the bolt position in the environmental map is calculated based on the bolt position information when the 3D model was designed and the position information of the 3D model in the environmental map (S170a5). This makes it possible to output the coordinate information of the bolt to be worked on.

[0033] After the bolt position information is read as described above, the distance between each bolt and the current position is calculated (S170b), and the bolt closest to the current position is identified (S170c). Then, based on the design information of the 3D model, other bolts associated with the identified bolt are output together (S170d). This process allows the first tightening bolt to be grouped together with other bolts that need to be tightened together with this tightening bolt. Therefore, this assumes that there are multiple target bolts. Next, the bolt closest to the current position is designated as the first tightening bolt, and the tightening order of the other bolts is displayed based on the design information of the 3D model (S170e). Then, in the 3D model placed on the environmental map, the bolts to be tightened (the bolts corresponding to the order) are displayed in a color that distinguishes them from the color of the 3D model (S170f).

[0034] After the position of the bolt to be tightened has been guided as described above, the bolt to be tightened (the bolt guided in S170) is tightened using an actual tool (S180). When tightening the bolt, the tightening position is determined, that is, it is determined whether the bolt to be tightened using the actual tool is the bolt guided in S170 (S190).

[0035] The determination of whether the bolt position is appropriate is performed as shown in Fig. 8. Specifically, the tool sensor 60 detects vibrations during torque wrench rotation (S190a), and the camera 30 is activated to read consecutive images that can be obtained during bolt tightening work (S190b). The reading of consecutive images is, for example, as shown in Fig. 9. That is, images 1 to 4 in Fig. 9 are consecutive images.

[0036] Next, to improve the accuracy of the image processing described below, images with high similarity are selected by extracting feature points. Specifically, feature point extraction processing is performed on each of the consecutive images, and images with the same number of feature points calculated after feature point matching are prioritized and extracted (S190c).

[0037] Next, a difference process is performed between the extracted consecutive images (S190d), and a difference area is detected. In Figure 9, the image showing the movement of only the handle portion shown in the middle represents the detection of the difference area (S190e). After the difference area is detected, a process is performed to represent the axis (axis of the tightening tool) passing through the center of this difference area with a thin line (straight line). The bottom part of Figure 9 shows the intersection of three thin lines (S190f). After the thin line process is completed, a process is performed to calculate the coordinates of the intersection point (center of rotation of the torque wrench) based on the consecutive images. This is the location represented as an intersection point in the diagram shown in the bottom part of Figure 9 (S190g). After the coordinates of the line intersection are calculated, it is determined whether the coordinates of this line intersection match the coordinates of the bolt position (S200). If it is determined in S200 that the bolt position is appropriate, the process proceeds to determining the set torque (S210). On the other hand, if it is determined in S200 that the bolt position is inappropriate, the worker visually checks the bolt position displayed on the system terminal 12 against the position of the tightening bolt, and performs the checking work manually.

[0038] The set torque determination is a determination of whether the tightening torque of the bolt is appropriate. Specifically, as shown in Figure 10, first, a determination is made as to whether the torque required for the bolt to be tightened can be set with the input torque wrench (S210a). Next, a determination of the set torque is made (S220).

[0039] To determine the set torque, the signal from the tool sensor 60 during the tightening operation is analyzed. In this embodiment, a torque wrench with a swing function is used, and the swing function is activated when the tightening torque reaches the set value. The tool sensor 60 detects the vibrations generated during this swing and outputs a waveform such as that shown in FIG. 11. In this embodiment, this waveform is analyzed and converted into a torque value. Specifically, the waveform is averaged to remove noise contained in the waveform and extract the analysis range (filtering). Absolute value processing is performed using zero as the base point of the waveform, and then integration processing is performed to determine the area within the waveform within the filter processing range. The obtained area is then averaged to remove maximum and minimum values, thereby determining the area within the waveform (S210b). The obtained area is fitted to the area-torque approximation curve shown in FIG. 12 to derive an estimated torque value (S210c).

[0040] The calculated torque estimate (converted torque) is compared with the set torque (S210d), and if they match, the determination of the set torque (tightening torque) is completed. On the other hand, if the converted torque does not match the set torque, the torque value of the torque wrench is reset, and the bolt tightening operation is carried out (S220). After it is determined that the bolt tightening work has been performed properly, a work record of the work performed is created and the process ends (S230).

[0041] The bolt tightening work management system 10 described above makes it possible to accurately record work positions and work conditions even at large work sites where a huge number of bolts are managed. Furthermore, even with analog torque wrenches, where it was previously difficult to digitize torque values, it has become possible to digitize tightening torque, making it possible to accurately record tightening values ​​in work records. [Explanation of symbols]

[0042] 10...Bolt tightening work management system, 12...System terminal, 14...Recording unit, 16...Processing unit, 18...Input unit, 20...Display unit, 22...Control unit, 24...Temporary memory unit, 30...Camera (imaging means), 40...Position sensor (position information acquisition means), 50...Database, 60...Tool sensor (tool acceleration acquisition means).

Claims

1. self-location detection on a map generated by error correction using an environmental map created based on VisualSLAM (VSLAM) implemented via an imaging means and location information obtained via a location information acquisition means for detecting location information of a system terminal held by the worker; A bolt tightening work management system characterized by having a control unit that determines whether a work position is appropriate based on a comparison between a predetermined work instruction position and the system's own position, and if the work instruction position is appropriate, specifies a bolt tightening position based on VSLAM and determines whether it is appropriate, and determines whether the tightening torque detected during bolt tightening is appropriate, and is able to output the results of each appropriate determination as a work record.

2. the location information acquisition means comprises a plurality of beacons whose installation positions are determined in advance, Calculating the distance between the beacon and the system terminal based on the strength of the radio waves output from each beacon and reaching the system terminal; 2. The bolt tightening operation management system according to claim 1, wherein the system terminal detects its own position on the environmental map based on the distances between at least three beacons and the system terminal.

3. The bolt tightening work management system according to claim 2, characterized in that the determination of the suitability of the bolt tightening position is made by detecting variations in the image of the tool used for tightening and deriving the coordinates of the center of rotation of the tool based on a difference area obtained by contour extraction.

4. 4. A bolt tightening work management system according to claim 1, wherein the determination of whether the tightening torque is appropriate is made by detecting vibrations when the torque wrench swings and converting these vibrations into torque.

Images