Rotation angle measuring device for engineering machinery, rotation angle measuring method for engineering machinery and storage medium

Abstract

In the present invention, a plurality of images generated by an imaging unit are acquired, and for each of the plurality of images, on the basis of the image and a previous image that is temporally adjacent to the image, a first rotation angle of a rotating body is calculated with reference to the position of the rotating body with respect to a base in the first image among the plurality of images, and a second rotation angle of the rotating body is calculated based on the first rotation angle. And a control unit that obtains a reference for the rotation angle of a predetermined place in the outside world on the basis of a plurality of first rotation angles obtained for each of the plurality of images, obtains a second rotation angle of the rotation body with respect to the reference on the basis of the first rotation angles obtained for each of the plurality of images and the reference, and controls the rotation of the rotation body on the basis of the second rotation angle obtained for each of the plurality of images. The imaging unit is provided so as to move in accordance with the rotation of the rotating body of the construction machine, and generates the plurality of images of the outside world arranged in a chronological order as seen from the rotating body.

Description

Technical Field

[0001] The present invention relates to a rotation angle measuring device for engineering machinery for obtaining the rotation angle of a rotating body in the engineering machinery, a rotation angle measuring method for engineering machinery, and a storage medium recording a rotation angle measuring program for engineering machinery. Background Art

[0002] In a construction machine, an operator (user, worker) sometimes rotates the rotating body while performing a predetermined operation. The construction machine includes a base, a rotating body rotatably connected to the base, and an operation mechanism provided on the rotating body and capable of performing the predetermined operation. Therefore, there is a case where it is desired to know the rotation angle of the rotating body. As a technology for obtaining the rotation angle of the rotating body, for example, a technology disclosed in Patent Document 1 is known.

[0003] The angle detection device of the construction machinery disclosed in Patent Document 1 includes: a camera, which is arranged on a rotating body that can rotate relative to a walking body and photographs the walking body; and an angle detection unit, which processes the image photographed by the camera, detects the edge of a predetermined part of the walking body, and calculates the relative angle of the rotating body with respect to the walking body.

[0004] Although the angle detection device of the construction machinery disclosed in the patent document 1 calculates the relative angle of the rotating body with respect to the walking body based on the predetermined part of the walking body, there is a need to calculate the rotation angle of the rotating body based on the direction in which the rotating body is mainly facing during operation. Construction machinery is used in various sites or has various specifications and modifications, and the previous methods based on specific objects or marks, such as the predetermined part of the walking body, are sometimes unable to detect the rotation angle.

[0005] Prior art literature

[0006] Patent Literature

[0007] Patent Document 1: Japanese Patent Publication No. 2017-58272 Summary of the invention

[0008] The present invention is completed in view of the above situation, and its purpose is to provide the following rotation angle measurement device for engineering machinery, rotation angle measurement method for engineering machinery and storage medium recording the rotation angle measurement program for engineering machinery, which are not based on specific objects or marks, etc., but can calculate the rotation angle of the rotating body based on the main direction of the rotating body during operation.

[0009] A rotation angle measuring device for construction machinery and a rotation angle measuring method for construction machinery reflecting one aspect of the present invention include a plurality of images generated by a camera unit, for each of the plurality of images, based on the image and a previous image adjacent to the image in time series, using the position of the rotation body in an initial image of the plurality of images relative to a base as a reference, a first rotation angle of the rotation body is obtained, based on the plurality of first rotation angles obtained for each of the plurality of images, a predetermined location in the outside world is obtained as a reference for the rotation angle, and for each of the plurality of images, based on the first rotation angles obtained for the image and the reference, a second rotation angle of the rotation body relative to the reference is obtained, the camera unit being provided on the construction machinery in a manner moving with the rotation of the rotation body and generating the plurality of images of the outside world viewed from the rotation body arranged in time series, the construction machinery including the base, the rotation body rotatably connected to the base, and a working mechanism provided on the rotation body and capable of performing a predetermined work. Furthermore, a storage medium reflecting another aspect of the present invention records a swing angle measurement program for a construction machine for causing a computer to function as such a swing angle measurement device for a construction machine.

[0010] The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description and accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is a block diagram showing the structure of the rotation angle measuring device for construction machinery in the embodiment.

[0012] Figure 2 It is a side view showing an example of a dismantling machine provided with the above-mentioned rotation angle measuring device for construction machinery.

[0013] Figure 3 FIG. 1 is a diagram for explaining an example of an image and feature points extracted from the image.

[0014] Figure 4 This is a diagram for explaining an example of first feature points and second feature points that correspond to each other between a first image and a second image.

[0015] Figure 5 This is a graph showing an example of the measured value and the analyzed value of the rotation angle of the rotating body.

[0016] Figure 6 This is a graph showing an example of a plurality of first rotation angles arranged in time series and a reference of a rotation angle obtained based on the first rotation angles.

[0017] Figure 7 This is a graph showing an example of a plurality of first rotation angles and second rotation angles arranged in time series.

[0018] Figure 8 This is a flowchart showing the operation of the swing angle measuring device for construction machinery.

[0019] Fig. 9 The graph shows an example of a plurality of first rotation angles arranged in time series when the rotating body is stationary and a reference of the rotation angle obtained based on the first rotation angles.

[0020] Fig.10 It means in Fig. 9 In the case shown, it is a diagram showing the result of determining whether the rotating body is stationary.

[0021] Fig.11 It means in Fig. 9 In the case shown, a plurality of first rotation angles are arranged in time series, and a graph showing a reference of a rotation angle obtained by excluding the first rotation angle when the rotating body is stationary.

[0022] Fig.12 This is a diagram for explaining manual setting of an exclusion area.

[0023] Fig.13 This is a flowchart showing the operation of the swing angle measuring device for construction machinery related to the automatic setting of the exclusion area. DETAILED DESCRIPTION

[0024] Hereinafter, one or more embodiments of the present invention will be described with reference to the accompanying drawings. However, the scope of the invention is not limited to the disclosed embodiments. In addition, the structures marked with the same symbols in the various figures represent the same structures, and their descriptions are appropriately omitted. In this specification, when a general term is used, it is represented by a reference symbol with the suffix omitted, and when a separate structure is indicated, it is represented by a reference symbol marked with a suffix.

[0025] The rotary angle measuring device for construction machinery in the embodiment comprises: an image acquisition unit, which acquires a plurality of images generated by a camera unit, the camera unit being arranged on the construction machinery in a manner that moves along with the rotation of a rotary body, and generates the plurality of images of the outside world viewed from the rotary body and arranged in a time series, the construction machinery comprising a base, the rotary body rotatably connected to the base, and an operating mechanism that is arranged on the rotary body and can perform a prescribed operation; and a first rotary angle calculation unit, which calculates, for each of the plurality of images, a value based on the image and a previous image adjacent to the image in the time series. , taking the position of the rotating body in the first image of the plurality of images relative to the base as a reference, the first rotation angle of the rotating body is obtained; a reference calculation unit, based on the plurality of first rotation angles obtained by the first rotation angle calculation unit for the plurality of images respectively, obtains a predetermined place in the outside world as a reference for the rotation angle; and a second rotation angle calculation unit, based on the first rotation angle obtained by the first rotation angle calculation unit for the image and the reference obtained by the reference calculation unit for the plurality of images respectively, obtains the second rotation angle of the rotating body relative to the reference. Hereinafter, taking a dismantled machine equipped with such a rotation angle measuring device for engineering machinery as an example, the rotation angle measuring device for engineering machinery, the rotation angle measuring method for engineering machinery installed in the rotation angle measuring device for engineering machinery, and the rotation angle measuring program for engineering machinery are described in more detail.

[0026] Figure 1 It is a block diagram showing the structure of the rotation angle measuring device for construction machinery in the embodiment. Figure 2 It is a side view showing an example of a dismantling machine provided with the above-mentioned rotation angle measuring device for construction machinery. Figure 3 FIG. 1 is a diagram for explaining an example of an image and feature points extracted from the image. Figure 3 A represents an example of an image, Figure 3 B means from Figure 3 Feature points extracted from the image shown in A. Figure 4 This is a diagram for explaining an example of first feature points and second feature points that correspond to each other between a first image and a second image. Figure 4 A represents the first feature point and the second feature point before filtering for removing erroneous correspondence, Figure 4 B represents the first feature point and the second feature point after the filtering process. Figure 5 This is a graph showing an example of the measured value and the analyzed value of the rotation angle of the rotating body. Figure 5 The horizontal axis is time, the vertical axis is angle, the dotted line represents the measured value, and the solid line represents the analyzed value. Figure 6This is a graph showing an example of a plurality of first rotation angles arranged in time series and a reference of a rotation angle obtained based on the first rotation angles. Figure 6 The horizontal axis is time and the vertical axis is angle. Figure 7 This is a graph showing an example of a plurality of first rotation angles and second rotation angles arranged in time series. Figure 7 A represents each curve showing a plurality of second rotation angles, Figure 7 B shows curves showing a plurality of first rotation angles. Figure 7 The horizontal axis is time and the vertical axis is angle.

[0027] The rotary angle measuring device D for construction machinery in the embodiment is, for example, Figure 1 As shown, the image acquisition unit 1, the control processing unit 2, the input unit 3, the output unit 4, the interface unit (IF unit) 5 and the storage unit 6 are provided in the construction machine HS, for example.

[0028] The construction machine HS using the swing angle measuring device D for construction machines is a machine (such as a construction machine) for performing work that can be performed by a work mechanism, for example, Figure 2 As shown, it includes: a lower walking body 101, which has a left walking track 101L and a right walking track (not shown) and travels; an upper frame 102, which is rotatably arranged on the lower walking body 101 and serves as a base; a machine room 103, which is arranged on the upper frame 102 and is internally provided with an engine (not shown) such as an internal combustion engine, and a hydraulic pump, etc., which generates power for driving a working attachment 104 described later; the working attachment 104, which is undulatingly arranged on the upper frame 102; operating components (not shown) such as operating levers, which are omitted from the illustration, used to operate the working attachment 104; and a cabin (operating room) 105, which is provided with the operating components.

[0029] The working attachment 104 is a working mechanism for performing a predetermined operation corresponding to the type of the construction machine HS. In the present embodiment, the construction machine HS is, for example, a demolition machine, and the working attachment 104 is, for example, a mechanism including a boom 141, a boom 142, and an openable crusher 143. The boom 141 is connected to the upper frame 102 so as to be swingable in the vertical direction, the boom 142 is connected to the distal end of the boom 141 so as to be swingable in the vertical direction, and the openable crusher 143 is connected to the distal end of the boom 142 and has a pair of crushing knives that can be opened and closed. The boom 141 rises and falls relative to the upper frame 102 by the telescopic action of the boom cylinder 144a. The boom 142 swings relative to the boom 141 by the telescopic action of the boom cylinder 144b. The openable crusher 143 opens and closes by the telescopic action of the crusher cylinder 144c. The boom cylinder 144a, the arm cylinder 144b, and the crusher cylinder 144c are respectively constituted by hydraulic cylinders of a hydraulic type driven by the hydraulic pump.

[0030] In addition, in this example, although the construction machine HS is a dismantling machine as an example, it is not limited to this, and it may be a hydraulic excavator, etc. In this example, the lower traveling body 101 corresponds to an example of a base body, the upper frame 102, the machine room 103 and the cabin 105 correspond to an example of a rotating body rotatably connected to the base body, and the working attachment 104 corresponds to an example of a working mechanism that is provided on the rotating body and can perform a predetermined work.

[0031] In the present embodiment, the construction machine HS further includes a camera unit 106, which is provided so as to move along with the rotation of the rotating body. The camera unit 106 generates a plurality of images of the outside world viewed from the rotating body, arranged in time series. Figure 2 In the example shown, the camera unit 106 is installed outside the cabin 105 as an example of the revolving body, for example, on the outer top surface of the canopy portion, so as to overlap with the field of view of the operator of the construction machine HS and to have its optical axis slightly directed downward. Such a camera unit 106 is, for example, a black-and-white digital camera or a color digital camera capable of generating video.

[0032] return Figure 1 The image acquisition unit 1 is a device connected to the control processing unit 2 and acquires the plurality of images generated by the camera unit 106 according to the control of the control processing unit 2, for example, a circuit for inputting and outputting data to and from an external device. In this embodiment, the image acquisition unit 1 acquires the video from the camera unit 106 as an example of the plurality of images arranged in time series.

[0033] In addition, the image acquisition unit 1 can, for example, acquire the multiple images from a storage medium such as a USB (Universal Serial Bus) memory or an SD (Secure Digital) card (registered trademark) storing the multiple images, or can acquire the multiple images from a storage medium such as a CD-R (Compact Disc Recordable) or a DVD-R (Digital Versatile Disc Recordable) storing the multiple images via a drive device, or can acquire the multiple images from a server device that manages the multiple images via a network.

[0034] Such an image acquisition unit 1 may be used also as the IF unit 5 described later, or may be separated from the IF unit 5 , for example.

[0035] The input unit 3 is a device connected to the control processing unit 2, and inputs various commands such as a command to start measuring the rotation angle, and various data required for operating the rotation angle measuring device D for construction machinery, such as a job name or date, into the rotation angle measuring device D for construction machinery. For example, it is a plurality of input switches, a keyboard, or a mouse that are assigned with prescribed functions. The output unit 4 is a device connected to the control processing unit 2, and outputs commands or data input from the input unit 3, and the rotation angle, etc. according to the control of the control processing unit 2. For example, it is a display device such as a CRT (Cathode Ray Tube) display, a liquid crystal display, and an organic EL (Electroluminescence) display, or a printing device such as a printer. The above-mentioned input unit 3 and output unit 4 can be used as an operation panel and a display panel in the construction machinery HS, for example, or can be separated from the operation panel and the display panel.

[0036] In addition, the input unit 3 and the output unit 4 may constitute a so-called touch screen. In the case of constituting such a touch screen, the input unit 3 is, for example, a position input device that detects an operation position and performs input, such as a resistive film method or a capacitive method, and the output unit 4 is a display device. With respect to the touch screen, the position input device is provided on the display surface of the display device, and one or more candidate input contents that can be input are displayed on the display device. If the user touches a display position that displays the input content that the user wants to input, the position is detected by the position input device, and the display content displayed at the detected position is input to the rotary angle measuring device D for construction machinery as the user's operation input content. Since the user can easily and intuitively understand the input operation, such a touch screen provides a rotary angle measuring device D for construction machinery that is easy for the user to operate.

[0037] The IF unit 5 is a circuit connected to the control processing unit 2, and performs data input and output with external devices according to the control of the control processing unit 2, and is, for example, an interface circuit of RS-232C, an interface circuit using the Bluetooth (registered trademark) standard, an interface circuit for infrared communication such as the IrDA (Infrared Data Association) standard, and an interface circuit using the USB (Universal Serial Bus) standard. In addition, the IF unit 5 is a circuit for communicating with external devices, and may be, for example, a data communication card or a communication interface circuit compliant with the IEEE802.11 standard.

[0038] The storage unit 6 is a circuit connected to the control processing unit 2 and stores various prescribed programs and various prescribed data according to the control of the control processing unit 2. The various prescribed programs include, for example, a control processing program, which includes a control program, a first rotation angle calculation program, a reference calculation program, and a second rotation angle calculation program. The control program is a program for controlling each part according to the functions of each part 1, 3 to 6 of the rotation angle measuring device D for construction machinery. The first rotation angle calculation program is a program for respectively calculating the first rotation angle of the rotating body based on the image and the previous image adjacent to the image in time series, with the position of the rotating body relative to the base in the initial image of the plurality of images as a reference, for each of the plurality of images acquired by the image acquisition unit 1. The reference calculation program is a program for calculating a prescribed place (first place) in the outside world as a reference for the rotation angle based on the plurality of first rotation angles obtained by the first rotation angle calculation program for each of the plurality of images. The second rotation angle calculation program is a program that calculates the second rotation angle of the rotating body relative to the reference for each of the plurality of images based on the first rotation angle calculated for the image by the first rotation angle calculation program and the reference calculated by the reference calculation program. The various specified data include, for example, a plurality of images in the previous period, the plurality of first rotation angles, the reference (a specified place in the outside world), and the plurality of second rotation angles, which are data required for executing the above-mentioned programs. Such a storage unit 6 includes, for example, a non-volatile storage element, i.e., a ROM (Read Only Memory) or a rewritable non-volatile storage element, i.e., an EEPROM (Electrically Erasable Programmable Read Only Memory). The storage unit 6 includes a so-called RAM (Random Access Memory) that serves as a working memory of the control processing unit 2, which stores data generated during the execution of the specified program. In addition, the storage unit 6 may also include a hard disk device that can store a large capacity.

[0039] The various predetermined programs described above may be stored in a storage medium such as a CD-ROM or a DVD-R, and stored in the storage unit 6 via a drive device connected to the IF unit 5 .

[0040] The control processing unit 2 is a circuit for controlling each part according to the functions of each part 1, 3 to 6 of the rotation angle measuring device D for construction machinery, and obtaining the second rotation angle of the rotating body relative to a predetermined place in the outside world that is obtained as a reference for the rotation angle. The control processing unit 2 is configured by, for example, a CPU (Central Processing Unit) and its peripheral circuits. In the control processing unit 2, by executing the control processing program, a control unit 21, a first rotation angle calculation unit 22, a reference calculation unit 23, and a second rotation angle calculation unit 24 are functionally configured.

[0041] The control unit 21 controls each unit 1 and 3 to 6 according to the function of the units 1 and 3 to 6 of the construction machine swing angle measuring device D, and controls the entire construction machine swing angle measuring device D. The control unit 21 stores the plurality of images acquired by the image acquisition unit 1 in the storage unit 6 .

[0042] The first rotation angle calculation unit 22 calculates the first rotation angle of the rotating body for each of the multiple images based on the image and a previous image adjacent to the image in time series, with the position of the rotating body relative to the base in the initial image of the multiple images as a reference.

[0043] More specifically, the first rotation angle calculation unit 22 extracts feature points (first feature points) from the image (first image) for each of the plurality of images, extracts feature points (second feature points) corresponding to the first feature points extracted from the first image from the previous image (second image) adjacent to the image in time series, and calculates the first rotation angle of the rotating body based on the movement between the first feature point and the second feature point, with the position of the rotating body relative to the base in the first image of the plurality of images as a reference. The feature point is a place in the image that can be distinguished from other places, such as a corner or edge of an object captured in the image.

[0044] In more detail, in the present embodiment, since the plurality of images acquired by the image acquisition unit 1 are videos generated by the camera unit 106, the first rotation angle calculation unit 22 first divides the video into frames to form the plurality of images, and respectively adds a serial number k from 1 to an integer as a label (first label, serial number of the image, image ID) to the plurality of images in a time series order, and stores the plurality of images in the storage unit 6 (k=1, 2, ..., K, where K is the total number of the plurality of images).

[0045] Next, the first rotation angle calculation unit 22 extracts feature points from the image in question (the first image) and extracts feature points from the previous image (the second image) adjacent to the image in time series, respectively, for the plurality of images. There are various methods for extracting the feature points, such as using A-KAZE. For example, in Figure 3 In the image shown in A, Figure 3 The characteristic point ● shown in B is extracted.

[0046] Next, the first rotation angle calculation unit 22 searches for and selects the feature points extracted from the second image corresponding to the feature points extracted from the first image (matching of feature points) through corresponding point search. There are many methods for the corresponding point search, such as kNN (k-Nearest Neighbor), for example, using brute-force matching. For example, between the image PC1 of the (k-1)th frame and the image PC2 of the kth frame, Figure 4 As shown in A, the feature points of image PC1 and the feature points of image PC2 are set to correspond to each other. Figure 4 In A, pairs of feature points that are set to correspond to each other are connected by line segments. Figure 4 The same is true for B.

[0047] Next, the first rotation angle calculation unit 22 implements a filtering process for removing erroneous correspondence of feature point pairs that are set to correspond to each other through the corresponding point search. For example, between adjacent frames, the amount of movement between the feature point pairs that correspond to each other is small, so when the amount of movement between the feature point pairs is greater than a predetermined threshold value (first threshold value) set in advance, the process of removing the feature point pairs (the first method of filtering process) is used as the filtering process. Or, for example, the amount of movement of the feature point pairs is generated by the rotation of the rotating body, and therefore, it mainly has a horizontal component (left-right direction) and almost no vertical component (up-down direction), so when the vertical component in the amount of movement of the feature point pairs is greater than a predetermined threshold value (second threshold value) set in advance, the process of removing the feature point pairs (the second method of filtering process) is used as the filtering process. Or, for example, in order to remove so-called outliers, when the normalized variable in the amount of movement of the feature point pairs is greater than a predetermined threshold value (third threshold value) set in advance, the process of removing the feature point pairs (the third method of filtering process) is used as the filtering process. Subtract the average movement amount of all feature point pairs from the movement amount of the feature point pair, and divide the subtraction result by the standard deviation of the movement amount of all feature point pairs, thereby obtaining the standardized variable of the feature point pair as the division result ((the standardized variable of the feature point pair) = ((the movement amount of the feature point pair) - (the average movement amount of all feature point pairs)) / (the standard deviation of the movement amount of all feature point pairs)). Alternatively, for example, the following processing (the filtering processing of the fourth method) is used as the filtering processing. In order to obtain the best result, a predetermined number of feature point pairs are selected from all feature point pairs, starting from the feature point pairs with the smallest movement amount. For example, in Figure 4 In the example shown in A, after the filtering process of the second method, the remaining Figure 4 B shows the feature points of image PC2 corresponding to the feature points of image PC1. Thus, the second feature points of the second image corresponding to the first feature points extracted from the first image are extracted. For example, the first to third thresholds are appropriately set in advance based on a plurality of samples.

[0048] Next, the first rotation angle calculation unit 22 calculates the average value of the movement of all feature point pairs after the filtering process as the movement between the first image and the second image, and converts the calculated movement between the first image and the second image into the rotation angle (angle change) Δγ between the first image and the second image. More specifically, the rotation angle (angle change) Δγ between the first image and the second image is calculated according to the following formula 1. That is, the average value of the multiplication results obtained by multiplying the movement between the first image and the second image by the angle of each pixel is calculated, thereby calculating the rotation angle (angle change) Δγ between the first image and the second image.

[0049] [Mathematical formula 1]

[0050]

[0051] Here, θ h is the horizontal angle of view of the camera unit 106, W is the number of pixels in the horizontal direction of the image generated by the camera unit 106, and x k，n is the x coordinate of the nth feature point extracted from the image of the kth frame when an xy orthogonal coordinate system is set in the image. The xy orthogonal coordinate system is as follows: Figure 3 and Figure 4 As shown in FIG. 1 , the upper left corner when viewed from the front is set as the coordinate origin, the horizontal direction is set as the x-axis, and the vertical direction is set as the y-axis, and N is the total number of feature points. In addition, an integer serial number n is assigned to each feature point after the filtering process in sequence from 1 as a label (second label, serial number of feature point, feature point ID). The same label (label) is assigned to the first feature point and the second feature point that correspond to each other.

[0052] Next, the first rotation angle calculation unit 22 calculates the first rotation angle γ of the rotating body obtained for the second image which is the previous image adjacent to the first image in time series. k－1 The first rotation angle γ of the rotating body related to the first image is obtained by adding the rotation angle Δγ between the first image and the second image. k (γ k =γ k－1 +Δγ).

[0053] Therefore, the rotation angle of the rotating body in the initial image among the multiple images is set to 0 (that is, the position of the rotating body in the initial image among the multiple images relative to the base is used as a reference), and the first rotation angle of the rotating body is calculated for each of the multiple images.

[0054] The first rotation angle of the rotating body thus obtained is expressed as an analysis value together with its actual measured value. Figure 5 The actual value is the measurement result measured by the rotary encoder. Figure 5 As shown, the analysis value is substantially consistent with the measured value, and the first rotation angle of the rotating body is estimated with high accuracy by the above method.

[0055] The reference calculation unit 23 obtains a predetermined place in the outside world as a reference for the rotation angle based on the plurality of first rotation angles obtained by the first rotation angle calculation unit 22 for the plurality of images respectively. More specifically, the reference calculation unit 23 obtains a straight line that approximates the plurality of first rotation angles relative to time by using a predetermined linear approximation method, and obtains the angle of the obtained straight line as a reference for the rotation angle. The plurality of first rotation angles are a plurality of first rotation angles arranged in a time series obtained by the first rotation angle calculation unit 22 for the plurality of images respectively. The linear approximation method is, for example, the least squares method or RANSAC (Random Sample Consensus). An example of the reference γ0 of the rotation angle obtained using the least squares method is shown in FIG. Figure 6 .exist Figure 6 In the example shown, the reference γ0 of the rotation angle is expressed with reference to the position of the rotating body relative to the base body in the first image among the plurality of images arranged in time series (γ1=0).

[0056] The second rotation angle calculation unit 24 calculates, for each of the plurality of images, a second rotation angle of the rotating body relative to the reference based on the first rotation angle calculated for the image by the first rotation angle calculation unit 22 and the reference calculated by the reference calculation unit 23. More specifically, the second rotation angle calculation unit 24 calculates, for each of the plurality of images, a second rotation angle of the rotating body relative to the reference by subtracting the reference calculated by the reference calculation unit 23 from the first rotation angle calculated for the image by the first rotation angle calculation unit 22.

[0057] One example is shown in Figure 7 A. For example, in the case of an operation such as dismantling a car by an open-close crusher, in which a rotating body is rotated left and right relative to a predetermined second direction in the outside world, the same operation is performed on each car to be dismantled. Therefore, the distribution of the first rotation angle of the rotating body is as follows: Figure 7 As shown in B, although the shapes are the same, they are not consistent because the initial positions at the beginning of the dismantling operation (the direction of the open and closed crusher at the beginning of the dismantling operation) are different. Figure 7In B, a curve GRA of the first rotation angle obtained based on a video A of a dismantling operation on a car a and a curve GRB of the first rotation angle obtained based on a video B of a dismantling operation on another car b are illustrated. Although the curves GRA and GRB have the same shape, they are not consistent. However, in the rotation angle measuring device D for construction machinery of the present embodiment, a prescribed place in the outside world is extrapolated as a prescribed second place in the outside world, and the second rotation angle of the rotating body is obtained with the direction of the rotating body toward the car to be dismantled as a reference for the rotation angle. Therefore, as Figure 7 As shown in A, the distributions of the second rotation angles of the rotational bodies for each of the vehicles to be dismantled are substantially the same, and the dismantling operations for each of the vehicles to be dismantled can be analyzed and compared.

[0058] The image acquisition unit 1, control processing unit 2, input unit 3, output unit 4, IF unit 5 and storage unit 6 described above may be constituted by, for example, a desktop, notebook or tablet computer. The computer may be incorporated into the construction machine HS, or, if the construction machine HS is equipped with a computer, the computer may also be used as the computer of the construction machine HS.

[0059] Next, the operation of this embodiment will be described. Figure 8 This is a flowchart showing the operation of the swing angle measuring device for construction machinery.

[0060] After the power supply of the construction machine rotation angle measuring device D is turned on, the necessary initialization of each part is performed and the operation is started. In the control processing unit 2, by executing the above-mentioned control processing program, the control unit 21, the first rotation angle calculation unit 22, the reference calculation unit 23 and the second rotation angle calculation unit 24 are functionally constituted.

[0061] exist Figure 8 In the embodiment, the swing angle measuring device D for construction machinery first obtains a plurality of images of the outside world viewed from the swing body arranged in time series from the camera unit 106 by the control unit 21 of the control processing unit 2, and stores them in the storage unit 6 (S1). In this embodiment, the video is obtained as described above. For example, a video of a predetermined length of time equivalent to the operation time from the start to the end of an operation is obtained.

[0062] Next, the rotation angle measuring device D for construction machinery uses the first rotation angle calculation unit 22 of the control processing unit 2 to divide the video into individual frames to form the multiple images arranged in time series, and adds integer serial numbers k from 1 to K to the above multiple images and stores them in the storage unit 6 (S2).

[0063] Next, the construction machine rotation angle measuring device D uses the first rotation angle calculation unit 22 to extract feature points from the first image PC(1) and store them in the storage unit 6, sets 2 as the loop counter k (k=2), and sets the initial value 0 as the first rotation angle γ1 of the rotation body related to the first image PC(1) (γ1=0, S3). The sequence number k is also used as the loop counter k.

[0064] Next, the swing angle measuring device D for construction machinery uses the first swing angle calculation unit 22 to extract feature points from the k-th image PC(k) and stores the extracted feature points in the storage unit 6 ( S3 ).

[0065] Next, the rotation angle measuring device D for construction machinery uses the first rotation angle calculation unit 22 to search for and select the feature points extracted from the (k-1)th image (second image) PC(k-1) corresponding to the feature points extracted from the kth image (first image) PC(k) (matching of feature points), and store them in the storage unit 6 (S5).

[0066] Next, the construction machine swing angle measuring device D uses the first swing angle calculation unit 22 to perform filtering processing for removing erroneous correspondence of feature point pairs corresponding to each other through the corresponding point search, and stores the results in the storage unit 6 ( S6 ).

[0067] Next, the rotation angle measuring device D for construction machinery uses the first rotation angle calculation unit 22 to calculate the average value of the movement of all feature point pairs after the filtering process as the movement between the kth image (first image) PC(k) and the (k-1)th image (second image) PC(k-1), and based on the calculated movement amount, calculates the rotation angle (angle change) Δγ and stores it in the storage unit 6 (S7).

[0068] Next, the construction machine rotation angle measuring device D uses the first rotation angle calculation unit 22 to calculate the first rotation angle γ of the rotation body obtained for the (k-1)th image (second image) PC(k-1) k－1 The first rotation angle γ of the rotating body related to the k-th image (first image) PC(k) is calculated by adding the rotation angle (angle change) Δγ obtained. k (γ k =γ k－1 +Δγ, S8).

[0069] Next, the construction machine rotation angle measuring device D uses the first rotation angle calculation unit 22 to determine whether the cycle count value k is the total number K of the plurality of images (S9). If the result of the determination is that the cycle count value k is the total number K, the construction machine rotation angle measuring device D then performs processing S11. On the other hand, if the result of the determination is that the cycle count value k is not the total number K, the construction machine rotation angle measuring device D increments the cycle count value k by 1 (k=k+1, S10), and returns the processing to processing S4.

[0070] In the process S11, the construction machine rotation angle measuring device D uses the reference calculation unit 23 of the control processing unit 2 to calculate the first rotation angles γ1 to γ1 obtained by the first rotation angle calculation unit 22 for the plurality of images PC(1) to PC(K). K , find a specified place in the outside world as the reference γ0 of the rotation angle.

[0071] Next, the construction machine rotation angle measuring device D uses the second rotation angle calculation unit 24 of the control processing unit 2 to calculate the first rotation angle γ obtained by the first rotation angle calculation unit 22 for the image PC(k) for each of the plurality of images PC(1) to PC(k). k The second rotation angle of the rotating body relative to the reference γ0 is obtained by the reference calculation unit 23, and the processing is terminated (S12). In addition, the engineering machinery rotation angle measuring device D outputs the second rotation angle to the output unit 4 as needed, and outputs the second rotation angle to an external device via the IF unit 5 as needed.

[0072] As described above, the rotation angle measuring device D for construction machinery in the embodiment, the rotation angle measuring method for construction machinery installed on the rotation angle measuring device D for construction machinery, and the rotation angle measuring program for construction machinery are not based on a specific object or mark, etc., but rather find a specified place in the outside world as a reference for the rotation angle, and find the second rotation angle of the rotating body relative to the reference. Therefore, the reference is set in the outside world facing the rotating body in operation, so the rotation angle of the rotating body can be found based on the direction in which the rotating body is mainly facing during operation.

[0073] The above-mentioned rotation angle measuring device D for construction machinery, the rotation angle measuring method for construction machinery, and the rotation angle measuring program for construction machinery can obtain the second rotation angle of the rotation body based on the estimated second position in the operation in which the rotation body is rotated left and right relative to a predetermined second position in the outside world. Therefore, when multiple operations of the same type are performed, even if the rotation body faces different directions at the beginning of each operation, each second rotation angle can be compared at each time in each operation from the beginning of each operation, and each distribution of the second rotation angle relative to the elapsed time can be compared in each operation.

[0074] In the above embodiment, the reference calculation unit 23 may exclude the first rotation angle of the rotating body relative to the base body (in the first rotation angle calculation unit 22) from the plurality of first rotation angles respectively obtained for the plurality of images. Figure 2 In the example shown, the first rotation angle when the lower walking body 101 is stationary is used to obtain the reference of the rotation angle. Thus, since the first rotation angle when the rotating body is stationary relative to the base is excluded and the reference of the rotation angle is obtained, the reference can be obtained with higher accuracy than when the exclusion is not performed.

[0075] Fig. 9 The graph shows an example of a plurality of first rotation angles arranged in time series when the rotating body is stationary and a reference of the rotation angle obtained based on the first rotation angles. Fig. 9 The horizontal axis is time and the vertical axis is angle. Fig.10 It means in Fig. 9 In the case shown, it is a diagram showing the result of determining whether the rotating body is stationary. Fig.10 The horizontal axis is time, and the vertical axis is the judgment result. "1.0" indicates that the judgment result is still, and "0.0" indicates that the judgment result is not still. Fig.11 It means in Fig. 9 In the case shown, a plurality of first rotation angles are arranged in time series, and a graph showing a reference of a rotation angle obtained by excluding the first rotation angle when the rotating body is stationary. Fig.11 The horizontal axis is time and the vertical axis is angle.

[0076] For example, if the rotating body is stationary due to some reasons such as the operator taking a short break, the first rotation angle of the rotating body may be, for example, Fig. 9As shown, it becomes approximately fixed. If the reference γ0 of the rotation angle is directly calculated based on the first rotation angle of the rotating body, it is impossible to correctly calculate the reference γ0 of the rotation angle based on the first rotation angle of the rotating body that has become approximately fixed. Therefore, the reference calculation unit 23 further calculates the dispersion of the movement amount based on a fixed number of images, and the fixed number of images are near the focus moment (focus timing) at which it is desired to determine whether the rotating body is stationary. When the calculated dispersion is below a preset threshold value (the fourth threshold value), it is determined that the rotating body is stationary at the focus moment. When the calculated dispersion exceeds the fourth threshold value, it is determined that the rotating body is not stationary at the focus moment. For example, the fourth threshold value is appropriately set in advance based on a plurality of samples. The fixed number of images near the focus moment are, for example, the image at the focus moment and 5 or 10 images before and after the image arranged in time series. Next, as described above, the reference calculation unit 23 calculates the reference γ0 of the rotation angle by excluding the first rotation angle when the rotating body is stationary relative to the base from the plurality of first rotation angles calculated by the first rotation angle calculation unit 22 for the plurality of images. Fig. 9 In the case shown in FIG. 1 , the result of determining whether the rotating body is stationary is indicated by Fig.10 The reference γ0 of the rotation angle obtained by excluding the first rotation angle when the rotating body is stationary is represented by Fig.11 .and Fig. 9 Compared with the reference γ0 of the rotation angle shown, Fig.11 The reference γ0 of the rotation angle shown is corrected.

[0077] In addition, although the camera unit 106 is installed outside the rotating body in the above-mentioned embodiment, Figure 2 In the example shown, it is installed outside the cabin 105 , but the camera unit 106 may be installed inside the rotating body, for example, inside the cabin 105 , in a manner capable of photographing the outside world.

[0078] In addition, in the above-mentioned embodiment, the swing angle measuring device D for construction machinery may also be as follows, for example Figure 1 As shown by the dotted line of , the storage unit 6 is functionally provided with an exclusion area storage unit 61 for storing a predetermined exclusion area, and the first rotation angle calculation unit 22 calculates the first rotation angle of the rotating body based on the image with the exclusion area removed and the previous image adjacent to the image in time series with the exclusion area removed for each of the plurality of images. Thus, when calculating the first rotation angle of the rotating body, the exclusion area is removed from the image and the image adjacent to the image, respectively, so that the amount of information processing can be reduced.

[0079] In the plurality of images arranged in time series, an image region in which the same subject is photographed at the same position does not contribute to the calculation of the movement amount between the first feature point and the second feature point, and such an image region is set as an exclusion region. The subject in the exclusion region is, for example, an object that is still photographed at the same position in the image by the camera unit 106 even if the rotating body rotates. In particular, as described above, when the camera unit 106 is installed inside the rotating body (in the Figure 2 In the example shown, the interior of the cabin 105 is shown, the frame (window frame), operating rod, floor, etc. of the rotating body become objects that are captured in the same position of the image by the camera unit 106 even if the rotating body rotates, and the subject of the exclusion area is at least one of the frame (window frame), operating rod, and floor of the rotating body. As a result, in the multiple images, the image area where the same subject is captured at the same position is set as the exclusion area, so that the area where the object that rotates as the rotating body rotates is captured can be excluded.

[0080] Such an exclusion area is set manually, for example, and stored in the exclusion area storage unit 61. For example, in the initial setting, the image generated by the camera unit 106 is acquired by the image acquisition unit 1, and the acquired image is output to the output unit 4. For example, the output unit 4 is a display device, such as a display Fig.12 The image shown is SP. Fig.12 2 is a diagram for explaining manual setting of an exclusion area. The operator (user) uses an input unit 3 such as a mouse to draw the outer contour of the area to be set as the exclusion area AR with a mouse pointer, thereby inputting the exclusion area. Fig.12 In the example shown, three areas where the operating lever is photographed, i.e., the first image area AR1 to the third image area AR3, are input as three exclusion areas AR1 to AR3. Then, the exclusion areas AR input in this way are stored in the exclusion area storage unit 61. The initial setting is implemented when the camera unit 106 is installed on the construction machine HS and the swing angle measuring device D for construction machinery is started to be used. In addition, even when the camera unit 106 is reinstalled on the construction machine HS or a predetermined time has passed, the exclusion areas AR can be set in the same manner as the initial setting.

[0081] In addition, although the setting of the exclusion area AR is performed manually in the above description, it may be performed automatically by the swing angle measuring device D for construction machinery according to the initial setting or the like. In this case, in the control processing unit 2 of the swing angle measuring device D for construction machinery, as shown in FIG. Figure 1As shown by the dotted line, the device further functionally includes an exclusion region extraction unit 25 that extracts the exclusion region based on the image and stores the extracted exclusion region in the exclusion region storage unit 61.

[0082] More specifically, for example, an image of the subject (e.g., an image of a joystick, etc.) representing the exclusion area is prepared in advance as a template and stored in the storage unit 6. Next, the exclusion area extraction unit 25 uses the image acquisition unit 1 to acquire the image generated by the camera unit 106, and searches and extracts the area that best matches the template from the acquired image through so-called template matching, and stores the extracted area as the exclusion area in the exclusion area storage unit 61.

[0083] Alternatively, for example, so-called geometric shape matching may be used instead of the above template matching. In this geometric shape matching, for example, an edge is used as the geometric shape, and the edge is extracted from the template and the image respectively, and the edge of the template is compared with the edge of the image, and the area having the edge that is most consistent with the edge of the template is extracted from the image as the exclusion area.

[0084] Alternatively, for example, the image acquisition unit 1 acquires two images PC(t1) and PC(t2) generated at different times t1 and t2 during the rotation of the rotating body. The exclusion area extraction unit 25 extracts feature points (third feature points) from one of the two images PC(t1) and PC(t2) acquired by the image acquisition unit 1, for example, PC(t1), and extracts feature points (fourth feature points) corresponding to the third feature points extracted from the one image PC(t1) from the other image PC(t2). When the extracted corresponding third feature points and fourth feature points are made consistent and the two images PC(t1) and PC(t2) are overlapped, the pixel values ​​of the one image PC(t1) and the pixel values ​​of the other image PC(t2) are compared at the same pixel positions for the pixels in the overlapping area, and the pixels whose pixel values ​​are consistent are extracted as the pixels of the exclusion area, and the area composed of the extracted pixels is stored in the exclusion area storage unit 61 as the exclusion area. When the same subject is photographed in the two images PC(t1) and PC(t2), the pixel values ​​are consistent, so the pixel values ​​are compared. In the extraction of the feature points or the corresponding point search, the same method as the first rotation angle calculation unit 22, such as A-KAZE or brute force matching, is used. After the corresponding point search, the filtering process used in the first rotation angle calculation unit 22 can also be implemented as needed. In order to make the two images PC(t1) and PC(t2) overlap with higher position accuracy, it is preferred to extract multiple third feature points and multiple fourth feature points. In addition, at least one of the two images PC(t1) and PC(t2) is affine transformed as needed. The area composed of the extracted pixels is an area composed of pixels that are continuous with each other among the extracted pixels, and the area is one or more. In the exclusion area, there may be a single color or multiple colors, so the pixel values ​​of the continuous pixels may be the same or different. Furthermore, pixels that are separated by a preset number of pixels (eg, one pixel or two pixels) may be grouped into one region.

[0085] Fig.13 This is a flowchart showing the operation of the swing angle measuring device for construction machinery related to the automatic setting of the exclusion area.

[0086] Regarding the automatic setting of the exclusion area, Fig.13In the process, it is first determined whether the rotation has started (S21). If the result of the determination is that the rotation has not started (No), the process returns to the process S21. On the other hand, if the rotation has started (Yes), an image is generated by the camera unit 106, and the image is stored (saved) (S22), and it is determined whether the rotation has ended (S23). If the result of the determination is that the rotation has not ended (No), the process returns to the process S22. On the other hand, if the rotation has started (Yes), the process S24 is then executed. The start and end of the rotation are implemented, for example, by an acceleration sensor that detects acceleration and is installed on the rotating body. If the acceleration is detected by the acceleration sensor, it is determined that the rotation has started. If the acceleration is not detected continuously for a predetermined period of time, it is determined that the rotation has ended. Furthermore, the start of the rotation can be detected by an increase in the detected value of the amount of operation of the operating lever or an increase in the angular velocity detected by the gyro sensor, and the end of the rotation can be detected by a decrease in the detected value of the amount of operation of the operating lever or a decrease in the angular velocity detected by the gyro sensor. In addition, the automatic setting of the exclusion area requires the two images PC(t1) and PC(t2), so when the two images PC(t1) and PC(t2) are generated and stored, it can be determined that the rotation is completed even if the rotation is in progress.

[0087] In the process S24, the camera unit 106 outputs two images PC(t1) and PC(t2) generated at different timings t1 and t2 during the rotation of the rotating body. The two images PC(t1) and PC(t2) are acquired by the image acquisition unit 1 and stored in the storage unit 6. Next, the exclusion area extraction unit 25 extracts the exclusion area AR based on the two images PC(t1) and PC(t2) acquired by the image acquisition unit 1 (S25), stores the extracted exclusion area AR in the exclusion area storage unit 61 and sets it (S26), and ends the process.

[0088] In addition, in the above-mentioned embodiment, although the swing angle measuring device D for construction machinery is integrally constituted with a computer, the swing angle measuring device D for construction machinery includes the above-mentioned embodiment and the above-mentioned variant, and may also be constituted by the following system, which includes a construction machinery HS, a portable terminal device and a management server connected in a manner that can communicate with each other via a communication network. In this case, the construction machinery HS sends the video generated by the camera unit 106 to the management server via the communication network, and the management server stores and manages the video. In addition, it may also be that the construction machinery HS sends the video generated by the camera unit 106 to the portable terminal device via the communication network, and the portable terminal device can store and display the video. The portable terminal device functions as the input unit 3 and the output unit 4 relative to the management server. The management server functions as the image acquisition unit 1, the control processing unit 2 and the storage unit 6.

[0089] This specification discloses various types of technologies as described above, but the main technologies are summarized as follows.

[0090] A rotation angle measuring device for construction machinery according to one embodiment includes: an image acquisition unit that acquires a plurality of images generated by a camera unit, the camera unit being disposed on the construction machinery in a manner that moves along with the rotation of a rotating body, and generating the plurality of images of the outside world viewed from the rotating body and arranged in a time series, the construction machinery including a base, the rotating body rotatably connected to the base, and an operating mechanism disposed on the rotating body and capable of performing a predetermined operation; and a first rotation angle calculation unit that calculates, for each of the plurality of images, a rotation angle based on the image and a previous image adjacent to the image in the time series. A first rotation angle of the rotating body is calculated based on the position of the rotating body in the initial image of the multiple images relative to the base; a reference calculation unit calculates a specified place (first place) in the outside world as a reference for the rotation angle based on the multiple first rotation angles calculated by the first rotation angle calculation unit for the multiple images respectively; and a second rotation angle calculation unit calculates the second rotation angle of the rotating body relative to the reference based on the first rotation angle calculated by the first rotation angle calculation unit for the image and the reference calculated by the reference calculation unit for each of the multiple images. Preferably, in the above-mentioned rotary angle measuring device for construction machinery, the base is a walking body, the working mechanism is a mechanism including a boom, a dipper and an openable crusher, the boom is connected to the rotary body so as to be swingable in the upward and downward directions, the dipper is connected to the boom so as to be swingable in the upward and downward directions, the openable crusher is connected to the dipper and has a pair of crushing knives that can be opened and closed, and the construction machinery is a dismantling machine. Preferably, in the above-mentioned rotary angle measuring device for construction machinery, the camera unit is installed on the outside of the rotary body. Preferably, in the above-mentioned rotary angle measuring device for construction machinery, the camera unit is installed on the inside of the rotary body in a manner that can capture the outside world.

[0091] This type of rotation angle measuring device for construction machinery does not use a specific object or mark as a reference, but instead finds a specified place in the outside world as a reference for the rotation angle, and finds the second rotation angle of the rotating body relative to the reference. Therefore, the reference is set in the outside world facing the rotating body in operation, so the rotation angle of the rotating body can be found based on the direction in which the rotating body mainly faces during operation.

[0092] In another embodiment, in the above-mentioned rotation angle measuring device for construction machinery, the specified operation is an operation performed by rotating the rotating body left and right relative to a specified second place in the outside world, and the specified place (first place) in the outside world is calculated as the specified second place in the outside world.

[0093] This kind of rotation angle measuring device for construction machinery can obtain the second rotation angle of the rotation body with reference to the estimated second position in the operation in which the rotation body is rotated left and right relative to a predetermined second position in the outside world. Therefore, when a plurality of operations of the same type are performed, even if the rotation body faces different directions at the start of each operation, each second rotation angle can be compared at each timing in each operation from the start of each operation, and each distribution of the second rotation angle with respect to the elapsed time can be compared in each operation.

[0094] In another embodiment, in the above-mentioned rotation angle measuring devices for construction machinery, the reference calculation unit obtains the reference of the rotation angle by excluding the first rotation angle when the rotating body is stationary relative to the base from the multiple first rotation angles obtained by the first rotation angle calculation unit for the multiple images respectively.

[0095] This construction machine rotation angle measuring device excludes the first rotation angle when the rotating body is stationary relative to the base body to obtain the reference of the rotation angle, so that the reference can be obtained with higher accuracy than when the exclusion is not performed.

[0096] In another embodiment, in the above-mentioned swing angle measuring device for construction machine, the reference calculation unit obtains the reference of the swing angle by using a predetermined linear approximation method.

[0097] Thus, it is possible to provide a swing angle measuring device for a construction machine that obtains the reference of the swing angle by using a predetermined linear approximation method.

[0098] In another embodiment, the above-mentioned rotation angle measuring devices for engineering machinery further include: an exclusion area storage unit for storing a specified exclusion area, wherein the first rotation angle calculation unit calculates the first rotation angle of the rotating body for each of the plurality of images based on the image with the exclusion area removed and a previous image adjacent to the image in a time series with the exclusion area removed.

[0099] This construction machine rotation angle measuring device removes the exclusion area from each of the image and an image adjacent to the image when obtaining the first rotation angle of the rotation body, thereby reducing the amount of information processing.

[0100] In another embodiment, in the above-mentioned rotary angle measuring device for construction machinery, the exclusion area is an image area in which the same subject is captured at the same position in the plurality of images. Preferably, in the above-mentioned rotary angle measuring device for construction machinery, the subject is an object that is captured in the same position of the image by the camera unit even when the rotary body rotates. Preferably, in the above-mentioned rotary angle measuring device for construction machinery, when the camera unit is installed inside the rotary body in a manner that can capture the outside world, the subject is at least one of the frame (window frame), operating rod and floor of the rotary body.

[0101] This construction machine swing angle measuring device sets the image area where the same object is photographed at the same position in the plurality of images as the exclusion area, and thus can exclude the area where the object that rotates as the swing body rotates is photographed.

[0102] Another embodiment of the invention relates to a method for measuring the rotation angle of an engineering machine, comprising: an image acquisition step of acquiring a plurality of images generated by a camera unit, wherein the camera unit is arranged on the engineering machine in a manner that moves with the rotation of a rotating body, and generates the plurality of images of the outside world viewed from the rotating body and arranged in a time series, wherein the engineering machine comprises a base, the rotating body rotatably connected to the base, and an operating mechanism arranged on the rotating body and capable of performing a prescribed operation; and a first rotation angle calculation step of calculating a rotation angle of the plurality of images based on the image and a previous image adjacent to the image in the time series. A first rotation angle of the rotating body is calculated based on the position of the rotating body in the initial image of the multiple images relative to the base; a reference calculation step is performed to calculate a specified place in the outside world as a reference for the rotation angle based on the multiple first rotation angles calculated by the first rotation angle calculation step for the multiple images respectively; and a second rotation angle calculation step is performed to calculate the second rotation angle of the rotating body relative to the reference based on the first rotation angle calculated by the first rotation angle calculation step for the image and the reference calculated by the reference calculation step for the multiple images respectively.

[0103] In another embodiment, in a storage medium (storage medium) recording (storing) a rotation angle measurement program for construction machinery, the rotation angle measurement program for construction machinery is a program for causing a computer to function as an image acquisition unit, a first rotation angle calculation unit, a reference calculation unit, and a second rotation angle calculation unit, wherein the image acquisition unit acquires a plurality of images generated by a camera unit, the camera unit is arranged on the construction machinery in a manner that moves with the rotation of the rotating body, and generates the plurality of images of the outside world viewed from the rotating body and arranged in a time series, the construction machinery including a base, the rotating body rotatably connected to the base, and an operating mechanism arranged on the rotating body and capable of performing a specified operation; the first rotation angle calculation unit The rotation angle calculation unit calculates the first rotation angle of the rotating body for each of the multiple images based on the image and a previous image adjacent to the image in a time series, taking the position of the rotating body in an initial image among the multiple images relative to the base as a reference; the reference calculation unit calculates a specified place in the outside world as a reference for the rotation angle based on the multiple first rotation angles calculated by the first rotation angle calculation unit for each of the multiple images; the second rotation angle calculation unit calculates the second rotation angle of the rotating body relative to the reference for each of the multiple images based on the first rotation angle calculated by the first rotation angle calculation unit for each of the images and the reference calculated by the reference calculation unit.

[0104] These methods and programs for measuring the rotation angle of construction machinery do not use a specific object or mark as a reference, but find a predetermined place in the outside world as a reference for the rotation angle, and find the second rotation angle of the rotating body relative to the reference. Therefore, the reference is set in the outside world facing the rotating body in operation, so the rotation angle of the rotating body can be found based on the direction in which the rotating body is mainly facing during operation. According to the present invention, a storage medium having such a program for measuring the rotation angle of construction machinery recorded therein can be provided.

[0105] This application is based on Japanese patent application No. 2022-105343 filed on June 30, 2022, and the contents are incorporated herein by reference.

[0106] In order to embody the present invention, the present invention is appropriately and fully described in the above contents with reference to the accompanying drawings and through the embodiments, but it should be recognized that those skilled in the art can easily change and / or improve the above embodiments. Therefore, as long as the changes or improvements implemented by those skilled in the art do not deviate from the scope of rights of the claims recorded in the claims, it is interpreted that the changes or improvements are included in the scope of rights of the relevant claims.

[0107] Industrial Applicability

[0108] According to the present invention, a construction machine rotation angle measuring device, a construction machine rotation angle measuring method, and a storage medium recording a construction machine rotation angle measuring program can be provided for obtaining a rotation angle of a rotating body in the construction machine.

Claims

1. A rotation angle measuring device for construction machinery, characterized in that include: an image acquisition unit that acquires a plurality of images generated by an imaging unit, the imaging unit being provided on the construction machine in a manner that moves along with the rotation of the rotating body, and generating the plurality of images of the outside world viewed from the rotating body arranged in a time series, the construction machine including a base, the rotating body rotatably connected to the base, and an operating mechanism provided on the rotating body and capable of performing a predetermined operation; A first rotation angle calculation unit is configured to calculate, for each of the plurality of images, a first rotation angle of the rotating body based on the image and a previous image adjacent to the image in time series, with the position of the rotating body relative to the base in an initial image among the plurality of images as a reference; a reference calculation unit that calculates a predetermined location in the outside world as a reference for a rotation angle based on the plurality of first rotation angles calculated by the first rotation angle calculation unit for the plurality of images; and The second rotation angle calculation unit calculates, for each of the plurality of images, a second rotation angle of the rotating body relative to the reference based on the first rotation angle calculated for the image by the first rotation angle calculation unit and the reference calculated by the reference calculation unit.

2. The rotary angle measuring device for construction machinery according to claim 1, characterized in that: The predetermined operation is an operation performed by rotating the rotating body leftward or rightward relative to a predetermined second direction in the outside world. The predetermined place in the outside world is estimated as a predetermined second place in the outside world.

3. The rotary angle measuring device for construction machinery according to claim 1, characterized in that: The reference calculation unit obtains the reference of the rotation angle by excluding the first rotation angle when the rotating body is stationary with respect to the base body from the plurality of first rotation angles obtained by the first rotation angle calculation unit for each of the plurality of images.

4. The rotary angle measuring device for construction machinery according to any one of claims 1 to 3, characterized in that: The reference calculation unit obtains the reference of the rotation angle by using a predetermined linear approximation method.

5. The rotary angle measuring device for construction machinery according to claim 1, characterized in that Also includes: The exclusion area storage unit stores a predetermined exclusion area, wherein: The first rotation angle calculation unit obtains a first rotation angle of the rotating body based on the image from which the exclusion region is removed and a previous image adjacent to the image in time series from which the exclusion region is removed, for each of the plurality of images.

6. The rotary angle measuring device for construction machinery according to claim 5, characterized in that: The exclusion area is an image area in which the same subject is captured at the same position in the plurality of images.

7. A method for measuring the rotation angle of an engineering machine, characterized in that include: An image acquisition step of acquiring a plurality of images generated by an imaging unit, the imaging unit being provided on the construction machine in a manner of moving along with the rotation of the rotating body, and generating the plurality of images arranged in time series of the outside world as seen from the rotating body, the construction machine comprising a base, the rotating body rotatably connected to the base, and an operating mechanism provided on the rotating body and capable of performing a predetermined operation; a first rotation angle calculation step, for each of the plurality of images, based on the image and a previous image adjacent to the image in time series, and taking the position of the rotating body relative to the base in the initial image of the plurality of images as a reference, to calculate a first rotation angle of the rotating body; a reference calculation step of calculating a predetermined location in the outside world as a reference for a rotation angle based on the plurality of first rotation angles calculated for the plurality of images in the first rotation angle calculation step; and The second rotation angle calculation step calculates, for each of the plurality of images, a second rotation angle of the rotating body relative to the reference based on the first rotation angle calculated for the image in the first rotation angle calculation step and the reference calculated in the reference calculation step.

8. A storage medium having recorded therein a rotation angle measurement program for construction machinery, characterized in that: The engineering machine rotation angle measurement program causes a computer to function as an image acquisition unit, a first rotation angle calculation unit, a reference calculation unit, and a second rotation angle calculation unit, wherein: The image acquisition unit acquires a plurality of images generated by a camera unit, the camera unit being provided on the construction machine in a manner of moving along with the rotation of the rotating body, and generating the plurality of images arranged in time series of the outside world as seen from the rotating body, the construction machine including a base, the rotating body rotatably connected to the base, and an operation mechanism provided on the rotating body and capable of performing a predetermined operation; The first rotation angle calculation unit calculates the first rotation angle of the rotating body for each of the plurality of images based on the image and a previous image adjacent to the image in time series, taking the position of the rotating body relative to the base in the first image among the plurality of images as a reference; The reference calculation unit calculates a predetermined location in the outside world as a reference of the rotation angle based on the plurality of first rotation angles calculated by the first rotation angle calculation unit for the plurality of images respectively; The second rotation angle calculation unit calculates, for each of the plurality of images, a second rotation angle of the rotating body relative to the reference based on the first rotation angle calculated for the image by the first rotation angle calculation unit and the reference calculated by the reference calculation unit.

Images