Method and Apparatus for Motion Counting Based on Excavator Pose Estimation

Abstract

According to one embodiment of the present disclosure, a motion counting method based on estimating the attitude of an excavator may include: acquiring an image of an excavator; detecting a predetermined first important point, a second important point, and a third important point in the image; generating a region of interest based on the location of the first important point and the location of the second important point; and tracking the location of the third important point and counting the motion of the excavator based on the positional relationship between the region of interest and the third important point.

Description

Technology Field

[0001] The present disclosure relates to a motion counting method and apparatus based on excavator attitude estimation. Background Technology

[0002] In cases where a worker must repeatedly perform the same task, such as removing air from an excavator cylinder, the cumulative number of movements must be counted.

[0003] In this case, methods are used in which an operator directly checks the excavator's operation visually to count it, or attaches sensors to the excavator to count its operation.

[0004] However, when the operator counts directly, there may be a problem where the accuracy of the count decreases due to factors such as reduced concentration, and in the case of the sensor attachment method, there may be the inconvenience of having to attach a sensor to each excavator performing the operation.

[0005] The aforementioned background technology is technical information that the inventor possessed for the derivation of the present invention or acquired during the process of deriving the present invention, and it cannot be considered as prior art disclosed to the general public prior to the filing of the present invention. The problem to be solved

[0006] Some embodiments according to the present disclosure aim to provide a motion counting method and apparatus based on excavator attitude estimation. The problems to be solved by the present invention are not limited to those mentioned above, and other problems and advantages of the present invention not mentioned can be understood from the following description and will be more clearly understood by embodiments of the present invention. Furthermore, it will be understood that the problems and advantages to be solved by the present invention can be realized by means and combinations thereof as set forth in the claims. means of solving the problem

[0007] As a technical means for achieving the technical problem described above, the first aspect of the present disclosure may provide a method for counting motions based on an excavator attitude estimation, comprising: acquiring an image of an excavator; detecting a predetermined first important point, a second important point, and a third important point in the image; generating a region of interest based on the location of the first important point and the location of the second important point; and tracking the location of the third important point and counting a specific motion of the excavator based on the positional relationship between the region of interest and the third important point.

[0008] A second aspect of the present disclosure provides an excavator attitude estimation-based motion counting device comprising: a memory in which at least one program is stored; and a processor that operates by executing said at least one program, wherein the processor acquires an image of an excavator, detects a first important point, a second important point, and a third important point determined in said image, generates a region of interest based on the location of said first important point and the location of said second important point, tracks the location of said third important point, and counts a specific motion of said excavator based on the positional relationship between said region of interest and said third important point.

[0009] A third aspect of the present disclosure may provide a computer-readable recording medium having a program for executing a method according to a first aspect on a computer.

[0010] In addition to this, other methods for implementing the present invention, other systems, and computer-readable recording media storing a computer program for executing said methods may be further provided.

[0011] Other aspects, features, and advantages other than those described above will become clear from the following drawings, claims, and detailed description of the invention. Effects of the invention

[0012] According to the means for solving the problem of the present disclosure described above, counting can be performed automatically without the excavator operator directly counting the operation of the excavator, thereby reducing the operator's fatigue and improving work efficiency.

[0013] The effects of the embodiments are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art from the description in this specification. Brief explanation of the drawing

[0014] FIG. 1 is a drawing for illustrating an example of a system for counting the operation of an excavator according to one embodiment of the present disclosure. FIG. 2 is a flowchart for schematically explaining the operation of an excavator attitude estimation-based motion counting device according to one embodiment of the present disclosure. FIG. 3 is a drawing for explaining an example of a captured image according to one embodiment of the present disclosure. FIG. 4 is a drawing for illustrating an example of a plurality of critical points detected according to one embodiment of the present disclosure. FIG. 5 is a drawing for illustrating an example of a region of interest according to one embodiment of the present disclosure. FIG. 6 is a drawing for explaining a method for counting specific operations of an excavator according to one embodiment of the present disclosure. FIG. 7 is a drawing for explaining an example of how visualization information according to one embodiment of the present disclosure is displayed to a user. FIG. 8 is a flowchart illustrating an example of a motion counting method based on excavator attitude estimation according to one embodiment of the present disclosure. FIG. 9 is a block diagram illustrating an example of the internal configuration of an excavator attitude estimation-based motion counting device according to one embodiment of the present disclosure. Specific details for implementing the invention

[0015] The advantages and features of the present disclosure and the methods for achieving them will become clear by referring to the embodiments described in detail together with the accompanying drawings. However, the present disclosure is not limited to the embodiments presented below, but can be implemented in various different forms and should be understood to include all modifications, equivalents, and substitutions that fall within the spirit and scope of the present disclosure. The embodiments presented below are provided to make the present disclosure complete and to fully inform those skilled in the art of the scope of the invention. In describing the present disclosure, detailed descriptions of related prior art are omitted where it is determined that such detailed descriptions may obscure the essence of the present invention.

[0016] The terms used herein are used merely to describe specific embodiments and are not intended to limit the disclosure. Unless otherwise defined, all terms used herein have the same meaning as generally understood by those skilled in the art to which this disclosure pertains.

[0017] In this specification, singular expressions include plural expressions unless the context clearly indicates otherwise. Furthermore, terms such as “comprising” or “having” are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0018] Additionally, terms including ordinal numbers, such as “first” or “second” as used herein, may be used to describe various components, but the components should not be limited by the terms. The terms are used solely for the purpose of distinguishing one component from another.

[0019] Phrases such as “in one embodiment,” “according to one embodiment,” “related to one embodiment,” or “according to an implementation of one embodiment” in this specification do not necessarily refer to the same embodiment. Furthermore, throughout this specification, “exemplars” are arbitrary distinctions to facilitate the description of the present disclosure, and each embodiment is not required to be mutually exclusive. For example, configurations mentioned for the description of one embodiment may be applied and / or implemented in other embodiments, and may be modified and applied and / or implemented to the extent that they do not depart from the scope of the present disclosure.

[0020] Some embodiments of the present disclosure may be represented by functional block configurations and various processing steps. Some or all of these functional blocks may be implemented by various numbers of hardware and / or software configurations that perform specific functions. For example, the functional blocks of the present disclosure may be implemented by one or more microprocessors or by circuit configurations for a specific function.

[0021] Additionally, for example, the functional blocks of the present disclosure may be implemented in various programming or scripting languages. The functional blocks may be implemented as algorithms executed on one or more processors. Furthermore, the present disclosure may employ prior art for electronic configuration, signal processing, and / or data processing, etc. Terms such as “mechanism,” “element,” “means,” and “configuration” may be used broadly and are not limited to mechanical and physical configurations. Additionally, terms such as “-part,” “-module,” etc. refer to a unit that processes at least one function or operation, which may be implemented in hardware or software, or as a combination of hardware and software.

[0022] Furthermore, the connecting lines or connecting members between the components depicted in the drawings are merely illustrative of functional connections and / or physical or circuit connections. In the actual device, connections between components may be represented by various alternative or added functional connections, physical connections, or circuit connections.

[0023] In addition, some components in the drawings may be depicted with their size or proportions slightly exaggerated. Also, components depicted in one drawing may not be depicted in another drawing.

[0024] The present disclosure will be described in detail below with reference to the attached drawings.

[0025] FIG. 1 is a drawing for illustrating an example of a system for counting the operation of an excavator according to one embodiment of the present disclosure.

[0026] Referring to FIG. 1, the system (1) may include an excavator (10), a camera (20), and a device (30). For example, the excavator (10), the camera (20), and the device (30) may be connected via wired or wireless communication to transmit and receive data to and from each other.

[0027] In the present disclosure, the size, shape, type, etc. of the excavator (10) is not limited. For example, the type of bucket, the connection relationship between the arm and the main body, etc., will not be limited to any one form.

[0028] In one embodiment, the excavator (10) may be a machine that performs work in a smart factory. The method of operation of the excavator (10) is not limited to unmanned or manned operation. Additionally, the smart factory may refer to an outdoor work site or an indoor work site.

[0029] The camera (20) can generate an image of the excavator.

[0030] The captured image may include an excavator, and may include the body and arm of a cylinder attached to the excavator. In other words, the angle, direction, etc., at which the captured image is taken will not be limited, and an image including the body and arm of the cylinder may be included in the captured image of the present disclosure.

[0031] In one embodiment, the camera (20) may refer to a camera installed in the smart factory as a component of the smart factory. For example, the camera (20) can automatically recognize an excavator, photograph the excavator, and transmit the captured image to an external device or an external server.

[0032] The device (30) may refer to a device that estimates the posture of the excavator (10) based on an image of the excavator (10) and uses this to count specific movements of the excavator (10). In the present disclosure, counting specific movements means calculating the cumulative number of movements of the excavator (10) whenever the excavator (10) performs a specific movement.

[0033] In one embodiment, the device (30) may be a device installed in a location adjacent to the excavator (10) and camera (20) inside a smart factory.

[0034] In another embodiment, the device (30) may be installed outside the smart factory and may be a device that transmits and receives data through communication with the excavator (10) and the camera (20).

[0035] Meanwhile, as an example, the device (30) may be a computing device that includes a display device, memory, and a processor. For example, the display device may be implemented as a touch screen to receive user input. Alternatively, the device (30) may further include a separate device (e.g., keyboard, mouse, etc.) for receiving user input. For example, the device (30) may be a smartphone, notebook PC, desktop PC, laptop, tablet computer, etc., but is not limited thereto.

[0036] As another example, the device (30) may mean a server. For example, the device (30) may be implemented in the form of a cloud server, but is not limited thereto.

[0037] Meanwhile, in FIG. 1, the camera (20) and the device (30) are depicted as being implemented as independent devices, but are not limited thereto. For example, the camera (20) may be included in the device (30). For example, the device (30) may include a shooting module that films an excavator in real time.

[0038] In the present disclosure, the device (30) may include an attitude estimation model used to estimate the attitude of an excavator (10). For example, code for implementing the attitude estimation model may be stored in the memory of the device (30).

[0039] In the present disclosure, a pose estimation model refers to an artificial intelligence model that performs at least one of the operations for estimating the pose of an object, such as joint position estimation, pose estimation, and motion recognition. The pose estimation model may include one or more artificial intelligence learning and inference models. Additionally, the artificial intelligence learning and inference models may include models commonly used in machine learning, models commonly used in deep learning, etc. For example, the pose estimation model may be a CNN (Convolutional Neural Networks)-based YOLO (You Only Look Once) model, but is not limited thereto.

[0040] Hereinafter, a method for counting the movements of the excavator (10) based on the excavator (10)'s posture estimated by the device (30) will be described in detail.

[0041] FIG. 2 is a flowchart for schematically explaining the operation of an excavator attitude estimation-based motion counting device according to one embodiment of the present disclosure.

[0042] FIG. 3 is a drawing for explaining an example of a captured image according to an embodiment of the present disclosure, FIG. 4 is a drawing for explaining an example of a plurality of important points detected according to an embodiment of the present disclosure, and FIG. 5 is a drawing for explaining an example of a region of interest according to an embodiment of the present disclosure. In addition, FIG. 6 is a drawing for explaining a method for counting specific operations of an excavator according to an embodiment of the present disclosure, and FIG. 7 is a drawing for explaining an example of visualization information being displayed to a user according to an embodiment of the present disclosure.

[0043] Referring to FIG. 2, first, the device (30) can acquire (210) an image of the excavator. For example, the device (30) can receive an image of the excavator taken in real time from the camera (20).

[0044] In one embodiment, the image may be an image including the body and arm of a cylinder attached to an excavator.

[0045] Referring to FIG. 3, the excavator (300) included in the image may be connected to a cylinder (310). In the present disclosure, the part connected to the main body of the excavator (300) may be defined as the body (320) of the cylinder (310), and the part connected to the body (320) may be defined as the arm (330) of the cylinder (310).

[0046] In one embodiment, the body (320) of the cylinder may have a fixed length. In other words, the length of the body (320) of the cylinder cannot be adjusted by user operation. In contrast, the arm (330) of the cylinder may have an adjustable length. In other words, depending on the operation of the excavator (300) by user operation, the arm (330) of the cylinder may be shortened and then lengthened back to its original length.

[0047] Referring again to FIG. 2, the device (30) can detect important points (220) in the image.

[0048] Here, the locations of the critical points targeted for detection can be determined in advance.

[0049] For example, referring to FIG. 4, the first important point (410) can be determined as any point on the body (401) of the cylinder attached to the excavator.

[0050] As described below, the first important point (410) is used to calculate the calibration ratio between the distance on the image and the actual distance, and its location can be determined as a point corresponding to a previously known actual distance. For example, the first important point (410) can be determined as a point where the body (401) of the cylinder is connected to the main body of the excavator.

[0051] Referring further to FIG. 4, the second important point (420) can be determined to be a point located at the connection point between the body (401) of the cylinder attached to the excavator and the arm (402) of the cylinder. In another sense, the second important point (420) may mean a reference point where the length of the arm (402) of the cylinder is adjusted.

[0052] In one embodiment, the third important point (430) may be determined to be a point located at the end of the arm (402) of the cylinder attached to the excavator.

[0053] Meanwhile, in one embodiment, the device (30) can detect a first important point, a second important point, and a third important point using a posture estimation model.

[0054] Here, the pose estimation model may be a skeleton-based pose estimation model that detects multiple important points and estimates the pose of the target object based on the connecting lines linking the detected important points.

[0055] Additionally, the attitude estimation model may be a model that has been pre-trained using an excavator image. The excavator image used to train the attitude estimation model may include a cylinder attached to the excavator and may be an image in which the locations of a first important point, a second important point, and a third important point are pre-marked.

[0056] In other words, a posture estimation model may refer to a model that receives an excavator image as input data and outputs data that identifies, detects, and tracks the first, second, and third critical points in the excavator image.

[0057] Referring again to FIG. 2, the device (30) can generate a region of interest (230) in an image.

[0058] Referring to FIG. 5, the region of interest (540) can be created based on the location of the first important point (510) and the location of the second important point (520).

[0059] More specifically, the device (30) can first correct the positions of the first important point (510), the second important point (520), and the third important point (530) included in the captured image through a linear regression process so that they are located on a single straight line. For convenience of explanation, the first important point (510), the second important point (520), and the third important point (530) shown in FIG. 5 are described as the important points whose positions have been corrected. As shown in FIG. 5, the important points whose positions have been corrected can be located on a single straight line.

[0060] Subsequently, the device (30) may use the positional relationship between the first important point (510) and the second important point (520) to calculate the calibration ratio between the distance on the image and the actual distance. Here, the distance on the image may mean a pixel distance, and the actual distance may mean a known distance (or length). For example, the actual distance may be the length of the body portion of the cylinder attached to the excavator. The device (30) may calculate the calibration ratio based on the distance on the image (511) between the first important point (510) and the second important point (520) and the actual distance (e.g., the length of the cylinder body). For example, the device (30) may calculate the actual distance (e.g., the actual length of the cylinder) per unit pixel distance.

[0061] Subsequently, the device (30) can calculate the size of the region of interest (540) based on the calculated calibration ratio. In the present disclosure, since the region of interest is set as an area associated with the operation performed by the excavator, the recognition range for the operation must be considered when calculating the size of the region of interest. In other words, the recognition range on the image can be calculated based on the calibration ratio and the recognition range for the operation, and the recognition range on the image can be calculated as the size of the region of interest.

[0062] In one embodiment, the operation performed by the excavator may be a operation to remove air contained in a cylinder attached to the excavator (air removal operation). For the air removal operation to be recognized as having been performed correctly once, the cylinder attached to the excavator must be compressed above a reference value and restored to a length approximately equal to its original length. That is, the recognized range for the air removal operation may include the operation of the cylinder arm until it has a length less than or equal to a first distance and the operation of the cylinder arm until it has a length greater than or equal to a second distance. The device (30) can calculate the minimum distance (521) and maximum distance (522) on the image based on the calibration ratio and the first distance and the second distance.

[0063] Afterwards, the device (30) can generate an area between the minimum distance (521) and the maximum distance (522) on the calculated image as a region of interest (540).

[0064] In one embodiment, the second important point (520) may be determined as a reference point for creating the region of interest (540). For example, the region of interest (540) may be the area between the second important point (520) as the origin and the circle with a radius of minimum distance (521), and the second important point (520) as the origin and the circle with a radius of maximum distance (522). Meanwhile, although the region of interest (540) in FIG. 5 is depicted as a sector-shaped area, the shape of the region of interest (540) is not limited thereto. The shape of the region of interest (540) may be various, such as a parallelogram, trapezoid, rectangle, semicircle, sector, etc., and any shape capable of displaying the area between minimum distance (521) and maximum distance (522) may be included in the region of interest (540) of the present disclosure.

[0065] Meanwhile, in the present disclosure, the device (30) can calculate a calibration ratio using some of the detected important points and dynamically generate a region of interest based on the calibration ratio. In other words, in the present disclosure, the region of interest may not be a region with a fixed size, but a region whose size is dynamically adjusted according to the angle and direction in which the image was taken, and the position, angle, and size of the excavator within the image.

[0066] Referring again to FIG. 2, the device (30) can track the location of a third important point and count the operation of the excavator (240).

[0067] Referring to FIG. 6, the device (30) can count the operation of the excavator based on the positional relationship between the area of ​​interest (640) and the third important point (630).

[0068] In one embodiment, when the operation of the excavator to be counted is an air bleeding operation, the device (30) can count that the air bleeding operation has been performed once in correspondence with the third important point (630) passing through the area of ​​interest (640) once in a round trip.

[0069] For example, when the arm of the cylinder is operated until it has a length less than or equal to the minimum distance of the region of interest (640), the third important point (630) may pass through the region of interest (640) once. In another sense, during the process of compressing the length of the arm of the cylinder, the third important point (630) may move from an area outside the region of interest (640) to an area inside the region of interest (640), and then move to an area between the region of interest (640) and the second important point.

[0070] Additionally, when the arm of the cylinder is operated until it has a distance greater than the maximum distance of the region of interest (640), the third important point (630) may pass through the region of interest (640) once. In another sense, during the process of the length of the arm of the cylinder being restored, the third important point (630) may move from the region between the region of interest (640) and the second important point to the region inside the region of interest (640), and then move to the region outside the region of interest (640).

[0071] Meanwhile, when the third important point (630) passes through the area of ​​interest (640) once, the direction in which the third important point (630) is moving is different from the direction in which the length of the cylinder arm is compressed and restored, so in order for the air removal operation to be counted once, the third important point (630) must pass through the area of ​​interest (640) once in a round trip.

[0072] Meanwhile, according to one embodiment, the device (30) can generate visualization information based on the counting result.

[0073] For example, referring to FIG. 7, the visualization information (730) may include the cumulative number of operations for the excavator's operation, the location of important points being detected in real time, and the region of interest being generated in real time.

[0074] The device (30) can display visualization information and provide it to the user (720).

[0075] For example, the device (30) may include a display module and may display visualization information to a user (720) using the device (30).

[0076] As another example, the excavator (710) may include a display module installed in the main body or the driver's seat. The device (30) transmits visualization information to the excavator (710) so that the visualization information can be displayed to the operator (720) who operates the excavator (710) through the display module installed in the excavator (710). The operator (720) can not only check the counting results in real time through the visualization information, but also determine whether the operation of the excavator is being performed correctly based on the area of ​​interest.

[0077] FIG. 8 is a flowchart illustrating an example of a motion counting method based on excavator attitude estimation according to one embodiment of the present disclosure.

[0078] The operations illustrated in FIG. 8 can be executed by the aforementioned device (30). Specifically, the operations illustrated in FIG. 8 can be executed by a processor included in the aforementioned device (30).

[0079] Referring to FIG. 8, first, in step 810, the device (30) can obtain an image of the excavator.

[0080] In one embodiment, the image may be captured from a camera installed in a smart factory.

[0081] In one embodiment, the image may be an image including the body and arm of a cylinder attached to the excavator.

[0082] Subsequently, in step 830, the device (30) can detect a predetermined first important point, a second important point, and a third important point in the image.

[0083] In one embodiment, the first important point may be any point located on the body of a cylinder attached to an excavator.

[0084] In one embodiment, the first important point may be a point used to calculate the calibration ratio between the distance on the image and the actual distance.

[0085] In one embodiment, the second important point may be a point located at the connection point between the body of the cylinder attached to the excavator and the arm of the cylinder.

[0086] In one embodiment, the second important point may be determined as a reference point for generating a region of interest.

[0087] In one embodiment, the third important point may be a point located at the end of the arm of the cylinder attached to the excavator.

[0088] In one embodiment, the device (30) can detect a first important point, a second important point, and a third important point using a skeleton-based attitude estimation model that has been learned using an excavator image including a cylinder.

[0089] Referring again to FIG. 8, in step 850, the device (30) can generate a region of interest based on the location of the first important point and the location of the second important point.

[0090] In one embodiment, the device (30) can correct the positions of the first important point, the second important point, and the third important point through a linear regression process so that the first important point, the second important point, and the third important point included in the image are located on a single straight line.

[0091] Additionally, the device (30) can calculate a calibration ratio based on the image distance between the corrected first important point and the corrected second important point and the actual distance between the first important point and the second important point.

[0092] Additionally, the device (30) can calculate the minimum and maximum distances on the image based on the calibration ratio and the recognition range for the operation.

[0093] Additionally, the device (30) can generate an area between the minimum distance and the maximum distance as a region of interest based on the corrected second important point.

[0094] Afterwards, in step 870, the device (30) tracks the location of the third important point and can count the operation of the excavator based on the positional relationship between the area of ​​interest and the third important point.

[0095] In one embodiment, the operation of the excavator may be the operation of removing air contained in a cylinder attached to the excavator. In one embodiment, the device (30) may count the operation of removing air as one time corresponding to the third important point passing through the area of ​​interest once in a round trip.

[0096] Meanwhile, in one embodiment, the device (30) can generate visualization information based on the counting result and display the visualization information.

[0097] FIG. 9 is a block diagram illustrating an example of the internal configuration of an excavator attitude estimation-based motion counting device according to one embodiment of the present disclosure.

[0098] Referring to FIG. 9, the operation counting device (hereinafter referred to as the 'device') (900) may include a processor (910), memory (920), an input / output interface (930), and a communication module (940). For convenience of explanation, FIG. 9 only illustrates components related to the present invention. Accordingly, other general-purpose components in addition to those illustrated in FIG. 9 may be included in the device (900). Furthermore, it is obvious to those skilled in the art that the processor (910), memory (920), input / output interface (930), and communication module (940) illustrated in FIG. 7 may be implemented as independent devices.

[0099] The processor (910) can process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. Here, the instructions may be provided from memory (920) or an external device. Additionally, the processor (910) can control the overall operation of other components included in the device (900).

[0100] The processor (910) can control at least some of the operations of the device (900) described above in FIGS. 1 to 8.

[0101] The processor (910) may be implemented as an array of multiple logic gates, or as a combination of a general-purpose microprocessor and memory storing a program that can be executed on the microprocessor. For example, the processor (910) may include a general-purpose processor, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a controller, a microcontroller, a state machine, etc. In some environments, the processor (910) may include an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a field programmable gate array (FPGA), etc. For example, the processor (910) may refer to a combination of processing devices such as a combination of a digital signal processor (DSP) and a microprocessor, a combination of multiple microprocessors, a combination of one or more microprocessors combined with a digital signal processor (DSP) core, or any other combination of such configurations.

[0102] The memory (920) may include any non-transient computer-readable recording medium. As an example, the memory (920) may include a permanent mass storage device such as a random access memory (RAM), read-only memory (ROM), disk drive, solid state drive (SSD), or flash memory. As another example, a permanent mass storage device such as a ROM, SSD, flash memory, or disk drive may be a separate permanent storage device distinct from the memory. Additionally, the memory (920) may store an operating system (OS) and at least one program code (e.g., code for the processor (910) to perform the operation described above with reference to FIGS. 1 through 8).

[0103] These software components may be loaded from a computer-readable recording medium separate from the memory (920). This separate computer-readable recording medium may be a recording medium that can be directly connected to the device (900) and may include, for example, a computer-readable recording medium such as a floppy drive, disk, tape, DVD / CD-ROM drive, memory card, etc. Alternatively, the software components may be loaded into the memory (920) via a communication module (940) that is not a computer-readable recording medium. For example, at least one program may be loaded into the memory (920) based on a computer program (e.g., a computer program for the processor (910) to perform the operation described with reference to FIGS. 1 through 8) which is installed by files provided through the communication module (940) by developers or a file distribution system that distributes installation files for the application.

[0104] The input / output interface (930) may be a means for interfacing with a device for input or output (e.g., keyboard, mouse, etc.) that may be connected to or included in the device (900). In FIG. 9, the input / output interface (930) is shown as an element configured separately from the processor (910), but is not limited thereto, and the input / output interface (930) may be configured to be included in the processor (910).

[0105] The communication module (940) may provide a configuration or function for the device (900) to communicate with an external device through a network. For example, control signals, commands, data, etc. provided under the control of the processor (910) may be transmitted to an external device via the communication module (940) and the network.

[0106] Unless explicitly stated or contrary to the order of the steps constituting the method according to the present invention, said steps may be performed in a suitable order. The present invention is not necessarily limited by the order in which said steps are described. The use of all examples or exemplary terms (e.g., etc.) in the present invention is merely for the purpose of describing the present invention in detail, and the scope of the present invention is not limited by said examples or exemplary terms unless limited by the claims. Furthermore, those skilled in the art will understand that various modifications, combinations, and changes may be made according to design conditions and factors within the scope of the claims or equivalents to which they are added.

[0107] Accordingly, the scope of the present invention should not be limited to the embodiments described above, and all scopes equivalent to or equivalently modified from the claims set forth below, as well as the claims set forth below, shall be considered to fall within the scope of the concept of the present invention.

Claims

Claim 1 A method for counting motion based on estimating the attitude of an excavator, comprising: acquiring an image of an excavator; detecting a predetermined first important point, a second important point, and a third important point in the image; generating a region of interest based on the location of the first important point and the location of the second important point; and tracking the location of the third important point and counting the motion of the excavator based on the positional relationship between the region of interest and the third important point; wherein the first important point is a point used to calculate a calibration ratio between a distance on the image and an actual distance. Claim 2 A method according to claim 1, wherein the image is captured from a camera installed in a Smart Factory. Claim 3 A method according to claim 1, wherein the image is an image including the body and arm of a cylinder attached to the excavator. Claim 4 A method according to claim 1, wherein the first important point is a point located on the body of a cylinder attached to the excavator. Claim 5 delete Claim 6 A method according to claim 1, wherein the second important point is a point located at the connection point between the body of the cylinder attached to the excavator and the arm of the cylinder. Claim 7 A method according to claim 1, wherein the second important point is determined as a reference point for generating the region of interest. Claim 8 A method according to claim 1, wherein the third important point is a point located at the end of the arm of the cylinder attached to the excavator. Claim 9 The method of claim 1, wherein the detecting step comprises the step of detecting the first important point, the second important point, and the third important point using a skeleton-based attitude estimation model learned using an excavator image including a cylinder. Claim 10 The method of claim 1, wherein the generating step comprises: a step of correcting the positions of the first important point, the second important point, and the third important point through a linear regression process so that the first important point, the second important point, and the third important point included in the image are located on a single straight line; a step of calculating a calibration ratio based on the distance on the image between the corrected first important point and the corrected second important point and the actual distance between the first important point and the second important point; a step of calculating a minimum distance and a maximum distance on the image based on the calibration ratio and the acceptance range for the operation; and a step of generating the area between the minimum distance and the maximum distance as the region of interest based on the corrected second important point. Claim 11 A method according to claim 1, wherein the operation of the excavator is a process of removing air contained in a cylinder attached to the excavator. Claim 12 In claim 11, the counting step counts the air removal operation as being performed once in correspondence with the third important point passing through the region of interest once in a round trip. Claim 13 A method according to claim 1, further comprising the step of generating visualization information based on a counting result and displaying the visualization information. Claim 14 A computer-readable recording medium storing a program for executing the method according to claim 1 on a computer. Claim 15 A motion counting device based on an excavator attitude estimation comprises: a memory storing at least one program; and a processor that operates by executing the at least one program, wherein the processor acquires an image of an excavator, detects a predetermined first important point, a second important point, and a third important point in the image, generates a region of interest based on the location of the first important point and the location of the second important point, tracks the location of the third important point, and counts a specific motion of the excavator based on the positional relationship between the region of interest and the third important point, wherein the first important point is a point used to calculate a calibration ratio between a distance on the image and an actual distance.

Images