Lifting analysis device, lifting analysis system, learning system, lifting analysis method, and learning method
The lift analysis system automates the collection and analysis of crane lifting data using an imaging unit and AI, addressing the inefficiencies and safety concerns of manual recording in conventional methods.
Patent Information
- Application Number
- JP2023074893
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-04-28
- Publication Date
- 2024-11-08
AI Technical Summary
The conventional method of managing crane lifting plans and recording lifting results is time-consuming and burdensome, requiring manual recording that can reduce safety and concentration among operators.
A lift analysis system that utilizes an imaging unit on the crane to capture images of the lifting process, combined with operation data, and an AI-based analysis to automatically collect and analyze lifting results, including load type, installation floor, and time, reducing the need for manual input.
Enables efficient and accurate collection of lifting results without manual recording, improving safety and reducing operator burden by automating the data collection and analysis process.
Smart Images

Figure 2024159112000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a lift analysis device, a lift analysis system, a learning system, a lift analysis method, and a learning method. [Background technology]
[0002] In the conventional method of managing the schedule and performance of crane lifting work, a crane lifting plan (planning the number of components to be installed per day and the order in which they will be installed) is made by comprehensively considering the structure and construction method of the building, site conditions, construction period, and the placement and number of cranes. When making a crane lifting plan, the installation time per piece of each component is estimated by referring to lifting data from similar sites. By estimating the installation time per piece of each component, it is possible to make a plan for how many pieces to lift per day. Once such a crane lifting plan is made and the site is actually in operation, the lifting results during operation are recorded by hand, and the plan is improved daily by comparing the recorded lifting results with the crane lifting plan. For example, the crane lifting plan may be revised by increasing the number of attached pieces or changing the setup of a time-consuming task. Patent Document 1 discloses that the usage schedule of a lifting device is registered in advance in a database, and the usage schedule of the lifting device is centrally managed. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2004-307132 A Summary of the Invention [Problem to be solved by the invention]
[0004] However, if the lifting results are to be utilized based on the contents of handwritten records, it takes time and effort to record them by hand and to compile the contents of the records. Furthermore, if an operator of a lifting device were to attempt to keep such handwritten records, he or she would have to record data in addition to performing lifting operations, which would increase the operator's burden and also pose an issue of ensuring safety due to a decrease in concentration.
[0005] The present invention has been made in consideration of the above circumstances, and its purpose is to provide a lifting analysis device, a lifting analysis system, a learning system, a lifting analysis method, and a learning method that can easily collect lifting performance data. [Means for solving the problem]
[0006] In order to solve the above-mentioned problems, one aspect of the present invention is to provide a method for manufacturing a semiconductor device comprising the steps of: The lifting analysis device has: a memory unit that stores a trained model that learns work content including at least one of the load type, installation floor, installation time, and work time in the work of transporting a load using the lifting device, using operation data of the lifting device, image data representing an image captured by an imaging unit that is attached to the lifting device and images an object to be lifted, and correct answer information including a correct answer for at least one of the load type, installation floor, installation time, and work time input by a checker who checks the work performed using the lifting device; an operation data acquisition unit that acquires the operation data of the lifting device, which is the subject of analysis; an image acquisition unit that acquires image data representing the image captured by the imaging unit that images the object to be lifted, which is the subject of analysis; and an analysis unit that obtains the work content as an analysis result by inputting the operation data that is the subject of analysis and the image data that is the subject of analysis into the trained model.
[0007] Moreover, one aspect of the present invention is a lifting analysis system having: a memory unit that stores a trained model that has learned work content including at least one of the load type, installation floor, installation time, and work time in the work of transporting a load using the lifting device, using operation data of the lifting device, image data representing an image captured by an imaging unit that is attached to the lifting device and images an object to be lifted, and correct answer information including a correct answer for at least one of the load type, installation floor, installation time, and work time input by a checker who checks the work performed using the lifting device; an operation data acquisition unit that acquires the operation data of the lifting device, which is the subject of analysis; an image acquisition unit that acquires image data representing an image captured by the imaging unit that images the object to be lifted, which is the subject of analysis; and an analysis unit that obtains the work content as an analysis result by inputting the operation data that is the subject of analysis and the image data that is the subject of analysis into the trained model.
[0008] Moreover, one aspect of the present invention is a learning system having a correct answer information acquisition unit that acquires correct answer information including a correct answer for at least one of the load type, installation floor, installation time, and work time input by a person who confirms work performed using a lifting device, and a learning unit that learns work content including at least one of the load type, installation floor, installation time, and work time in work of transporting a load using the lifting device, using at least one of the load information regarding a load estimated based on image data that is the subject of learning and the operation data of the subject of learning, and the correct answer information.
[0009] Moreover, one aspect of the present invention is a lifting analysis method in which an operation data acquisition unit acquires operation data of a lifting device, which is the subject of analysis, and an image acquisition unit acquires image data representing an image captured by an imaging unit that captures an image of an object to be lifted by the lifting device, which is the subject of analysis, and an analysis unit uses the operation data of the lifting device, image data representing an image captured by an imaging unit that captures an image of an object to be lifted by the lifting device, and correct answer information including correct answers for at least one of a load type, installation floor, installation time, and work time input by a checker who checks the work performed using the lifting device, to input the operation data to be analyzed and the image data to be analyzed into a learned model that has learned work content including at least one of a load type, installation floor, installation time, and work time in the work of carrying a load using the lifting device, thereby obtaining the work content as an analysis result.
[0010] Moreover, one aspect of the present invention is a learning method in which a correct answer information acquisition unit acquires correct answer information including a correct answer for at least one of the load type, installation floor, installation time, and work time input by a checker who checks work performed using a lifting device, and a learning unit learns work content including at least one of the load type, installation floor, installation time, and work time in the work of transporting a load using the lifting device using the correct answer information and at least one of the load information regarding the load estimated based on the image data that is the subject of learning and the operation data of the subject of learning. Effect of the Invention
[0011] As described above, according to the present invention, lifting records can be easily collected. [Brief description of the drawings]
[0012] [Figure 1] 1 is a schematic block diagram showing the configuration of a lifting management system S according to an embodiment of the present invention. [Diagram 2] 4 is a flow chart illustrating the operation of the lifting management system S in the learning phase. [Diagram 3] 4 is a flow chart illustrating the operation of the lifting management system S in the execution phase. [Figure 4] 13 is a diagram showing an example of a screen in which an image equivalent to the work content of the crane 100 viewed from above is displayed based on operation data. FIG. [Diagram 5] 13 is a diagram for explaining a flow of analyzing a lifting task and storing the analysis results in the management server 300. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0013] Hereinafter, a lifting management system S according to one embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a schematic block diagram showing the configuration of a lifting management system S according to one embodiment of the present invention. The lifting management system S includes a crane 100, a terminal device 200, and a management server 300. The crane 100 is equipped with an analysis device 150, which wirelessly communicates with the terminal device 200 and the management server 300. The crane 100 is a lifting device used at a construction site. The crane 100 may be another type of lifting device, such as a tower crane or a mobile crane.
[0014] "crane" The crane 100 includes a camera 101 , a movable part 102 , an actuator 103 , a control part 104 , a group of sensors 105 , and an analysis device 150 . The camera 101 is attached to the jib of the crane 100, and captures images of the area below the jib as its imaging range. The camera 101 continuously captures images of the area below the jib while work is being performed by operating the crane 100. The work of the crane 100 can be divided into several work sections, for example, there are four work sections: rigging, lifting, adjustment, and rigging removal. By capturing images with the camera 101 while each work section is being performed, it is possible to capture images of the object being lifted by the jib, the location where rigging is performed, the object being lifted, the location where rigging is performed, and the like. Slinging is the first task in a unit operation, and involves hanging the load to be transported in this unit operation on the hook of crane 100. Lifting is the task of using crane 100 to pull up the load that has been hung from the hook in the slinging task, and moving it to the designated position from which it is to be removed. Adjustment is the task of adjusting the position and orientation of the transported object so that the load that has been moved to the designated position in the lifting task is placed correctly. Removing the load is the task of removing the hook from the load that has been placed in the correct position in the adjustment task. The object is a suspended load. The suspended load may be a beam such as a main beam or a sub-beam, or a building material other than a beam.
[0015] The movable part 102 includes, for example, a rotating body, a lifting frame, a jib, a wire, a hook, and the like. The rotating body is attached to the upper end of the mast of the crane 100 and can rotate, for example, horizontally with the mast as the rotation axis. A driver's seat is provided on the rotating body. A jib is also attached to the rotating body. A hook block equipped with a hook is suspended by a wire from the tip of the jib. The actuator 103 includes, for example, a motor that moves the movable part 102 . The control unit 104 includes, for example, a PLC (Programmable Logic Controller) that controls the actuator 103 . The sensor group 105 is a plurality of sensors, and includes sensors corresponding to various measurement items, such as a sensor for detecting the lifting load, a sensor for detecting the jib (boom) length, a sensor for detecting the jib (boom) angle, a sensor for detecting approach, a sensor for detecting wind speed, and a sensor for detecting faults.
[0016] 《Analyzer》 Analysis device 150 includes image data acquisition unit 151, operation data acquisition unit 152, communication unit 153, storage unit 154, analysis unit 155, and control unit 156. Analysis device 150 is provided, for example, in the crane cab of crane 100 (inside the rotating body). The image data acquisition unit 151 acquires image data from the camera 101, which represents an image captured by the camera 101 capturing an image of an object to be lifted, which is the subject of analysis. The operation data acquiring unit 152 acquires operation data of the lifting device (crane 100) to be analyzed from the control unit 104. The operation data acquiring unit 152 collects the operation data, for example, every second.
[0017] Here, the operational data includes the load value of the suspended load, the crane operating radius, the rotation angle, the hook lifting height, etc. Furthermore, the operational data may be, for example, a control signal output from the control unit 104 to the actuator 103 or the like, an output signal from the sensor group 105, or the like. The operation data may also include, for example, position information of the movable part 102 such as hook coordinates, current and voltage values supplied to the actuator 103, wind speed detected by the sensor group 105, as well as alarms, fault information, lifting height, etc.
[0018] The communication unit 153 wirelessly communicates with the management server 300. The communication unit 153 communicates with the management server 300 via a mobile line or Wi-Fi (registered trademark). For example, the communication unit 153 receives a trained model transmitted from the management server 300 and writes it in the storage unit 154. In addition, the communication unit 153 transmits image data acquired by the image data acquisition unit 151 and operation data acquired by the operation data acquisition unit 152 to the management server 300.
[0019] The storage unit 154 stores various types of data. For example, the storage unit 154 stores a trained model generated by the management server 300. The trained model is a trained model that has learned the work content, including at least one of the load type, installation floor, installation time, and work time, in the work of transporting a load using a lifting device, using operation data of a lifting device (e.g., crane 100), image data representing an image captured by an imaging unit that is attached to the lifting device and captures an image of an object to be lifted, and correct answer information including correct answers for at least one of the load type, installation floor, installation time, and work time input by a checker who checks the work performed using the lifting device.
[0020] The analysis unit 155 inputs the operational data to be analyzed and the image data to be analyzed into the trained model, thereby obtaining the work content as the analysis result. In addition, the analysis unit 155 re-learns the period from the start of construction at the construction site being analyzed to the completion of construction of the specified floor, and then uses the re-learned trained model to analyze the construction of the remaining floors.
[0021] The control unit 156 controls each unit of the analysis device 150. In addition, the control unit 156 transmits the results of the analysis performed by the analysis unit 155, the image data, and the operation data to the management server 300 via the communication unit 153.
[0022] The analysis device 150 functions as an analysis platform for AI (Artificial Intelligence). The image data acquisition unit 151, operation data acquisition unit 152, communication unit 153, analysis unit 155, and control unit 156 of the analysis device 150 may be configured, for example, by a processing unit such as a CPU (Central Processing Unit) or a dedicated electronic circuit.
[0023] <<Terminal device 200>> The terminal device 200 includes a communication unit 201, a storage unit 202, an input unit 203, an output unit 204, and a control unit 205. The terminal device 200 is used, for example, by a checker who checks the status of work performed using a lifting device at a construction site. The communication unit 201 has a function of wirelessly communicating with the analytical device 150 and a function of wirelessly communicating with the management server 300 . The storage unit 202 stores various types of data.
[0024] The input unit 203 is an input device such as a touch panel, and receives operational input from a person at the construction site who checks the operation information of the crane 100. The contents of the operational input include, for example, the type of the load lifted by the crane 100 (which may be a main girder, a secondary girder, or the name of the material), the name of the suspended material, the installation floor number (installation floor), the installation time, the work time, and the like. The output unit 204 outputs various data. The output unit 204 is, for example, a liquid crystal display panel, and displays the various data. The control unit 205 controls each unit in the terminal device 200. The control unit 205 also acquires operational data from the analysis device 150 via the communication unit 201, and causes the output unit 204 to output (display) the acquired operational data.
[0025] The communication unit 153 and the control unit 205 of the terminal device 200 may be configured, for example, by a processing unit such as a CPU (Central Processing Unit) or a dedicated electronic circuit.
[0026] 《Management Server》 The management server 300 has a communication unit 301 , a memory unit 302 , an image data acquisition unit 303 , an operation data acquisition unit 304 , a learning unit 305 , a relearning unit 306 , a correct answer information acquisition unit 307 , and a continuous hanging information acquisition unit 308 . The communication unit 301 wirelessly communicates with the terminal device 200 and the analysis device 150 . The storage unit 302 stores various types of data. For example, the storage unit 302 stores a trained model generated by the training unit 305. The storage unit 302 also stores a trained model retrained by the re-training unit 306. Furthermore, the storage unit 302 receives the analysis results, image data, and operation data transmitted from the analysis device 150 via the communication unit 301, and stores them.
[0027] The image data acquisition unit 303 acquires the image data transmitted from the analysis device 150 via the communication unit 301 . The operational data acquisition unit 304 acquires the operational data transmitted from the analysis device 150 via the communication unit 301 .
[0028] The learning unit 305 uses the load information, operation data, and correct answer information related to the load estimated based on the image data to learn the work content including at least one of the load type, installation floor, installation time, and work time in the work of transporting a load using a lifting device. This learning is performed using AI functions. The learning unit 305 uses the multiple hoisting information acquired by the multiple hoisting information acquisition unit 308, the operation data, and the image data to learn the relationship between the load information regarding the load estimated based on the image data and the work content estimated based on the operation data.
[0029] The trained model generated by the learning unit 305 estimates work content when image data and operation data are input. The trained model stores information used to construct the model. For example, when the trained model is a model based on a DNN (Deep Neural Network), information used to construct the model is stored that indicates the number of units in each of the input layer, intermediate layer, and output layer, the number of layers in the intermediate layer, the coupling coefficients and bias values between units, and the activation function. In addition, the trained model is not limited to DNN, and any training model such as CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), or a combination of CNN and RNN, HMM (Hidden Markov Model), or SVM (Support Vector Machine) may be used.
[0030] When a construction site other than the construction site at which operation data, image data, and correct answer information were collected to generate the trained model is used as the analysis target, the re-learning unit 306 re-learns the trained model (the trained model generated by the learning unit 305) using the operation data, image data, and correct answer information collected at the construction site to be analyzed.
[0031] The correct answer information acquisition unit 307 acquires correct answer information including the correct answer for at least one of the load type, installation floor, installation time, and work time input by a person who confirms the work performed using the lifting device. When the load information includes information indicating that the load is a beam, the hoisting information acquisition unit 308 acquires hoisting information including the determination result for at least one of beam information indicating whether the beam is a main beam or a minor beam and hoisting information determining the number of hoists.
[0032] The communication unit 301, image data acquisition unit 303, operation data acquisition unit 304, learning unit 305, relearning unit 306, correct answer information acquisition unit 307, and continuous hanging information acquisition unit 308 of the management server 300 may be composed of a processing device such as a CPU (Central Processing Unit) or a dedicated electronic circuit.
[0033] Memory unit 154, memory unit 202, and memory unit 302 are configured with a storage medium, such as a hard disk drive (HDD), flash memory, electrically erasable programmable read only memory (EEPROM), random access read / write memory (RAM), read only memory (ROM), or any combination of these storage media. The storage unit 154, storage unit 202, and storage unit 302 may be implemented using, for example, a non-volatile memory.
[0034] Next, there will be described the operation of the lifting control system S. The operation of the lifting control system S is roughly divided into a learning phase and an execution phase. Learning Phase FIG. 2 is a flow chart illustrating the operation of the lifting control system S in the learning phase. When the crane 100 is operating at the construction site, the camera 101 captures an image of the area below the jib at a predetermined frame rate and sequentially outputs the captured image data (step S11). The frame rate may be any value, but is preferably an interval at which multiple frames are captured for one work section of the crane 100. For example, the frame rate is an interval at which multiple frames can be captured for a slinging operation, or multiple frames can be captured while the jib is rotating with a load suspended. The image data acquisition unit 151 acquires the image data sequentially output from the camera 101.
[0035] When the crane 100 is operated, the PLC of the control unit 156 generates and outputs operation data (step S12). The operation data acquisition unit 152 acquires the operation data output from the PLC of the control unit 156.
[0036] At a stage where a trained model has not been generated, in the analysis device 150, the analysis unit 155 does not perform analysis, and the control unit 156 transmits image data and operational data to the management server 300 via the communication unit 153 (step S13). The image data and the operation data are transmitted from the communication unit 201 at each predetermined timing. The predetermined timing may be at regular intervals or at the timing when the image data and the operation data are generated.
[0037] On the other hand, when operation data is generated in the analysis device 150, the terminal device 200 receives the generated operation data and displays it on the screen by the output unit 204. A checker using the terminal device 200 checks the actual operation status of the crane 100 and also checks the operation data displayed on the screen, and inputs correct answer information indicating a correct answer for at least one of the items of the load type, installation floor, installation time, and work time for the operation data from the input unit 203. The load type is input by the checker through the input unit 203 after visually checking the load actually suspended by the crane 100.
[0038] The installation floor is determined by the inspector visually checking which floor of the building being constructed the load actually suspended by the crane 100 is transported to and removed from, and the floor number to which the removed load (e.g. a beam) is attached, and this is input by the inspector via the input unit 203. The attachment time is the time when the load actually suspended by the crane 100 is attached, which is visually confirmed by the inspector, and is input by the inspector via the input unit 203. This time may be, for example, set on the screen of the terminal device 200 as a button that reads "input current time as attachment time," and the time when the inspector taps this button may be input as the attachment time, or the attachment time may be input by tapping numbers on the touch panel in the time input field.
[0039] The work time is calculated by measuring the elapsed time from the time when a checker who has confirmed that a certain sorting work has started presses the "Start work" button when the sorting work starts, provided on the touch panel of the terminal device 200, to the time when a checker who has confirmed that a certain sorting work has ended presses the "End work" button when the sorting work ends. The work time for each sorting work is specified by receiving an input as to which sorting work the calculated work time should be set as. Although the task time will be described as being measured for each divided task, the task time may also be measured for tasks that are defined by a division other than the divided tasks.
[0040] The terminal device 200 transmits the correct answer information input from the input unit 203 and the operational data to which the correct answer information has been added to the management server 300 (step S14). Here, the checker checks the operation data received in sequence on the screen while continuously checking the work status continuously performed by the crane 100, checks the displayed operation data and the work content being performed at the actual construction site, and then inputs the check results as correct answer information in sequence. As a result, data in which the operation data and correct answer information are associated with each other is transmitted in sequence from the terminal device 200 to the management server 300.
[0041] The learning unit 305 receives image data and operation data from the analysis device 150, and receives correct answer information together with the operation data from the terminal device 200. Based on the operation data received from the analysis device 150, the learning unit 305 identifies correct answer information for the operation data that corresponds to the operation data received from the analysis device 150, and learns the identified correct answer information, image data, and operation data as teacher data.
[0042] Such learning may be performed at a stage where image data and operational data obtained from the analysis device 150 and correct answer information and operational data obtained from the terminal device 200 have been accumulated for a certain period of time. Then, a trained model can be obtained by performing learning. Such trained models may be stored in different trained models according to the scale of the site or the type of building to be constructed, or different trained models may be stored for each model of the crane 100. This allows multiple types of trained models to be prepared according to the characteristics of the construction site, and a trained model close to the characteristics of the construction site to be analyzed in the future can be selected and applied to the analysis of lifting at the construction site. This allows for more accurate analysis.
[0043] Once the trained model is generated, the management server 300 transmits the trained model to the analysis device 150 (step S15). The analysis device 150 stores the trained model obtained from the management server 300 in the memory unit 154.
[0044] When the analysis device 150 acquires the trained model from the management server 300, the analysis device 150 stores the acquired trained model in the memory unit 154.
[0045] Here, we have described the case of generating a new trained model, but it is also possible to retrain an already trained model. Re-learning may be performed, for example, when a trained model generated based on data collected at a construction site different from the current analysis target construction site is used at the current analysis target construction site, after a certain period of time has passed since the start of use. For example, the external shape of a suspended load may differ slightly from construction site to construction site, so adjustments may be required when using a trained model generated at another construction site at the next construction site. An example of a certain period of time is the period from the start of construction to the construction of the lower floors. Re-learning is performed during construction of the lower floors, and during construction of the remaining floors (middle to higher floors), the analysis unit 155 uses the trained model obtained by re-learning to perform analysis. This improves the proficiency of the trained model. Furthermore, when the proficiency of the trained model reaches a certain level, the trained model may be used for analysis from the beginning of construction at another construction site without re-learning.
[0046] When re-learning, the analysis unit 155 inputs image data and operation data acquired in response to the operation of the crane 100 into the learned model, and obtains analysis results such as the type of suspended load, installation floor, installation time, and work time. The control unit 156 transmits the analysis results, image data, and operation data to the management server 300 (step S21). Meanwhile, during the relearning period, the checker checks the actual operating status and operation data of the crane 100 and inputs correct answer information. The terminal device 200 transmits the input correct answer information and operation data to the management server 300 (step S22).
[0047] The re-learning unit 306 of the management server 300 compares the analysis results obtained from the analysis unit 155 with the correct answer information obtained from the terminal device 200 for the trained model that is the same as the trained model used in the analysis unit 155 among the trained models stored in the storage unit 302, and updates the coefficients, etc. of the trained model based on the comparison result so that the analysis results approach the correct answer information. During the re-learning period, the trained model is updated and the re-learning progresses by sequentially acquiring the analysis results and the correct answer information and performing re-learning.
[0048] The management server 300 performs the above-mentioned re-learning during the construction period of the lower floors, and transmits the re-learned trained model to the analysis device 150 (step S23) when construction of the middle floors begins, and ends the re-learning. The analysis device 150 stores the re-learned trained model in the storage unit 154. The analysis unit 155 performs analysis using the re-learned trained model.
[0049] The analysis unit 155 inputs the image data acquired by the image data acquisition unit 151 and the operation data acquired by the operation data acquisition unit 304 into a learned model, thereby obtaining data representing the work content and analyzing the work content. The control unit 156 transmits the results of the analysis performed by the analysis unit 155, and the image data and operational data used in the analysis, to the management server 300 via the communication unit 201 (step S13).
[0050] Execution Phase FIG. 3 is a flow chart illustrating the operation of the lifting management system S in the execution phase. In the execution phase, the analysis device 150 uses the trained model received from the management server 300 to estimate the work content of the crane 100 at the construction site. The analysis unit 155 acquires image data captured by the camera 101 while the crane 100 is in operation via the image data acquisition unit 151 (step S51), acquires operation data generated while the crane 100 is in operation via the operation data acquisition unit 152 (step S52), inputs the image data and operation data into the trained model, and obtains analysis results such as the type of suspended load, installation floor, installation time, and work time. The control unit 156 transmits the analysis results, image data, and operation data to the management server 300 (step S53). In this execution phase, there is no need to input correct answer information from the terminal device 200 .
[0051] While the crane 100 is operating, the above-mentioned steps S51 to S53 are repeated, and the work content obtained as the analysis result is transmitted to the management server 300. As a result, the estimated work content is accumulated in the memory unit 302 of the management server 300.
[0052] Next, the operation of the lifting control system S in the execution phase will be further described. In the above-described embodiment, it is possible to distinguish the suspended load and estimate the work situation by using not only the image obtained from the camera 101 but also the image data and the operation data in combination. FIG. 4 is a diagram showing an example of a screen in which an image equivalent to the work content of the crane 100 viewed from above is displayed based on the operation data. In this figure, the position of the mast of the crane 100 is represented as Pc. The rotation angle of the crane 100's rotating body can be detected based on the detection results of the sensor group 105 for the start and end positions of the rotation of the rotating body. That is, by grasping the rotation angle of the rotating body, the rotation angle of the jib in the rotation direction can be grasped. Also, the number of rotations of the motor that winds and unwinds the wire is obtained from the sensor group 105, and the angle of the jib is obtained from the sensor group 105, and the position of the jib in the height direction can be estimated from a combination of these. As a result, when the rotating body (jib) faces the material storage area Rs, which is an area where materials are stored, the jib is lowered, and a certain time has passed, it can be estimated that slinging is being performed. Also, by associating the identification information of the material storage area Rs with the material name and storing it in advance in the memory unit 302 of the management server 300 with respect to what materials are stored in the material storage area Rs, it is possible to narrow down to a certain degree what materials have been slinged from among a large number of materials (beams and other than beams, etc.).
[0053] In addition, the attachment position can be estimated by estimating the position where the ball was removed (for example, position Pd). Here, when a load is lifted, it may be hoisted in series. When at least one of the slinging positions and the release positions overlap, it can be assumed that the load is hoisted in series. It is also possible to infer whether the load is hoisted in series or not based on the weight of the load. When the weight of a load hoisted from one material storage area Rs is greater than the weight of the greatest material stored in the material storage area Rs, it can be assumed that the load is hoisted in series. In this way, it is possible to narrow down the materials and estimate the work category by combining the measurement results obtained from the sensor group 105 and the operation data such as the elapsed time. In addition, after narrowing down the materials, the materials can be further narrowed down from the image data of the camera 101.
[0054] When identifying suspended loads using only image data, it is difficult to distinguish between parts that have similar appearances. Also, when identifying suspended loads or estimating work conditions using only operation data, the weight of the suspended load, its attachment position (position on the plane and position in the height direction in three-dimensional space), whether it is a continuous suspension, etc. can be ascertained by referring to the operation data, but since no image is used, it is not possible to estimate the suspended load based on its color or shape, etc., and therefore the accuracy may not necessarily be as good as an analysis based on image data. However, by using both image data and operation data in combination, it is possible to improve the accuracy of identifying the suspended load, and also to improve the accuracy of estimating the work situation.
[0055] For example, even if it is possible to determine from image data that the suspended load is a beam, it may be difficult to distinguish whether it is a main girder or a sub-girder. After estimating that it is a beam based on image data, it is possible to estimate whether it is a main girder or a sub-girder based on weight and installation coordinates based on operation data. For example, if the position where the ball is removed is the position where the main girder is to be installed, it can be estimated that it is a main girder, and if the position where the ball is removed is the position where the sub-girder is to be installed, it can be estimated that it is a sub-girder. In addition, while it is difficult to determine from image data where a suspended load is to be attached, the attachment position can be estimated by using attachment coordinates based on operational data. In addition, when multiple suspended loads overlap in the height direction when viewed from above, it is difficult to identify the number of suspended pieces from image data; however, the number of suspended pieces can be estimated by referring to the weight of the load based on the operation data.
[0056] Next, the operation of the lifting control system S in the execution phase will be further described. 5 is a diagram explaining the flow of analyzing one lifting task using image data and operation data, and storing the analysis results in the management server 300. Here, one lifting task is one cycle of rigging, lifting, adjustment, and removing the rigging. Such a cycle is executed every time a lift is performed. The analysis unit 155 acquires image data from the camera 101 via the image data acquisition unit 151 (step S101), and acquires operation data from the sensor group 105 and the like via the operation data acquisition unit 152 (step S201). The analysis unit 155 inputs the operation data into the trained model to determine whether or not a hanging jig is present (step S202). If it is determined that a hanging jig is present, the weight measurement result obtained from the weight sensor of the sensor group 105 is stored in the storage unit 154 as the weight of the hanging jig (step S301).
[0057] Next, the analysis unit 155 determines the operation time during which the work of operating the crane is performed (step S203). That is, the analysis unit 155 identifies data corresponding to each of the classification work of slinging, lifting, adjustment, and lifting off from the operation data acquired by the operation data acquisition unit 152 using the learned model, thereby identifying data corresponding to one lifting task (one cycle) using the crane (step S302). For example, the analysis unit 155 identifies, from the operation data, a slinging start time which is the time when slinging is started, a slinging end time which is the time when slinging is ended, a lifting start time which is the time when lifting is started, a lifting end time which is the time when lifting is ended, an adjustment start time which is the time when adjustment is started, an adjustment end time which is the time when adjustment is ended, a lifting start time which is the time when lifting is started, and a lifting end time which is the time when lifting is ended. Furthermore, the analysis unit 155 uses the camera image acquired by the image data acquisition unit 151 to acquire the XY coordinates of the attachment position of the load as an analysis result from the trained model (step S303).
[0058] Next, the analysis unit 155 extracts image data at the time determined to be lifting (image data captured during the period determined to be lifting) (step S102). Here, since the lifting period lasts for a certain amount of time, multiple pieces of image data are also acquired. The analysis unit 155 inputs each of the extracted multiple pieces of image data into the trained model, and determines the type of the suspended load from the image included in the image data (step S103). Types of suspended loads include, for example, pillars, beams (main beams and minor beams), temporary structures, floors, frameworks, exteriors, interiors, equipment, and others.
[0059] The analysis unit 155 determines the type of the lifted load by taking a majority vote based on the respective determination results of the multiple image data (step S104). If the result of the majority vote indicates that the load is a type other than a beam, such as a pillar, a structure, or a floor (step S105), the analysis unit 155 stores in the storage unit 154 the type of the load based on the determination result (pillar, temporary structure, floor, structure, exterior, interior, equipment, other, etc.) (step S307). On the other hand, when the analysis unit 155 determines that the load is a beam as a result of the majority vote (step S106), it re-determines whether the load is a girder or a sub-beam by inputting the operation data into the trained model (step S204). The analysis unit 155 determines whether the load is a girder or a sub-beam based on the operation data, and stores the determination result of either the girder or the sub-beam as the type of suspended load in the storage unit 154 (step S307). Here, even if it is not possible to detect that it is a beam from the image, it may be possible to detect whether it is a main girder or a sub-beam from the operation data. For example, image data may be captured in a state where the appearance of the suspended load is difficult to see, such as when the surroundings are dark, there is reflection due to the influence of external light, or it is difficult to distinguish the suspended load from the background. Even in such a case, it is possible to determine whether it is a main girder or a sub-beam from the slinging position, the slinging removal position, etc.
[0060] Furthermore, when it is determined in step S104 that the beam is a beam, the analysis unit 155 re-determines whether the beam is a main girder or a sub-girder by inputting the operation data corresponding to the lifting to the trained model (step S204).Then, the analysis unit 155 stores the suspended load type indicating whether the beam is a main girder or a sub-girder in the storage unit 154 (step S307).
[0061] Next, the analysis unit 155 acquires the result of determining whether the load is a multiple-suspension load or not, which is obtained by inputting the operation data into the trained model, and, if the load is a multiple-suspension load, the count result of the number of multiple-suspension loads (step S205). The analysis unit 155 then stores the result of determining whether the load is a multiple-suspension load or not, and, if the load is a multiple-suspension load, the number of multiple-suspension loads as the number of components in the storage unit 154 (step S308). Here, the weight of the load is also stored in the storage unit 154 together with the number of components (step S309). Next, the analysis unit 155 acquires the result of determining the installation floor, which is obtained by inputting the operational data into the trained model, and stores in the memory unit 154 the vertical coordinate (Z coordinate value) representing the installation floor (step S310) and the installation floor (step S311). In this manner, one lifting task can be stored in the memory unit 154. When the analysis result of one lifting task is obtained, the control unit 156 transmits the analysis result together with the image data and operation data to the management server 300 via the communication unit 153. As a result, the analysis result, image data, and operation data for one lifting task are stored in the management server 300 in association with each other. Then, every time the next task is executed, the above-mentioned process is repeated. In this way, the working status of the crane 100 can be analyzed and stored in the memory unit 302.
[0062] According to the embodiment described above, image data and operation data are input into a trained model to analyze the crane's work status, making it possible to easily collect detailed information about the crane's work content that would be difficult to collect by hand. Furthermore, when records are handwritten, the records must be accurate, but the accuracy depends on the operator, and if a button is forgotten, the reliability of the recorded contents decreases. However, according to the above-described embodiment, the operator does not need to input the lifting performance, so it is possible to avoid a decrease in the reliability of the recorded contents due to operator operational errors, etc. It is also possible to automate and improve the collection of performance data during crane lifting operations.
[0063] Furthermore, according to the embodiment described above, by attaching a camera 101 and a group of sensors 105, etc. to the crane 100, camera images and operation data can be acquired, and the AI of the analysis device 150 can automatically analyze the type of load, installation floor, installation time, work time, etc. Furthermore, according to the above-described embodiment, the processes from data acquisition to AI analysis are performed on the analysis device 150 side, not on the management server 300 side, so there is no data loss due to communication problems, compared to the case where image data and operation data are sent from the crane 100 side to the management server 300 and AI analysis is performed on the management server 300. For example, it is possible to avoid a case where AI analysis cannot be performed because image data does not reach the management server 300 due to communication problems, or a case where AI analysis cannot be performed because operation data does not reach the management server 300.
[0064] According to the embodiment described above, in the learning process, in the early stages of construction, the learned model can be trained while correcting the analysis results by the analysis unit 155 using corrective information based on the results of visual confirmation by a checker. As the learning progresses, it becomes possible to perform automatic estimation without inputting corrective information.
[0065] Furthermore, according to the above-described embodiment, the image data and operational data used in the learning phase are stored in the management server 300, so that they can be reused in the future after AI technology has developed.
[0066] In the above-described embodiment, the analysis device 150 and the management server 300 are configured as a single device by containing the respective functions in a single housing. However, at least one of the functions may be provided in a device in a different housing, and the devices may communicate with each other to configure the devices as a single system (computer system).
[0067] The analysis device 150 or the management server 300 in the above-described embodiment may be realized by a computer. In that case, a program for realizing this function may be recorded in a computer-readable recording medium, and the program recorded in the recording medium may be read into a computer system and executed to realize the function. The term "computer system" as used herein includes hardware such as an OS and peripheral devices. The term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into a computer system. The term "computer-readable recording medium" may also include a medium that dynamically holds a program for a short period of time, such as a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line, and a medium that holds a program for a certain period of time, such as a volatile memory inside a computer system that is a server or client in that case. The above-described program may be a program for realizing a part of the above-described function, or may be a program that can realize the above-described function in combination with a program already recorded in the computer system, or may be a program that is realized using a programmable logic device such as an FPGA (Field Programmable Gate Array).
[0068] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and designs that do not deviate from the gist of the present invention are also included.
[0069] The "Sustainable Development Goals (SDGs)" are 17 international goals that were adopted at the United Nations Summit in September 2015. The autonomous mobile control system according to this embodiment can contribute to the achievement of the 17 SDGs, for example, goal 9 "Build resilient infrastructure, promote industry, innovation and infrastructure." [Explanation of symbols]
[0070] 100...crane, 101...camera, 102...moving part, 103...actuator, 104...control part, 105...sensor group, 150...analysis device, 151...image data acquisition part, 152...operation data acquisition part, 153...communication part, 154...storage part, 155...analysis part, 156...control part, 200...terminal device, 201...communication part, 202...storage part, 203...input part, 204...output part, 205...control part, 300...management server, 301...communication part, 302...storage part, 303...image data acquisition part, 304...operation data acquisition part, 305...learning part, 306...relearning part, 307...correct answer information acquisition part, 308...multiple hoisting information acquisition part, S...lifting management system
Claims
1. a storage unit that stores a learned model that has learned work content including at least one of a load type, an installation floor, an installation time, and a work time in a work of transporting a load using the lifting device, using operation data of the lifting device, image data representing an image captured by an imaging unit that is attached to the lifting device and captures an image of an object to be lifted, and correct answer information including a correct answer for at least one of a load type, an installation floor, an installation time, and a work time input by a checker who checks the work performed using the lifting device; An operation data acquisition unit that acquires operation data of the lifting device to be analyzed; an image acquisition unit that acquires image data representing an image captured by an imaging unit that captures an image of the lifted object, which is an analysis target; An analysis unit that obtains the work content as an analysis result by inputting the operation data to be analyzed and the image data to be analyzed into the trained model; A lifting analysis device having the above structure.
2. A correct answer information acquisition unit acquires correct answer information including a correct answer for at least one of a load type, an installation floor, an installation time, and a work time input by a person who confirms the work performed using the lifting device, The operation data acquisition unit acquires operation data to be learned, The image acquisition unit acquires image data captured by the imaging unit and which is a learning target, The lifting analysis device is A learning unit that learns work content including at least one of a load type, an installation floor, an installation time, and a work time in a work of transporting a load using the lifting device, using load information regarding a load estimated based on the image data that is the object of learning, the operation data of the object of learning, and the correct answer information, The storage unit stores a trained model trained by the training unit. The lifting analyzer of claim 1 .
3. When the information indicating that the load is a beam is included in the load information, the beam information indicates whether the beam is a main beam or a sub-beam, and the continuous suspension information includes at least one of the judgment results determined for the number of continuous suspensions. The learning unit is Using the continuous hoisting information acquired by the continuous hoisting information acquisition unit, the operation data, and the image data, a relationship between the load information regarding the load estimated based on the image data and the work content estimated based on the operation data is learned. The lifting analyzer of claim 2.
4. A re-learning unit that re-learns the trained model using the operation data, the image data, and the correct answer information collected at the construction site to be analyzed when a construction site different from the construction site at which the operation data, the image data, and the correct answer information are collected is used as an analysis target to generate the trained model, The analysis unit performs the re-learning for the period from the start of construction to the completion of the construction of a predetermined floor at the construction site to be analyzed, and then performs analysis using the re-learned trained model during the construction of the remaining floors. The lifting analyzer of claim 3.
5. a storage unit that stores a learned model that has learned work content including at least one of a load type, an installation floor, an installation time, and a work time in a work of transporting a load using the lifting device, using operation data of the lifting device, image data representing an image captured by an imaging unit that is attached to the lifting device and captures an image of an object to be lifted, and correct answer information including a correct answer for at least one of a load type, an installation floor, an installation time, and a work time input by a checker who checks the work performed using the lifting device; An operation data acquisition unit that acquires operation data of the lifting device to be analyzed; an image acquisition unit that acquires image data representing an image captured by an imaging unit that captures an image of the lifted object, which is an analysis target; An analysis unit that obtains the work content as an analysis result by inputting the operation data to be analyzed and the image data to be analyzed into the trained model; A lifting analysis system having:
6. a correct answer information acquisition unit that acquires correct answer information including a correct answer for at least one of a load type, an installation floor, an installation time, and a work time input by a checker who checks the work performed using the lifting device; a learning unit that learns work details including at least one of a load type, an installation floor, an installation time, and a work time in a work of transporting a load using the lifting device, using at least one of load information regarding a load estimated based on image data that is a learning target and operation data of the learning target, and the correct answer information; A learning system having the above structure.
7. The operation data acquisition unit acquires operation data of the lifting device to be analyzed, An image acquisition unit acquires image data representing an image captured by an imaging unit that captures an image of an object to be analyzed that is lifted by the lifting device, The analysis unit obtains the work content as an analysis result by inputting the operation data to be analyzed and the image data to be analyzed into a trained model that has learned work content including at least one of the load type, the installation floor, the installation time, and the work time in the work of transporting a load using the lifting device, using operation data of the lifting device, image data representing an image captured by an imaging unit that captures an image of an object to be lifted that is attached to the lifting device, and correct answer information including a correct answer for at least one of the load type, the installation floor, the installation time, and the work time input by a checker who checks the work performed using the lifting device. Lift analysis method.
8. The correct answer information acquisition unit acquires correct answer information including a correct answer for at least one of a load type, an installation floor, an installation time, and a work time input by a checker who checks the work performed using the lifting device, and A learning unit learns work details including at least one of a load type, an installation floor, an installation time, and a work time in a work of transporting a load using the lifting device, using at least one of load information regarding a load estimated based on image data that is a learning target and operation data of the learning target, and the correct answer information. How to learn.
Citation Information
Patent Citations
System and method for management of material lifting
JP2004307132A
Cited By
Attachment status acquisition system
WO2025263096A1
