Work determination device and its program, as well as work content recording system and work content recording method.

Abstract

To provide a work determination device that can determine whether the intended task is being performed based on sound. [Solution] The work determination device 2 is an object position presentation system comprising: a sound collection processing unit 21 that collects sound as sound data; a determination processing unit 22 that estimates the work content from the sound data collected by the sound collection processing unit 21 using a trained model that estimates the work content from the sound data and determines whether or not the work content exists; and an output unit (communication unit 24) that outputs the determination result of the determination processing unit 22.

Description

Technical Field

[0001] The present invention relates to a work determination device and a program thereof, as well as a work content recording system and a work content recording method.

Background Art

[0002] Tunnel construction completes one cycle (tunnel cycle time) by performing drilling, charging, blasting, mucking, trimming, primary spraying, support erection, secondary spraying, scaling, and rock bolting. Conventionally, various methods have been disclosed to manage the progress of this tunnel cycle time. Patent Document 1 discloses a method for determining the operating status of machines in a tunnel by associating the ON / OFF state of the power supply of a machine with the work performed by the machine. Patent Document 2 discloses a method for determining the work process based on the average value or statistical variation (for example, standard deviation) of the current supplied to heavy tunnel construction machinery from the outside. Patent Document 3 discloses a method for determining work content using a machine learning model from images of construction machines working inside a tunnel.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0004] Even if a machine is powered on, it may not be performing any construction work. Therefore, the methods described in Patent Documents 1 and 2 may not be able to accurately manage the work process of construction machinery. The method described in Patent Document 3 has a problem in that it requires the construction of a communication environment in order to stably transmit and receive large amounts of video data from a camera terminal that photographs construction machinery to a terminal that determines the work content, resulting in high implementation costs.

[0005] This invention has been made in view of the above problems, and aims to provide a work determination device and its program that can determine whether a target work is being performed by sound, as well as a work content recording system and work content recording method that can record work content more accurately. [Means for solving the problem]

[0006] To solve the aforementioned problems, the present invention provides a work determination device for determining predetermined work content by sound, comprising a sound collection processing unit and a determination processing unit. The sound collection processing unit collects sounds from the site as sound data. The determination processing unit uses a trained model that estimates the work content from the sound data to estimate the work content from the sound data collected by the sound collection processing unit and determines whether or not the work content exists. This trained model can be created, for example, by a method based on an AI model creation method.

[0007] With this configuration, the present invention uses a trained model to determine the work content by sound, making it possible to determine the work content more accurately from the work sounds at the site.

[0008] In the work determination device, the trained model is a model that has learned work content corresponding to sound data for predetermined unit time lengths, the sound collection processing unit comprises a sound collection unit and a division unit, and the determination processing unit may comprise a unit time work estimation unit, a recording unit and a work content determination unit. The sound collection unit collects sound data. The division unit divides the sound data collected by the sound collection unit for each unit time length. The unit time work estimation unit uses the trained model to estimate the work content for each unit time length from the sound data divided by the division unit. The recording unit records the work content estimated by the unit time work estimation unit as a log. The work content determination unit determines the work content for each predetermined time length based on the proportion of sound data estimated as predetermined work content in the log for the predetermined time length each time a log of a predetermined time length is recorded.

[0009] With this configuration, the present invention divides sound data into predetermined unit time lengths, for example, every second, and determines the work content. This allows the present invention to reduce the computational load using a trained model. Furthermore, the present invention determines the work content for each predetermined time length from the proportion of sound data estimated to be predetermined work content in a log of a predetermined time length. This allows the present invention to reduce misjudgment of work content caused by dividing the data into short intervals of seconds.

[0010] Furthermore, the present invention can also be operated by a program that makes the computer function as a work determination device by utilizing the hardware resources of the computer, such as the CPU, memory, and hard disk.

[0011] Furthermore, the present invention relates to a work content recording system for recording work content using construction machinery, comprising: a transmitter; a receiving device; a work determination device; and a processing device communicateably connected to the receiving device and the work determination device. The transmitter is attached to each piece of construction machinery and emits radio waves. The receiving device receives the radio waves emitted by the transmitter. The work determination device determines the work content from the sound. The processing device comprises: an analysis unit; a determination recording unit; and a supplementation unit. The analysis unit analyzes the radio waves received from the receiving device and identifies the construction machinery in operation. The determination recording unit determines and records the current work content based on the construction machinery and cycle information indicating the work procedure of the construction work identified by the analysis unit. The supplementation unit supplements the work content recorded by the determination recording unit based on the determination result output from the work determination device.

[0012] With this configuration, the present invention makes it possible to determine whether heavy machinery is actually operating and performing work by supplementing the work content. This makes it possible to obtain information such as whether the heavy machinery is actually on standby and the process is not progressing, or whether multiple processes are being carried out simultaneously, which helps in more accurate estimation of work time.

[0013] Furthermore, the present invention relates to a work content recording method for recording work content using construction machinery, and includes a receiving step, an analysis step, a work determination step, a determination recording step, and a supplementation step. In the receiving step, a receiving device receives radio waves from a transmitter attached to the construction machinery. In the analysis step, a processing device, which is communicably connected to the receiving device and the work determination device, analyzes the radio waves received in the receiving step and identifies the construction machinery in operation. In the work determination step, the work determination device determines the work content from the sound. In the determination recording step, the processing device determines and records the current work content based on cycle information indicating the work procedure of the construction machinery and construction work identified in the analysis step. In the supplementation step, the processing device supplements the work content recorded in the determination recording step with the determination result output from the work determination device.

[0014] According to such a procedure, the present invention can accurately determine and record the work content from the construction machine identified by the transmitter and the work content determined by the work determination device based on sound. [Effects of the Invention]

[0015] According to the present invention, it is possible to accurately determine the work content using a model that has learned the sound of the work content in advance. [Brief Description of the Drawings]

[0016] [Figure 1] This is a schematic configuration diagram of a work content recording system according to an embodiment. [Figure 2] This is a functional block diagram of the work content recording system according to the embodiment. [Figure 3] This is an explanatory diagram for explaining a process of determining "contact alignment". [Figure 4] This is an example of an input screen for cycle information. [Figure 5] This is a flowchart showing the flow of processing when learning a learning model. [Figure 6] This is a flowchart showing the operation of the work determination device in the embodiment. [Figure 7] This is a flowchart showing the operation of the AI-based work content determination process in Fig. 6. [Figure 8] This is a flowchart showing the operation of the processing device in the embodiment. [Figure 9] This is a flowchart (1 / 3) showing the operation of the work content determination process in Fig. 8. [Figure 10] This is a flowchart (2 / 3) showing the operation of the work content determination process in Fig. 8. [Figure 11] This is a flowchart (3 / 3) showing the operation of the work content determination process in Fig. 8. [Figure 12] This is a flowchart showing the operation of the work content complementing process in Fig. 8. [Modes for Carrying Out the Invention]

[0017] Embodiments of the present invention will be described below with reference to the drawings as appropriate. In the drawings below, identical or similar components are denoted by the same reference numerals, and redundant explanations are omitted as appropriate. Furthermore, the size and shape of components may be represented schematically, modified or exaggerated, for the sake of clarity.

[0018] [Overall configuration of the site management system] Referring to Figure 1, the overall configuration of the field management system 100 according to this embodiment will be described. Figure 1 is a schematic diagram of the field management system 100 according to this embodiment.

[0019] The site management system (work content recording system) 100 is a system for recording work content using construction machinery. The site management system 100 comprises a transmitter 1, a work determination device 2, a receiving device 3, and a processing device 4. Furthermore, the site management system 100 also includes a display device 5 (51, 52, ..., 5 N ) is equipped with.

[0020] The work determination device 2, the receiving device 3, the processing device 4, and the display device 5 are connected wirelessly or via wired communication over a network. The network can be, for example, the Internet, but is not limited to that. For example, communication over the network can be achieved by implementing relay devices such as routers, mesh networks, and bridges (not shown). Alternatively, for example, data communication between the work determination device 2 and the receiving device 3 may be performed using Bluetooth®.

[0021] In this embodiment, the example of a breaker performing a test run inside a tunnel is described as construction machine 6. A breaker is a work machine in which a hydraulic breaker (breaker attachment) is attached to the end of the arm of an excavator (backhoe, hydraulic shovel). Note that the hydraulic breaker (breaker attachment) alone may also be referred to as the breaker. A hydraulic breaker is a device that crushes rocks, concrete, and other relatively hard materials by operating with supplied hydraulic pressure. Note that here only one breaker is shown as construction machine 6, but there are multiple heavy machines, such as heavy machines actually performing work and heavy machines on standby.

[0022] Transmitter 1 is attached to the construction machine 6 and emits radio waves for wireless communication. For example, transmitter 1 can be a beacon equipped with an acceleration sensor that emits a beacon signal as a radio wave when an acceleration above a predetermined threshold is detected. The beacon signal includes unique identification information (ID) that identifies transmitter 1. Transmitter 1 does not have to be a beacon equipped with an acceleration sensor; for example, it may be a power-supplied beacon that emits a beacon signal when it detects power supply.

[0023] The work determination device 2 is attached to the construction machine 6 and determines the type of work being performed by the attached construction machine. For example, the work determination device 2 can be configured as an edge device equipped with Android® or the like. Of course, the work determination device 2 can also be implemented as a general-purpose computer equipped with a CPU (Central Processing Unit), memory, etc. The work determination device 2 is activated in AI determination mode (AI determination mode) when the construction machine 6 is in operation. It uses a learning model to determine the work content from the collected sound data and transmits the determination result to the processing device 4. The detailed configuration of the work determination device 2 will be described later, referring to Figure 2.

[0024] The receiving device 3 receives radio waves transmitted by the transmitter 1. Although only one receiving device 3 is shown in this illustration, multiple receiving devices 3 may be arranged at predetermined intervals in the direction from the tunnel face towards the tunnel entrance. The receiving devices 3 can also be attached to the inner wall of the tunnel. The receiving device 3 converts the radio waves received from the transmitter 1 into data of a predetermined protocol and transmits the converted data to the processing device 4 via the network. The data transmitted by the receiving device 3 includes, for example, the ID of the received transmitter 1 and the ID of the receiving device 3 itself.

[0025] The processing unit 4 is a server device that analyzes the data received from the receiving unit 3, determines the work content, and records it. The processing unit 4 determines the current work content based on the cycle information indicating the work procedures of the construction machinery and construction work from the data received from the receiving unit 3, and further supplements the work content with the work determination result determined by the work determination device 2. The detailed configuration of the processing unit 4 will be described later with reference to Figure 2.

[0026] Display device 5 displays the work content recorded by processing device 4. Display device 5 acquires and displays the work content from processing device 4 via the network. Display device 5 can connect to multiple devices (51, 52, ..., 5) via the network. N ) This allows for connection. This enables the work manager to check the work status from any location. The following provides a more detailed explanation of the work determination device 2 and the processing device 4.

[0027] [Configuration of the work determination device] The configuration of the work determination device 2 will be explained with reference to Figure 2 (and Figure 1 as appropriate). As shown in Figure 2, the work determination device 2 comprises a sound collection processing unit 21, a determination processing unit 22, a learning model parameter storage unit 23, and a communication unit 24.

[0028] The sound collection processing unit 21 collects sound as sound data. The sound collection processing unit 21 comprises a sound collection unit 210 and a splitting unit 211. The sound collection unit 210 collects sound data via a microphone (not shown in the diagram). The microphone is pre-installed in a position where it can collect sounds generated when the work determination device 2 performs its work. Specifically, the sound collection unit 210 collects sound data from the construction machine 6 (in this case, a breaker) when it performs its check-in operation. The sound collection unit 210 outputs the collected sound data to the splitting unit 211.

[0029] The splitting unit 211 divides the sound data collected by the sound collection unit 210 into predetermined unit time lengths. Specifically, the splitting unit 211 divides the sound data into segments of approximately 1 second (1 to 3 seconds) along the time axis. This unit time length is the same as the length of the sound data used when training (machine learning) the trained model, which is identified by the parameters stored in the learning model parameter storage unit 23 described later. The splitting unit 211 outputs the split sound data to the judgment processing unit 22.

[0030] The determination processing unit 22 uses a pre-trained model (trained model) to estimate the work content from the sound data collected by the sound collection processing unit 21, and determines whether or not a predetermined work content is present. The determination processing unit 22 comprises a unit time work estimation unit 220, a log recording unit 221, and a work content determination unit 222.

[0031] The unit time work estimation unit 220 uses a trained model to estimate the work content for each unit time length from the sound data divided by the division unit 211. The trained model is a machine learning model that has been pre-programmed to take sound data as input and output the work content and its confidence level. The unit time work estimation unit 220 takes the sound data as input data and performs calculations on a trained model using parameters that have been previously stored in the trained model parameter storage unit 23. As a result of the calculations of the trained model, the unit time work estimation unit 220 estimates whether the sound data is a sound (breaker sound) generated from the construction machine 6 or another sound (other sound), and the confidence level of that determination. The unit time work estimation unit 220 outputs the estimation result to the log recording unit 221.

[0032] The logging unit 221 records the estimated results for each unit time estimated by the unit time work estimation unit 220 as log information L. The log information L is recorded in a storage medium such as memory (not shown in the diagram). The logging unit 221 notifies the work content determination unit 222 that the estimation result has been recorded in the log information L.

[0033] The work content determination unit 222 determines the work content for each predetermined time period based on the proportion of sound data estimated to represent predetermined work content in the log of the predetermined time period each time a log of a predetermined time period is recorded as log information L. The work content determination unit 222 determines the work content of a log interval as a single group when a predetermined time length (for example, 40 to 50 seconds) of log data has been accumulated in the log information L. Here, if the proportion of "breaker sounds" recorded in the log information L is 20% or more, the work content determination unit 222 determines that the work content for the predetermined time length is a "successful hit". If the proportion of "breaker sounds" is less than 20%, the work content determination unit 222 determines that the work content for the predetermined time length is not a "successful hit".

[0034] Here, referring to Figure 3, we will specifically explain the process of determining the work content from the log information L. If the unit time length for dividing the sound data is set to 1 second, the division unit 211 divides and generates audio files such as "br19-1115.wav" every 1 second. In the example in Figure 3, the unit time work estimation unit 220 performs an AI judgment for each 1 second of sound data, and the result (log information) is shown. For example, "File: br19-1115.wav, Predicted: other, Confidence: 1.00" indicates that the inference result (Predicted) based on the 1 second of audio data "br19-1115.wav" from 19:11:15 is not the sound of a circuit breaker (other), and its confidence level (Confidence) is 100% (1.00). The following "File: br19-1116.wav, Predicted: breaker, Confidence: 0.80" indicates that the inference result (Predicted) based on the 1-second audio data "br19-1116.wav" from 19:11:16 is a circuit breaker sound, and its confidence level (Confidence) is 80% (0.80).

[0035] Then, the work content determination unit 222 determines that, for example, if the proportion of "breaker sound" in the 40-second inference result from "File: br19-1115.wav, Predicted: other, Confidence: 1.00" to "File: br19-1155.wav, Predicted: other, Confidence: 1.00" is 20% or more, this 40-second time interval is determined to be a "winner". For example, if the 40-second log is divided into 40 sound data (each sound data is 1 second of audio data), the 40 audio data are input one by one into the trained model, and the results of determining whether or not it is a "breaker sound" are collected for 40 of them. If 8 or more of them are determined to be a "breaker sound", the 40-second time interval is determined to be a "winner". Returning to Figure 2, we will continue the explanation of the configuration of the work determination device 2.

[0036] After determining the work content, the work content determination unit 222 deletes the log recorded in log information L. After this, a new log will be recorded in log information L. When the work content determination unit 222 determines that a predetermined work content (in this case, "winning") has been performed, it instructs the communication unit 24 to notify the processing unit 4 that the predetermined work content has been performed.

[0037] The learning model parameter storage unit 23 stores in advance the parameters of a trained model that takes sound data as input and outputs a judgment result (breaker sound or other sound, confidence level). The learning model parameter storage unit 23 can be made of a general storage medium such as semiconductor memory.

[0038] The communication unit (output unit) 24 notifies the processing unit 4 when the determination processing unit 22 determines that a predetermined operation has been performed. In this case, the communication unit 24 transmits an ID or other identifier for the construction machine 6 to the processing unit 4 via the network.

[0039] With the configuration described above, the work determination device 2 can use AI to determine the content of a task based on sound. Furthermore, since the work determination device 2 determines the content of a task using sound data obtained by dividing the sound along the time axis, it can determine the content of a task from short-term sound data and reduce the size of the learning model. As a result, the computational load on the work determination device 2 is reduced, and determination can be performed even when using an inexpensive, waterproof and dustproof small terminal. In addition, since the work determination device 2 does not need to transmit the original sound data, there is no need for a large communication infrastructure. Furthermore, the work determination device 2 can be operated by a program (work determination program) that causes a computer (not shown in the diagram) to function as one of the parts within the aforementioned device.

[0040] [Processing unit configuration] Next, the configuration of the processing unit 4 will be described with reference to Figure 2 (and Figure 1 as appropriate). As shown in Figure 2, the processing unit 4 comprises a storage unit 40, a communication unit 41, and a control unit 42. The storage unit 40 stores various types of information necessary for processing by the processing unit 4. The storage unit 40 can be made up of a general storage medium such as a hard disk. The memory unit 40 stores the transmitter DB400 (DB stands for database, the same applies hereafter), the receiver DB401, the AI ​​judgment DB402, the operation information 403, the related information 404, and the cycle information 405.

[0041] The transmitter DB400 stores transmitter information for each transmitter 1, which is information related to the transmitter 1. Transmitter information includes, but is not limited to, an ID that identifies the transmitter 1, the type of transmitter 1 (accelerometer type, power supply type, etc.), and an ID that identifies the construction machine 6 to which the transmitter 1 is attached.

[0042] The receiving device DB401 stores receiving device information for each receiving device 3. This receiving device information includes, but is not limited to, an ID that identifies the target receiving device 3, the location of the receiving device 3, or a threshold value related to the intensity of the radio waves received by the receiving device 3. The location of the receiving device 3 can be identified, for example, using well-known map information.

[0043] The AI ​​Judgment DB 402 stores the work content (hit detection) determined by the work determination device 2 in chronological order. For example, the AI ​​Judgment DB 402 stores information such as hit detection, start time XX:XX:XX, and end time XX:XX:XX, associated with the ID of the transmitter 1 (or the ID of the construction machine).

[0044] The operational information DB 403 stores operational information for each construction machine 6. The operational information for construction machine 6 is information generated as an analysis result by the analysis unit 420, which will be described later. The operational information for construction machine 6 includes, but is not limited to, the operational status of construction machine 6, an ID that identifies the receiving device 3 that received radio waves from the transmitter 1 attached to construction machine 6, an ID that identifies construction machine 6, the ID of the transmitter 1 attached to construction machine 6, and the time when the radio waves were received.

[0045] The operating status of construction machinery 6 is information indicating whether or not construction machinery 6 is in operation (working). Furthermore, if transmitter 1 is an acceleration-type beacon, the operating information of construction machinery 6 may include the type of work being performed, the work duration, the start time of the work, and the end time of the work. Furthermore, if transmitter 1 is a power supply-type beacon, the operating information of construction machinery 6 may include the power supply duration, the power supply start time, and the power supply end time of construction machinery 6.

[0046] Related information 404 is information that associates the work performed in tunnel construction with a single type of construction machine 6 that performs that work. If a specific construction machine 6 is in operation, the determination recording unit 421, described later, can determine that the work associated in related information 404 is the current work. In this embodiment, the association between the work content registered in related information 404 and the construction machine 6 is as shown in the table below, but the association is not limited to this.

[0047] [Table 1]

[0048] Cycle information 405 is information that shows the work procedures for tunnel construction. One cycle of tunnel construction work proceeds as follows, for example: drilling → charging → blasting → excavation → check → primary shotcrete → support structure installation → secondary shotcrete → surface preparation → rock bolting (sometimes abbreviated as "RB"), and the cycle is repeated. These work contents can be appropriately selected or rearranged depending on various circumstances such as the location of the tunnel construction and the properties of the ground. The related information 404 and cycle information 405 can be input into the processing unit 4 in advance and their contents can be changed as appropriate (e.g., do not perform secondary spraying). This cycle information 405 is pre-configured via an external terminal (for example, display device 5). An example screen for configuring the cycle information 405 is shown in Figure 4. The example screen in Figure 4 includes fields for "Work Content," "Priority," "Color," "Area," "Starting Point," "Previous Work Conditions," and "Duration."

[0049] The "Work Details" column registers the work performed during tunnel construction, and displays the name of the work. The work procedure shown in cycle information 405 corresponds to the order in which the work details are registered in the "Work Details" column. The "Priority" column displays the priority assigned to each task, along with a numerical value for that priority. When the receiving device 3 receives radio waves from multiple types of transmitters 1 attached to multiple types of construction machinery 6, it can determine that the task with the highest priority among the multiple types of tasks is the current task.

[0050] The "Color" field registers the color used to display the work content on the display device 5, and the sample color is displayed. For illustrative purposes, in this embodiment, color differences are indicated by differences in hatching. The "Area" field registers the location where the work will be performed, and displays the name of that location. For example, since drilling is performed at the tunnel face, if "Drilling" is registered in the "Work Details" field, "Tunnel Face" will be registered in the "Area" field. The location to be registered in the "Area" field can be selected from the options in the pull-down menu.

[0051] The "Starting Point" column displays the work that serves as the starting point for one cycle, as indicated by cycle information 405. The work to be registered in the "Starting Point" column can be selected by enabling one of the radio buttons provided for each work. As shown in Figure 4, if blasting is selected as the starting point, the work for one cycle of tunnel construction can be set to blasting → excavation → check marking → primary spraying → support structure installation → secondary spraying → surface preparation → rock bolting → drilling → propellant charging.

[0052] The "Previous Work Conditions" column registers the preliminary work content for the target work content, and displays the name of the preliminary work content. If the radio waves received by the receiving device 3 are from a transmitter 1 attached to a construction machine 6 that performs multiple types of work, it becomes possible to determine the current work content based on the preliminary work content.

[0053] The "Previous Work Conditions" field serves to narrow down the possible current work content, which can be estimated from multiple types based on the operation of a specific construction machine 6. The work content registered in the "Previous Work Conditions" field can be selected from the options in the pull-down menu.

[0054] In the "Duration" column, the time assigned to the target work item for which the preceding work item is registered in the "Previous Work Conditions" column is registered and displayed as the duration. The duration serves as the estimated time for completion of the target work item. If the receiving device 3 receives a radio wave from the transmitter 1 of the construction machine 6 associated with the target work item in related information 404, after the previously determined preceding work item, it becomes possible to determine that the current work item is the target work item.

[0055] In this case, if the receiver 3 finishes receiving radio waves before the end of the duration, the work time for the target task will be the time from the end of the preceding task to the end of the receiver 3's radio wave reception. If the receiver 3 finishes receiving radio waves after the end of the duration, the work time for the target task will be the time from the end of the preceding task to the end of the duration.

[0056] The work content registered in the "Duration" field can be selected from the options in the pull-down menu. For example, the options include selecting a time in one-hour increments. As shown in Figure 4, if "Drilling" is registered in the "Previous Work Conditions" field and "2 hours" is registered in the "Duration" field for the "Charging" work content, then the work content performed within 2 hours after the completion of drilling will be determined to be charging, and it will be possible to determine that charging is the current work content that is continuing. Returning to Figure 2, we will continue the explanation of the configuration of the processing unit 4.

[0057] The communication unit 41 receives various data from the work determination device 2 and the receiving device 3 via the network. Here, the communication unit 41 receives an ID from the work determination device 2 when a predetermined work content (winning) is output as an AI determination result. The communication unit 41 stores the AI ​​determination result and time in the AI ​​determination DB. Furthermore, the communication unit 41 receives the ID of the transmitter 1 from the receiving device 3.

[0058] The control unit 42 controls various operations of the processing unit 4. The control unit 42 comprises an analysis unit 420, a determination recording unit 421, a completion unit 422, and a display control unit 423. The analysis unit 420 analyzes the radio waves received from the transmitter 1 by processing the data received from the receiving device 3. If transmitter 1 is an acceleration-type beacon, the radio waves emitted by transmitter 1 are radio waves caused by vibrations of the construction machine 6. For example, the radio waves emitted by transmitter 1 have a waveform that follows the waveform of the acceleration of the construction machine 6. Alternatively, the radio waves emitted by transmitter 1 may have a predetermined waveform (e.g., a sine wave) and their frequency may change in response to vibration (acceleration). Alternatively, the radio waves may exhibit a constant waveform (with constant frequency and amplitude) when the acceleration (vibration) exceeds a predetermined value.

[0059] The analysis unit 420, for example, performs an analysis to determine if the corresponding construction machine 6 is operating by referring to the transmitter DB 400, based on the fact that radio waves are being emitted from the transmitter 1. The analysis unit 420 may also perform a Fourier transform to analyze the frequency characteristics contained in the radio waves. The analysis unit 420 may also determine the intensity (amplitude) of the radio waves received by the receiving device 3 and, based on the determined intensity, identify the location of the construction machine 6 or the operating status of the monitored object. Furthermore, the analysis unit 420 may identify the ID of the construction machine 6 to which the transmitter 1 is attached by identifying the ID of the transmitter 1 that emitted the radio waves. This allows the analysis unit 420 to identify the construction machine 6 that is in operation.

[0060] The determination and recording unit 421 determines the current work content based on the construction machinery 6 and cycle information 405 indicating the work procedures for the construction work identified by the analysis unit 420, and records it as a tunnel construction record. The determination process of the determination recording unit 421 will be explained in detail later in the description of the operation of the processing unit 4 in Figure 8.

[0061] The supplementary unit 422 supplements the work content determined by the determination recording unit 421 based on the determination result output from the work determination device 2, and corrects the tunnel construction record. The AI ​​judgment DB 402 stores the ID and time of construction machinery 6 that have been determined to be "successful" in association with each other. Therefore, the supplementary unit 422 determines that even if the judgment recording unit 421 has determined that the current process is "successful" based on the cycle information 405, it is not "successful" if it has not been determined to be "successful" in the AI ​​judgment DB. Furthermore, even if the AI ​​judgment DB has determined that it is a "winner," the interpolation unit 422 may determine that there is an error in the AI ​​judgment depending on the work performed in the previous and current processes. The interpolation process in this interpolation unit 422 will be explained in detail later in the description of the operation of the processing unit 4 shown in Figure 8.

[0062] The display control unit 423 displays the work time for each task, which has been determined by the determination recording unit 421 and supplemented by the supplementation unit 422, on the display device 5. The display control unit 423 controls multiple display devices 51, 52, ..., 5 N Among these, the processing unit 4 transmits the work time for each task to the requested display device 5. This allows the processing unit 4 to visualize the work content via the display device 5, even from a remote location. With the configuration described above, the processing unit 4 can analyze the radio waves emitted by the transmitter 1 and identify the work content. Furthermore, the processing unit 4 can supplement the work content identified by the analysis results from the transmitter 1 with the results of the AI ​​judgment performed by the work judgment device 2, thereby identifying the work content with greater accuracy. Furthermore, the processing unit 4 can be operated using a program that causes a computer (not shown in the diagram) to function as one of the components within the aforementioned device.

[0063] [Field Management System Operation] The operation of the field management system 100 according to this embodiment will be described. The following describes the operation of the work determination device 2 and the processing device 4, which are the main components of the field management system 100. Before describing the operation of each device, please refer to Figure 5 and explain the learning process of the learning model used for AI judgment, which is stored in the learning model parameter storage unit 23 of the work judgment device 2. Note that this learning process can be operated by a general-purpose computer. The processes performed by the computer will be described in order below.

[0064] [Training process for learning models] In step S1, the sounds emitted by the construction machine 6, which is a circuit breaker, while it is in operation are input as learning sounds. In addition, in step S1, sounds other than those of the circuit breaker are also input as learning sounds. In step S2, the learning sound input in step S1 is divided into predetermined unit time lengths. This unit time length is the same as the time length used by the division unit 211 of the work determination device 2 to divide the sound data, which is approximately 1 second.

[0065] In step S3, training data is created from the sounds divided in step S2. Specifically, sound data of unit time lengths obtained by dividing the sound emitted when the circuit breaker is operating is associated with a value ("1") indicating that the sound is the circuit breaker sound (correct sound). In addition, sound data of unit time lengths obtained by dividing sounds other than the circuit breaker is associated with a value ("0") indicating that the sound is a sound other than the circuit breaker. In step S4, the parameters of the learning model are determined by performing deep learning (machine learning) using the training data created in step S3. Here, the segmented sound data is input to the learning model, and the model is trained to output a value that indicates either the circuit breaker sound or a sound other than the circuit breaker sound.

[0066] In step S5, the parameters of the learned model acquired in step S4 are recorded. The learning process ends in step S6. The parameters obtained through this learning process are pre-stored in the learning model parameter storage unit 23 of the work determination device 2.

[0067] [Operation of the work determination device] The operation of the work determination device 2 will be explained with reference to Figure 6 (and Figure 2 as appropriate for the configuration). In step S10, the work determination device 2 is activated in an AI determination mode (AI determination mode) when the construction machine 6 (breaker) is activated. For example, the work determination device 2 is connected to the cigarette lighter socket of the construction machine 6 with a cable, and when the construction machine 6 is activated, power is supplied to the work determination device 2, and the stored program (work determination program) is activated in AI determination mode. Alternatively, the AI ​​determination mode may be activated when the main power of the work determination device 2 is turned on.

[0068] The following steps S11 to S13 are repeatedly executed while the AI ​​judgment mode is activated. In step S11, the sound collection unit 210 acquires sound data via a microphone (not shown). Specifically, when the operator of the breaker starts the contact work, work noise is generated. The sound collection unit 210 collects the work noise using a microphone (not shown).

[0069] In step S12, the splitting unit 211 divides the sound data acquired in step S11. For example, the splitting unit 211 divides the sound data into 1-second intervals with a predetermined unit time length. In step S13, the determination processing unit 22 performs AI-based work content determination processing on the sound data divided in step S12 and determines the work content. By repeating steps S11 to S13 described above, the work determination device 2 can determine whether or not the breaker, which is the construction machine 6, is performing a ground clearance test.

[0070] Referring to Figure 7, the AI-based work content determination process in step S13 will be explained in more detail. In step S130, the unit time work estimation unit 220 uses the trained model described in Figure 5 to estimate the work content for each unit time length from the sound data divided in step S12 (Figure 6). In step S131, the logging unit 221 records the estimated results of the work content for each unit time length estimated in step S130 into a log file (log information L). In step S132, the logging unit 221 determines whether or not logs have been accumulated for the sound data of a predetermined duration (for example, 40 to 50 seconds) collected in step S11. These logs of a predetermined duration form a single group used to estimate the nature of the work performed. If no logs of the predetermined length have accumulated (No in step S132), the determination processing unit 22 returns to step S130.

[0071] On the other hand, if logs of a predetermined length of time have accumulated (Yes in step S132), in step S133, the work content determination unit 222 determines whether the proportion of breaker sounds is 20% or more based on the determination results (breaker sounds or other sounds) recorded in the log information L. If the proportion of breaker sounds is less than 20% (No in step S133), the AI-based work content determination process is terminated, and the operation returns to step S11 in Figure 6.

[0072] On the other hand, if the proportion of breaker sounds is 20% or more (Yes in step S133), in step S134, the work content determination unit 222 determines that a hit detection is being performed and notifies the processing unit 4 via the communication unit 24 that a hit detection has occurred. At this time, the work content determination unit 222 notifies the communication unit 24 of the ID that identifies the construction machine 6, and the communication unit 24 transmits that ID as data. After data transmission, the work determination device 2 returns to step S11 in Figure 6. Through the above operations, the work determination device 2 can use sound to AI-determine the content of the hit detection operation and notify the processing device 4 of the occurrence of the hit detection.

[0073] [Operation of the processing unit] The operation of the processing unit 4 will be explained with reference to Figure 8 (and Figure 2 as appropriate for the configuration). In step S20, the communication unit 41 receives data transmitted from the receiving device 3 (reception step). This data includes an ID that identifies the transmitter 1. In step S21, the analysis unit 420 identifies the construction machine 6 to which the transmitter 1 that emitted the radio waves (beacon signal) is attached, based on the data received in step S20 (analysis step). By referring to the transmitter DB 400, the construction machine 6 can be identified from the ID of the transmitter 1. In step S22, the analysis unit 420 generates or updates the operation information of the construction machine 6 identified in step S21. The operation information is stored in the operation information DB 403 and includes the type of work (work content), total operating time, work start time, work end time, etc. In step S23, the determination recording unit 421 executes a process to determine the work content based on the cycle information 405 (work determination step) and record it (determination recording step).

[0074] Here, with reference to Figures 9 to 11, the process of determining the work content in step S23 will be explained in more detail. In step S2301, the determination recording unit 421 determines whether the data received in step S20 (Figure 8) contains the ID of the transmitter 1 (beacon) attached to the explosive transport vehicle. If the ID of the explosive transport vehicle's beacon is included (Yes in step S2301), then in step S2302, the determination recording unit 421 determines whether the received data includes the ID of the beacon attached to the rock drill equipped on the drill jumbo, or the ID of the beacon on the water pump equipped on the drill jumbo.

[0075] If the ID of the rock drill or water pump beacon is included (Yes in step S2302), in step S2303, the determination recording unit 421 interprets this as explosive transport during drilling and makes a provisional determination that the current work is drilling. If neither the ID of the rock drill nor the water pump is included (No in step S2302), in step S2304, the determination recording unit 421 refers to the "charge" entry in the cycle information 405 to determine whether or not it is within the duration after the completion of drilling.

[0076] Here, if it is within the duration (Yes in step S2304), in step S2305, the judgment recording unit 421 considers it to be transporting explosives after drilling and tentatively determines that the current work is charging. On the other hand, if it is not within the duration (No in step S2304), in step S2306, the judgment recording unit 421 maintains the previous judgment result and determines that the current work is the work that has already been determined.

[0077] Furthermore, if the ID of the explosive transport vehicle's beacon is not included (No in step S2301), in step S2307, the determination recording unit 421 determines whether the data received in step S20 includes the ID of the rock drill beacon on the drill jumbo, or the ID of the water pump beacon on the drill jumbo. If the ID of a rock drill or water pump beacon is included (Yes in step S2307), the determination recording unit 421 considers that drilling is in progress before explosive transport and provisionally determines in step S2303 that the current work is drilling. On the other hand, if neither the ID of a rock drill nor a water pump is included (No in step S2307), the work content determination process is terminated.

[0078] Furthermore, in step S2308, the determination recording unit 421 determines whether the received data includes the ID of the power supply trolley's beacon. If the ID of the power supply trolley's beacon is included (Yes in step S2308), in step S2309, the determination recording unit 421 refers to the "blasting" entry in the cycle information 405 and determines whether it is within the duration after the completion of the explosive charge.

[0079] Here, if it is within the duration (Yes in step S2309), in step S2310, the determination recording unit 421 tentatively determines that the current work is blasting. On the other hand, if it is not within the duration (No in step S2309), in step S2306, the determination recording unit 421 maintains the previous determination result and determines that the current work is a previously determined work. On the other hand, if the ID of the power trolley's beacon is not included (No in step S2308), the work content determination process is terminated.

[0080] Furthermore, as shown in Figure 10, in step S2311, the determination recording unit 421 determines whether or not the received data contains the ID of the wheel loader's beacon. If the ID of the wheel loader's beacon is included (Yes in step S2311), in step S2312, the determination recording unit 421 tentatively determines that the current work is a slippage. On the other hand, if the ID of the wheel loader's beacon is not included (No in step S2311), the work content determination process is terminated.

[0081] Furthermore, in step S2313, the determination recording unit 421 determines whether the received data contains the ID of the breaker's beacon. If the ID of the breaker's beacon is included (Yes in step S2313), in step S2314, the determination recording unit 421 refers to the "Hit-out" entry in the cycle information 405 to determine whether it is within the duration after the end of the slippage. If it is within the duration (Yes in step S2314), in step S2315, the determination recording unit 421 tentatively determines that the current work is hit-out. On the other hand, if it is not within the duration (No in step S2314), in step S2316, the determination recording unit 421 refers to the "Scaling" entry in the cycle information 405 to determine whether it is within the duration after the end of the secondary spraying.

[0082] If it is within the duration (Yes in step S2316), the determination recording unit 421 tentatively determines that the current work is surface preparation. On the other hand, if it is not within the duration (No in step S2316), in step S2306, the determination recording unit 421 maintains the previous determination result and determines that the current work is a previously determined work. On the other hand, if the breaker beacon ID is not included (No in step S2313), the work content determination process is terminated.

[0083] Furthermore, in step S2318, the determination unit 12 determines whether the received data includes the ID of the concrete pump beacon equipped on the spraying machine. If the ID of the concrete pump beacon is included (Yes in step S2318), in step S2319, the determination recording unit 421 refers to the "primary spraying" entry in the cycle information 405 and determines whether it is within the duration after the end of the slippage.

[0084] If it is within the duration (Yes in step S2319), in step S2320, the determination recording unit 421 tentatively determines that the current work is primary spraying. On the other hand, if it is not within the duration (No in step S2319), in step S2321, the determination recording unit 421 refers to the "secondary spraying" entry in the cycle information 405 to determine whether or not it is within the duration after the completion of the shoring construction.

[0085] If it is within the duration (Yes in step S2321), in step S2322, the determination recording unit 421 provisionally determines that the current work is secondary spraying. On the other hand, if it is not within the duration (No in step S2321), in step S2306, the determination recording unit 421 maintains the previous determination result and determines that the current work is a previously determined work. On the other hand, if the concrete pump beacon ID is not included (No in step S2318), the work content determination process is terminated.

[0086] Furthermore, as shown in Figure 11, in step S2323, the determination recording unit 421 determines whether the received data contains the ID of the mangauge beacon equipped on the spraying machine. If the mangauge beacon ID is included (Yes in step S323), in step S2324, the determination recording unit 421 provisionally determines that the current work is shoring installation. On the other hand, if the mangauge beacon ID is not included (No in step S2323), the work content determination process is terminated.

[0087] Furthermore, in step S2325, the determination recording unit 421 determines whether the received data contains the ID of the mortar pump beacon equipped on the drill jumbo. If the ID of the mortar pump beacon is included (Yes in step S2325), in step S2326, the determination recording unit 421 tentatively determines that the current work is a rock bolt (RB). On the other hand, if the ID of the mortar pump beacon is not included (No in step S2325), the work content determination process is terminated.

[0088] Next, in step S2327, the determination recording unit 421 makes a final determination based on the priority set in the cycle information 405, targeting at least one of the work contents provisionally determined in steps S2303, S2306, S2310, S2312, S2315, S2317, S2320, S2322, S2324, and S2326, and the work contents maintained in step S2306. Based on the final determination, one current work contents is determined and recorded as a tunnel construction record. Returning to Figure 8, we will continue the explanation of the operation of the processing unit 4.

[0089] In step S24, the completion unit 422 performs a work content completion process (completion step) that completes the work content determined in step S23 based on the determination result output from the work determination device 2. Now, with reference to Figure 12, the process of completing the work content in step S24 will be explained in more detail. In step S2401, the judgment recording unit 421 determines whether or not there is a determination of hitch in the AI ​​judgment DB 402 at the time when hitch was determined in the tunnel construction record. If there is no determination of hitch (No in step S2401), in step S2402, the judgment recording unit 421 corrects the tunnel construction record to indicate that hitch was not performed at that time, and returns to step S25 (Figure 8).

[0090] On the other hand, if a determination is made regarding the presence of the object (Yes in step S2401), in step S2403, the determination recording unit 421 determines whether the previous or current work process in the tunnel construction record is performing any of the following operations: "excavation", "presentation", or "excavation / presentation". If these predetermined operations have not been performed (No in step S2403), in step S2404, the judgment recording unit 421 determines that there is an error in the AI ​​judgment, discards the judgment result recorded in the AI ​​judgment DB 402, and returns to step S25 (Figure 8). On the other hand, if any of the following operations are being performed (Yes in step S2403), in step S2405, the judgment recording unit 421 determines whether there is a gap since the previous "extrusion" process (no processing after "extrusion") in the tunnel construction record at the judgment time recorded in the AI ​​judgment DB 402. If there is a gap since the previous "extrusion" process (Yes in step S2405), in step S2406, the judgment recording unit 421 sets the current process to "fitting," maintains this state for 1 minute, and returns to step S25 (Figure 8).

[0091] On the other hand, if there is no gap since the previous "slip-out" process (No in step S2405), in step S2407, the judgment recording unit 421 determines whether or not the "slip-out" is continuing. If the "slip-out" is not continuing (No in step S2407), in step S2408, the judgment recording unit 421 sets the current process to "hit-out," maintains this state for 1 minute, and returns to step S25 (Figure 8). On the other hand, if the "slip-out" is continuing (Yes in step S2407), in step S2409, the determination recording unit 421 sets the current process to "slip-out / contact," maintains this state for 1 minute, and returns to step S25 (Figure 8). In steps S2406, S2408, and S2409, the state is maintained for 1 minute because the AI ​​judgment result from AI judgment DB 402 is obtained at predetermined time intervals (for example, 40 to 50 seconds). Furthermore, it does not need to be exactly 1 minute as long as the interval is longer than the predetermined time interval. Returning to Figure 8, we will continue the explanation of the operation of the processing unit 4.

[0092] In step S25, the display control unit 423 displays the work time for each work item, which has been determined by the determination recording unit 421 and supplemented by the supplementation unit 422, as a tunnel construction record on the requested display device 5. Through the above operations, the processing unit 4 can generate tunnel construction records that are more accurate than before, based on the occurrence of hit detection determined by the work determination device 2 using AI.

[0093] [effect] According to this embodiment, even when multiple construction machines 6 are in operation, the work determination device 2 can accurately determine the work content by using AI determination based on a model that has been pre-trained on the sounds of the work content. Furthermore, the processing unit 4 can generate highly accurate work content records by supplementing the work content records from the transmitter 1 with the AI ​​judgment results from the work judgment device 2.

[0094] Although embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above, and can be modified as appropriate without departing from its spirit. For example, in the above embodiment, the work determination device 2 is provided as independent hardware on the construction machine 6. However, the work determination device 2 may also be operated on the receiving device 3 as software that serves as the work determination program. [Explanation of Symbols]

[0095] 100. On-site management system (work content recording system) 1 Transmitter 2. Work determination device 20 Location information acquisition section 21 Sound Collection Processing Unit 210 Sound collection section 211 Split section 22 Determination Processing Unit 220 Unit Time Work Estimation Unit 221 Log Recording Section 222 Work content judgment section 23 Learning Model Parameter Storage Unit 24. Communication section (output section) 3. Receiving device 4 Processing Unit 40 Storage section 400 Transmitter Database 401 Receiving Equipment DB 402 AI judgment DB 403 Operation Information Database 404 Related Information 405 Cycle Information 41 Communications Department 42 Control Unit 420 Analysis Department 421 Judgment Record Section 422 Supplementary section 423 Display Control Unit 5 Display device 6. Construction machinery

Claims

1. A work determination device that determines predetermined work content by sound, A sound collection processing unit that collects the aforementioned sound as sound data, A determination processing unit that uses a trained model to estimate the work content from sound data, estimates the work content from the sound data collected by the sound collection processing unit, and determines whether or not the work content exists, An output unit that outputs the determination result of the determination processing unit, A work determination device characterized by comprising the following features.

2. The aforementioned trained model is a model that has learned the work content corresponding to sound data for each predetermined unit time length, The aforementioned sound collection processing unit is A sound collection unit for collecting the aforementioned sound data, The system includes a division unit that divides the sound data collected by the sound collection unit into units of the unit time length, The determination processing unit, A unit time work estimation unit that estimates the work content for each unit time length from the sound data divided by the division unit using the trained model, A recording unit that records the work content estimated by the unit time work estimation unit as a log, Each time a log of a predetermined length is recorded, a work content determination unit determines the work content for each predetermined length based on the proportion of sound data estimated to be a predetermined work content in the log of the predetermined length, The work determination device according to claim 1, characterized by comprising the following:

3. A program for causing a computer to function as a work determination device according to claim 1 or claim 2.

4. A work content recording system for recording the details of work performed using construction machinery, A transmitter that is attached to each of the aforementioned construction machines and emits radio waves, A receiving device that receives radio waves transmitted by the aforementioned transmitter, A work determination device according to claim 1 or claim 2 that determines the content of work from sound, The system comprises a receiving device and a processing device that is communicatively connected to the work determination device, The aforementioned processing apparatus is An analysis unit analyzes radio waves received from the receiving device and identifies the construction machine in operation, Based on the cycle information indicating the work procedures of the construction machinery and construction work identified by the analysis unit, the determination and recording unit determines and records the current work content. A supplementation unit that supplements the work content recorded in the judgment recording unit based on the judgment result output from the work judgment device, A work content recording system characterized by having the following features.

5. A method for recording work details using construction machinery, A receiving step in which a receiving device receives radio waves from a transmitter attached to the construction machine, A processing device, which is communicatively connected to the receiving device and the work determination device, performs an analysis step in which it analyzes the radio waves received in the receiving step and identifies the construction machine that is in operation, A work determination device according to claim 1 or claim 2 includes a work determination step in which it determines the content of work from sound, The processing device determines and records the current work content based on cycle information indicating the work procedures of the construction machinery and construction work identified in the analysis step, and The processing device performs a supplementation step in which it supplements the work content recorded in the determination recording step based on the determination result output from the work determination device, A method for recording work content, characterized by including the following.

Images