Work management device and work management method

Abstract

To enable more accurate work management by accurately estimating work being performed using a trained model built through machine learning.SOLUTION: A work management device (3) is provided, comprising: a first output unit (312) configured to output a detection class estimated to be appearing in work at the time of acquisition of a captured image by inputting the captured image to an indicator estimation model (321), and a work estimation unit (314) configured to estimate work being performed at the time of acquisition of the captured image on the basis of an output result of the first output unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a work management device and the like. [Background technology]

[0002] The plumbing work management system of Patent Document 1 estimates the current work status of plumbing work from characteristic images detected from captured images that include the work area in the plumbing work, and determines the content of the notification to those involved in the plumbing work based on the estimated current work status. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-107341 Summary of the Invention [Problem to be solved by the invention]

[0004] However, Patent Document 1 does not specifically disclose a trained model constructed using machine learning. One aspect of the present invention aims to realize a work management device or the like that can accurately manage work by accurately estimating work that is currently being performed or work that is scheduled to be performed using a trained model constructed using machine learning. [Means for solving the problem]

[0005] A work management device according to one aspect of the present invention includes an image acquisition unit that acquires a photographed image including a work area in plumbing work; a first output unit that outputs at least one indicator that is estimated to have appeared in the work at the time the photographed image is acquired by inputting the photographed image into an indicator estimation model that is constructed using machine learning to estimate at least one indicator that is appearing in the work at a certain time out of a plurality of indicators that indicate each of a plurality of things involved in the work in the plumbing work, or a combination thereof; and a first output unit that identifies the work at a certain time within the predetermined work time at the time the photographed image is acquired based on pre-prepared data that associates each work that may be performed in the plumbing work with the combination of indicators and the combinations of indicators that appear at each of the times the photographed image is acquired within the predetermined work time, output over time by the first output unit. and a second output unit that outputs work schedule data indicating the relationship between at least one work piece estimated to be performed in the next specified work time and the time period in which the work piece will be performed by inputting the performance data into a work estimation model constructed using machine learning to estimate the work piece to be performed in the next specified work time from the performance of the work piece in the specified work time. The indicator estimation model is constructed by performing machine learning using item images indicating items used in the plumbing work and the indicators as training data. The work estimation model is constructed by performing machine learning using past work schedule data indicating the relationship between at least one work piece indicated as being scheduled to be performed in a construction plan for past plumbing work and the time period in which the work piece will be performed, past performance data indicating the relationship between at least one work piece performed in the past plumbing work and the time period in which the work piece will be performed, and data on factors that influenced the work in the past plumbing work.

[0006] A work management method according to one aspect of the present invention includes an image acquisition step of acquiring a photographed image including a work area in plumbing work; a first output step of inputting the photographed image into an index estimation model constructed using machine learning to estimate at least one index appearing in work at a certain point in time from a plurality of indexes indicating each of a plurality of things involved in work in the plumbing work, or a combination thereof, thereby outputting at least one index estimated to have appeared in work at the time the photographed image was acquired; and a second output step of identifying work at a certain time within the predetermined work time that is the time the photographed image was acquired, based on data prepared in advance that associates each work that may be performed in the plumbing work with a combination of the indexes, and the combinations of indices that appear at each time the photographed image was acquired within the predetermined work time, which are output over time in the first output step. and a second output process of outputting work schedule data indicating the relationship between at least one work piece estimated to be performed in the next specified work time and the time period in which the work piece will be performed by inputting the performance data into a work estimation model constructed using machine learning to estimate the work piece to be performed in the next specified work time from the performance of the work piece in the specified work time. The indicator estimation model is constructed by performing machine learning using as training data item images of items used in the plumbing work and the indicators, and the work estimation model is constructed by performing machine learning using as training data past work schedule data indicating the relationship between at least one work piece indicated as being scheduled to be performed in a construction plan for past plumbing work and the time period in which the work piece will be performed, past performance data indicating the relationship between at least one work piece performed in the past plumbing work and the time period in which the work piece was performed, and data on factors that influenced the work in the past plumbing work. [Effects of the Invention]

[0007] According to one aspect of the present invention, by using a trained model constructed by machine learning to accurately estimate work that is currently being performed or that is scheduled to be performed, the work can be managed with high accuracy. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram showing an example of a schematic configuration of a waterworks construction management system. [Figure 2] FIG. 10 is a diagram showing an example of an output result of a first output unit. [Figure 3] 10A and 10B are diagrams illustrating an example of a task for which a task status determination unit determines a task status. [Figure 4] 10A and 10B are diagrams illustrating an example of a display of a result of work state determination by a work state determination unit. [Figure 5] FIG. 10 is a diagram showing an example of display of performance data created by a data creation unit. [Figure 6] FIG. 10 is a diagram showing a display example of work schedule data output by a second output unit. [Figure 7] 10 is a diagram showing an example of the relationship between the determination result by the work state determination unit and the notification content by the display control unit. FIG. [Figure 8] 10 is a flowchart illustrating an example of processing in the work management device. DETAILED DESCRIPTION OF THE INVENTION

[0009] [Configuration of waterworks construction management system] 1 is a block diagram showing an example of a schematic configuration of a plumbing work management system 1 according to an embodiment of the present invention. The plumbing work management system 1 manages various plumbing work tasks (e.g., progress of the plumbing work) for installing pipes. The plumbing work management system 1 includes a camera 2, a work management device 3, and a display device 4.

[0010] The camera 2 photographs various tasks being performed at one or more plumbing construction sites. The camera 2 may be capable of photographing the work area where the various tasks are performed. Examples of the camera 2 include a camera attached to the helmet or work clothes (e.g., the arm part of the work clothes) of a worker (person involved in the plumbing construction) to photograph the work at hand, or a camera mounted on various work tools (e.g., a backhoe). Examples of the camera 2 include a camera installed near the work area. The camera 2 includes a photographing unit 21 and a communication unit 22.

[0011] The photographing unit 21 photographs at least the work area in the plumbing work. The photographing unit 21 may photograph still images or may photograph moving images. Multiple photographs may be taken within the same plumbing work. The work area (and therefore the camera 2) may be fixed, or may be movable with each photograph taken as a unit. In other words, the photographing unit 21 creates a photographed image including the work area each time a photograph is taken.

[0012] The communication unit 22 is communicatively connected to the work management device 3 and transfers data of the images captured by the imaging unit 21 to the work management device 3. The communication between the communication unit 22 and the work management device 3 is wireless, but may also be wired.

[0013] The work management device 3 manages various types of work that are being carried out as plumbing work or various types of work that are scheduled to be carried out. When managing work at multiple plumbing work sites, the work management device 3 manages the work for each site. Details of the work management device 3 will be described later.

[0014] The display device 4 displays various information related to the plumbing work to the manager who manages the plumbing work (those involved in the plumbing work). For example, the display device 4 may display the work status determination results output from the work management device 3, performance data showing the performance of various works during a specified work period, or work schedule data showing the schedule of various works to be performed during the next specified work period.

[0015] The display device 4 is a monitor or personal computer on which a manager manages various tasks. However, the display device 4 may also be an electronic device (e.g., a tablet or smartphone) carried by a worker. In this case, the display device 4 can display various information related to plumbing work to the worker. The plumbing work management system 1 may also be equipped with an audio output device (e.g., a speaker) that outputs the various information by voice.

[0016] In this embodiment, the predetermined work time indicates the total work time specified for one day (e.g., 8:00 to 16:30), and the next predetermined work time indicates the total work time specified for the next day (e.g., 8:00 to 16:30). However, the predetermined work time may also indicate, for example, the work time from the start of work to the time when the photographed image is acquired, in which case the next predetermined work time indicates the work time after the time when the photographed image is acquired.

[0017] [Configuration of the work management device] The work management device 3 includes a control unit 31 and a memory unit 32. The control unit 31 comprehensively controls each unit of the work management device 3. The memory unit 32 stores various programs and various data used by the control unit 31. The memory unit 32 stores, for example, an index estimation model 321 and a work estimation model 322, which will be described later. The memory unit 32 may be realized by a storage device separate from the work management device 3, but connected to the work management device 3 so as to be able to communicate with it.

[0018] The control unit 31 includes an image acquisition unit 311 , a first output unit 312 , a position identification unit 313 , an activity estimation unit 314 , an activity state determination unit 315 , a data creation unit 316 , a second output unit 317 , and a display control unit 318 .

[0019] The image acquisition unit 311 acquires the captured image transferred from the camera 2, which includes the area to be worked on in the plumbing work.

[0020] The first output unit 312 inputs the captured image acquired by the image acquisition unit 311 to the index estimation model 321 stored in the storage unit 32. The index estimation model 321 is a trained model constructed using machine learning to estimate at least one index that appears in work at a certain point in time from among multiple indexes for evaluating the work status in plumbing work. In this embodiment, the index is referred to as a detection class that needs to be detected from the captured image to evaluate the work status.

[0021] Examples of detection classes include single objects such as various pipes (pipes) used in plumbing work, workers, ordinary people other than workers, various work tools used in plumbing work, and trenches where pipes are placed. A detection class may be a combination of multiple such single objects, and for example, a combination of a trench and a worker in the trench may be set as one detection class.

[0022] Examples of various pipes include straight pipes and irregularly shaped pipes. Examples of various work implements (e.g., work machines (work vehicles) and tools) include asphalt cutters, backhoes, dump trucks, rammers, cones (or poles), sheet piles, ratchet pawls, sleepers, and ladders. Furthermore, the work implements set as detection classes may be parts of the work implements; for example, a bucket and a sling belt that are parts of a backhoe may be set as detection classes.

[0023] As the detection class, an object that allows the work state to be determined may be set. For example, as the detection class, an object that characterizes the performance of appropriate work (standard work) and an object that characterizes dangerous work may be set. An example of an object that characterizes the performance of standard work is the working posture of a worker when performing standard work. An example of an object that characterizes dangerous work is a groove where no safety cone is installed.

[0024] The index estimation model 321 is a trained model constructed to extract and output the above-described detection classes contained in the captured image as a result of inputting the captured image. The index estimation model 321 may be, for example, a neural network (e.g., a convolutional neural network (CNN)) including at least an input layer, an intermediate layer, and an output layer.

[0025] In the indicator estimation model 321, the input layer is a layer to which captured images are input. The intermediate layer is a layer in which parameters are learned based on an area image including the work area, an item image showing items used in plumbing work, and a detection class. The output layer is a layer that outputs at least one detection class.

[0026] For example, data in which a detection class is associated as correct answer data with each of a plurality of region images and each of a plurality of article images is prepared as training data. The index estimation model 321 is constructed by sequentially inputting the training data and learning parameters (weights and biases) so as to minimize the loss function. The index estimation model 321 is constructed, for example, by a model generation device (not shown), but is not limited to this and may also be constructed, for example, by the control unit 31. In this case, the control unit 31 also functions as a learning device that constructs the index estimation model 321 by machine learning using the training data.

[0027] In addition, the other trained models described below may also be constructed by sequentially inputting training data and learning parameters to minimize the loss function, and may be constructed by a model generation device or by the control unit 31.

[0028] The area image may be a photographed image taken by the camera 2, a differential image, or a composite image. A differential image is, for example, an image obtained by extracting the difference between two photographed images taken over time. A composite image is, for example, an image in which various background images (virtual background images) prepared in advance are embedded in the background area of ​​the photographed work area. The item image may be an image of the various pipes and work tools described above. As the item image, multiple images of one pipe and one work tool with their sizes changed may be prepared. In this way, by preparing processed images in addition to the photographed images, it is possible to prepare diverse training data from a small number of photographed images. Furthermore, the area image and the item image may include images suitable for estimating (inferring) the detection class.

[0029] The index estimation model 321 constructed in this way causes the computer to function such that, when a captured image is input to the input layer, the computer performs calculations in the intermediate layer and outputs from the output layer at least one detection class that is estimated to have appeared in the work at the time the captured image was acquired. That is, the first output unit 312 inputs the captured image to the index estimation model 321, and outputs at least one detection class that is estimated to have appeared in the work at the time the captured image was acquired. Note that the time when the captured image is acquired is the time when the image acquisition unit 311 acquires the image, but it may also be interpreted as the time when the work area is photographed by the camera 2.

[0030] FIG. 2 is a diagram showing an example of the output result of the first output unit 312. Reference numeral 201 in FIG. 2 indicates the output result from the start of work to time T1 on a certain day, and reference numeral 202 indicates the output result from the start of work to the end of work on a certain day. In reference numerals 201 and 202, the vertical axis indicates the number of each detection class, and the horizontal axis indicates the work time. For example, each of the detection classes described above is assigned a number, such as "0" for the worker, "1" for the straight pipe, "2" for the irregular pipe, "3" for the backhoe, and "4" for the sling belt. In the example of FIG. 2, the numbers "0" to "7" are shown as the detection class numbers, but numbers are assigned to the detection classes as many as the number of preset detection classes.

[0031] For example, consider a case where a worker, an irregular-shaped pipe, a backhoe, and a sling belt are included in a captured image acquired at time T1. In this case, the first output unit 312 outputs a combination of "0" and "2" to "4" corresponding to these detection classes at time T1, as indicated by reference numeral 201 in FIG. 2. Because the first output unit 312 inputs captured images into the index estimation model 321 over time from the start of work, as indicated by reference numeral 201, at time T1, it can output the combination of detection classes at each time point from the start of work (8:00) to time T1. As a result, at the end of work for the day, the first output unit 312 can output the combination of detection classes at each time point from the start of work to the end of work for the day, as indicated by reference numeral 202.

[0032] The index estimation model 321 outputs a numerical value (e.g., a numerical value greater than or equal to 0 and less than or equal to 1) for each detection class as a result of the above calculation. The larger the numerical value of a detection class, the higher the probability that an object represented by that detection class is included in the captured image. Therefore, the index estimation model 321 estimates a detection class that indicates a numerical value equal to or greater than a predetermined threshold as the detection class included in the captured image. The index estimation model 321 may output, as an output result, the above numerical value, for example, together with the detection class, as the accuracy (reliability) of the estimation of the detection class.

[0033] The position identification unit 313 identifies the position of the detection class output by the first output unit 312 in the captured image. The position identification unit 313 can identify the position of the detection class in the captured image by, for example, inputting the captured image into a position estimation model constructed to estimate the position of the detection class in the captured image using training data in which article images are associated with detection classes. The position estimation model may be stored in the storage unit 32 and may be a trained model implemented by, for example, R-CNN (Regions with CNN).

[0034] The activity estimation unit 314 estimates the activity being performed at the time of acquisition of the captured image based on the output result of the first output unit 312. Specifically, the activity estimation unit 314 estimates the activity being performed at the time of acquisition of the most recently acquired captured image by the image acquisition unit 311 by analyzing the combinations of detection classes that appear at each of the times of acquisition of the captured image output over time by the first output unit 312. That is, the activity estimation unit 314 estimates the activity currently being performed from the combination of detection classes estimated based on the most recently acquired captured image by the image acquisition unit 311 and the combination of detection classes estimated based on the captured image at at least one point immediately before that (past captured image).

[0035] Each task that may be performed in plumbing work is associated with a combination of detection classes and stored in the memory unit 32. Based on the output result (combination of detection classes) at each point in time from the first output unit 312, the task estimation unit 314 identifies the task corresponding to the combination as the task being performed at the time the photographed image was acquired.

[0036] For example, in the work process of hanging and installing a straight pipe, there is a task of removing a foreign object from a receiving groove. For this task, for example, a combination of a worker, a straight pipe, a receiving groove, and an object (foreign object) present in the receiving groove is associated as a detection class. The task estimation unit 314 determines whether the above task is being performed by determining the presence or absence of a worker, a straight pipe, a receiving groove, and an object located in the receiving groove at each successive point in time in the captured image based on the output result of the first output unit 312.

[0037] The activity estimation unit 314 may estimate an activity using an performed activity estimation model constructed to estimate the activity being performed at the time the captured image was acquired, using training data that associates each activity with a combination of detection classes. In this case, the activity estimation unit 314 can estimate the activity by inputting the output result of the first output unit 312 into the performed activity estimation model. The performed activity estimation model may be stored in the storage unit 32, and may be a trained model implemented using, for example, CNN.

[0038] Furthermore, when the position identification unit 313 identifies the position of a detection class in the captured image, the work estimation unit 314 may estimate the work being performed in different areas within the captured image based on the position of the detection class. The work estimation unit 314 may determine whether the captured image includes multiple scenes of plumbing work being performed, for example, based on the type of estimated detection class and the position of the identified detection class. When the work estimation unit 314 determines that plumbing work is being performed in different areas within the captured image, it estimates the work being performed in each area.

[0039] The work state determination unit 315 determines the appropriateness of the work state of the work estimated by the work estimation unit 314 by analyzing the combinations of detection classes that appear at each point in time when the captured images are acquired, which are output over time by the first output unit 312. That is, the work state determination unit 315 determines the appropriateness of the work state of the work that is estimated to be currently being performed from the combination of detection classes estimated based on the most recently captured image acquired by the image acquisition unit 311 and the combination of detection classes estimated based on the captured image at least one point in time immediately before that. This determination result indicates the performance of each work.

[0040] Specifically, the work status determination unit 315 determines whether the work status is appropriate by determining whether the work procedures are appropriate, whether safety is guaranteed, and whether there are any delays in the work.

[0041] Whether or not a work procedure is appropriate may be determined, for example, based on the combination of detection classes at each successive point in time in the captured image and the position or change in position over time of a certain detection class in the captured image at each point in time.

[0042] Fig. 3 is a diagram showing an example of the work for which the work status is determined by the work status determination unit 315. Fig. 3 illustrates an example of the work performed in the work process from pipe hanging and installation to joint connection check, divided into the case of a straight pipe and the case of a special-shaped pipe.

[0043] As shown in FIG. 3 , when a foreign object is being removed from the receiving groove, the work status determination unit 315 determines the appropriateness of the work procedure by, for example, determining the presence or absence of an object in the receiving groove at each successive point in time in the captured image (i.e., the change in the object's position). The work status determination unit 315 may also perform similar processing when a foreign object is being removed from a region within approximately 30 cm of the outer edge of the insertion port and from the inner surface of the receiving port. That is, the work status determination unit 315 may determine the appropriateness of the work procedure by, for example, determining the presence or absence of an object in these positions at each successive point in time (i.e., the change in the object's position). In other words, the work status determination unit 315 may determine that the area of ​​the object occupying the receiving groove, etc., decreases over time in the captured image, and may determine that the area has become equal to or smaller than a predetermined area. In this case, the work status determination unit 315 may determine that the work procedure for removing the foreign object from the receiving groove, etc., is appropriate.

[0044] Furthermore, the work status determination unit 315 can determine whether the work procedure is appropriate based on the captured image in this way, and can also determine whether the work quality is ensured. For example, if the captured image shows that there are almost no objects in the receiving groove, the work status determination unit 315 can determine that the work quality of the work of removing foreign objects from the receiving groove and the like is good.

[0045] Similarly, for the other tasks shown in FIG. 3, the task status determination unit 315 can determine the suitability of the task procedure and task quality by analyzing the combination of detection classes and the positions of the detection classes in the captured image.

[0046] In addition, whether or not safety is guaranteed may be determined, for example, based on the combination of detection classes at each successive point in time in the captured image and the position or change in position over time of a certain detection class in the captured image at each point in time.

[0047] Here, examples of accidents that should be prevented in each work include the following: (Accident 1) An ordinary person falls into a ditch. (Disaster 2) A work vehicle (e.g., a dump truck or backhoe) comes into contact with a worker or a member of the public. (Workers and members of the public are collectively referred to as "persons.") (Accident 3) When a pipe was being lifted, it fell and came into contact with a person. (Disaster 4) The contents of the bucket fell and came into contact with a person. (Disaster 5) Ditch collapse. (Disaster 6) A pipe placed at the scene came into contact with a person. (Disaster 7) Tools etc. fell into a ditch and came into contact with a person. (Accident 8) A worker falls from a ladder.

[0048] The work state determination unit 315 may determine whether or not the above-mentioned disasters (Disaster 1) to (Disaster 8) are likely to occur based on the combination of detection classes and the position or change in position over time of a certain detection class, as described above. Below is a processing example when it is determined that the above-mentioned disasters (Disaster 1) to (Disaster 8) are likely to occur.

[0049] When a ditch, a safety cone, and an ordinary person are extracted as the output result (detection class) of the indicator estimation model 321, the work state determination unit 315 may determine whether or not the above-mentioned disaster (Disaster 1) has occurred. For example, consider a case where the work state determination unit 315 detects an ordinary person for a predetermined time within a closed area formed by a curve connecting safety cones (or a curve surrounding the periphery of a ditch) in a captured image. In this case, the work state determination unit 315 determines that the above-mentioned disaster (Disaster 1) may occur and that safety is not guaranteed (i.e., a dangerous state). The work state determination unit 315 may detect the above-mentioned curve and ordinary person from the captured image after performing a bird's-eye view transformation on the captured image.

[0050] If a worker or a helmet is extracted as the detection class instead of a general person, the occurrence of the above-mentioned (Disaster 1) disaster may be determined. In this case, if a helmet is detected in the closed space for a predetermined period of time, the work state determination unit 315 may presume that a worker is present in the closed space and determine that a dangerous situation exists. Even if a groove is extracted as the detection class but a safety cone is not extracted, the work state determination unit 315 may determine that a dangerous situation exists in which the above-mentioned (Disaster 1) disaster may occur.

[0051] When a work vehicle and a person are extracted as detection classes, the work status determination unit 315 may determine whether or not the above-mentioned (Disaster 2) disaster has occurred. For example, when the distance between a work vehicle or a part of it (e.g., a bucket) and a person in a captured image is equal to or less than a predetermined value, the work status determination unit 315 may determine that a dangerous state exists in which the above-mentioned (Disaster 2) disaster may occur. The captured image may be acquired from a camera 2 installed on a work vehicle or a camera 2 installed at a location where plumbing work is being carried out. The work status determination unit 315 may also perform bird's-eye view transformation on the captured image and then detect the work vehicle, person, and the above-mentioned distance from the captured image. The work status determination unit 315 may also detect the direction of a person's face in the captured image, and when a work vehicle is not positioned in that direction, determine that a dangerous state exists in which the above-mentioned (Disaster 2) disaster may occur.

[0052] When a pipe and a sling belt are extracted as detection classes, the work status determination unit 315 may determine whether or not the above-mentioned (Disaster 3) disaster has occurred. For example, the work status determination unit 315 may detect the inclination of the pipe with respect to the ground in the captured image, and if the inclination is equal to or greater than a predetermined angle (e.g., 30°), determine that a dangerous state exists in which the above-mentioned (Disaster 3) disaster may occur. The work status determination unit 315 may also calculate the hoisting angle between two sling belts in the captured image, and if the hoisting angle is equal to or greater than a predetermined angle (e.g., 60°), determine that a dangerous state exists in which the above-mentioned (Disaster 3) disaster may occur. If the hoisting angle is equal to or greater than the predetermined angle, there is a risk of the pipe, which is the suspended load, skidding. The work status determination unit 315 may also determine that a dangerous state exists in which the above-mentioned (Disaster 3) disaster may occur even if only one sling belt is detected in the captured image.

[0053] When a bucket and a person are extracted as the detection class, the work status determination unit 315 may determine whether or not the above-mentioned disaster (Disaster 4) has occurred. For example, when a person is located under the bucket or within a predetermined range around the bucket in the captured image, the work status determination unit 315 may determine that a dangerous state exists in which the above-mentioned disaster (Disaster 4) may occur. Furthermore, the work status determination unit 315 may detect the relative positions of the bucket and the person from the captured image after performing a bird's-eye view transformation on the captured image. Furthermore, when the inclination of the opening surface of the bucket with respect to the ground in the captured image is equal to or greater than a predetermined angle, the work status determination unit 315 may determine that a dangerous state exists in which the above-mentioned disaster (Disaster 4) may occur.

[0054] When a ditch and an sheet pile are extracted as detection classes, the work status determination unit 315 may determine whether or not the above-mentioned disaster (Disaster 5) has occurred. For example, when the shape of a ditch in a photographed image is a predetermined shape (a shape that may collapse), the work status determination unit 315 may determine that the above-mentioned disaster (Disaster 5) is in a dangerous state where the above-mentioned disaster (Disaster 5) may occur. Furthermore, when the angle of the sheet pile driven into the ditch with respect to the ground in the photographed image is equal to or less than a predetermined angle (when the degree of tilting of the sheet pile is large), the work status determination unit 315 may determine that the above-mentioned disaster (Disaster 5) is in a dangerous state where the above-mentioned disaster (Disaster 5) may occur. Furthermore, when a rod that prevents the sheet pile from tilting cannot be detected in the photographed image, the work status determination unit 315 may determine that the above-mentioned disaster (Disaster 5) is in a dangerous state where the above-mentioned disaster (Disaster 5) may occur. Furthermore, even when a ditch is extracted as a detection class but an sheet pile is not extracted, the work status determination unit 315 may determine that the above-mentioned disaster (Disaster 5) is in a dangerous state where the above-mentioned disaster (Disaster 5) may occur.

[0055] When a pipe, a sleeper, and a ratchet are extracted as detection classes, the work status determination unit 315 may determine whether or not the above-mentioned disaster (Disaster 6) has occurred. The sleepers and ratchet are used to prevent the pipe from rolling. For example, when the position of a pipe does not change for a predetermined time in the captured image, the work status determination unit 315 may determine that the pipe is waiting to be joined. When the work status determination unit 315 determines that the positional relationship between the pipe waiting to be joined and the sleeper and ratchet is not a predetermined positional relationship in the captured image, the work status determination unit 315 may determine that a dangerous state exists in which the above-mentioned disaster (Disaster 6) may occur.

[0056] Also, consider a case where pipes are extracted as a detection class, and the pipes are located outside the trench (on the ground) for a predetermined time in the captured image, but sleepers or ratchets are not extracted as a detection class. In this case, the work state determination unit 315 cannot detect sleepers or ratchets required for the pipes placed on the ground in the captured image, and may determine that this is a dangerous state in which the above-mentioned disaster (Disaster 6) may occur.

[0057] When the outer periphery (line) of a ditch and a tool are extracted as detection classes, the work status determination unit 315 may determine whether or not the above-mentioned disaster (Disaster 7) has occurred. For example, when a tool is located on the outer periphery of a ditch in a captured image, the work status determination unit 315 may determine that a dangerous state exists in which the above-mentioned disaster (Disaster 7) may occur. Furthermore, the work status determination unit 315 may detect the outer periphery of the ditch and the tool from the captured image after performing bird's-eye view transformation on the captured image.

[0058] When a ladder, a ditch sidewall, and a worker are extracted as detection classes, the work state determination unit 315 may determine whether or not the above-mentioned disaster (Disaster 8) has occurred. For example, the work state determination unit 315 may calculate the angle between the ladder and the ditch sidewall on which the ladder is installed in the captured image, and when the angle is equal to or smaller than a predetermined angle, determine that a dangerous state exists in which the above-mentioned disaster (Disaster 8) may occur. Furthermore, when a worker is performing work other than climbing up and down the ladder in the captured image, the work state determination unit 315 may determine that a dangerous state exists in which the above-mentioned disaster (Disaster 8) may occur.

[0059] When detecting an object from a captured image, the work status determination unit 315 may create three-dimensional point cloud data of the captured image and detect the object from the three-dimensional point cloud data, or may detect the object by edge detection processing. Also, the work status determination unit 315 may measure the actual distance between objects using a distance measurement sensor (e.g., a TOF (Time of Flight) sensor).

[0060] The work state determination unit 315 may also determine the suitability of a work procedure using a procedure estimation model constructed to estimate the suitability of the work procedure using training data in which a combination of detection classes and the positions or position changes over time of the detection classes are associated with the suitability of the work procedure.The work state determination unit 315 may also determine the safety using a safety estimation model constructed to estimate the safety using training data in which a combination of detection classes and the positions or position changes over time of the detection classes are associated with the safety of the work.The work state determination unit 315 may also determine the suitability of the work procedure using a quality estimation model constructed to estimate the quality of the work using training data in which a combination of detection classes and the positions or position changes over time of the detection classes are associated with the quality of the work.The procedure estimation model, safety estimation model, and quality estimation model may be stored in the storage unit 32 and may be trained models realized, for example, by CNN.

[0061] Furthermore, whether or not a task is delayed may be determined by comparing the task time estimated by the task estimation unit 314 with task process data indicating the relationship between the tasks to be performed that day and the time periods during which the tasks are performed. The task status determination unit 315 may determine that a task is delayed when it determines that a predetermined time or more has passed since the scheduled task time for the task. The predetermined time may be set for each task based on the performance of the task, etc. Furthermore, the predetermined time may be set in stages, allowing the degree of delay to be determined in stages. The task process data may be stored in the memory unit 32.

[0062] The display control unit 318 controls the display device 4 to display various information on the display device 4. For example, the display control unit 318 may display the work status determination result described above in real time as soon as the determination result is obtained. By displaying the work status determination result, the display control unit 318 can function as a warning unit that warns the manager or worker when the work status determination unit 315 determines that the work status is negative.

[0063] Fig. 4 is a diagram showing an example of the display of the work status determination result by the work status determination unit 315. Reference numeral 401 in Fig. 4 is a diagram showing an example of a display of whether the work procedure is appropriate, whether safety is guaranteed, and whether there is a delay in the work. In the example of reference numeral 401, if the work status is appropriate, it is marked as "normal," and if not, it is marked as "abnormal." Reference numeral 402 in Fig. 4 is a diagram showing an example of a display of the results of determining for each site whether there is a delay in the work.

[0064] The example of reference numeral 401 indicates that the work procedures currently being carried out in a work process of joining pipes at a certain site are appropriate and that there are no delays in the work, but that safety is not guaranteed. In other words, this example warns that safety is not guaranteed. The example of reference numeral 402 indicates whether or not there are delays in the work, step by step, for each site. The example of reference numeral 402 indicates that there are no delays in the work at sites A and C, that there is a slight delay at site B, and that there is a clear delay at site D. In other words, this example warns that there are delays in the work at least at site D.

[0065] Furthermore, when the work state determination unit 315 determines that safety is not ensured in order to prevent the above-mentioned (Disaster 1) to (Disaster 8), the display control unit 318 may cause the display device 4 to display an image for calling the attention of the manager or worker. For example, the display control unit 318 may perform the following call of attention. Note that this call of attention may be realized by outputting a voice or a warning sound from a sound output device. (Disaster 1) Warning that safety cones have not been installed. (Disaster 2) Warning that there are work vehicles nearby. In the case of (Disaster 3): A warning that the pipe is tilted significantly, or that there is only one sling belt. In the case of (Disaster 4): A warning that a bucket is nearby, or that the bucket is tilting too much. In the case of (Disaster 5): A warning that the sheet piles have not been driven into the ditch (encouraging people to drive the sheet piles into the ditch). Or, a warning that the sheet piles driven into the ditch are tilted (not driven almost vertically). In the case of (Disaster 6): A warning that sleepers and ratchet stops have not been installed. In the case of (Disaster 7): A warning that tools may fall into the ditch. In the case of (Disaster 8): Warning that the ladder is unstable. Warning that the worker is performing work other than climbing up and down the ladder.

[0066] The work status determination unit 315 does not have to determine all of whether the work procedure is appropriate, whether the work quality is guaranteed, whether safety is guaranteed, and whether a delay has occurred in the work. The work status determination unit 315 may determine at least one of these work statuses. The display control unit 318 may display the determination result of the work status determined by the work status determination unit 315.

[0067] The data creation unit 316 analyzes the combinations of detection classes that appear at the respective times when captured images were acquired during all work hours on a certain day, which are output over time by the first output unit 312. By analyzing these combinations of detection classes, the data creation unit 316 creates performance data that indicates the relationship between at least one task performed during all work hours on a certain day and the time period during which that task was performed. This performance data indicates the work performance for one day (work performance during a specified work time), and is a combination of the performance of each task performed on that day.

[0068] In this embodiment, the activity estimation unit 314 analyzes the combination of detection classes to estimate the activities performed at the time of capturing the captured images. Therefore, the data creation unit 316 can create performance data using the estimation results of one day's worth of activities estimated by the activity estimation unit 314.

[0069] Furthermore, the data creation unit 316 may create procedure suitability data and risk occurrence data by analyzing the combinations of the above-mentioned detection classes. The procedure suitability data is data indicating the suitability of the work procedures for each work during all work hours in a day. The risk occurrence data is data indicating the occurrence of dangerous work during all work hours in a day.

[0070] In this embodiment, the work state determination unit 315 analyzes the combination of detection classes to determine the appropriateness of the work state of each work at the time of acquisition of the captured image. Therefore, the data creation unit 316 can create procedure appropriateness data using the determination results of the work state determination unit 315 on the appropriateness of the work procedures for each work task for one day. Furthermore, the data creation unit 316 can create risk occurrence presence / absence data using the determination results of the work state determination unit 315 on the safety of each work task for one day.

[0071] Furthermore, the data creation unit 316 may analyze the combination of detection classes to create performance data up to the time of capturing the captured image, data on the appropriateness of procedures, and data on the presence or absence of a risk. In this case, the data creation unit 316 can create these data in real time.

[0072] In this embodiment, the data creation unit 316 may use the work estimation result by the work estimation unit 314 to create performance data indicating the relationship between the work estimated to have been performed up to the time point when the photographed image was acquired and the time period in which the work was performed. Furthermore, the data creation unit 316 may use the determination result of the suitability of the work status for each work by the work status determination unit 315 to create procedure suitability data indicating the suitability of the work procedures for each work up to the time point when the photographed image was acquired. Furthermore, the data creation unit 316 may use, for example, the determination result of the suitability of the work status to create risk occurrence presence / absence data indicating the suitability of the occurrence of dangerous work up to the time point when the photographed image was acquired.

[0073] As described above, the work status determination unit 315 can also determine whether work quality is ensured based on the captured images. Therefore, the data creation unit 316 may create quality data indicating whether the work quality of each work task for one day or up to the time the captured images were acquired by analyzing the combination of detection classes, just like the procedure suitability data and risk occurrence data.

[0074] The display control unit 318 may, for example, cause the performance data created by the data creation unit 316 to be displayed in real time on the display device 4. Fig. 5 is a diagram showing an example of displaying performance data. In Fig. 5, the performance time of a work process (e.g., asphalt cutting) including at least one work is shown as the work performance time. The same applies to Fig. 6.

[0075] In the example of Figure 5, work process data indicating the relationship between the work to be performed that day and the time period in which the work will be performed ("Work Schedule" in Figure 5) is shown as an implementation schedule, along with performance data ("Work Results" in Figure 5) created by the data creation unit 316 up to the time the photographed image was acquired (time T1).

[0076] Furthermore, the display control unit 318 may cause the display device 4 to display information based on the procedure suitability data and the risk occurrence data in real time, as shown in Fig. 5. The example in Fig. 5 shows that a dangerous task occurred in time period Ta based on the risk occurrence data, and that a task performed using an incorrect procedure occurred in time period Tb based on the procedure suitability data. Furthermore, a comparison with the work process data shows that the task was delayed in time periods Tc and Td. Note that if a task with poor quality is identified in the quality pass / fail data, the display control unit 318 may indicate that the task occurred in the performance data in Fig. 5.

[0077] In this way, the display control unit 318 displays performance data and the like on the display device 4, allowing the manager to grasp the progress and status of the work in real time. Furthermore, if the work status is determined to be unsafe, the manager can give instructions or guidance to the worker while visually checking the captured images acquired by the image acquisition unit 311. Furthermore, if it is determined that the status is not safe, the manager can also visually check the captured images that show dangerous work.

[0078] When a day's worth of work is completed, the display control unit 318 can display the day's performance data on the display device 4 as a performance process chart as shown in Fig. 5. By displaying the day's performance data together with the work process data, the display control unit 318 can display delayed work and the time periods during which they occurred. The display control unit 318 can also display, together with the day's performance data, time periods during which dangerous work occurred, time periods during which work was performed using incorrect procedures, and time periods during which work of poor quality occurred.

[0079] Furthermore, the display control unit 318 may display procedure suitability data, risk occurrence data, and quality pass / fail data separately from the performance data. In this case, the display control unit 318 may display the procedure suitability data in a table format, such as a work procedure chart, risk occurrence data in a dangerous work chart, and quality pass / fail data in a quality control chart. In this case, the display control unit 318 can notify the manager or worker of work that was performed using an incorrect procedure, work that involved dangerous work, and work that is of poor quality.

[0080] The data creation unit 316 does not have to create all of the procedure suitability data, risk occurrence data, and quality pass / fail data. The data creation unit 316 may create at least one of the procedure suitability data, risk occurrence data, and quality pass / fail data. The display control unit 318 only needs to display the data created by the data creation unit 316.

[0081] The second output unit 317 inputs the one-day performance data created by the data creation unit 316 to the task estimation model 322 stored in the storage unit 32. The task estimation model 322 is a trained model constructed using machine learning to estimate tasks to be performed during all task hours on the following day based on the performance of tasks during all task hours on a given day.

[0082] The task estimation model 322 is a trained model constructed so that, as a result of inputting one day's worth of performance data, it outputs task schedule data indicating the relationship between at least one task that is estimated to be performed during all work hours on the following day and the time period in which that task will be performed. The task estimation model 322 may be, for example, a neural network (e.g., CNN) including at least an input layer, an intermediate layer, and an output layer.

[0083] In the work estimation model 322, the input layer is a layer to which one day's worth of performance data is input. The middle layer may be a layer in which parameters are trained based on past work schedule data, past performance data, and factor data. The past work schedule data is data indicating the relationship between at least one work scheduled to be performed in a past plumbing work construction plan and the time period during which the work was performed. The past performance data is data indicating the relationship between at least one work performed in a past plumbing work construction project and the time period during which the work was performed. The factor data is data that affected the work in the past plumbing work construction project. Furthermore, when one day's worth of performance data is input to the input layer, the output layer causes the computer to function so that, after calculations by the middle layer, the work schedule data is output from the output layer. That is, the second output unit 317 outputs the work schedule data by inputting captured images to the work estimation model 322.

[0084] For example, multiple pieces of the above-mentioned past work schedule data, past performance data, and factor data are prepared as training data. Examples of information included in the factor data include the following information. (1) Information indicating day or night. (2) Weather information (e.g., sunny, cloudy, rainy, or foggy). (3) Information about the trench (e.g., trench width, trench depth, soil cover, offset value (distance from the sidewalk to the location where the pipe is laid), or information indicating the presence or absence of spring water, etc.). (4) Information about the soil (e.g., information indicating the hardness of the soil, whether excavated soil will be used for backfilling, or the thickness of the asphalt pavement). (5) Road information (e.g., information indicating the number of lanes, the presence or absence of sidewalks, the slope angle, whether the road is closed, or the width of the work zone). (6) Pipe information (e.g., pipe type, nominal diameter, number of straight / irregular pipes, pipe alignment, presence or absence of buried objects, location of buried objects, or information indicating whether buried objects need to be removed, etc.). (7) Information on the work machinery (e.g., information indicating the model, horsepower, number of units, or bucket model number, etc.) (8) Information on the work content (e.g., information indicating the track record of manual digging or excavation using a backhoe, or the track record of work other than excavation, etc.). (9) Tool information (e.g., information indicating the model number of a lever hoist, etc.). (10) Information about workers (e.g., number of workers, age, gender, or years of work experience). (11) Information on the surrounding environment (information indicating the relative positions of the material storage area and the construction site, etc.).

[0085] The past performance data may include, for example, information indicating the past performance values ​​of work hours for plumbing work performed by a specific contractor performing the work to be estimated. The past performance data may also include information indicating the past performance values ​​of work hours for plumbing work performed nationwide by contractors other than the specific contractor. The factor data may also include the results of analysis of incorrect work procedures, dangerous work, and causes of dangerous work that occurred in these plumbing works (information based on past accident cases). When analyzing sling belts, the analysis results may include information such as the number of sling belts, the lifting angle, and the attachment position relative to the pipe.

[0086] The past work schedule data may be, for example, work schedule data including work procedures that were indicated as work to be carried out in past construction plans for plumbing work by the specific contractor.

[0087] The intermediate layer of the work estimation model 322 may further be a layer in which parameters are learned based on work change-related data and past progress data as training data. The work change-related data is data that indicates the relationship between the results of a change in the work process order when the work process order is changed in past plumbing work and the change in work time before and after the change in the work process order. In other words, the work change-related data is data that indicates how work time has changed (for example, how much it has been reduced) as a result of reviewing the work process order. The past progress data is data that indicates the progress rate (progress rate) of work within a specified work time relative to the entire work of past plumbing work, associated with past performance data. The progress rate can also be referred to as the achievement rate.

[0088] In this case, when one day's worth of performance data is input to the input layer, the middle layer of the work estimation model 322 executes the following estimation process as part of the above calculation. That is, the middle layer of the work estimation model 322 estimates the progress rate of the work indicated by the input performance data relative to the overall plumbing work, based on the progress rate of the work indicated by the past progress data. In terms of estimating this progress rate, the work progress rate associated with the past performance data can be defined as correct data.

[0089] The middle layer estimates the work process sequence for the entire work time of the next day based on the estimated progress rate and the work change related data. For example, if the estimated progress rate is lower than the planned progress rate, the middle layer can execute a calculation to create work schedule data including a work process sequence that can improve the progress rate, taking into account the relationship between changes in the work process sequence and changes in work time in past plumbing work projects. Note that the estimated progress rate being lower than the planned progress rate means that the work is behind schedule.

[0090] In this way, the task estimation model 322 constructed using past progress data and task change-related data as training data can output task schedule data that revises the tasks for the next day based on the progress of the tasks on a given day.

[0091] Of the past performance data described above, only data with a progress rate equal to or greater than a predetermined threshold may be used as training data. In this case, past performance data with a low progress rate is not used as training data, allowing the task estimation model 322 to output task schedule data in which a decrease in the progress rate is suppressed. Therefore, in this case, the task estimation model 322 can output task schedule data with higher reliability.

[0092] Furthermore, the second output unit 317 may input procedure suitability data into the work estimation model 322, thereby identifying in the work schedule data any work that is estimated to have a possibility of error in the work procedure throughout all work hours on the following day. Furthermore, the second output unit 317 may input risk occurrence data into the work estimation model 322, thereby identifying in the work schedule data any work that is estimated to have a possibility of dangerous work throughout all work hours on the following day. Furthermore, the second output unit 317 may input quality suitability data into the work estimation model 322, thereby identifying in the work schedule data any work that is estimated to have poor quality throughout all work hours on the following day.

[0093] For example, if the work schedule data contains an identical task that has been identified as having an error in the work procedure in the procedure suitability data, the work estimation model 322 identifies that task in the work schedule data. Similarly, if the work schedule data contains an identical task that has been identified as having a dangerous task in the risk occurrence data, the work estimation model 322 identifies that task in the work schedule data. Furthermore, if the work schedule data contains an identical task that has been identified as having poor quality in the quality pass / fail data, the work estimation model 322 identifies that task in the work schedule data.

[0094] The display control unit 318 may, for example, cause the work schedule data output by the second output unit 317 to be displayed on the display device 4. FIG. 6 is a diagram showing an example of how work schedule data is displayed. In the example of FIG. 6, the work schedule data estimated to be optimal as the next day's work schedule output by the second output unit 317 is shown as an implementation schedule for the next day. The example of FIG. 6 shows that a dangerous task may occur in time period Tp based on the risk occurrence data, and that an error in the work procedure may occur in time period Tq based on the procedure suitability data. The example of FIG. 6 also shows that the quality may be poor in time period Tr based on the quality pass / fail data.

[0095] The display control unit 318 may display, separately from the scheduled work data, the work that the second output unit 317 estimates may be dangerous, the work that the second output unit 317 estimates may have an error in the work procedure, and the work that the second output unit 317 estimates may have poor quality. In this case, the display control unit 318 can notify the manager or worker of these works.

[0096] Then, as described above, by displaying the work schedule data, which is the output result of the second output unit 317, on the display device 4, the display device 4 can function as a suggestion device that suggests revising the work schedule for the next day.

[0097] <Example of using the work status judgment results> FIG. 7 is a diagram showing an example of the relationship between the determination results by the work status determination unit 315 and the notification content by the display control unit 318. As described above, the work status determination unit 315 outputs the determination results for the work procedure, work quality, safety, and work time (presence or absence of delay) for each work. As shown in FIG. 7, the notification content can be determined based on a combination of the determination results for the work procedure, work quality, safety, and work time for each work. In the "Work Time" column in FIG. 7, "Short" indicates that the time required for the work is shorter than the time indicated in the work process data, and "Long" indicates that the time required for the work is longer than the time indicated in the work process data. Furthermore, "Standard" indicates that the time required for the work is approximately equal to the time indicated in the work process data.

[0098] For example, as shown in Fig. 7, consider a case where the work status determination unit 315 determines that the work procedure is appropriate, the work quality is good, and the work is safe, but the work time is determined to be shorter than planned. In this case, it is possible to suggest that the work time from the next day onwards be made shorter than the planned work time and that the workers be allocated to other work sites. On the other hand, if it is determined that the work time is longer than planned, it is possible to suggest that the work time from the next day onwards be made longer than the current work time and that the workers be added.

[0099] 7, for work that the work status determination unit 315 determines to have an error in the work procedure or to have poor work quality, the unit can instruct the user to review the work. For work that the work status determination unit 315 determines to be dangerous, the unit can issue a warning about the dangerous work and suggest ways to improve it. Even when the work status determination unit 315 makes these determinations, if it determines that the work time is shorter or longer than planned, the unit may make the above-mentioned suggestions regarding the work time.

[0100] The display control unit 318 may cause the display device 4 to display notification content corresponding to the determination result of the work status determination unit 315. In this case, too, the display device 4 may function as a suggestion device that proposes revising the work schedule for the next day. For example, the display control unit 318 may cause the display device 4 to display improvement methods associated with each dangerous task. Furthermore, the second output unit 317 outputs work schedule data and the like as a result of inputting performance data and the like into the work estimation model 322. Therefore, the display control unit 318 may make a suggestion regarding the length of work time by displaying the work schedule data output by the second output unit 317 that reflects the length of work time for that day. Furthermore, the display control unit 318 may issue a warning about dangerous work by displaying the work schedule data that identifies the dangerous work and that is output by the second output unit 317. The display control unit 318 may also cause the display device 4 to display the notification content in real time when the determination result by the work status determination unit 315 is obtained.

[0101] <Example of output from the second output section> For example, the second output unit 317 may create a dangerous work list with captured images based on the results of the analysis of the history of dangerous work and the causes of the dangerous work, as well as images taken when the dangerous work occurred. This dangerous work list can be used to assist in creating a safety management form. The second output unit 317 may also create work procedure data (work procedure documents) based on the results of the analysis of the past performance data, the history of incorrect work procedures, the history of dangerous work, and the causes of the dangerous work. The second output unit 317 may create the work procedure data in a format that allows reference to past work procedures, for example.

[0102] Furthermore, the task estimation model 322 may be constructed so as to estimate tasks to be performed in the next specified task time based on the task results for a specified task time. For example, the task estimation model 322 is not limited to outputting task schedule data for the next day, but may also output task schedule data for the day after that. Furthermore, the task estimation model 322 may receive input of performance data up to the time of acquisition of the captured image, and output task schedule data thereafter.

[0103] Furthermore, it is sufficient if at least one of the procedure suitability data, risk occurrence data, and quality data is input to the task estimation model 322, rather than all of these data. That is, the second output unit 317 only needs to output at least one of tasks estimated to be potentially dangerous, tasks estimated to have a potential error in the task procedure, and tasks estimated to have poor quality. The display control unit 318 only needs to display the tasks output by the second output unit 317.

[0104] [Processing in the work management device] Fig. 8 is a flowchart showing an example of processing (work management method) in the work management device 3. As shown in Fig. 8, the image acquisition unit 311 acquires a captured image from the camera 2 (S1: image acquisition step). The first output unit 312 inputs the captured image acquired by the image acquisition unit 311 to the index estimation model 321 (S2). The first output unit 312 outputs the estimated detection class as an output result of the index estimation model 321 (S3: output step, first output step).

[0105] The position identification unit 313 estimates the position of the detection class output by the first output unit 312 in the captured image (S4). The work estimation unit 314 analyzes the combination of detection classes output over time by the first output unit 312 to estimate the work being performed at the time the captured image was acquired (i.e., the work currently being performed) at the position of the detection class identified by the position identification unit 313 (S5: work estimation step). When the work estimation unit 314 determines that the captured image includes scenes of multiple plumbing works being performed based on the positions of the detection classes, it estimates the work being performed for each plumbing work.

[0106] The work state determination unit 315 determines the work state of the work estimated by the work estimation unit 314 by analyzing the combinations of detection classes output over time by the first output unit 312 (S6). The display control unit 318 causes the display device 4 to display the work state determination result by the work state determination unit 315 (S7).

[0107] Furthermore, the data creation unit 316 creates performance data (performance process chart) up to the time point when the most recent captured image was acquired by analyzing the combinations of detection classes output over time by the first output unit 312 (S8). The display control unit 318 displays the performance data created by the data creation unit 316 on the display device 4 in real time (S9). Note that in S8, the data creation unit 316 may create procedure suitability data (e.g., a work procedure chart), data on whether or not a hazard has occurred (e.g., a hazardous work chart), and quality pass / fail data (e.g., a quality control chart) up to the time point when the most recent captured image was acquired. In this case, the display control unit 318 may display the procedure suitability data, hazard occurrence data, and quality pass / fail data on the display device 4.

[0108] The control unit 31 determines whether or not the work for one day has been completed (S10). The control unit 31 may determine that the work for one day has been completed, for example, when the transmission of captured images from the camera 2 has ceased for a predetermined period of time or when a user input has been received.

[0109] If the control unit 31 determines that one day's worth of work has been completed (YES in S10), the data creation unit 316 creates one day's worth of performance data (e.g., an implementation schedule) (S11: data creation step). The second output unit 317 inputs one day's performance data created by the data creation unit 316 to the work estimation model 322 (S12). The second output unit 317 outputs the next day's work schedule data, which estimates the work to be performed on the next day and the time period during which the work will be performed, as the output result of the work estimation model 322 (S13: second output step). The display control unit 318 displays the next day's work schedule data output by the second output unit 317 on the display device 4 (S14). On the other hand, if the control unit 31 determines that one day's worth of work has not been completed (NO in S10), the process returns to S1.

[0110] In S11, the data creation unit 316 may create one day's worth of procedure suitability data (e.g., a work procedure chart), risk occurrence data (e.g., a dangerous work chart), and quality pass / fail data (e.g., a quality control chart). In this case, the second output unit 317 may input these data into the work estimation model 322 to identify, in the work schedule data, work that is estimated to have a possible error in the work procedure, work that is estimated to have a possible dangerous work, and work that is estimated to have poor quality. The display control unit 318 may then display the work schedule data in which these work tasks are identified on the display device 4. The display control unit 318 may display the above-mentioned work tasks estimated by the work estimation model 322 on the display device 4, separately from the work schedule data.

[0111] The display control unit 318 may also display on the display device 4 the performance data, procedure suitability data, risk occurrence data, and quality pass / fail data for one day created in S11.

[0112] [Modification of the work management device] In this embodiment, the work management device 3 has been described as outputting the work status determination result and also outputting the work schedule data. However, the work management device 3 may have only the first function of outputting the work status determination result, or may have only the second function of outputting the work schedule data.

[0113] The work management device 3 that realizes the first function may include an image acquisition unit 311, a first output unit 312 that outputs the detection class estimated by the index estimation model 321, and a work estimation unit 314 that estimates the work currently being performed based on the detection class.

[0114] Furthermore, the work management device 3 that realizes the second function may include the following components in addition to the image acquisition unit 311 and the first output unit 312. The work management device 3 may include a data creation unit 316 that creates performance data by analyzing the detection class output by the first output unit 312, and a second output unit 317 that outputs work schedule data by inputting the performance data into the work estimation model 322.

[0115] [Major effects] (1) The work management device 3 having the first function can automatically estimate the work currently being performed simply by acquiring a photographed image. This allows a manager or worker to easily manage the work currently being performed.

[0116] There are uncertainties in plumbing work, such as planned work being suddenly changed on the day, but the work management device 3 allows managers or workers to manage the work currently being carried out, so they can also keep track of work that has been suddenly changed.

[0117] Furthermore, the work management device 3 only needs to be able to acquire images captured by the camera 2, and does not necessarily need to be installed at the plumbing work site or implemented by electronic equipment carried by a worker. The same applies to the display device 4 that displays the output results of the work management device 3. Therefore, a manager such as a site supervisor who manages the plumbing work can remotely grasp the work currently being performed, as estimated by the work management device 3, even if they are not at the site.

[0118] The work management device 3 also includes a work status determination unit 315 that determines whether the estimated work status is appropriate by analyzing the detection class. This allows the manager or worker to understand whether the current work status is appropriate. The manager's management items in daily plumbing work are diverse (e.g., whether the work procedures are appropriate, whether the work quality is good or bad, safety, and whether there are any delays in work time). The work management device 3 determines whether the work status indicated by these management items is appropriate, allowing the manager or worker to easily manage multiple management items.

[0119] Furthermore, in plumbing work, checking the progress and details of daily work (e.g., checking safety) requires time and experience. The work management device 3 can estimate the appropriateness of the work being performed and its status simply by acquiring photographed images, so managers or workers can easily check the progress and details of daily work without relying on experience.

[0120] Furthermore, by acquiring photographed images from each of a plurality of plumbing work sites, the work management device 3 can estimate the work being performed and the appropriateness of the work status for each site. In the future, it is expected that labor shortages will lead to an increase in cases where one manager will be required to manage multiple sites simultaneously, but the work management device 3 allows managers or workers to easily and simultaneously manage the work being performed at multiple sites and the appropriateness of the work status.

[0121] (2) The work management device 3 having the second function can automatically create work schedule data for the next day simply by acquiring photographed images. Therefore, the work management device 3 can easily increase the likelihood that the work for the next day will proceed as planned.

[0122] Furthermore, in plumbing work, reviewing the work schedule according to the work situation requires time and experience. The work management device 3 automatically outputs the work schedule data for the next day simply by acquiring photographed images, so the manager can easily improve work efficiency without relying on experience.

[0123] [Software implementation example] The control block (particularly the control unit 31) of the work management device 3 may be realized by a logic circuit (hardware) formed on an integrated circuit (IC chip) or the like, or may be realized by software.

[0124] In the latter case, the work management device 3 includes a computer that executes instructions from a program, which is software that realizes each function. This computer includes, for example, one or more processors and a computer-readable recording medium storing the program. The object of the present invention is achieved when the processor in the computer reads and executes the program from the recording medium. The processor may be, for example, a central processing unit (CPU). The recording medium may be a "non-transitory tangible medium," such as a read-only memory (ROM), tape, disk, card, semiconductor memory, or programmable logic circuit. The device may also include a random access memory (RAM) for loading the program. The program may be supplied to the computer via any transmission medium capable of transmitting the program (such as a communication network or broadcast waves). One aspect of the present invention may also be realized in the form of a data signal embedded in a carrier wave, in which the program is embodied by electronic transmission.

[0125] [Other Descriptions of the Disclosure] A work management device according to one embodiment of the present invention comprises an image acquisition unit that acquires a photographed image including a work area in plumbing work; a first output unit that outputs at least one indicator that is estimated to have appeared in the work at the time the photographed image is acquired by inputting the photographed image into an indicator estimation model constructed using machine learning to estimate at least one indicator that is appearing in the work at a certain point in time from among multiple indicators for evaluating the work status in the plumbing work; and a work estimation unit that estimates the work being performed at the time the photographed image is acquired based on the output result of the first output unit.

[0126] A work management method according to one embodiment of the present invention includes an image acquisition process for acquiring a photographed image including a work area in plumbing work; an output process for outputting at least one indicator estimated to have appeared in the work at the time the photographed image was acquired by inputting the photographed image into an indicator estimation model constructed using machine learning to estimate at least one indicator that appears in the work at a certain point in time from among multiple indicators for evaluating the work status in the plumbing work; and a work estimation process for estimating the work being performed at the time the photographed image was acquired based on the output result of the output process.

[0127] According to the above configuration, by inputting a photographed image into the index estimation model, it is possible to estimate the combination of indices at the time the photographed image was acquired. Therefore, based on the estimated combination of indices, it is possible to estimate the work being performed at the time the photographed image was acquired. Therefore, those involved in the plumbing work can manage the work being performed.

[0128] In a work management device according to one aspect of the present invention, the work estimation unit may estimate the work being performed at the time when the most recent captured image was acquired by the image acquisition unit by analyzing a combination of indicators that appear at each of the time when the captured image was acquired, which are output over time by the first output unit.

[0129] According to the above configuration, the task currently being performed can be estimated by analyzing a combination of indices estimated from the current and immediately previous captured images.

[0130] A work management device according to one embodiment of the present invention may include a work state determination unit that determines the appropriateness of the work state of the work estimated by the work estimation unit by analyzing the combination of indicators that appear at each of the times when the captured images are acquired, which are output over time by the first output unit.

[0131] According to the above configuration, the appropriateness of the current work status can be determined by analyzing a combination of indices estimated from the current and immediately previous captured images. Therefore, those involved in the plumbing work can understand the appropriateness of the current work status.

[0132] In a work management device according to one embodiment of the present invention, the work status determination unit may determine the appropriateness of the work status by determining at least one of whether the work procedure is appropriate, whether safety is guaranteed, and whether there is a delay in the work.

[0133] According to the above configuration, it is possible to determine the current work status, such as whether the work procedures are appropriate, whether safety is ensured, and / or whether the work is delayed. Therefore, those involved in the plumbing work can grasp the appropriateness of the work procedures, safety, and / or the progress of the work.

[0134] The work management device according to one aspect of the present invention may include a warning unit that warns people involved in the plumbing work when the work status determination unit determines that the work status is negative.

[0135] According to the above configuration, it is possible to warn people involved in the plumbing work that the current work conditions are undesirable.

[0136] A work management device according to one embodiment of the present invention includes a position identification unit that identifies the position of the indicator output by the first output unit in the captured image, and when the plumbing work is being carried out in different areas within the captured image, the work estimation unit may estimate the work being carried out in each of the different areas based on the position of the indicator identified by the position identification unit.

[0137] According to the above configuration, when a captured image includes a scene in which a plurality of tasks are being performed, each of the plurality of tasks can be estimated from the captured image.

[0138] A work management device according to one embodiment of the present invention may include a data creation unit that creates performance data indicating the relationship between at least one work performed during a specified work time and the time period in which the work was performed by analyzing the combinations of indicators that appear at each of the times when the captured images were acquired within the specified work time, as output over time by the first output unit.

[0139] According to the above configuration, the user can grasp the performance (progress) of work performed during a predetermined work time (for example, a work time set per day (for example, 8:00 to 16:30)).

[0140] In one aspect of the work management device of the present invention, the data creation unit may create at least one of procedure suitability data indicating the suitability of work procedures during a specified work time, and hazard occurrence data indicating whether dangerous work has occurred during the specified work time, by analyzing the combinations of indicators that appear at each of the points in time at which the captured images are acquired within a specified work time, which are output over time by the first output unit.

[0141] According to the above configuration, those involved in the plumbing work can understand whether the work procedures for the work carried out during a specified work period are appropriate and / or whether any dangerous work has occurred.

[0142] A work management device according to one embodiment of the present invention comprises an image acquisition unit that acquires a photographed image including a work area in plumbing work; a first output unit that outputs at least one indicator that is estimated to have appeared in the work at the time the photographed image is acquired by inputting the photographed image into an indicator estimation model constructed using machine learning to estimate at least one indicator that appears in the work at a certain point in time from among multiple indicators for evaluating the work status in the plumbing work; a data creation unit that creates actual data indicating the relationship between at least one work performed during a specified work time and the time period in which the work is performed by analyzing the combinations of indicators that appear at each of the times the photographed images are acquired within the specified work time, as output over time by the first output unit; and a second output unit that outputs work schedule data indicating the relationship between at least one work estimated to be performed in the next specified work time and the time period in which the work is performed by inputting the actual data into a work estimation model constructed using machine learning to estimate the work to be performed in the next specified work time from the actual work performance during the specified work time.

[0143] A work management method according to one embodiment of the present invention includes an image acquisition process for acquiring a photographed image including a work area in plumbing work; a first output process for inputting the photographed image into an index estimation model constructed using machine learning to estimate at least one index appearing in the work at a certain point in time from among multiple indexes for evaluating the work status in the plumbing work, thereby outputting at least one index estimated to have appeared in the work at the time the photographed image was acquired; a data creation process for creating performance data indicating the relationship between at least one work performed during a specified work time and the time period during which the work was performed, by analyzing the combinations of indexes that appear at each of the times the photographed images were acquired within the specified work time, which are output over time in the first output process; and a second output process for inputting the performance data into a work estimation model constructed using machine learning to estimate the work to be performed in the next specified work time from the performance of the work during the specified work time, thereby outputting work schedule data indicating the relationship between at least one work estimated to be performed in the next specified work time and the time period during which the work was performed.

[0144] According to the above configuration, work schedule data for the next specified work time (e.g., work time specified for the next day (e.g., 8:00-16:30)) can be created taking into account the work results for the current specified work time. This increases the likelihood that work will proceed as planned in the next specified work time. In addition, outputting the work schedule data can suggest revisions to the work schedule for the next specified work time.

[0145] In one aspect of the work management device of the present invention, the data creation unit creates at least one of procedure suitability data indicating the suitability of work procedures during a specified work time and hazard occurrence data indicating whether dangerous work will occur during the specified work time by analyzing the combinations of indicators that appear at each of the time points at which the captured images are acquired within a specified work time, as output over time by the first output unit, and the second output unit may (1) input the procedure suitability data into the work estimation model to identify in the work schedule data work that is estimated to have a possibility of an error in the work procedures during the next specified work time, or (2) input the hazard occurrence data into the work estimation model to identify in the work schedule data work that is estimated to have a possibility of dangerous work during the next specified work time, or (3) identify both work that is estimated to have a possibility of an error in the work procedures in (1) and work that is estimated to have a possibility of dangerous work during (2) in the work schedule data.

[0146] According to the above configuration, work that may result in an error in the work procedure at the next specified work time and / or dangerous work that may occur at the next specified work time can be reflected in the work schedule data. Therefore, by checking the work schedule data, people involved in plumbing work can know in advance work that may result in an error in the work procedure and / or dangerous work that may occur.

[0147] An index estimation model according to one embodiment of the present invention comprises an input layer to which a photographed image including a work area in plumbing work is input, an output layer that outputs at least one of a plurality of indices for evaluating the work status in the plumbing work, and an intermediate layer in which parameters are learned based on an image including the work area and an image showing items used in the plumbing work, and the index. When the photographed image is input to the input layer, the computer is caused to function such that, after calculations by the intermediate layer, the output layer outputs at least one index that is estimated to have appeared in the work at the time the photographed image was obtained.

[0148] According to the above configuration, by inputting a photographed image into the index estimation model, it is possible to estimate the combination of indices at the time the photographed image was acquired. Therefore, it is possible to estimate the work being performed at the time the photographed image was acquired based on the estimated combination of indices.

[0149] A work estimation model according to one embodiment of the present invention comprises an input layer to which actual data indicating the relationship between at least one work performed during a specified work time and the time period during which that work is performed is input; an output layer to output work schedule data indicating the relationship between at least one work performed during the next specified work time and the time period during which that work is performed; and an intermediate layer in which parameters are learned based on past work schedule data indicating the relationship between at least one work scheduled to be performed in a construction plan for past plumbing work and the time period during which that work is performed, past actual data indicating the relationship between at least one work performed in the past plumbing work and the time period during which that work is performed, and factor data that influenced the work in the past plumbing work.When the actual data is input to the input layer, the computer is caused to function so that the work schedule data is output from the output layer after calculations by the intermediate layer.

[0150] With the above configuration, work schedule data for the next specified work time can be created taking into account the work results for the current specified work time. Therefore, it is possible to increase the possibility that the work will proceed as planned in the next specified work time. In addition, by outputting the work schedule data, it is possible to suggest a review of the work schedule for the next specified work time.

[0151] In a work estimation model according to one embodiment of the present invention, the intermediate layer further learns the parameters based on work change-related data indicating the relationship between the results of changing the work process order when the work process order is changed in the past plumbing work and the change in work time before and after the change in the work process order, and past progress data corresponding to the past performance data indicating the progress of work at the specified work time for the entire work of the past plumbing work, and when the performance data is input to the input layer, the intermediate layer may, as part of the calculation, estimate the progress of the work indicated by the performance data with respect to the entire plumbing work, and estimate the work process order for the next specified work time based on the estimated progress and the work change-related data.

[0152] According to the above configuration, it is possible to output work schedule data in which the work for the next predetermined work time has been revised in accordance with the progress of the work during the current predetermined work time.

[0153] [Additional Notes] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. [Explanation of symbols]

[0154] 3 Work management device 311 Image Acquisition Unit 312 First output section 313 Location identification part 314 Work Estimation Department 315 Working condition determination unit 316 Data Creation Department 318 Display control unit (warning unit) 317 Second Output Section 321 Index Estimation Model 322 Work Estimation Model

Claims

1. an image acquisition unit that acquires a photographed image including a work area in plumbing work; a first output unit that outputs at least one index that is estimated to have appeared in the work at the time the photographed image is acquired by inputting the photographed image into an index estimation model that is constructed using machine learning to estimate at least one index that is appearing in the work at a certain time out of a plurality of indexes that indicate each of a plurality of things involved in the work in the waterworks work or a combination thereof; a data creation unit that creates performance data indicating the relationship between at least one task performed during the specified work period and the time period in which the task was performed by identifying the task performed at a certain time within the specified work period that is the time of acquisition of the photographed image, based on data prepared in advance that associates each task that can be performed during the waterworks work with a combination of the indicators, and the combinations of indicators that appear at each time point when the photographed image is acquired within the specified work period, which are output over time by the first output unit; a second output unit that outputs work schedule data indicating a relationship between at least one work that is estimated to be performed in the next predetermined work time and a time zone in which the work is to be performed by inputting the performance data into a work estimation model that is constructed using machine learning to estimate a work that should be performed in a next predetermined work time from the performance data of the work in the next predetermined work time, The index estimation model is constructed by performing machine learning using an article image representing an article used in the plumbing work and the index as training data, The work estimation model is constructed by performing machine learning using as training data past work schedule data showing the relationship between at least one work scheduled to be performed in a construction plan for past plumbing work and the time period during which the work was performed, past performance data showing the relationship between at least one work performed in the past plumbing work and the time period during which the work was performed, and factor data that affected the work in the past plumbing work, a work management device.

2. The work management device described in claim 1 is provided with a work estimation unit that estimates the work being performed at the time the captured image is acquired based on data that corresponds each work that can be performed in the pre-prepared plumbing work with a combination of the indicators and the output result of the first output unit.

3. a work state determination unit that determines whether or not a delay is occurring in the work as a measure of the suitability of the work state estimated by the work estimation unit by analyzing a combination of indices that appear at each time point when the photographed images are acquired, the combination being output over time by the first output unit; 3. The work management device according to claim 2, wherein the work status determination unit determines whether or not a delay has occurred in the work by comparing the work time of the work estimated by the work estimation unit with work process data indicating a relationship between the work to be performed at a predetermined work time and a time period in which the work is performed.

4. The work management device according to claim 3 , further comprising a warning unit that warns a person involved in the plumbing work when the work status determination unit determines that the work status is not good.

5. an image acquisition step of acquiring a photographed image including a work area in plumbing work; a first output step of inputting the photographed image into an index estimation model constructed using machine learning to estimate at least one index appearing in the work at a certain point in time from among a plurality of indexes indicating each of a plurality of things involved in the work in the waterworks work or a combination thereof, and outputting at least one index that is estimated to have appeared in the work at the time the photographed image was acquired; a data creation process for creating performance data showing the relationship between at least one task performed during the specified work period and the time period in which the task was performed by identifying the task performed at a certain time within the specified work period that is the time of acquisition of the photographed image, based on data prepared in advance that associates each task that can be performed during the waterworks work with a combination of the indicators, and the combinations of indicators that appear at each time point when the photographed image is acquired within the specified work period, which are output over time in the first output process; a second output step of inputting the performance data into a task estimation model constructed using machine learning to estimate tasks to be performed in the next predetermined task time from the performance data of the tasks in the next predetermined task time, and outputting task schedule data indicating a relationship between at least one task estimated to be performed in the next predetermined task time and a time zone for performing the task; The index estimation model is constructed by performing machine learning using an article image representing an article used in the plumbing work and the index as training data, A work management method in which the work estimation model is constructed by performing machine learning using as training data past work schedule data showing the relationship between at least one work scheduled to be performed in a construction plan for past waterworks work and the time period during which the work is performed, past performance data showing the relationship between at least one work performed in the past waterworks work and the time period during which the work is performed, and factor data that influenced the work in the past waterworks work.

Images