Concrete compaction work analysis device, analysis method and program

Abstract

To provide a concrete compaction work analysis device for analyzing concrete compaction work performed by a worker with higher accuracy than before.SOLUTION: In a system 1, using images obtained by imaging devices 10a and 10b that perform imaging of a worker performing compaction work with a vibrator, a concrete compaction work analysis device 30 evaluates whether or not compaction for concrete is properly performed, by analyzing the positions, postures, or movements of the vibrator and the worker in image data of the images.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a technique for analyzing the compaction work of poured concrete by a worker. [Background technology]

[0002] Compaction is a process of vibrating concrete poured in a formwork to distribute the concrete evenly and to prevent voids from forming in the formwork. For example, Patent Document 1 discloses a method in which recognition objects i1 to i3 are attached to a vibration unit 21 of a vibrator 2 for compaction, three-dimensional coordinate values ​​of the recognition objects i1 to i3 are obtained from captured images of the compaction work, and the insertion depth and insertion period of the vibrator 2 are calculated based on the relationship between the coordinate values ​​of the concrete surface and the coordinate values ​​of the tip F of the vibrator 2. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6781439 Summary of the Invention [Problem to be solved by the invention]

[0004] The present invention aims to use AI analysis to analyze the degree of concrete filling performed by workers during concrete compaction work with greater accuracy than conventional methods. [Means for solving the problem]

[0005] In order to solve the above-mentioned problems, the present invention provides a method for obtaining an image of a worker, a vibrator, and a space in which the worker holds the vibrator and performs a concrete compaction operation, the method comprising: obtaining an image of the worker, a vibrator, and a space in which the worker holds the vibrator and performs a concrete compaction operation; extracting an image of the worker corresponding to the worker and an image of the vibrator corresponding to the vibrator from the image data; analyzing the extracted image of the worker and the vibrator; to the concrete a determination unit that determines whether or not compaction of concrete is being performed based on an analysis result of the worker image object and the vibrator image object and the determined whether or not vibration is being applied; The determination unit determines whether or not the vibration is applied based on both whether or not the tip of the vibrator image object is inserted into concrete in the image data and a signal indicating whether or not electricity is being applied to the vibrator. A compaction operation analysis device is provided.

[0006] The system may include an estimation unit that estimates concrete areas that have been compacted, concrete areas that are estimated to have not been compacted, and concrete areas that are estimated to have an insufficient level of compaction based on the judgment results by the judgment unit, and a display unit that visibly displays at least one of the concrete areas that are estimated to have been compacted, concrete areas that are estimated to have not been compacted, and concrete areas that are estimated to have an insufficient level of compaction based on the estimation by the estimation unit.

[0007] The acquisition unit may acquire the image data captured by a stereo camera or a plurality of monocular cameras.

[0008] The display unit may display any one of the regions in at least one of a planar shape, a cross-sectional shape, and a three-dimensional shape.

[0009] The estimation unit may estimate concrete areas that have been compacted, concrete areas that are estimated to have not been compacted, and concrete areas that are estimated to have an insufficient degree of compaction based on attributes of the vibrator and / or attributes of the concrete in addition to the judgment result by the judgment unit.

[0010] The analysis unit may perform skeleton estimation on the worker image object, and analyze a position, posture, or movement of the worker image object.

[0011] The analysis unit may perform skeleton estimation on the vibrator image object in addition to the worker image object, and analyze a position, posture, or movement of the vibrator image object.

[0012] The extraction unit may extract from the image data a plurality of worker image objects corresponding to a plurality of workers working simultaneously and a plurality of vibrator image objects corresponding to the vibrators, and the judgment unit may judge whether or not compaction of the concrete is being performed based on analysis results of the plurality of worker image objects and the plurality of vibrator image objects and the presence or absence of vibration for each of the identified plurality of vibrators.

[0015] The apparatus may further include a model generation unit that generates a learning model by performing machine learning based on teacher data using a group of data representing the position, posture or movement of the worker image object and the vibrator image object, and a group of data representing the presence or absence of vibration by the vibrator as explanatory variables, and a group of data representing whether or not compaction of the concrete is being performed as an objective variable, and the judgment unit may use the generated learning model to judge whether or not compaction of the concrete is being performed.

[0016] the image data is a plurality of image data captured by a plurality of imaging devices each having a different imaging position and / or imaging direction, the acquisition unit acquires the image data from the plurality of imaging devices, and the extraction unit extracts the worker image object and the vibrator image object, which are identical targets, from the plurality of image data; The analysis unit may be configured to superimpose and analyze the worker image object of the same target and the vibrator image object of the same target, which are extracted from the plurality of image data, respectively.

[0017] The image data may be video data.

[0018] The system may further include a warning unit that issues a warning when the degree of compaction judged by the judgment unit is likely to fall short of a predetermined quality.

[0019] The warning by the warning unit may be given by a display on a display unit or by a voice to the operator.

[0020] a coordinate conversion unit that converts at least one of a concrete region estimated to have been compacted, a concrete region estimated to have not been compacted, and a concrete region estimated to have been insufficiently compacted into three-dimensional coordinate values; The image forming apparatus may further include an output unit that outputs the three-dimensional coordinate values ​​of any concrete region after conversion by the coordinate conversion unit as CIM (Construction Information Modeling / Management) data or BIM (Building Information Modeling) data.

[0021] The coordinate conversion unit may convert into three-dimensional coordinate values ​​using three-dimensional coordinate values ​​of positions where the multiple imaging devices are installed, directional angles of the multiple imaging devices at the time of imaging, and three-dimensional coordinate values ​​of singular points in image data acquired by the multiple imaging devices.

[0022] The present invention also provides a method for producing a method for manufacturing a semiconductor device comprising the steps of: The computer acquiring image data of a worker, a vibrator, and a space in which the worker holds the vibrator and performs a concrete compaction operation; The computerextracting, from the image data, a worker image object corresponding to the worker and a vibrator image object corresponding to the vibrator; The computer analyzing the extracted worker image object and vibrator image object; The computer a step of determining whether or not vibration is being applied from the vibrator to the concrete, the step of determining whether or not vibration is being applied based on both whether or not a tip of the vibrator image object is inserted into concrete in the image data and a signal indicating whether or not electricity is being applied to the vibrator; The computer The present invention provides a method for analyzing compaction work, comprising a step of determining whether or not compaction of concrete is being performed based on the analysis results of the worker image object and the vibrator image object and the identified presence or absence of vibration.

[0023] Further, the present invention provides a method for obtaining an image of a worker, a vibrator, and a space in which the worker holds the vibrator and performs a concrete compaction operation, the method comprising the steps of: acquiring image data of a worker, a vibrator, and a space in which the worker holds the vibrator and performs a concrete compaction operation; extracting a worker image object corresponding to the worker and a vibrator image object corresponding to the vibrator from the image data; analyzing the extracted worker image object and vibrator image object; to the concrete Steps to determine whether or not vibration is present determining whether or not the vibration is applied based on both whether or not a tip of the vibrator image object is inserted into concrete in the image data and a signal indicating whether or not electricity is being applied to the vibrator. and a step of determining whether or not compaction of the concrete is being performed based on the analysis results of the worker image object and the vibrator image object and the identified presence or absence of vibration. Effect of the Invention

[0024] According to the present invention, the degree of concrete filling caused by concrete compaction work performed by workers can be analyzed more accurately than in the past. [Brief description of the drawings]

[0025] [Figure 1] FIG. 1 is a block diagram showing an example of the overall configuration of a system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram showing an example of a hardware configuration of the compaction operation analysis device according to the embodiment. [Diagram 3] FIG. 2 is a block diagram showing an example of a functional configuration of a compaction operation analysis device. [Figure 4] FIG. 13 is a diagram illustrating an example of skeletal structure estimation for a worker. [Diagram 5] FIG. 13 is a diagram illustrating skeleton estimation for a vibrator. [Figure 6] 4 is a flowchart showing the operation of the compaction operation analysis device. [Figure 7] 1 is a diagram illustrating an example of a plurality of imaging devices, a directional angle of each imaging device when capturing an image, and a positional relationship with a singular point; [Figure 8] FIG. 4 is a diagram illustrating an example of the insertion depth of a vibrator into concrete. [Figure 9] FIG. 13 is a diagram showing an example of a display when the compaction operation analysis device displays a concrete compaction area. [Figure 10] FIG. 13 is a diagram showing a display example when displaying the vibrator insertion position in a CIM. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] An example of an embodiment of the present invention will be described. [composition] FIG. 1 is a block diagram showing an example of the overall configuration of a system 1 according to an embodiment of the present invention. As shown in FIG. 1, the system 1 includes a plurality of imaging devices, a network 20 including a wireless communication network or a wired communication network, and a compaction operation analysis device 30. Of the imaging devices 10a and 10b, the imaging device 10a is installed at a site where concrete is poured, and is connected to the network 20, for example, by wire. On the other hand, the imaging device 10b is attached to the head or shoulder of a worker performing compaction operation or other workers around him (hereinafter simply referred to as workers), and is connected to the network 20, for example, wirelessly. In FIG. 1, one imaging device 10a installed at the concrete pouring site and one imaging device 10b attached to a worker are illustrated, but there may be a plurality of each of them.

[0027] A worker inserts a vibrator into the concrete poured into the formwork at a predetermined interval for a predetermined period of time, and applies appropriate vibrations to the concrete, thereby compacting the concrete. This eliminates any voids in the formwork that are not filled with concrete, resulting in a high quality concrete structure. The compaction work using the vibrator relies on the experience and intuition of the worker, and the work quality may vary. In the system 1 according to this embodiment, the image capturing device 10a, 10b captures the image of the worker holding the vibrator and performing the compaction work, and the compaction work analysis device 30 analyzes the positions, postures, or movements of the worker and the vibrator in the image data, thereby evaluating whether the concrete is being compacted properly. That is, according to this embodiment, image analysis is performed on the worker in addition to the vibrator, so that the quality of the concrete compaction work can be analyzed with higher accuracy than when only the vibrator is image-analyzed.

[0028] FIG. 2 is a diagram showing the hardware configuration of the compaction operation analysis apparatus 30. The compaction operation analysis apparatus 30 is physically configured as a computer device including a processor 3001, a memory 3002, a storage 3003, a communication device 3004, an input device 3005, an output device 3006, and a bus connecting these devices. Each of these devices operates with power supplied from a power source (not shown). In the following description, the word "apparatus" can be read as a circuit, a device, a unit, and the like. The hardware configuration of the compaction operation analysis apparatus 30 may be configured to include one or more of the devices shown in FIG. 2, or may be configured without including some of the devices.

[0029] Each function of the compaction work analysis device 30 is realized by loading a specific software (program) onto hardware such as a processor 3001 and a memory 3002, so that the processor 3001 performs calculations, controls communications via a communication device 3004, acquires data transmitted from other devices, and controls at least one of the reading and writing of data in the memory 3002 and the storage 3003.

[0030] The processor 3001 controls the entire computer by running, for example, an operating system. The processor 3001 may be configured with a central processing unit (CPU) including an interface with peripheral devices, a control device, an arithmetic unit, a register, and the like.

[0031] The processor 3001 reads out a program (program code), a software module, data, etc. from at least one of the storage 3003 and the communication device 3004 into the memory 3002, and executes various processes according to the read out programs. As the program, a program that causes a computer to execute at least a part of the operations described below is used. The functional blocks of the compaction operation analysis device 30 may be realized by a control program stored in the memory 3002 and operated in the processor 3001. The various processes may be executed by one processor 3001, or may be executed simultaneously or sequentially by two or more processors 3001. The processor 3001 may be implemented by one or more chips. The program may be transmitted to the compaction operation analysis device 30 from another device on the cloud via a telecommunication line, or may be installed in the memory 3002 or the storage 3003.

[0032] The memory 3002 is a computer-readable recording medium, and may be composed of at least one of, for example, a Read Only Memory (ROM), an Erasable Programmable ROM (EPROM), an Electrically Erasable Programmable ROM (EEPROM), a Random Access Memory (RAM), etc. The memory 3002 may also be called a register, a cache, a main memory (primary storage device), etc. The memory 3002 can store a program (program code), a software module, etc. that can be executed to implement the method according to this embodiment.

[0033] Storage 3003 is a computer-readable recording medium, and may be composed of at least one of, for example, an optical disk such as a CD-ROM (Compact Disc ROM), a hard disk drive, a solid state drive, a flexible disk, a magneto-optical disk (e.g., a compact disk, a digital versatile disk, a Blu-ray (registered trademark) disk), a smart card, a flash memory (e.g., a card, a stick, a key drive), a floppy (registered trademark) disk, a magnetic strip, etc. Storage 3003 may be referred to as an auxiliary storage device.

[0034] The communication device 3004 is hardware (transmission / reception device) for performing communication between computers via at least one of wired and wireless communication, and performs communication with the imaging devices 10a and 10b via the network 20. Also, when a mobile terminal or the like is used as the output device 3006 separately from the compaction operation analysis device 30, communication between the computer and the output device 3006 is performed via the network 20.

[0035] The input device 3005 is an input device (e.g., a keyboard, a mouse, a microphone, a switch, a button, etc.) that accepts input from the outside. The output device 3006 is an output device (e.g., a mobile terminal, smart glasses, a display, a speaker, an LED lamp, etc.) that performs output to the outside. The input device 3005 and the output device 3006 may be integrated into one (e.g., a touch panel).

[0036] The compaction operation analysis device 30 may be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), a field programmable gate array (FPGA), etc., and some or all of the functional blocks may be realized by the hardware. For example, the processor 3001 may be implemented using at least one of these pieces of hardware.

[0037] 3 is a diagram showing an example of the functional configuration of the compaction operation analysis device 30. Each function realized by the compaction operation analysis device 30 is realized by loading predetermined software (programs) onto hardware such as a processor 3001 and a memory 3002, causing the processor 3001 to perform calculations, control communication by a communication device 3004, and control at least one of reading and writing of data in the memory 3002 and the storage 3003.

[0038] In FIG. 3, the acquisition unit 31 acquires various data from outside the compaction operation analysis device 30. For example, the acquisition unit 31 acquires image data representing images captured by the imaging devices 10a and 10b (i.e., images of a worker, a vibrator, and a space in which the worker holds the vibrator and performs the compaction operation of concrete) from the imaging devices 10a and 10b via the network 20. This image data may be, for example, time-sequential video data, or a collection of still image data captured at a predetermined time interval (for example, one second) using a time-plus function. Each of the imaging devices 10a and 10b is configured by an imaging device using a stereo camera or at least a pair of monocular cameras to enable distance measurement using parallax. A plurality of such imaging devices 10a and 10b are installed so that image data of the compaction operation in the construction target area can be acquired.

[0039] The extraction unit 32 extracts an image object corresponding to a worker (hereinafter referred to as a worker image object) and an image object corresponding to a vibrator (hereinafter referred to as a vibrator image object) by applying a shape recognition algorithm such as pattern matching to the above image data.

[0040] The analysis unit 33 analyzes the worker image object and the vibrator image object extracted by the extraction unit 32. More specifically, the analysis unit 33 performs skeletal estimation on the worker image object and the vibrator image object to analyze the position, posture, or movement of the worker image object and the vibrator image object. Skeletal estimation is a technique for recognizing the positions of a plurality of parts s such as the joints of the human body H and the parts constituting the face (eyes, nose, ears, etc.) and estimating the skeleton of the human body H from the positional relationship of these parts s, as exemplified in FIG. 4. Skeletal estimation for the human body can be realized in a state where so-called markers are not provided on the human body (markerless). On the other hand, when applying skeletal estimation to the vibrator, it is desirable to attach a plurality of markers m, for example, different in color or shape, to each part of the vibrator B, as exemplified in FIG. 5. The analysis unit 33 can estimate the position, posture, or movement of the vibrator B by recognizing the position of each marker m attached to the vibrator B. Furthermore, even if the marker m is not attached to the vibrator, it is possible to simultaneously estimate and analyze the behavior and movements of the target worker and the object held by the worker in real time by using artificial intelligence to simultaneously recognize people and objects from the image, so the marker m on the vibrator is not a mandatory requirement.

[0041] The identifying unit 34 identifies whether or not vibration is being applied to the concrete by the vibrator (hereinafter, referred to as the presence or absence of vibration). Specifically, the identifying unit 34 may identify whether or not vibration is being applied based on a signal indicating whether or not electricity is being applied to the vibrator, or may identify whether or not vibration is being applied based on whether or not the tip of the vibrator image object is inserted into the concrete in the image data, or may identify whether or not vibration is being applied by taking both of these factors into consideration. When identifying whether or not vibration is being applied based on a signal indicating whether or not electricity is being applied to the vibrator, a power on / off signal is transmitted from the vibrator to the compaction operation analysis device 30, for example, wirelessly, and the acquiring unit 31 acquires this on / off signal.

[0042] The determination unit 35 determines whether or not compaction of the concrete is being performed based on the analysis result of the worker image object and the vibrator image object by the analysis unit 33 and the presence or absence of vibration specified by the specification unit 34. This determination is performed using a learning model stored in the storage unit 36.

[0043] The learning model is generated by the model generation unit 37 according to supervised learning using, for example, a neural network. Specifically, the model generation unit 37 generates the learning model by performing machine learning based on teacher data in which a data group including data representing the position, posture, or movement of the vibrator image object and / or the worker image object, and a data group representing the presence or absence of vibration by the vibrator are used as explanatory variables, and a data group representing whether or not compaction of the concrete is being performed is used as an objective variable. When generating the learning model, an image of the worker performing compaction is captured to obtain a data group (explanatory variables) including data representing the position, posture, or movement of the vibrator image object and / or the worker image object, and a data group (explanatory variables) representing the presence or absence of vibration is obtained based on a signal representing the on or off of the vibrator. Furthermore, at this time, the worker or an observer of the work, etc. inputs data (objective variables) representing whether or not compaction of the concrete is being performed to the system 1 by a predetermined operation. As a result, teacher data consisting of a set of explanatory variables and objective variables is generated.

[0044] At a concrete pouring site, since there are many reinforcing bars and obstacles, it may be difficult to accurately identify whether the vibrator is inserted into the concrete by image recognition alone. In contrast, according to the above learning model, image recognition is performed on the vibrator and / or the worker, and the positions, postures, or movements of the vibrator and the worker are used as explanatory variables, so it is possible to accurately identify whether compaction is being performed according to the position, posture, or movement specific to the worker when performing compaction work. Furthermore, at a concrete pouring site, even if the vibrator is inserted into the concrete, vibration by the vibrator may not be performed, that is, compaction work may not be performed. Furthermore, the vibrator may be switched on and vibrating when the vibrator is not inserted into the concrete, and in this case, compaction work is also not being performed. Therefore, according to the above learning model, it is possible to accurately determine whether compaction of the concrete is being performed when the vibrator is inserted into the concrete and vibration by the vibrator is being performed. Furthermore, for images of the vibrator and worker, imaging devices can be installed in multiple locations and directions, and images of the same object can be obtained from multiple directions using these multiple imaging devices and integrated, making it possible to recognize the position, posture, or movement of the vibrator and worker even if there are rebars or other obstacles between a specific imaging device and the object.

[0045] The storage unit 36 ​​stores the learning model generated by the model generation unit 37. The judgment unit 35 inputs the analysis result by the analysis unit 33 (i.e., data representing the position, posture, or motion of the vibrator image object and / or the worker image object) and the presence or absence of vibration identified by the identification unit 34 as explanatory variables to the learning model, and as a result, obtains data on the presence or absence of compaction (objective variable) output from the learning model.

[0046] The estimation unit 38 estimates concrete regions where compaction has been performed, based on the determination result by the determination unit 35. Furthermore, the estimation unit 38 estimates concrete regions where compaction has been estimated to have not been performed and concrete regions where the estimation unit 38 has estimated that the degree of compaction is insufficient, based on the determination result by the determination unit 35.

[0047] The display unit 39 displays at least one of the concrete regions estimated by the estimation unit 38 to have been compacted, the concrete regions estimated by the estimation unit 38 to have not been compacted, and the concrete regions estimated by the estimation unit 38 to have an insufficient degree of compaction so as to be visible to a worker or a manager. Any of the regions displayed by the display unit is displayed in at least one of a planar shape, a cross-sectional shape, and a three-dimensional shape.

[0048] The warning unit 40 issues a warning to a worker or manager when the degree of compaction judged by the judgment unit 35 is insufficient and there is a risk that the concrete filling degree will not be satisfied. This warning is issued by displaying on the display unit 39 or by outputting a voice to the worker or manager.

[0049] The coordinate conversion unit 41 converts at least one of the concrete areas estimated to have been compacted, the concrete areas estimated to have not been compacted, and the concrete areas estimated to have been insufficiently compacted into three-dimensional coordinate value data that can be used in CIM (Construction Information Modeling / Management) or BIM (Building Information Modeling).

[0050] The output unit 42 outputs the three-dimensional coordinate values ​​of any area after conversion by the coordinate conversion unit 41 as CIM data that can be used in CIM, or as BIM data that can be used in BIM.

[0051] [Operation] Next, the operation of this embodiment will be described. At the time when the process shown in Fig. 6 is started, the learning model generated by the model generating unit 37 is stored in the storage unit 36. In addition, in the process shown in Fig. 6, the presence or absence of vibration by the vibrator is determined based on a signal indicating whether or not the power supply to the vibrator is on or off.

[0052] First, the imaging devices 10a and 10b take images during the period when compaction work is performed at the concrete pouring site. At this time, since there are many reinforcing bars and obstacles at this concrete pouring site, the imaging devices 10a, 10a... are installed at multiple locations at the site so as to take images in multiple different imaging directions, and the imaging devices 10b, 10b... are attached to multiple workers, so that there is no gap in the imaging range. In other words, the imaging device 10a mainly takes images to identify the positions of each worker and vibrator in the entire concrete pouring site. The imaging device 10b attached to the workers around the worker performing the compaction work mainly takes images to identify the posture and movement of the worker performing the compaction work and the vibrator held by the worker. The imaging device 10b attached to the worker performing the compaction work mainly takes images to identify the posture and movement of the vibrator held by the worker. The multiple image data thus captured are transmitted from each of the imaging devices 10a, 10b to the compaction work analysis device 30. At the same time, a signal indicating whether the power supply to the vibrator is on or off is transmitted from the vibrator to the compaction work analysis device 30. The acquisition unit 31 of the compaction work analysis device 30 acquires these data or signals (Step S11 in FIG. 6).

[0053] Next, the extraction unit 32 of the compaction operation analysis device 30 extracts an image object corresponding to a concrete surface together with a worker image object and a vibrator image object from the acquired image data (step S12). At this time, the extraction unit 32 extracts the worker image object and the vibrator image object of the same target from the multiple image data captured by the multiple imaging devices 10a, 10a,... or 10b, 10b... having different imaging positions and / or imaging directions. In order to identify the worker image object and the vibrator image object of the same target, the extraction unit 32 uses the three-dimensional coordinate values ​​of the positions where the multiple imaging devices 10a, 10a... are installed, the direction angle Θ at the time of imaging by these imaging devices 10a, 10a, and the three-dimensional coordinate values ​​of the singular point M in the image shown by the image data captured by these imaging devices 10a, 10a..., as exemplified in FIG. 7. The singular point M in the image shown by the image data here may be, for example, a structure with a distinctive appearance installed at the concrete pouring site, or a marker indicating a singular point, and may be, for example, an AR (Augmented Reality) marker. These three-dimensional coordinate values ​​and direction angles are input and stored in the compaction operation analysis device 30 in advance.

[0054] Next, the analysis unit 33 of the compaction operation analysis device 30 performs skeleton estimation on the worker image object and the vibrator image object extracted by the extraction unit 32, and analyzes the positions, postures, or movements of the worker image object and the vibrator image object (step S13). As a result, for the vibrator, as exemplified in Fig. 8, the length of the vibrator inserted into the concrete is calculated from the difference between the insertion state of the vibrator at a certain time T1 and the insertion state of the vibrator at a certain time T2. At this time, this analysis is performed in a state where the worker image object and the vibrator image object of the same target extracted by the extraction unit 32 are superimposed. As a result, the analysis is prevented from being performed on the worker image object and the vibrator image object of the same target in duplicate.

[0055] Next, the determination unit 34 of the compaction operation analysis device 30 determines whether or not vibration is being applied by the vibrator based on a signal indicating whether or not electricity is being applied to the vibrator (step S14). The determination unit 34 may determine whether or not vibration is being applied based on whether or not the tip of the vibrator image object is inserted into the concrete in the image data, or may determine whether or not vibration is being applied by taking both into account. This determines the start and end times of vibration, and the difference between these times becomes the vibration application time by the vibrator.

[0056] Next, the judgment unit 35 of the compaction work analysis device 30 inputs the analysis results of the vibrator image object and / or the worker image object by the analysis unit 33 and the presence or absence of vibration identified by the identification unit 34 into a learning model, and judges whether or not the compaction work on the concrete is appropriate at the position of the imaged worker and vibrator (step S15).

[0057] Next, the estimation unit 38 of the compaction operation analysis device 30 estimates the concrete region where compaction has been performed based on the determination result by the determination unit 35 (step S16).

[0058] Next, the display unit 39 of the compaction operation analysis device 30 displays at least one of the concrete area estimated by the estimation unit 38 to have been compacted, the concrete area estimated by the estimation unit 38 to have not been compacted, and the concrete area estimated by the estimation unit 38 to have an insufficient degree of compaction so that the worker or manager can visually confirm (step S17). At this time, the coordinate conversion unit 41 performs conversion into three-dimensional coordinate values ​​using the three-dimensional coordinate values ​​of the positions where the imaging devices 10a and 10b are installed, the directional angles of the imaging devices 10a and 10b at the time of imaging, and the three-dimensional coordinate values ​​of the singular points in the image data acquired by the imaging devices 10a and 10b, as described above. The display unit 39 displays each area in three dimensions based on these three-dimensional coordinate values.

[0059] FIG. 9 is a diagram showing an example of display when the compaction operation analysis device displays the area of ​​concrete compaction. These areas are integrated with a CIM model or BIM model prepared in advance, and are displayed in different colors depending on at least one of the planar shape, cross-sectional shape, and three-dimensional shape of the area where compaction is performed. At this time, when an operation is performed to select one of the colored areas, the date and time when compaction was performed, the vibration time, and the three-dimensional compaction coordinate value are displayed. Furthermore, at this time, the warning unit 40 of the compaction operation analysis device 30 warns the worker or manager when the degree of compaction judged by the judgment unit 35 is insufficient and there is a possibility that the specified compaction operation has not been performed. For example, if the vibration time is long, there is a possibility that the concrete material will separate, so the warning unit 40 may issue a warning before the appropriate vibration time is reached.

[0060] Furthermore, the output unit 42 of the compaction operation analysis device 30 outputs the three-dimensional coordinate values ​​of any of the areas converted by the coordinate conversion unit 41 to an external device as CIM data usable in CIM (or BIM data usable in BIM). This allows the external device to manage the compaction operation using a three-dimensional display. Here, FIG. 10 is a diagram showing a display example when displaying the insertion position of the vibrator on the external device. In FIG. 10, the compaction position is displayed together with the reinforcing bar arrangement position that has been previously CIM modeled. In the example of FIG. 10, an insertion point A where the vibrator does not come into contact with the reinforcing bar is displayed as a possible insertion position for compaction based on the reinforcing bar arrangement position, diameter, and pitch of the reinforcing bar C and the cross-sectional area of ​​the vibrator in the short side direction (horizontal cross section).

[0061] According to the above-mentioned embodiment, since image analysis of the worker is performed in addition to the vibrator, it is possible to estimate the degree of filling of concrete by the compaction work with higher accuracy than when image analysis is performed only with the vibrator. Also, it is possible to visualize the compaction state in three dimensions in CIM or BIM.

[0062] [Variations] The above-described embodiment may be modified as follows. The estimation unit 38 may estimate the area of ​​concrete where compaction has been performed based on the attributes of the vibrator and / or the attributes of the concrete in addition to the determination result by the determination unit 35. Specifically, the estimation unit 38 stores the attributes of the vibrator (size, vibration range, vibration strength performance, etc.) and the attributes of the concrete (components and filling property (performance obtained by the interaction between the fluidity and material separation resistance when vibration compaction is applied)) in association with an appropriate vibration time, and estimates the area of ​​concrete where compaction has been performed from the relationship between the attributes of the vibrator and / or the attributes of the concrete at the pouring site and the vibration time of the vibrator.

[0063] At a concrete pouring site, compaction may be performed simultaneously at multiple positions. Therefore, the extraction unit 27 may extract multiple worker image objects corresponding to multiple workers working simultaneously and multiple vibrator image objects corresponding to multiple vibrators from the image data, link each worker image object with a vibrator image object, and the determination unit 35 may determine whether or not compaction of the concrete is being performed based on the analysis results of the multiple worker image objects and the multiple vibrator image objects and the presence or absence of vibration specified for each vibrator. [Explanation of symbols]

[0064] 10a, 10b: imaging device, 20: network, 30: compaction work analysis device, 3001: processor, 3002: memory, 3003: storage, 3004: communication device, 3005: input device, 3006: output device, 31: acquisition unit, 32: extraction unit, 33: analysis unit, 34: identification unit, 35: judgment unit, 36: model generation unit, 37: memory unit, 38: estimation unit, 39: display unit, 40: warning unit, 41: coordinate conversion unit, 42: output unit, s: part, H: human body, m: marker, B: vibrator, M: marker, X: image object, Θ: direction angle, C: reinforcing bar, A: vibrator insertion point.

Claims

1. an acquisition unit that acquires image data of a worker, a vibrator, and a space in which the worker holds the vibrator and performs concrete compaction work; an extracting unit that extracts, from the image data, a worker image object corresponding to the worker and a vibrator image object corresponding to the vibrator; an analysis unit that analyzes the extracted worker image object and the extracted vibrator image object; An identification unit that identifies whether or not vibration is applied to the concrete by the vibrator; a determination unit that determines whether or not compaction of concrete is being performed based on an analysis result of the worker image object and the vibrator image object and on the specified presence or absence of the vibration, The identification unit is a compaction work analysis device that identifies whether or not vibration is applied based on both whether or not the tip of the vibrator image object is inserted into concrete in the image data and a signal indicating whether electricity is on or off to the vibrator.

2. An estimation unit that estimates a concrete area where compaction has been performed, a concrete area where compaction has not been performed, and a concrete area where compaction has been estimated to be insufficient based on a judgment result by the judgment unit; a display unit that visibly displays at least one of a concrete region that is estimated to have been compacted, a concrete region that is estimated to have not been compacted, and a concrete region that is estimated to have an insufficient degree of compaction based on the estimation by the estimation unit; and The compaction operation analysis device according to claim 1 , comprising:

3. The acquisition unit acquires the image data captured by a stereo camera or a plurality of monocular cameras. The compaction operation analysis device according to claim 1 or 2, further comprising:

4. The display unit displays any one of the regions in at least one of a planar shape, a cross-sectional shape, and a three-dimensional shape. The compaction operation analysis device according to claim 2 .

5. The estimation unit is A concrete region where compaction has been performed is estimated based on the judgment result by the judgment unit, as well as on an attribute of the vibrator and / or an attribute of the concrete. The compaction operation analysis device according to claim 2 or 4.

6. The analysis unit performs skeleton estimation for the worker image object and analyzes a position, a posture, or a movement of the worker image object. The compaction operation analysis device according to any one of claims 1 to 5.

7. The analysis unit performs skeleton estimation on the vibrator image object in addition to the worker image object, and analyzes a position, a posture, or a movement of the vibrator image object. The compaction operation analysis device according to claim 6.

8. the extraction unit extracts, from the image data, a plurality of worker image objects corresponding to a plurality of workers working simultaneously and a plurality of vibrator image objects corresponding to the vibrator; The determination unit determines whether or not compaction of concrete is being performed based on analysis results of the plurality of worker image objects and the plurality of vibrator image objects and the presence or absence of vibration for each of the identified plurality of vibrators. The compaction operation analysis device according to any one of claims 1 to 7.

9. a model generation unit that generates a learning model by performing machine learning based on teacher data in which a data group representing a position, a posture, or a movement of the worker image object and the vibrator image object and a data group representing the presence or absence of vibration by the vibrator are used as explanatory variables, and a data group representing the presence or absence of compaction of concrete is used as an objective variable; The determination unit determines whether or not compaction of concrete is being performed by using the generated learning model. The compaction operation analysis device according to any one of claims 1 to 8.

10. The image data is a plurality of image data captured by a plurality of imaging devices each having a different imaging position and / or imaging direction, the acquisition unit acquires the image data from the plurality of imaging devices; the extraction unit extracts the worker image object and the vibrator image object of the same target from the plurality of image data, The analysis unit superimposes and analyzes the worker image object and the vibrator image object of the same object extracted from the plurality of image data. The compaction operation analysis device according to any one of claims 1 to 9.

11. The image data is video data. The compaction operation analysis device according to any one of claims 1 to 10.

12. a warning unit that issues a warning when the degree of compaction judged by the judgment unit is likely to fall short of a predetermined quality; The compaction operation analysis device according to any one of claims 1 to 11, comprising:

13. The warning unit issues a warning by displaying a message on a display unit or by giving a voice to the operator. The compaction operation analysis device according to claim 12.

14. a coordinate conversion unit that converts at least one of a concrete region estimated to have been compacted, a concrete region estimated to have not been compacted, and a concrete region estimated to have been insufficiently compacted into three-dimensional coordinate values; and an output unit that outputs the three-dimensional coordinate values ​​of any of the areas after the conversion by the coordinate conversion unit as CIM (Construction Information Modeling / Management) data or BIM (Building Information Modeling) data. The compaction operation analysis device according to claim 2 .

15. The coordinate conversion unit performs conversion into the three-dimensional coordinate values ​​using three-dimensional coordinate values ​​of positions where the plurality of imaging devices are installed, direction angles at the time of imaging by the plurality of imaging devices, and three-dimensional coordinate values ​​of a singular point in image data acquired by the plurality of imaging devices. The compaction operation analysis device according to claim 14.

16. A step of acquiring image data by a computer capturing an image of a worker, a vibrator, and a space in which the worker holds the vibrator and performs a concrete compaction operation; A step in which a computer extracts, from the image data, a worker image object corresponding to the worker and a vibrator image object corresponding to the vibrator; A step in which a computer analyzes the extracted worker image object and the vibrator image object; a step of determining whether or not vibration is being applied from the vibrator to the concrete by the computer, the step determining whether or not vibration is being applied based on both whether or not a tip of the vibrator image object is inserted into concrete in the image data and a signal indicating whether or not electricity is being applied to the vibrator; a step of determining by a computer whether or not compaction of concrete is being performed based on a result of analysis of the worker image object and the vibrator image object and on the identified presence or absence of the vibration; A method for analyzing compaction work comprising the steps of:

17. On the computer, acquiring image data of a worker, a vibrator, and a space in which the worker holds the vibrator and performs a concrete compaction operation; extracting, from the image data, a worker image object corresponding to the worker and a vibrator image object corresponding to the vibrator; analyzing the extracted worker image object and vibrator image object; a step of determining whether or not vibration is being applied from the vibrator to the concrete, the step of determining whether or not vibration is being applied based on both whether or not a tip of the vibrator image object is inserted into concrete in the image data and a signal indicating whether or not electricity is being applied to the vibrator; determining whether or not compaction of concrete is being performed based on the analysis results of the worker image object and the vibrator image object and the identified presence or absence of the vibration; A program for executing the above.

Images