Concrete joint surface evaluation device, concrete joint surface evaluation program, and concrete joint surface evaluation method

Abstract

To provide a concrete joint surface evaluation device, a method, and a program for accurately evaluating a joint surface of previously poured concrete where concrete will be poured and joined later with a small amount of calculation.SOLUTION: In a concrete joint surface evaluation device 1 that uses a trained model 10 to infer an evaluation result for a joint surface and display the evaluation result, the trained model 10 repeats convolution processing to generate a feature map a plurality of times. The plurality of convolution processing includes individual intra-channel convolution processing and inter-channel convolution processing. The intra-channel individual convolution processing generates a plurality of new channels for each channel that constitutes the feature map input to the intra-channel individual convolution processing. The inter-channel convolution processing involves performing convolution processing in a direction in which the channels are overlapped, and generating new channels by repeating this processing a plurality of times to generate a new feature map.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a concrete construction joint surface evaluation device, a concrete construction joint surface evaluation program, and a concrete construction joint surface evaluation method. [Background technology]

[0002] When constructing a structure using concrete, it is common to first pour a portion of concrete, then, after the first concrete has hardened, pour a second concrete in contact with the first concrete, thereby integrating the first and second concretes. As the first concrete hardens, fine particles contained in the cement and aggregate may rise to the surface along with the water in the concrete, forming a thin film called laitance. If such laitance forms on the joint surface of the first concrete and the second concrete to be poured, the integrity of the concrete may be reduced. Furthermore, if the joint surface is smooth, shear forces may not be sufficiently transmitted between the first concrete and the second concrete. Therefore, to improve the integrity of the first concrete and the second concrete, the joint surface of the first concrete is treated by scraping the surface with a wire brush or by spraying high-pressure water to expose the aggregate.

[0003] For the above-mentioned joint surfaces, the condition of the joint surface to which surface treatment should be performed has not been quantitatively determined. Instead, workers observe the joint surface and determine whether to perform surface treatment or whether to terminate the surface treatment if surface treatment is being performed based on their own intuition. For this reason, for example, if the worker is not skilled, the surface treatment of the joint surface may not be performed sufficiently, resulting in poor integrity between the previously poured concrete and the subsequently poured joint concrete. Therefore, it is desirable to quantitatively evaluate the condition of the joint surface of the previously poured concrete to which the later concrete will be joined.

[0004] In this regard, for example, Patent Document 1 describes a method for evaluating concrete joint surfaces, in which the surface of previously poured and hardened concrete, which is the joint surface that will come into contact with newly poured concrete, is photographed from different positions using two digital cameras, the amount of unevenness in a predetermined direction of the joint surface at predetermined intervals from the two images is measured in the direction perpendicular to the joint surface, the shape line drawn by the above measured values ​​in a predetermined range is treated as a waveform and subjected to a Fourier transform, a power spectrum is calculated, the sum of the values ​​of this power spectrum in a predetermined range in the frequency domain of this power spectrum (the reciprocal of the wavelength) is calculated, and the shear strength of the joint surface is evaluated based on this sum.

[0005] In order to evaluate the condition of such construction joint surfaces with greater accuracy, it is conceivable to apply artificial intelligence technology. More specifically, for example, a convolutional neural network (CNN) is used as a learning model, and this learning model is subjected to deep learning using images of the construction joint surface and expert evaluations of the construction joint surface captured in the images as training data to generate a trained model. By inputting images of the construction joint surface to be evaluated into the trained model, it may be possible to evaluate the construction joint surface. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-145174 Summary of the Invention [Problem to be solved by the invention]

[0007] The above-described CNN is constructed so that a convolution process is performed on an input image to generate a feature map multiple times. In each convolution process, a feature map with multiple (e.g., C) channels superimposed is input, and multiple (e.g., D) filters are applied to the feature map to generate multiple (e.g., D) channels corresponding to the number of filters. A new feature map is generated by superimposing these multiple channels. Here, in each convolution process, when a filter is applied to the feature map, information is mixed both within the channel and between channels to generate a channel corresponding to the filter. Therefore, each filter has a three-dimensional shape in three directions, including the two directions within the plane in which each channel of the feature map extends and the direction in which the channels are superimposed. In other words, each filter is constructed as a set of planar kernels of a certain size (e.g., K × K), the number of which is equal to the number of channels of the input feature map (e.g., C). Therefore, the number of parameters required for one convolution process is a relatively large value, for example, C x K x K x D, which is the product of four numbers. Therefore, a correspondingly large memory is required when executing the trained model. Furthermore, the amount of calculation required when actually executing the trained model can also be large, corresponding to the number of parameters.

[0008] The work of evaluating the joint surfaces of pre-placed concrete is basically carried out at the construction site. Therefore, the device for quantitatively evaluating the joint surfaces is necessarily realized by a device such as a tablet or smartphone that can be easily brought to the construction site and easily handled. Such devices that can be easily brought to the construction site, such as a tablet or smartphone, often have limited computing resources. Here, as described above, if the evaluation of the joint surface is realized using a trained model, for example, using CNN, and the size of the feature map and kernel is appropriately designed in the convolution process to construct the trained model, the number of parameters may become so large and the amount of calculation may become so large that it may be difficult to execute on a device with limited computing resources such as the one described above.

[0009] It is desired to accurately evaluate the joint surface of previously poured concrete where subsequent concrete will be poured and joined, with a small amount of calculation.

[0010] The problem to be solved by the present invention is to provide a concrete joint surface evaluation device, a concrete joint surface evaluation program, and a concrete joint surface evaluation method that can accurately evaluate, with a small amount of calculation, the joint surface of previously poured concrete where concrete will be poured and joined later. [Means for solving the problem]

[0011] The present invention employs the following means to solve the above problems. That is, the present invention provides a concrete joint surface evaluation device for evaluating a joint surface of previously poured concrete to which concrete will be subsequently poured and joined, the device comprising: an input image acquisition unit that acquires an image of the joint surface as an input image; a construction joint surface evaluation result inference unit that inputs an image based on the input image into a trained model that has been trained using previously prepared learning input images of the joint surface and evaluation results of the joint surface for the learning input images as training data, and infers the evaluation results of the joint surface captured in the image based on the input image; and a display unit that displays the inferred evaluation results, the trained model being constructed so that a convolution process is performed multiple times to generate a feature map, and the multiple convolution processes are individual convolution processes within a channel. Provided is a concrete joint surface evaluation device comprising: an inter-channel convolution process, wherein the intra-channel individual convolution process generates a plurality of new channels by applying kernels individually provided for each of the channels constituting the feature map input to the intra-channel individual convolution process, and generates the new feature map by overlaying the new plurality of channels; and the inter-channel convolution process performs convolution process on pixels located at the same coordinates in each of the plurality of channels constituting the feature map generated by the intra-channel individual convolution process in the direction in which the channels are overlaid, thereby generating the new channel, repeated a plurality of times; and generates the new feature map by overlaying the generated new channels.

[0012] According to the above configuration, the joint surface of the previously poured concrete to which the concrete will be poured and joined later is evaluated by inputting an image based on the input image of the joint surface into a trained model that has been trained using the training input image of the joint surface and the evaluation results of the joint surface for the training input image as training data, and inferring the evaluation results of the joint surface captured in the image. This allows for accurate and quantitative evaluation of the joint surface. Here, the trained model is constructed so that a feature map is generated by performing convolution processing multiple times, and the multiple convolution processing includes individual intra-channel convolution processing and inter-channel convolution processing. First, in the intra-channel individual convolution process, a kernel individually assigned to each channel constituting the feature map input to the intra-channel individual convolution process is applied to generate multiple new channels, and the new multiple channels are then overlapped to generate a new feature map. In this process, since the kernel is assigned to each channel constituting the feature map, each kernel has a fixed size and is configured as a plane (e.g., like a single image). Therefore, the number of parameters required in the intra-channel individual convolution process is C×K×K(×1), where C is the number of channels in the input feature map and K×K is the size of each kernel. In inter-channel convolution, convolution is performed on pixels located at the same coordinates in each of the multiple channels that make up the feature map generated by the intra-channel individual convolution, in the direction in which the channels are overlapped, to generate a new channel. This process is repeated multiple times, and the multiple new channels are overlapped to generate a new feature map. In this process, the so-called filter required to generate a new channel is a collection of 1x1 kernels, each equivalent to one pixel, equal to the number of channels in the feature map generated by the intra-channel individual convolution. Therefore, the number of parameters required in inter-channel convolution is C(x1x1)xD, where C is the number of channels in the input feature map and D is the number of channels in the newly generated feature map. In this way, in the above-described configuration, the number of required parameters can be significantly reduced compared to a typical CNN that requires the number of parameters equivalent to the multiplication of four values, and the amount of calculation can also be reduced accordingly. Furthermore, in the intra-channel convolution process, a planar kernel is applied to the input feature map. Therefore, in the intra-channel convolution process, the calculation is performed within the channel, not between the channels that make up the feature map. Therefore, while information is mixed within a channel, information is not mixed between channels. On the other hand, in the subsequent inter-channel convolution process, a 1x1 kernel is applied to each channel, so information is not mixed within a channel. However, by performing convolution in the direction in which the channels overlap using a filter composed of these 1x1 kernels, equal to the number of channels in the feature map generated by the intra-channel convolution process, information is mixed between channels. In this way, information is not mixed between channels in the intra-channel convolution process, and information is not mixed within channels in the inter-channel convolution process. This configuration reduces the computational load by preventing information mixing between channels, while information is mixed both within and between channels in the intra-channel convolution process and the inter-channel convolution process. Therefore, the accuracy of the concrete joint surface evaluation is basically not compromised. In this way, it is possible to accurately evaluate the joint surface of previously poured concrete where subsequent concrete will be poured and joined, with a small amount of calculation.

[0013] In one aspect of the present invention, the trained model performs an attention map generation process and an attention part extraction process after multiple convolution processes, and the attention map generation process generates an attention map based on the feature map input to the attention map generation process, configured so that each pixel has a value closer to 1 if it is something that should be given special attention for the evaluation of the joint surface, and closer to 0 if it is not something that should be paid attention to, and the attention part extraction process multiplies each of the channels that make up the feature map input to the attention map generation process by the attention map for each pixel.

[0014] According to the above configuration, after multiple convolution processes, an attention map generation process and an attention portion extraction process are executed. In the attention map generation process, an attention map is generated based on the feature map input to the attention map generation process so that each pixel in each channel has a value closer to 1 if it is a pixel that should be given priority and attention for evaluating the construction joint surface, and a value closer to 0 if it is not a pixel that should be given attention. In addition, in the attention portion extraction process, the attention map is multiplied by each channel that constitutes the feature map input to the attention map generation process. By this process, in each channel of the feature map, pixels that are considered to be effective in evaluating the construction joint surface have a larger value, and pixels that are not considered to be effective in evaluating the construction joint surface have a smaller value. This makes it possible to evaluate the construction joint surface more accurately.

[0015] In another aspect of the present invention, the joint surface evaluation result inference unit divides the input image into a plurality of regions to generate a plurality of divided images, inputs each of the plurality of divided images into the trained model as an image based on the input image, and infers the evaluation result of the joint surface for each of the plurality of divided images, and the display unit displays the input image for each of the plurality of regions in a color corresponding to the evaluation result corresponding to that region.

[0016] According to the above configuration, it is possible to extract areas of the entire input image where the evaluation of the joint surface is high or low.

[0017] In another aspect of the present invention, the joint surface evaluation result inference unit divides the input image into multiple regions and further grayscales the images, thereby generating each of the multiple divided images as a single-channel image.

[0018] According to the above configuration, a trained model can be generated efficiently.

[0019] In another aspect of the present invention, the evaluation result of the joint surface is either one or both of the degree of unevenness on the joint surface and the degree of laitance on the joint surface.

[0020] According to the above-described configuration, the joint surface can be evaluated by evaluating either or both of the degree of unevenness on the joint surface and the degree of laitance on the joint surface.

[0021] The present invention also provides a concrete joint surface evaluation program for evaluating a joint surface of previously poured concrete to which concrete will be poured and joined later, the program including: an input image acquisition function for acquiring an image of the joint surface as an input image; a construction joint surface evaluation result inference function for inferring the evaluation result of the construction joint surface captured in the image based on the input image to a trained model that has been trained using, as training data, a previously prepared learning input image of the construction joint surface and an evaluation result of the construction joint surface for the learning input image as teacher data; and a display function for displaying the inferred evaluation result. The trained model is constructed so that a convolution process is performed to generate a feature map multiple times, and the multiple convolution processes are performed using a channel. a plurality of channels constituting the feature map input to the intra-channel individual convolution processing, each of which applies a kernel individually provided for each channel constituting the feature map to generate a plurality of new channels, and overlays the plurality of new channels to generate the new feature map; and a plurality of channels convolution processing, each of which performs convolution processing between pixels located at the same coordinates in each of the plurality of channels constituting the feature map generated by the intra-channel individual convolution processing, in the direction in which the channels are overlaid, to generate the new channel, repeating this process a plurality of times, and overlaying the plurality of new channels generated to generate the new feature map.

[0022] According to the above-described configuration, similar to that described with respect to the concrete joint surface evaluation device, it is possible to accurately evaluate the joint surface of previously poured concrete where later concrete will be poured and joined, with a small amount of calculation.

[0023] The present invention also provides a concrete joint surface evaluation method for evaluating a joint surface of previously poured concrete to which concrete will be poured and joined later, the method comprising: an input image acquisition step of acquiring an image of the joint surface as an input image; a construction joint surface evaluation result inference step of inputting an image based on the input image into a trained model trained using previously prepared learning input images of the joint surface and evaluation results of the construction joint surface for the learning input images as training data, and inferring the evaluation results of the construction joint surface captured in the image based on the input image; and a display step of displaying the inferred evaluation results, wherein the trained model is constructed so that a process of performing convolution processing to generate a feature map is repeated multiple times, and the multiple convolution processings are individual convolution processing within a channel. and inter-channel convolution processing, wherein the intra-channel individual convolution processing generates a plurality of new channels by applying kernels individually provided for each of the channels constituting the feature map input to the intra-channel individual convolution processing, and generates the new feature map by overlaying the new plurality of channels, and the inter-channel convolution processing performs convolution processing between pixels located at the same coordinates of each of the plurality of channels constituting the feature map generated by the intra-channel individual convolution processing in the direction in which the channels are overlaid, to generate the new channel, repeating this process a plurality of times, and overlaying the generated new channels to generate the new feature map.

[0024] According to the above-described configuration, similar to that described with respect to the concrete joint surface evaluation device, it is possible to accurately evaluate the joint surface of previously poured concrete where later concrete will be poured and joined, with a small amount of calculation. [Effects of the Invention]

[0025] According to the present invention, it is possible to provide a concrete joint surface evaluation device, a concrete joint surface evaluation program, and a concrete joint surface evaluation method that can accurately evaluate the joint surface of previously poured concrete where concrete will be poured and joined later, with a small amount of calculation. [Brief explanation of the drawings]

[0026] [Figure 1] 1 is a block diagram of a concrete construction joint surface evaluation device according to an embodiment of the present invention. [Figure 2] 1 is an example of an input image of a concrete joint surface, which is input to the concrete joint surface evaluation device. [Figure 3] 10 is an example of a divided image generated based on the input image. [Figure 4] FIG. 1 is a conceptual block diagram of a trained model provided in the concrete joint surface evaluation device. [Figure 5] FIG. 10 is a diagram showing the processing flow in the input processing unit of the trained model. [Figure 6] FIG. 10 is a diagram showing the flow of processing in the first convolution processing unit of the trained model. [Figure 7] FIG. 10 is an explanatory diagram of individual intra-channel convolution processing and inter-channel convolution processing. [Figure 8] FIG. 10 is a diagram showing the flow of processing in the second convolution processing unit of the trained model. [Figure 9] FIG. 10 is a diagram showing the processing flow in the attention processing unit of the trained model. [Figure 10] FIG. 10 is a diagram showing the processing flow in the fully connected processing unit of the trained model. [Figure 11]10 is an example of an output result of the concrete joint surface evaluation device. [Figure 12] 1 is a flowchart of a concrete joint surface evaluation method in the concrete joint surface evaluation device. DETAILED DESCRIPTION OF THE INVENTION

[0027] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. When constructing a structure using concrete, a portion of the concrete may be poured first, and after the first concrete has hardened, a second concrete may be poured in contact with the first concrete, thereby integrating the first concrete with the second concrete. The concrete joint surface evaluation device of this embodiment evaluates the joint surface of such first-placed concrete where the second concrete will be poured and joined.

[0028] If laitance is formed on the joint surface, the integrity of the previously poured concrete and the subsequently poured concrete is reduced. Therefore, the concrete joint surface evaluation device of this embodiment evaluates the degree of laitance on the joint surface. More specifically, the concrete joint surface evaluation device of this embodiment evaluates the presence or absence of laitance on the joint surface. On the other hand, even if the joint surface is smooth, the integrity of the previously poured concrete and the subsequently poured concrete is reduced. Therefore, the concrete joint surface evaluation device may evaluate the degree of unevenness of the joint surface. Alternatively, the concrete joint surface evaluation device may evaluate both the degree of unevenness of the joint surface and the degree of laitance of the joint surface.

[0029] FIG. 1 is a block diagram of a concrete construction joint evaluation device. The concrete construction joint surface evaluation device 1 of this embodiment includes an input image acquisition unit 2, a mesh division unit 3, and a display unit 4. The construction joint surface evaluation result inference unit 3 includes a trained model 10. The concrete joint surface evaluation device 1 is a computer equipped with hardware such as a processor and memory, and is functionally equipped with the above-described configuration by executing the concrete joint surface evaluation program according to this embodiment. That is, by executing the concrete joint surface evaluation program, the concrete joint surface evaluation device 1 causes the computer to realize the input image acquisition function, mesh division function, and display function corresponding to the input image acquisition unit 2, mesh division unit 3, and display unit 4, respectively.

[0030] In particular, in this embodiment, the concrete joint evaluation device 1 is a device such as a tablet or smartphone that can be easily brought to a construction site where concrete is actually poured. Therefore, the concrete joint evaluation program as described above can be realized as an application installed on the device. Such devices often have limited computational resources. The concrete construction joint surface evaluation device 1, concrete construction joint surface evaluation program, and concrete construction joint surface evaluation device method according to the present embodiment can be easily executed even on such devices with limited computational resources, thanks to the configurations described below.

[0031] FIG. 2 is an example of an input image of a concrete joint surface, which is input to the concrete joint surface evaluation device in this embodiment. A worker who wishes to evaluate a construction joint surface captures an image of the construction joint surface to be evaluated, obtaining an image such as that shown in Fig. 1. For example, the worker can obtain an image such as that shown in Fig. 1 by activating a camera attached to the device. Alternatively, the worker may activate a concrete construction joint surface evaluation program, and the concrete construction joint surface evaluation program may activate a camera to capture an image such as that shown in Fig. 1. The input image acquisition unit 2 acquires, as an input image 100, an image of the construction joint surface as described above.

[0032] FIG. 3 shows an example of a segmented image generated based on an input image. As will be explained later, the construction joint surface evaluation result inference unit 3 evaluates the construction joint surface using the trained model 10. The construction joint surface evaluation result inference unit 3 first processes the input image 100 to a state appropriate for evaluation in the trained model 10. Specifically, the construction joint surface evaluation result inference unit 3 divides the input image 100 into a plurality of regions of a predetermined size, cuts out the image for each region, and divides it into a plurality of images. The construction joint surface evaluation result inference unit 3 further grayscales each of the images thus generated. In this way, the construction joint surface evaluation result inference unit 3 generates a plurality of divided images 101 from the input image 100. The size of the divided images 101 can be set appropriately. In this embodiment, the size of the divided images 101 is 81 × 145. Each divided image 101 is grayscaled, and therefore is a one-channel image.

[0033] The joint surface evaluation result inference unit 3 inputs each of the images, i.e., divided images 101, based on the input image 100 generated in this manner into a trained model 10 as described below, and evaluates the joint surface for each divided image 101 within the area captured within that divided image 101.

[0034] The trained model 10 is trained using as training data a pre-prepared training input image of the joint surface and the evaluation results of the joint surface for the training input image as training data. In this case, the learning input image is, for example, an image of a concrete joint surface captured in a previous construction project such as pouring a concrete joint. The learning input image is an image that has the same size as the segmented image 101 and has been converted to grayscale and set to one channel.

[0035] The evaluation result of the construction joint surface for the learning input image is, for example, the evaluation result of the construction joint surface by an expert for the learning input image. In this embodiment, the concrete construction joint surface evaluation device 1 evaluates the degree of laitance on the construction joint surface, particularly the presence or absence of laitance. Therefore, if the construction joint surface captured in the learning input image does not have laitance and the condition of the construction joint surface is good, for example, a value of 1 may be set as the evaluation result. Also, if the construction joint surface captured in the learning input image has laitance and the condition of the construction joint surface cannot be said to be good, for example, a value of 0 may be set as the evaluation result.

[0036] As will be described later, the trained model 10 is configured to perform convolution processing, followed by full connection processing, to output an evaluation result of the pour joint surface. Therefore, a training input image is input to a training model having a similar configuration to infer an evaluation result. The values ​​of the filter parameters and weights of the training model are adjusted by backpropagation or gradient descent so that this inferred evaluation result is close to the evaluation result by an expert corresponding to the training input image, i.e., the training data. This allows machine learning of the training model. As a result, when a training input image is input, the training model is trained to output an inference result, i.e., an evaluation result, close to the corresponding training data.

[0037] In this way, a trained model 10 is generated as a trained model that is used as a program module that is part of artificial intelligence software, with learning parameters such as filter parameters and weights learned. As described above, the trained model 10 is deep-learned so that when an image (divided image 101) of a joint surface is input, it estimates the evaluation result of the joint surface corresponding to the image.

[0038] In the above-described learning using an image as input, the efficiency and accuracy of learning can be improved by ingenuity. For example, when there is one piece of learning data as a combination of a learning input image and training data, a new learning input image can be generated by rotating the learning input image or flipping it vertically, horizontally, or vertically, and then combining this with training data corresponding to the original learning input image, thereby augmenting and generating multiple pieces of training data. Increasing the training data in this way and improving the diversity of the training data makes the trained model 10 less susceptible to overtraining and improves the generalization performance of the trained model 10.

[0039] Next, a specific configuration of the trained model 10 will be described. Figure 4 is a conceptual block diagram of the trained model. The trained model 10 includes an input processing unit 11, a first convolution processing unit 12, a second convolution processing unit 13, an attention processing unit 14, and a fully connected processing unit 15.

[0040] FIG. 5 is a diagram showing the flow of processing in the input processing unit of the trained model. When an image (divided image 101) is input to the construction joint surface evaluation result inference unit 3, the input processing unit 11 receives the input image through input reception processing 111.

[0041] In FIG. 5 and in FIGS. 6, 8, 9, and 10 used in the following description, the three numbers listed in the "input" column for each process indicate the channel height, channel width, and number of channels of the image (feature map) input to that process. The three numbers listed in the "output" column for each process indicate the channel height, channel width, and number of channels of the image (feature map) output from that process. For example, in the input reception process 111, in this embodiment, a one-channel image measuring 81 × 145 is input to the trained model 10, so the values ​​"81," "145," and "1" are listed as the "input" of the input reception process 111. Furthermore, because the input reception process 111 passes the received image to the subsequent rescaling process 112 without processing, the same values ​​as the "input," "81," "145," and "1," are listed as the "output" of the input reception process 111. Hereinafter, unless specifically necessary, explanations of these values ​​for each process will be omitted.

[0042] It should be noted that the image sizes and the number of channels are not limited to the numbers shown in the examples, and other numbers may be used. Also, in the processes described below, specific numbers are used as examples for the sizes and numbers of kernels and filters, but it goes without saying that in practice these are not limited to the numbers shown in the examples, and other numbers may be used.

[0043] The input processing unit 11 rescales each pixel of the divided image 101 received by the input reception processing unit 111 through the rescaling processing unit 112, for example by dividing the pixel by the maximum brightness value of all pixels, so that the pixel value becomes a value between 0 and 1. The rescaled image is input to the first convolution processing unit 12, which will be described next.

[0044] FIG. 6 is a diagram illustrating the flow of processing in the first convolution processing unit of the trained model. The first convolution processing unit 12 executes a plurality of convolution processes, including a pre-convolution process 121, an intra-channel individual convolution process 122, and an inter-channel convolution process 123.

[0045] The first convolution processing unit 12 first performs pre-convolution processing 121 on the rescaled image. The pre-convolution processing 121 includes four filters, each of which is made up of a single kernel, each of which has a size of 3 × 3. The pre-convolution processing 121 applies each of these four filters to the rescaled image, and then applies a ReLU (Rectified Linear Unit) activation function to create four channels, thereby generating a feature map made up of the four channels.

[0046] The first convolution processing unit 12 then performs intra-channel individual convolution processing 122 . FIG. 7 is an explanatory diagram of the intra-channel individual convolution processing and the inter-channel convolution processing. The intra-channel individual convolution processing 122 is provided with a two-dimensional kernel 20, for example, 3 × 3 in size. The number of kernels 20 provided in the intra-channel individual convolution processing 122 is four, which is the number of channels of the input feature map M21 generated in the previous convolution processing, i.e., the pre-convolution processing 121. In other words, the intra-channel individual convolution processing 122 is provided with a kernel 20 corresponding to each channel of the input feature map M21.

[0047] The first convolution processing unit 12 applies a kernel 20, which is individually provided for each channel constituting the input feature map M21, by an intra-channel individual convolution process 122, and then applies a ReLU activation function to generate multiple new channels, and then generates a new feature map M22 by overlapping the new multiple channels.

[0048] In the above processing, a planarly configured kernel 20 is applied to the input feature map, and processing is performed by applying the kernel 20 for each channel. That is, the intra-channel individual convolution processing 122 is performed within the channels that make up the feature map M21, and is not performed between channels. Therefore, in the intra-channel individual convolution processing 122, information is mixed within a channel, but information is not mixed between channels. In this way, in the intra-channel individual convolution processing 122, processing is performed for each channel, so the number of channels generated by the intra-channel individual convolution processing 122 is equal to the number of channels of the input feature map M21, for example, 4.

[0049] Following the intra-channel individual convolution process 122, the first convolution processing unit 12 executes an inter-channel convolution process 123 using the feature map M22 generated by the intra-channel individual convolution process 122 as an input. The inter-channel convolution process 123 includes a filter 21 in which a kernel 22 of 1×1 size is stacked, the number of which is equal to the number of channels of the input feature map M22, for example, four. The inter-channel convolution process 123 includes, for example, four filters 21.

[0050] In the inter-channel convolution process 123, each of these four filters 21 is applied to the feature map M22 generated by the intra-channel individual convolution process 122. By applying the above-described filters 21 to the feature map M22, convolution processing is performed between pixels located at the same coordinates in each of the multiple channels that make up the feature map M22 in the direction in which the channels are overlapped. Then, the inter-channel convolution process 123 generates new channels by applying a ReLU activation function. In this way, the inter-channel convolution process 123 repeats the process of generating new channels multiple times, i.e., the number of times equal to the number of filters 21, and overlaps the generated new channels in the number corresponding to the number of filters 21 to generate a feature map M23 consisting of multiple channels, for example, four channels.

[0051] In the above processing, the size of each kernel 22 is 1×1. Therefore, even if a filter 21 having such a kernel 22 is applied to a feature map M22, processing is not performed between pixels within the same channel that constitutes the feature map M22. In other words, the direction of the convolution processing in the inter-channel convolution processing 123 is limited to the direction in which the channels are overlapped, and convolution processing is performed only between channels. In this way, in the inter-channel convolution processing 123, information is mixed between channels, but information is not mixed within a channel.

[0052] For example, in CNN, the filter applied to the feature map is composed of a kernel with a certain size (2x2 or more) stacked as many times as the number of channels that make up the feature map. Therefore, the convolution process in CNN is performed simultaneously both within and between channels. This allows CNN to mix information both within and between channels. In contrast, in the above configuration, the convolution processing is divided into an intra-channel individual convolution processing 122, which is performed only within a channel and not between channels, and an inter-channel convolution processing 123, which is performed only between channels and not between channels. In this way, by limiting the direction of the convolution processing in each convolution processing, the required amount of calculation is reduced. Furthermore, by executing the inter-channel convolution processing 123 following the intra-channel individual convolution processing 122, information is mixed both within and between channels, similar to CNN. Therefore, even when the direction of the convolution processing is limited as described above, the evaluation accuracy of the concrete joint surface of the entire trained model 10 is unlikely to be impaired.

[0053] In this way, the first convolution processing unit 12 performs a plurality of convolution processes 121, 122, and 123. The second convolution processing unit 13, which will be described later, also performs a plurality of convolution processes 131, 132, and 133. In this way, the trained model 10 is constructed so that the convolution processes 121, 122, 123, 131, 132, and 133 are performed to generate feature maps M21, M22, M23, M31, M32, and M33, and this process is repeated a plurality of times.

[0054] The first convolution unit 12 performs maximum value pooling 124 on the feature map M23 generated by the inter-channel convolution 123. The maximum value pooling 124 is performed with a pool size of, for example, 2×2. Thereafter, the first convolution unit 12 executes a batch normalization process 125.

[0055] FIG. 8 is a diagram illustrating the flow of processing in the second convolution processing unit of the trained model. The second convolution processing unit 13 executes a plurality of convolution processes. Similar to the first convolution processing unit 12, the plurality of convolution processes include a pre-convolution process 131, an intra-channel individual convolution process 132, and an inter-channel convolution process 133.

[0056] The second convolution processing unit 13 first performs pre-convolution processing 131 on the feature map output by the first convolution processing unit 12. The pre-convolution processing unit 131 is provided with eight filters, each consisting of four 3×3 kernels, for example. The pre-convolution processing unit 131 applies each of these eight filters to the feature map output by the first convolution processing unit 12, and then applies a ReLU activation function to create eight channels, thereby generating a feature map M31 (see FIG. 7) consisting of eight channels.

[0057] The second convolution processing unit 13 then executes intra-channel individual convolution processing 132 . 7, the intra-channel individual convolution processing 132 is provided with a kernel 30 having a size of, for example, 3 × 3. The number of kernels 30 provided in the intra-channel individual convolution processing 132 is 8, which is the number of channels of the input feature map M31 generated in the previous convolution processing, i.e., the pre-convolution processing 131. In other words, the intra-channel individual convolution processing 132 is provided with kernels 30 corresponding to each of the channels of the input feature map M31.

[0058] The second convolution processing unit 13 applies kernels 30 individually provided for each channel constituting the input feature map M31 by intra-channel individual convolution processing 132, and then applies a ReLU activation function to generate multiple new channels, and then overlays the new multiple channels to generate a new feature map M32.

[0059] In the above processing, a planar kernel 30 is applied to the input feature map, and processing is performed by applying the kernel 30 for each channel. That is, the intra-channel individual convolution processing 132 is performed within the channels that make up the feature map M31, and is not performed between channels. Therefore, in the intra-channel individual convolution processing 132, information is mixed within a channel, but information is not mixed between channels. In this way, in the intra-channel individual convolution processing 132, processing is performed for each channel, so the number of channels generated by the intra-channel individual convolution processing 132 is equal to the number of channels of the input feature map M31, for example, 8.

[0060] Following the intra-channel individual convolution process 132, the second convolution processing unit 13 executes an inter-channel convolution process 133 using the feature map M32 generated by the intra-channel individual convolution process 132 as an input. The inter-channel convolution process 133 includes filters 31 in which kernels 32 each having a size of 1×1 are stacked, the number of which is equal to the number of channels of the input feature map M32, for example, eight. The inter-channel convolution process 133 includes, for example, eight filters 31.

[0061] In the inter-channel convolution process 133, each of these eight filters 31 is applied to the feature map M32 generated by the intra-channel individual convolution process 132. By applying the above-described filters 31 to the feature map M32, convolution processing is performed between pixels located at the same coordinates in each of the multiple channels that make up the feature map M32 in the direction in which the channels are overlapped. The inter-channel convolution process 133 then generates new channels by applying a ReLU activation function. In this way, the inter-channel convolution process 133 repeats the process of generating new channels multiple times, i.e., the number of times equal to the number of filters 31, and overlaps the generated new channels in the number corresponding to the number of filters 31 to generate a feature map M33 consisting of multiple channels, for example, eight channels.

[0062] In the above processing, the size of each kernel 32 is 1×1. Therefore, even if a filter 31 having such a kernel 32 is applied to a feature map M32, processing is not performed between pixels within the same channel that constitutes the feature map M32. In other words, the direction of the convolution processing in the inter-channel convolution processing 133 is limited to the direction in which the channels are overlapped, and convolution processing is performed only between channels. In this way, in the inter-channel convolution processing 133, information is mixed between channels, but information is not mixed within a channel.

[0063] In this way, the second convolution processing unit 13, like the first convolution processing unit 12, is configured to limit the direction of the convolution processing to reduce the amount of calculation, while preventing the evaluation accuracy of the joint surface from being compromised.

[0064] The second convolution processing unit 13 performs maximum value pooling processing 134 on the feature map M33 generated by the inter-channel convolution processing 133. The maximum value pooling processing 134 is performed with a pool size of 2×2, for example. Thereafter, the second convolution unit 13 executes a batch normalization process 135.

[0065] FIG. 9 is a diagram showing the flow of processing in the processing unit of interest. The attention processing unit 14 first executes an attention map generation process 141. The input of the attention map generation process 141 is the feature map finally generated after the multiple convolution processes 121, 122, 123, 131, 132, and 133 described above in the first convolution processing unit 12 and the second convolution processing unit 13. More precisely, in this embodiment, the input of the attention map generation process 141 is the feature map that is the output of the batch normalization process 135 that is last executed after the multiple convolution processes 121, 122, 123, 131, 132, and 133.

[0066] The attention processing unit 14 generates an attention map based on the feature map input to the attention map generation process 141, using the attention map generation process 141, so that each pixel has a value closer to 1 if it is something that should be given special attention for evaluating the joint surface, and closer to 0 if it is something that does not require attention. Specifically, the attention map generation process 141 has one filter in which a 1x1 kernel is stacked on top of the number of channels of the input feature map, for example, 8. The attention map generation process 141 applies this one filter to the input feature map, and then applies a sigmoid activation function to generate a feature map as an image with only one channel. The sigmoid activation function outputs a value between 0 and 1. By utilizing this property, the attention map output by the attention map generation process 141 can be generated as described above, for example, so that pixels in the attention map have a value close to 1 when they show high attention, and have a value close to 0 when they show low attention.

[0067] Next, the attention processing unit 14 executes an attention portion extraction process 142 that multiplies the feature map input to the attention processing unit 14, that is, the output of the batch normalization process 135, by the attention map. Specifically, in the attention portion extraction process 142, the attention map generated in the attention map generation process 141 is multiplied for each pixel for each channel constituting the feature map input to the attention map generation process 141. In this manner, for each channel constituting the feature map, if the corresponding pixel (at the same coordinates) in the attention map has a value close to 1, the pixel value of the pixel is multiplied by a value close to 1, thereby maintaining the pixel's value at approximately the same level. Also, if the corresponding pixel (at the same coordinates) in the attention map has a value close to 0, the pixel value of the pixel is multiplied by a value close to 0, thereby maintaining the pixel's value close to 0. In this manner, pixels corresponding to features related to laitance in the attention map that should be focused on are emphasized more than pixels corresponding to features that do not need to be focused on. This allows the trained model 10 to ignore unnecessary information and focus on features essential to the trained model 10's purpose of evaluating the construction joint surface.

[0068] FIG. 10 is a diagram showing the flow of processing in the full connection processor. The fully connected processor 15 converts the feature map output as a result of the focused processor 14 into a one-dimensional vector through one-dimensional vector conversion processing 151. In this embodiment, the feature map output as a result of the focused processor 14 has eight channels, each of which has a size of 20 × 36. Therefore, the one-dimensional vector conversion processing 151 generates a vector having 20 × 36 × 8 = 5760 elements.

[0069] Thereafter, the fully connected processing unit 15 takes the vector generated as described above as input, and sequentially executes the following processes as fully connected processes 152, 153, and 154: it calculates a weighted sum of the values ​​of each element in the previous layer, applies a ReLU activation function, and stores the sum as the value of an element in the next layer.

[0070] After executing the batch normalization process 155, the full connection processor 15 executes the final full connection process and performs the output process 156, which applies a sigmoid activation function. In this embodiment, the trained model 10 has only one output node. This output node stores the evaluation result of the construction joint surface for the image (segmented image 101) input to the trained model 10 as a value between 0 and 1. As already described, in this embodiment, the concrete construction joint surface evaluation device 1 evaluates the degree of laitance on the construction joint surface, particularly the presence or absence of laitance. Therefore, if the construction joint surface captured in the training input image does not have laitance and the condition of the construction joint surface is good, a value close to 1, for example, may be stored in the output node. Furthermore, if the construction joint surface captured in the training input image has laitance and the condition of the construction joint surface cannot be said to be good, a value of 0, for example, may be stored in the output node.

[0071] In this way, the construction joint surface evaluation result inference unit 3 inputs an image (segmented image 101) based on the input image into the trained model 10, and infers the evaluation result of the construction joint surface captured in the image based on the input image. In the concrete construction joint surface evaluation device 1 of this embodiment, the trained model 10 evaluates the degree of laitance of the input image (segmented image 101), but as already explained, instead of this, the degree of unevenness on the construction joint surface may be evaluated, or both the degree of unevenness on the construction joint surface and the degree of laitance on the construction joint surface may be evaluated together. The joint surface evaluation result inference unit inputs each of the multiple divided images 101 into the trained model 10 as an image based on the input image, and infers the evaluation result of the joint surface for each of the multiple divided images 101.

[0072] The display unit 4 displays the inferred evaluation results on a display device or the like (not shown). FIG. 11 shows an example of the output result of the concrete construction joint evaluation device. The display unit 4 generates an output image 110, for example, as shown in Fig. 11. More specifically, the display unit 4 generates the output image 110 so as to display, for example, the input image 100, for each of a plurality of regions corresponding to each of the divided images 101, in a color according to an evaluation result obtained by inputting the divided images 101 corresponding to the region into the trained model 10. As already explained, when the output result of the trained model 10 is such that a value close to 1 is stored in the output node if the condition of the joint surface is good, or a value of 0 is stored in the output node if the condition of the joint surface cannot be said to be good, the output result is compared with a judgment threshold having a value of, for example, 0.5, and if the output result is greater than the judgment threshold, the condition of the joint surface is deemed to be good and the relevant area is displayed as is in the input image 100, i.e., in a grayscale state; if the output result is smaller than the judgment threshold, the condition of the joint surface cannot be said to be good and the color tone of the relevant area is changed, for example, to a redder shade.

[0073] Next, a concrete construction joint evaluation method using the above-mentioned concrete construction joint evaluation device will be described with reference to Figures 1 to 11 and Figure 12. Figure 12 is a flowchart of the concrete construction joint evaluation method.

[0074] In the concrete joint surface evaluation method, first, an image of the concrete joint surface is acquired as an input image 100 (step S1: input image acquisition step). The construction joint surface evaluation result inference unit 3 divides the input image 100 into a plurality of regions of a predetermined size, cuts out the image for each region, divides it into a plurality of images, and generates divided images 101. The construction joint surface evaluation result inference unit 3 inputs each of the images based on the input image 100, i.e., the divided images 101, into a trained model 10 that has been trained using a pre-prepared learning input image in which the construction joint surface is captured and the evaluation result of the construction joint surface for the learning input image as training data, and infers the evaluation result of the construction joint surface captured in the image (divided image 101) based on the input image (step S2: construction joint surface evaluation result inference process). Finally, the display unit 4 displays the inferred evaluation result (step S3: display step).

[0075] The concrete joint surface evaluation device 1 as described above evaluates the joint surface of previously poured concrete to which concrete will be poured and joined later, and includes an input image acquisition unit 2 that acquires an image of the joint surface as an input image 100, a trained model 10 that has been trained using, as training data, a prepared input image of the joint surface, and an evaluation result of the joint surface for the training input image as teacher data, and inputs an image (divided image 101) based on the input image 100 to infer an evaluation result of the joint surface captured in the image based on the input image 100, and a display unit 4 that displays the inferred evaluation result, and the trained model 10 is constructed so that convolution processes 121, 122, 123, 131, 132, and 133 are performed multiple times to generate a feature map, and the multiple convolution processes 121, 122, 123, 131, 132, and 133 are performed to generate a feature map. The image processing system includes intra-channel individual convolution processes 122, 132 and inter-channel convolution processes 123, 133. The intra-channel individual convolution processes 122, 132 generate multiple new channels by applying kernels 20, 30 provided individually to each of the channels constituting the feature maps M21, M31 input to the intra-channel individual convolution processes 122, 132, respectively, and generate new feature maps M22, M32 by overlapping the new multiple channels. The inter-channel convolution processes 123, 133 perform convolution processes on pixels located at the same coordinates of each of the multiple channels constituting the feature maps M22, M32 generated by the intra-channel individual convolution processes 122, 132 in the direction in which the channels are overlapped to generate new channels. This process is repeated multiple times, and the generated new channels are overlapped to generate new feature maps M23, M33.

[0076] According to the above configuration, the joint surface of previously poured concrete to which the next concrete will be poured and joined is evaluated by inputting an image (segmented image 101) based on input image 100 of the joint surface into trained model 10, which has been trained using training input images of the joint surface and evaluation results of the joint surface for the training input images as training data, and inferring the evaluation results of the joint surface captured in the image. This allows for accurate and quantitative evaluation of the joint surface. Here, the trained model 10 is constructed to repeat multiple times the convolution processes 121, 122, 123, 131, 132, and 133 to generate a feature map, and the multiple convolution processes 121, 122, 123, 131, 132, and 133 include individual intra-channel convolution processes 122 and 132 and inter-channel convolution processes 123 and 133. First, in the intra-channel individual convolution processes 122 and 132, kernels 20 and 30, which are provided individually for each of the channels constituting the feature maps M21 and M31 input to the intra-channel individual convolution processes 122 and 132, are applied to generate new channels, and new feature maps M22 and M32 are generated by overlapping the new channels. In this process, the kernels 20 and 30 are provided to be applied to each of the channels constituting the feature maps M21 and M31, and therefore each has a planar configuration (e.g., like a single image) so that it has a certain size. Therefore, the number of parameters required in the intra-channel individual convolution processes 122 and 132 is C×K×K(×1), where C is the number of channels in the input feature maps M21 and M31 and K×K is the size of each kernel. Furthermore, in the inter-channel convolution processes 123 and 133, convolution processing is performed on pixels located at the same coordinates in each of the multiple channels constituting the feature maps M22 and M32 generated by the intra-channel individual convolution processes 122 and 132 in the direction in which the channels are overlapped to generate new channels. This process is repeated multiple times, and the multiple new channels generated are overlapped to generate new feature maps M23 and M33. In this process, the so-called filters 21 and 31 required to generate a new channel are a collection of kernels 22 and 32, each equivalent to one pixel, i.e., 1×1 in size, equal to the number of channels in the feature maps M22 and M32 generated by the intra-channel individual convolution processes 122 and 132. Therefore, the number of parameters required in the inter-channel convolution processes 123 and 133 is calculated by multiplying the number of channels in the input feature map by C and the number of channels in the newly generated feature maps M23 and M33, and then multiplying these two values ​​by C(×1×1)×D. In this way, in the above-described configuration, the number of required parameters can be significantly reduced compared to a typical CNN that requires the number of parameters equivalent to the multiplication of four values, and the amount of calculation can also be reduced accordingly. Furthermore, in the intra-channel individual convolution processes 122 and 132, the planar kernels 20 and 30 are applied to the input feature maps M21 and M31. Therefore, in the intra-channel individual convolution processes 122 and 132, the calculations are not performed between the channels constituting the feature maps M21 and M31, but are performed within the channels. As a result, while information is mixed within a channel, information is not mixed between channels. On the other hand, in the subsequent inter-channel convolution processes 123 and 133, a 1×1 kernel is applied to each channel, so information is not mixed within a channel. However, convolution processing is performed in the direction in which the channels overlap by filters 21 and 31, which are configured by gathering the same number of 1×1 kernels 22 and 32 as the number of channels of the feature maps M22 and M32 generated by the intra-channel individual convolution processes 122 and 132, thereby mixing information between channels. In this way, the intra-channel individual convolution processes 122, 132 do not mix information between channels, and the inter-channel convolution processes 123, 133 do not mix information within channels, thereby reducing the amount of calculation, but the intra-channel individual convolution processes 122, 132 and the inter-channel convolution processes 123, 133 share the work of mixing information both within and between channels. Therefore, basically, the evaluation accuracy of the construction joint surface is not impaired. In this way, it is possible to accurately evaluate the joint surface of previously poured concrete where subsequent concrete will be poured and joined, with a small amount of calculation.

[0077] For example, in the convolution process applied to a typical CNN, if the number of channels C of the input feature map is 32, the kernel size is 3 × 3, and the number of channels D of the output feature map is 64, then 32 × 3 × 3 × 64 = 18,432 parameters are required just to implement the filter for this convolution process. In contrast, consider the case where the above convolution processing is divided into individual intra-channel convolution processing and inter-channel convolution processing. In this case, C × K × K = 32 × 3 × 3 = 288 parameters are required to implement a filter for performing individual intra-channel convolution processing (see Figure 7). Furthermore, C × 1 × 1 × D = 32 × 1 × 1 × 64 = 2048 parameters are required to implement a filter for performing inter-channel convolution processing. Therefore, the total number of parameters is 288 + 2048 = 2336, which is approximately 1 / 7.9 of the original number.

[0078] In addition, the trained model 10 executes an attention map generation process 141 and an attention part extraction process 142 after multiple convolution processes 121, 122, 123, 131, 132, and 133. The attention map generation process 141 generates an attention map based on the feature map input to the attention map generation process 141, so that each pixel has a value closer to 1 if it is something that should be given special attention for evaluating the joint surface, and closer to 0 if it is something that does not require attention. The attention part extraction process 142 multiplies each of the channels that make up the feature map input to the attention map generation process 141 by the attention map for each pixel.

[0079] According to the above configuration, after the multiple convolution processes 121, 122, 123, 131, 132, and 133, the attention map generation process 141 and the attention portion extraction process 142 are executed. In the attention map generation process 141, an attention map is generated based on the feature map input to the attention map generation process 141 so that each pixel in each channel has a value closer to 1 if it is a pixel that requires emphasis and attention for evaluating the pour joint surface, and a value closer to 0 if it is not a pixel that requires attention. In addition, in the attention portion extraction process 142, the attention map is multiplied by each channel constituting the feature map input to the attention map generation process 141. As a result of this processing, in each channel of the feature map, pixels that are considered to be effective in evaluating the pour joint surface have a larger value, and pixels that are not considered effective in evaluating the pour joint surface have a smaller value. This enables more accurate evaluation of the pour joint surface.

[0080] In addition, the joint surface evaluation result inference unit 3 divides the input image 100 into multiple regions to generate multiple divided images 101, inputs each of the multiple divided images 101 into the trained model 10 as an image based on the input image 100, and infers the evaluation result of the joint surface for each of the multiple divided images 101.The display unit 4 displays the input image 100 for each of the multiple regions in a color corresponding to the evaluation result corresponding to that region.

[0081] According to the above configuration, it is possible to extract areas of the entire input image 100 where the evaluation of the pour joint surface is high or low.

[0082] Furthermore, the construction joint surface evaluation result inference unit 3 divides the input image 100 into a plurality of regions and further converts them into grayscale images, thereby generating a plurality of divided images 101 each as a one-channel image.

[0083] According to the above configuration, the trained model 10 can be generated efficiently.

[0084] The evaluation results of the joint surface are either the degree of unevenness on the joint surface or the degree of laitance on the joint surface, or both.

[0085] According to the above-described configuration, the joint surface can be evaluated by evaluating either or both of the degree of unevenness on the joint surface and the degree of laitance on the joint surface.

[0086] The concrete joint surface evaluation program as described above evaluates the joint surface of previously poured concrete to which concrete will be poured and joined later, and is for realizing the following in a computer: an input image acquisition function (input image acquisition unit 2) for acquiring an image of the joint surface as an input image 100; a construction joint surface evaluation result inference function (construction joint surface evaluation result inference unit 3) for inputting an image (divided image 101) based on the input image 100 into a trained model 10 that has been trained using a previously prepared learning input image of the joint surface and an evaluation result of the joint surface for the learning input image as training data; and a display function (display unit 4) for displaying the inferred evaluation result. The trained model 10 is constructed so that convolution processes 121, 122, 123, 131, 132, and 133 are performed multiple times to generate a feature map, and the multiple convolution processes 1 The processors 21, 122, 123, 131, 132, and 133 include intra-channel individual convolution processes 122 and 132 and inter-channel convolution processes 123 and 133. The intra-channel individual convolution processes 122 and 132 generate new channels by applying kernels 20 and 30, respectively, to the channels constituting the feature maps M21 and M31 input to the intra-channel individual convolution processes 122 and 132, and overlay the new channels to generate new feature maps M22 and M32. The inter-channel convolution processes 123 and 133 perform convolution processes between pixels located at the same coordinates in each of the channels constituting the feature maps M22 and M32 generated by the intra-channel individual convolution processes 122 and 132, in the direction in which the channels are overlaid, to generate new channels. This process is repeated multiple times, and the generated new channels are overlaid to generate new feature maps M23 and M33.

[0087] According to the above-described configuration, similar to that described with respect to the concrete joint surface evaluation device 1, it is possible to accurately evaluate the joint surface of previously poured concrete where subsequent concrete will be poured and joined, with a small amount of calculation.

[0088] The concrete joint surface evaluation method as described above evaluates the joint surface of previously poured concrete to which concrete will be poured and joined later, and includes an input image acquisition step (step S1) of acquiring an image of the joint surface as an input image 100, a construction joint surface evaluation result inference step (step S2) of inputting an image (divided image 101) based on the input image 100 to a trained model 10 that has been trained using, as training data, a previously prepared learning input image of the joint surface and an evaluation result of the construction joint surface for the learning input image as teacher data, and inferring an evaluation result of the construction joint surface captured in the image based on the input image 100, and a display step (step S3) of displaying the inferred evaluation result. The trained model 10 is constructed so that convolution processes 121, 122, 123, 131, 132, and 133 are performed multiple times to generate a feature map, and the multiple convolution processes 121, 122, 123, and 133 are repeated multiple times to generate a feature map. 1, 132, 133 comprise intra-channel individual convolution processes 122, 132 and inter-channel convolution processes 123, 133. The intra-channel individual convolution processes 122, 132 generate multiple new channels by applying kernels 20, 30 provided individually to each of the channels constituting the feature maps M21, M31 input to the intra-channel individual convolution processes 122, 132, respectively, and overlay the new multiple channels to generate new feature maps M22, M32. The inter-channel convolution processes 123, 133 perform convolution processes between pixels located at the same coordinates of each of the multiple channels constituting the feature maps M22, M32 generated by the intra-channel individual convolution processes 122, 132 in the direction in which the channels are overlaid to generate new channels, repeating this process multiple times, and overlaying the generated new channels to generate new feature maps M23, M33.

[0089] According to the above-described configuration, similar to that described with respect to the concrete joint surface evaluation device 1, it is possible to accurately evaluate the joint surface of previously poured concrete where subsequent concrete will be poured and joined, with a small amount of calculation.

[0090] The concrete joint surface evaluation device, concrete joint surface evaluation program, and concrete joint surface evaluation method of the present invention are not limited to the above-described embodiments explained with reference to the drawings, and various other modifications are possible within the technical scope. For example, in the above embodiment, two convolution processing units that perform intra-channel individual convolution processing and inter-channel convolution processing are provided: the first convolution processing unit 12 and the second convolution processing unit 13. Alternatively, the trained model may be configured to include three or more convolution processing units, and to perform each of intra-channel individual convolution processing and inter-channel convolution processing three or more times. In addition to this, it is possible to select and discard the configurations given in the above embodiments, or to change them to other configurations as appropriate. [Explanation of symbols]

[0091] 1. Concrete joint evaluation device 2. Input image acquisition section 3. Joint surface evaluation result inference section 4 Display 10 Pre-trained models 20, 30 individual convolution kernels within channels 121, 122, 123, 131, 132, 133 Convolution processing Individual convolution processing within 122 and 132 channels 123, 133 Inter-channel convolution processing 141 Attention map generation process 142 Attention Extraction Processing 100 input images 101 Segmented images (images based on input image) Feature maps input to individual convolution processing within M21 and M31 channels M22, M32: New feature maps (generated by individual convolution within channels) M22, M32: New feature maps (generated by inter-channel convolution)

Claims

1. A concrete joint surface evaluation device that evaluates the joint surface of previously poured concrete to which concrete will be joined later, An input image acquisition unit that acquires an image of the joint surface as an input image; A construction joint surface evaluation result inference unit that inputs an image based on the input image into a trained model that has been trained using a previously prepared learning input image in which the construction joint surface is captured and an evaluation result of the construction joint surface for the learning input image as training data, and infers the evaluation result of the construction joint surface captured in the image based on the input image; a display unit that displays the inferred evaluation result; Equipped with The trained model is constructed so as to repeat a convolution process to generate a feature map multiple times; The plurality of convolution processes include intra-channel individual convolution processes and inter-channel convolution processes; the intra-channel individual convolution processing applies kernels individually provided for each of the channels constituting the feature map input to the intra-channel individual convolution processing to generate a plurality of new channels, and then generates the new feature map by overlapping the new plurality of channels; The inter-channel convolution process generates new channels by performing convolution process on pixels located at the same coordinates of each of the plurality of channels constituting the feature map generated by the intra-channel individual convolution process in a direction in which the channels are overlapped, and repeats this process multiple times to generate new channels, and overlaps the generated new channels to generate new feature maps. A concrete joint surface evaluation device characterized by:

2. The trained model performs an attention map generation process and an attention portion extraction process after the plurality of convolution processes; The attention map generation process generates an attention map based on the feature map input to the attention map generation process, in which each pixel has a value closer to 1 if it is something that should be given emphasis and attention for the evaluation of the joint surface, and a value closer to 0 if it is something that should not be given attention, The attention portion extraction process multiplies each of the channels constituting the feature map input to the attention map generation process by the attention map for each pixel.

2. The concrete construction joint surface evaluation device according to claim 1.

3. The construction joint surface evaluation result inference unit divides the input image into a plurality of regions to generate a plurality of divided images, inputs each of the plurality of divided images into the trained model as an image based on the input image, and infers the evaluation result of the construction joint surface for each of the plurality of divided images, The display unit displays the input image for each of the plurality of regions in a color according to the evaluation result corresponding to the region.

3. The concrete joint surface evaluation device according to claim 1 or 2.

4. The construction joint surface evaluation result inference unit divides the input image into a plurality of regions and further grayscales the input image, thereby generating each of the plurality of divided images as a one-channel image.

4. The concrete construction joint surface evaluation device according to claim 3.

5. The evaluation result of the joint surface is either one or both of the degree of unevenness on the joint surface and the degree of laitance on the joint surface.

3. The concrete joint surface evaluation device according to claim 1 or 2.

6. A concrete joint surface evaluation program for evaluating a joint surface of a previously placed concrete to be joined by a subsequent concrete joint, On the computer, An input image acquisition function that acquires an image of the joint surface as an input image; A construction joint surface evaluation result inference function that inputs an image based on the input image into a learned model that has been trained using a previously prepared learning input image in which the construction joint surface is captured and the evaluation result of the construction joint surface for the learning input image as training data, and infers the evaluation result of the construction joint surface captured in the image based on the input image; a display function for displaying the inferred evaluation result; This is to realize the following: The trained model is constructed so as to repeat a convolution process to generate a feature map multiple times; The plurality of convolution processes include intra-channel individual convolution processes and inter-channel convolution processes; the intra-channel individual convolution processing applies kernels individually provided for each of the channels constituting the feature map input to the intra-channel individual convolution processing to generate a plurality of new channels, and then generates the new feature map by overlapping the new plurality of channels; The inter-channel convolution process generates new channels by performing convolution process on pixels located at the same coordinates of each of the plurality of channels constituting the feature map generated by the intra-channel individual convolution process in a direction in which the channels are overlapped, and repeats this process multiple times to generate new channels, and overlaps the generated new channels to generate new feature maps. A concrete joint surface evaluation program characterized by:

7. A concrete joint surface evaluation method for evaluating a joint surface of previously poured concrete where concrete is subsequently poured and joined, An input image acquisition step of acquiring an image of the joint surface as an input image; A construction joint surface evaluation result inference process, which inputs an image based on the input image into a trained model trained using a previously prepared learning input image in which the construction joint surface is imaged and an evaluation result of the construction joint surface for the learning input image as training data, and infers the evaluation result of the construction joint surface imaged in the image based on the input image; a display step of displaying the inferred evaluation result; Equipped with The trained model is constructed so as to repeat a convolution process to generate a feature map multiple times; The plurality of convolution processes include intra-channel individual convolution processes and inter-channel convolution processes; the intra-channel individual convolution processing applies kernels individually provided for each of the channels constituting the feature map input to the intra-channel individual convolution processing to generate a plurality of new channels, and then generates the new feature map by overlapping the new plurality of channels; The inter-channel convolution process generates new channels by performing convolution process on pixels located at the same coordinates of each of the plurality of channels constituting the feature map generated by the intra-channel individual convolution process in a direction in which the channels are overlapped, and repeats this process multiple times to generate new channels, and overlaps the generated new channels to generate new feature maps. A method for evaluating a concrete joint surface.

Images