Reinforcement inspection device, learning device, reinforcement inspection system, and reinforcement inspection method

Abstract

This reinforcement inspection device (2) comprises: an acquisition unit (221) that acquires image information; a selection unit (223) that selects a designated learning model from among a plurality of learning models for inferring a sub-component member appearing in the image information, the learning models being provided with respect to each external appearance feature including the type, color, and shape of the sub-component member; and an inference unit (224) that infers a sub-component member from the image information using the selected learning model.

Description

[Technical Field]

[0001] The present disclosure relates to a reinforcement inspection device, a learning device, a reinforcement inspection system, and a reinforcement inspection method. [Background technology]

[0002] In the construction of reinforced concrete structures, an inspection (reinforcement inspection) is conducted to check whether the reinforcing bars in a reinforcement structure in which multiple reinforcing bars are arranged are arranged as designed. For example, Patent Document 1 describes a reinforcement as-built management system that uses at least one of pattern matching and machine learning to generate data such as the type of reinforcing bars, the number of reinforcing bars, the reinforcing bar arrangement pitch, the length of the reinforcing bars, the thickness of the reinforcing bars, and the shape and position of joints within a set imaging range. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-27058 Summary of the Invention [Problem to be solved by the invention]

[0004] Actual reinforced concrete structures are composed of not only the main reinforcing bars, which are the main reinforcing bars, but also various other components. Components other than the main reinforcing bars in reinforced concrete structures include components such as joints that are installed as part of the main reinforcing bars, and also components such as shear reinforcement that are installed as separate components from the main reinforcing bars. However, in conventional reinforcement inspections, although technology has been proposed to automatically detect the main reinforcing bars, there is no technology that can automatically detect various other components apart from the main reinforcing bars, and inspections have traditionally been performed manually by inspectors.

[0005] The reinforcement as-built management system described in Patent Document 1 automatically detects main reinforcement and the joints attached to some of the main reinforcement, but does not anticipate the detection of components separate from the main reinforcement, such as shear reinforcement. For this reason, even with the reinforcement as-built management system described in Patent Document 1, it is expected that inspectors will still have to manually inspect components that are attached separate from the main reinforcement in reinforcement structures.

[0006] The present disclosure is intended to solve the above-mentioned problems, and aims to provide a reinforcement inspection device, a learning device, a reinforcement inspection system, and a reinforcement inspection method that can automatically detect secondary components other than the main reinforcement in a reinforcement structure. [Means for solving the problem]

[0007] The bar arrangement inspection device according to the present disclosure is a bar arrangement inspection device for inspecting a reinforcement structure in which a plurality of reinforcing bars are arranged as main reinforcements and which includes secondary components other than the main reinforcements, the bar arrangement inspection device comprising: an acquisition unit for acquiring image information; Based on the design data of the structure including the reinforcement structure, A learning model for inferring secondary components shown in image information, which is a learning model provided for each appearance feature including color and shape for secondary components of the same type. From inside , To infer secondary components included in reinforced structures Learning model Automatically The system includes a selection unit that selects a learning model, and an inference unit that infers a sub-component from image information using the selected learning model. [Effects of the Invention]

[0008] According to the present disclosure, a specified learning model is selected from a plurality of learning models provided for each appearance feature of a secondary component, including the type, color, and shape, for inferring the secondary component shown in image information, and the selected learning model is used to infer the secondary component from the image information. This allows the reinforcement inspection device according to the present disclosure to automatically detect secondary components provided other than the main reinforcement in a reinforcement structure. [Brief explanation of the drawings]

[0009] [Figure 1]1 is a block diagram showing the configuration of a bar arrangement inspection system according to a first embodiment. [Figure 2] 1 is a block diagram showing a hardware configuration for realizing the functions of the bar arrangement inspection device according to the first embodiment. FIG. [Figure 3] FIG. 10 is a screen diagram showing an example of an operation screen. [Figure 4] FIG. 10 is a diagram showing an example of registered contents of a selection candidate database (hereinafter referred to as DB). [Figure 5] FIG. 10 is a screen diagram showing an example of image information obtained by photographing a reinforcement structure. [Figure 6] 3 is a flowchart showing a reinforcement bar arrangement inspection method according to the first embodiment. [Figure 7] FIG. 10 is a diagram illustrating an example of learning data. [Figure 8] FIG. 2 is a block diagram showing the configuration of a learning unit. [Figure 9] FIG. 10 is a diagram showing the evaluation results of the learning model. [Figure 10] FIG. 10 is an explanatory diagram illustrating a schematic diagram of a matching determination of an embedding vector in a learning model. [Figure 11] 10 is a flowchart showing a process for creating a preset model. [Figure 12] 10 is a flowchart illustrating a process for creating a custom model. DETAILED DESCRIPTION OF THE INVENTION

[0010] Embodiment 1 FIG. 1 is a block diagram showing the configuration of a reinforcement inspection system 1 according to a first embodiment. In FIG. 1, the reinforcement inspection system 1 is a system in which a reinforcement inspection device 2 and a learning device 3 are connected via communication, and inspects a reinforcement structure before concrete is poured. The reinforcement inspection device 2 acquires image information showing the reinforcement structure, and detects secondary components in the reinforcement structure using a learning model for inferring secondary components in the reinforcement structure that are shown in the acquired image information. The learning device 3 creates a learning model used by the reinforcement inspection device 2 to detect secondary components.

[0011] The reinforcement inspection device 2 inspects at least the secondary components in the reinforcement structure using image information of the reinforcement structure captured by the camera device, and outputs the inspection results to the display unit 23. For example, the reinforcement inspection device 2 inspects the number of each of multiple types of secondary components included in the reinforcement structure, or inspects whether the secondary components have been created as designed. The reinforcement inspection device 2 is, for example, a tablet terminal, a smartphone, or a personal computer (PC). The learning device 3 is, for example, a server that provides the reinforcement inspection device 2 with a learning model used to detect secondary components.

[0012] A reinforced concrete structure is composed of multiple main reinforcements, which are arranged reinforcing bars, and secondary components other than the main reinforcements. The main reinforcements are the main reinforcing bars that bear the load of a building or the like that is based on the reinforced concrete structure, and are also called distribution reinforcements. For example, a reinforced concrete structure may have a structure in which multiple flat surfaces are provided with multiple main reinforcements arranged in a lattice pattern.

[0013] A secondary component is a component that is provided to support the main reinforcement in a reinforced structure or to give the reinforced structure a function different from that of the main reinforcement, and is a part of the main reinforcement or a component that is completely separate from the main reinforcement. For example, secondary components include shear reinforcement, lap joints, spacer blocks, sheath pipes, and compression joints. Hereinafter, it is assumed that the reinforced structure inspected by the reinforcement inspection system 1 is provided with at least one of shear reinforcement, lap joints, spacer blocks, sheath pipes, and compression joints.

[0014] Shear reinforcement is a reinforcing bar that restrains and reinforces the main reinforcement. By attaching shear reinforcement to the main reinforcement, the shear force acting on the main reinforcement is suppressed. Shear reinforcement can be U-shaped, V-shaped, or C-shaped, for example. U-shaped or V-shaped shear reinforcement has longer reinforcing bars on both sides and is sometimes installed to surround the main reinforcement that constitutes a column, etc. C-shaped shear reinforcement has shorter reinforcing bars on both sides than U-shaped or V-shaped shear reinforcement, and for example, one end is connected to one main reinforcement that is spaced apart, and the other end is connected to the other main reinforcement.

[0015] The ends of the shear reinforcement are hooked to connect to the main reinforcement bars. There are variations in hook shape, such as right-angle hooks, acute-angle hooks, or semicircular hooks. In addition, the rebars that make up the main reinforcement and shear reinforcement are sometimes coated with resin to improve corrosion resistance, resulting in a different color from the base metal. For example, rebars coated with epoxy resin can be blue or green, and those with special surface treatments can even be gray. As such, shear reinforcement has a variety of appearance characteristics, including color and shape. Furthermore, the appearance characteristics of shear reinforcement vary depending on the manufacturer and model.

[0016] A lap splice is a joint formed by overlapping the ends of two reinforcing bars, which are the main reinforcing bars. Secondary components also include components that are part of the main reinforcing bars. In other words, the reinforcing bars that make up a lap splice are sometimes coated with resin to improve corrosion resistance, and their appearance is a different color from the base metal.

[0017] Lap splices also come in a variety of shapes depending on the type of rebar and the intended use of the reinforced structure. For example, the length of overlap between the ends of two rebars in a lap splice varies depending on the intended use of the reinforced structure. Lap splices vary in shape depending on whether the rebar ends have hooks or not, and also by the shape of the hooks. Examples of hook shapes include right-angle hooks, acute-angle hooks, and semicircular hooks. Thus, lap splices come in a variety of appearance features, including color and shape.

[0018] Spacer blocks are components used to maintain the cover of rebar and prevent the rebar from becoming distorted during work. "Cover" is the minimum distance from the concrete surface to the rebar. Spacer blocks consist of spacers and bar supports. Spacers ensure the cover of rebar on the sides, while bar supports ensure the cover of rebar in the horizontal direction.

[0019] Spacer blocks are made of concrete, steel, or plastic, and come in colors that correspond to their material. Spacer blocks also come in a variety of shapes, including dice shapes, grooved shapes with grooves for rebar, and inverted V shapes. Spacer blocks have a variety of appearance features, including color and shape. Furthermore, the appearance features of spacer blocks vary by manufacturer and model.

[0020] A sheath pipe is a metal pipe through which a steel wire is passed. For example, a sheath pipe is made of galvanized steel sheet and has a silver appearance. Sheath pipes are installed at various positions in a reinforcement structure depending on their use, and their lengths also vary. In addition, sheath pipes are colored according to their material. As such, sheath pipes have various appearance characteristics, including color and shape. Furthermore, the appearance characteristics of sheath pipes vary depending on the manufacturer and the model.

[0021] A compression joint is a joint where the ends of two reinforcing bars, which are the main reinforcement, are butt-joined. In other words, a compression joint is a secondary component that is installed as part of the main reinforcement. In addition, the reinforcing bars that make up a compression joint are sometimes coated with resin to improve corrosion resistance, and their appearance is a different color from the base metal.

[0022] The shape of compression joints is subject to reinforcement inspection. For example, the law requires that the diameter of the bulge where the ends of rebars are butted together must be at least 1.4 times the diameter of the rebar, and that the length of the butt-jointed part must be at least 1.1 times the diameter of the rebar. Furthermore, the eccentricity of the rebar in a compression joint from its central axis must be no more than one-fifth the diameter of the rebar, and the deviation of the compression surface from the top of the bulge must be no more than one-fourth the diameter of the rebar. As such, compression fittings come in a variety of appearance features, including color and shape.

[0023] As shown in Fig. 1, the reinforcement arrangement inspection device 2 includes a communication unit 21, a calculation unit 22, a display unit 23, an operation input unit 24, and a memory unit 25. The learning device 3 includes a communication unit 31, a calculation unit 32, and a memory unit 33. Although Fig. 1 shows a case where the reinforcement arrangement inspection device 2 and the learning device 3 are separate devices, the reinforcement arrangement inspection device 2 may also include the learning device 3. In this case, the communication units 21 and 31 may be communication devices that exchange data within a common device. Furthermore, the memory units 25 and 33 may be memory areas constructed in a common storage device.

[0024] Furthermore, the reinforcement inspection system 1 or the reinforcement inspection device 2 equipped with the learning device 3 may provide a reinforcement inspection service in the form of SaaS (Software as a Service) to a user terminal (not shown in FIG. 1) that can communicate with the reinforcement inspection device 2. For example, a reinforcement inspection application for providing the reinforcement inspection service is executed on the reinforcement inspection device 2, and the user terminal can receive the reinforcement inspection service on a web browser without having to install a dedicated application for the service.

[0025] The user terminal transmits image information of the reinforcement structure, captured by a camera provided on the terminal, to the reinforcement inspection device 2. The reinforcement inspection device 2 uses the image information received from the user terminal to inspect the secondary components and the like shown in the image information and returns the inspection results to the user terminal. The user terminal receives the inspection results from the reinforcement inspection device 2 and can display the inspection results in an appropriate manner on a display unit (not shown) provided on the user terminal.

[0026] The communication unit 21 communicates with the learning device 3 via a communication line. For example, the communication unit 21 can communicate via a communication line with the learning device 3 that is capable of communication using a communication method such as LTE, 3G, 4G, or 5G. The communication unit 31 communicates with the reinforcement bar arrangement inspection device 2 via a communication line. For example, like the communication unit 21, the communication unit 31 can communicate via a communication line with the reinforcement bar arrangement inspection device 2 that is capable of communication using a communication method such as LTE, 3G, 4G, or 5G.

[0027] The calculation unit 22 controls the overall operation of the reinforcement bar arrangement inspection device 2. The calculation unit 22 includes an acquisition unit 221, a preprocessing unit 222, a selection unit 223, and an inference unit 224. The calculation unit 22 executes the reinforcement bar arrangement inspection application, thereby realizing the functions of the acquisition unit 221, the preprocessing unit 222, the selection unit 223, and the inference unit 224.

[0028] The calculation unit 32 controls the overall operation of the learning device 3. The calculation unit 32 includes a data acquisition unit 321, a learning unit 322, and a search unit 323. The calculation unit 32 executes a learning application, thereby realizing the functions of the data acquisition unit 321, the learning unit 322, and the search unit 323.

[0029] The display unit 23 is a display device included in the bar arrangement inspection device 2. The display unit 23 is, for example, an LCD (Liquid Crystal Display) or an organic EL (Electroluminescence) display device.

[0030] The operation input unit 24 is an input device that accepts operations on an operation screen, described below, displayed on the display unit 23. When the reinforcement bar arrangement inspection device 2 is a smartphone or a tablet terminal, the operation input unit 24 is, for example, a touch panel that is provided integrally with the screen of the display unit 23. When the reinforcement bar arrangement inspection device 2 is a PC, the operation input unit 24 is, for example, a mouse or a keyboard.

[0031] The storage unit 25 stores, for example, a reinforcement inspection application and information used for arithmetic processing by the calculation unit 22. The storage unit 25 stores, as information used for arithmetic processing, for example, image information, position information of objects shown in the image information, and a learning model acquired from the learning device 3. The storage unit 25 is a storage device provided in a computer functioning as the reinforcement inspection device 2, and includes storage such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), or a memory 103 in FIG. 2 (described later), etc.

[0032] The storage unit 33 stores, for example, a selection candidate DB 331, a pre-learning model DB 332, and a learning DB 333 in addition to a learning application. The storage unit 33 is a storage device provided in a computer that functions as the learning device 3, and includes, for example, a storage such as an HDD or SSD, or a memory 103 in FIG. 2 described later.

[0033] The selection candidate DB 331 stores one or more preset models. A preset model is a first learning model provided for each appearance feature, including the type, color, and shape, of a secondary component, for inferring the secondary component shown in the image information acquired by the acquisition unit 221. The preset model is created in advance by the learning device 3 before being specified by the selection unit 223. These preset models are stored in the selection candidate DB 331, linked to each appearance feature, including the type, color, and shape of the secondary component.

[0034] The pre-training model DB 332 stores pre-training models. The pre-training models are models trained to infer objects shown in image information. For example, the pre-training models are models trained using a large amount of training data set such as COCO (Common Objects in Context). As a training method, for example, stochastic gradient descent may be used to set parameters of a neural network.

[0035] The learning DB 333 stores learning data including image information and positional information of objects shown in the image information. The positional information of objects shown in the image information is set, for example, using the operation input unit 24 of the reinforcement arrangement inspection device 2. When the learning device 3 receives information specifying a learning model from the reinforcement arrangement inspection device 2, it reads out a pre-learning model from the pre-learning model DB 332 and reads out learning data from the learning DB 333. The learning device 3 inputs the learning data into the pre-learning model to create a custom model for inferring secondary components shown in the image information. The custom model is created for each appearance feature of a secondary component, including its type, color, and shape, and is a second learning model for inferring secondary components shown in the image information acquired by the acquisition unit 221.

[0036] 2 is a block diagram showing a hardware configuration that realizes the functions of the reinforcement bar arrangement inspection device 2. For example, the reinforcement bar arrangement inspection device 2 has, as its hardware configuration, a communication interface 100, an input / output interface 101, a processor 102, and a memory 103. The functions of the acquisition unit 221, preprocessing unit 222, selection unit 223, and inference unit 224 provided in the reinforcement bar arrangement inspection device 2 are realized by executing a reinforcement bar arrangement inspection application in this hardware configuration.

[0037] The communication interface 100 outputs the learning model received from the learning device 3 via the communication line to the processor 102, and transmits designation information for the learning model generated by the processor 102 to the learning device 3 via the communication line. The processor 102 reads and writes data from the storage unit 25 in FIG. 1 via the input / output interface 101. Furthermore, the processor 102 acquires image information from an external device via the input / output interface 101. The external device is, for example, a camera device that captures images of the reinforcement structure, or an external storage device that stores image information captured by the camera device.

[0038] The programs constituting the reinforcement inspection application for realizing the functions of the acquisition unit 221, preprocessing unit 222, selection unit 223, and inference unit 224 are stored in the storage unit 25. The processor 102 reads the programs stored in the storage unit 25 via the input / output interface 101, loads them into the memory 103, and executes the programs loaded into the memory 103. In this way, the processor 102 realizes the functions of the acquisition unit 221, preprocessing unit 222, selection unit 223, and inference unit 224. The memory 103 is, for example, a RAM (Random Access Memory).

[0039] When the functions of the learning device 3 are realized using the hardware configuration shown in Figure 2, the functions of the data acquisition unit 321, learning unit 322, and search unit 323 provided in the learning device 3 are realized by executing a learning application in the above hardware configuration.

[0040] The communication interface 100 outputs designation information for the learning model received from the bar arrangement inspection device 2 via the communication line to the processor 102, and transmits the learning model searched by the processor 102 via the communication line to the bar arrangement inspection device 2. The processor 102 reads and writes data from the memory unit 33 in Fig. 1 via the input / output interface 101.

[0041] A program constituting a learning application for realizing each function of data acquisition unit 321, learning unit 322, and search unit 323 is stored in storage unit 33. Processor 102 reads the program stored in storage unit 33 via input / output interface 101, loads it into memory 103, and executes the program loaded into memory 103. In this way, processor 102 realizes each function of data acquisition unit 321, learning unit 322, and search unit 323.

[0042] The functional components of the reinforcement bar inspection device 2 will be described. The acquisition unit 221 acquires image information. For example, the acquisition unit 221 is connected to a camera device via wireless communication or wired communication, and receives image information (still images or video) of the reinforcement structure from the camera device. The information acquired by the acquisition unit 221 is output to the preprocessing unit 222. The acquisition unit 221 may also output the acquired information to the memory 103 shown in FIG. 2 for storage.

[0043] The camera device is assumed to be, for example, a monocular camera, but may also be a stereo camera or an infrared camera. A stereo camera or infrared camera also obtains distance information between the reinforcement structure and the camera device. The acquisition unit 221 may acquire this distance information in addition to the image information. When the reinforcement inspection device 2 is a smartphone or tablet terminal, the camera device may be a camera included in the smartphone or tablet terminal.

[0044] The preprocessing unit 222 preprocesses the image information acquired by the acquisition unit 221 so that it is in a form suitable for inference processing performed by the inference unit 224. For example, the preprocessing unit 222 normalizes the image information. Normalization is a process of adjusting pixel values ​​on the screen of the display unit 23 that displays the image information to values ​​within a certain range. When the image information is a color image, the color value of the ith pixel in the image takes a value from 0 to 255, r i (red), g i (green) and b i (blue) (r i ,g i ,b i ) can be expressed as

[0045] When a learning model is created by deep learning (hereinafter referred to as DL), DL generally handles color values ​​of 0 to 255 in the range of 0 to 1. Therefore, the preprocessing unit 222 calculates the color value (r i ,g i ,b i ) is the normalized color value (r i Hat,g i Hat,b iIn addition, if the image information is a moving image consisting of frame images shot at a constant frame rate, normalization is performed for each frame image. This allows the inference unit 224 to smoothly infer the sub-components. TIFF0007814535000001.tif15166

[0046] Furthermore, the preprocessing unit 222 may perform normalization after orienting the image indicated by the image information. An orientated image is an image in which the camera device is positioned at a constant distance from the reinforcement structure and the reinforcement structure is positioned directly in front of the camera device. For example, the preprocessing unit 222 specifies the four corner vertices of any rectangle on the reinforcing bars arranged in a lattice pattern in the image information, and estimates a transformation matrix using the position coordinates of the specified four vertices. Then, based on the estimated transformation matrix, the preprocessing unit 222 converts the image indicated by the image information into an orientated image in which the plane of the inspection target is positioned directly in front of the camera device.

[0047] The selection unit 223 is provided for each appearance feature including the type, color, and shape of the secondary component, and selects a specified learning model from among multiple learning models for inferring the secondary component shown in the image information. For example, the selection unit 223 outputs display control information for displaying an operation screen to the display unit 23. The display unit 23 displays the operation screen in accordance with the display control information from the selection unit 223. This operation screen is a screen on which an operation to specify a learning model is performed.

[0048] When a learning model is designated by an operation received using the operation input unit 24 based on the operation screen, the selection unit 223 creates designation information for the learning model and transmits the designation information to the learning device 3 via a communication line using the communication unit 21. When the learning device 3 receives the designation information from the reinforcement arrangement inspection device 2, it searches for the learning model indicated by the designation information from among the multiple learning models stored in the storage unit 33, and returns information indicating the learning model found as a result of the search to the reinforcement arrangement inspection device 2.

[0049] Furthermore, the learning models to be selected include preset models and custom models. The preset model is a learning model for inferring secondary components of a reinforcement structure from image information, and is created in advance by the learning device 3. For example, a large amount of learning data set such as COCO is used to create the preset model, and the selection candidate DB 331 stores preset models whose evaluation results are above the allowable lower limit. For example, the precision with respect to the correct data and the recall of the inference results are used for the evaluation.

[0050] Like the preset models, the custom models are learning models for inferring secondary components in a reinforcement structure from image information. However, creation of the custom model is initiated by the learning device 3 when the selection unit 223 accepts the designation of the custom model. To create a custom model, it is not possible to prepare a large data set as learning data, but image information captured on-site and in which the positions of the secondary components are identified is used as learning data. This allows the reinforcement inspection device 2 to infer secondary components using highly rated preset models and to infer secondary components using custom models suited to the situation on-site.

[0051] Fig. 3 is a screen diagram showing an example of operation screen 23A. Selection unit 223 outputs display control information to display unit 23 for displaying an operation screen on which an operation for specifying a preset model and a custom model is performed. Display unit 23 displays operation screen 23A as shown in Fig. 3 in accordance with the display control information from selection unit 223. Operation screen 23A displays selection buttons 23A-1, 23A-2, 23A-3, ... for selecting each of a plurality of preset models.

[0052] In FIG. 3, selection button 23A-1 is a selection button for specifying a preset model for inferring a shear reinforcement having the appearance characteristics of brown and U-shaped. Selection button 23A-2 is a selection button for specifying a preset model for inferring a shear reinforcement having the appearance characteristics of blue and C-shaped. Selection button 23A-3 is a selection button for specifying a preset model for inferring a reinforcing bar lap splice having the appearance characteristics of blue and cylindrical. These selection buttons slide to visible and selectable positions on operation screen 23A by operating slide bar 23Ab using operation input unit 24.

[0053] For example, an inspector of a reinforcement structure refers to design data for a structure including the reinforcement structure, identifies a secondary component to be used in the reinforcement structure, and uses the operation input unit 24 to press a selection button for a learning model corresponding to the identified secondary component. Operation information indicating which selection button was pressed is output from the operation input unit 24 to the selection unit 223. The selection unit 223 creates specification information for a preset model corresponding to the operation information, and transmits the specification information to the learning device 3 via the communication line using the communication unit 21.

[0054] Fig. 4 is a diagram showing an example of the registered contents of the selection candidate DB 331. As shown in Fig. 4, the storage unit 33 included in the learning device 3 stores the selection candidate DB 331, in which a plurality of preset models are stored and linked to appearance features including the type, color, and shape of the secondary component. Upon receiving the above-mentioned specification information from the reinforcement arrangement inspection device 2, the learning device 3 searches for the preset model indicated by the specification information from the plurality of preset models stored in the selection candidate DB 331, and returns information indicating the preset model as a search result to the reinforcement arrangement inspection device 2.

[0055] For example, when the selection buttons 23A-1, 23A-2, and 23A-3 shown in FIG. 3 are pressed, the learning device 3 searches the selection candidate DB 331 for models A1, A2, and B1 shown in FIG. 4 and returns information indicating the search results for models A1, A2, and B1 to the reinforcement arrangement inspection device 2. The information indicating a learning model such as model A1 is, for example, a parameter required to construct a neural network that functions as a learning model, such as a weighting coefficient for a node. When the selection unit 223 receives the information indicating models A1, A2, and B1 via the communication line from the communication unit 21, the selection unit 223 outputs the received information to the inference unit 224 and further stores it in the storage unit 25. This allows the selection unit 223 to accurately select a preset model for inferring a secondary component.

[0056] Although Figure 4 shows model B1 for predicting a lap joint of rebars with the external characteristics of being blue and cylindrical, the shape of the lap joint is not limited to the shape of the rebars on which the lap joint is formed. For example, the shape of the lap joint includes the length over which the ends of the two rebars overlap. Furthermore, the shape of the lap joint also includes the presence or absence of hooks at the ends of the rebars. Furthermore, if the rebars have hooks, the shape of the hooks may also include, for example, right-angle hooks, acute-angle hooks, or semicircular hooks.

[0057] The shape of the compression joint includes the shape of the rebar where the compression joint is formed, as well as the shape of the bulge at the butt-jointed end of the rebar. The shape of the bulge is determined by, for example, the diameter of the bulge, the length of the bulge where the butt-jointed end of the rebar is, the eccentricity of the bulge from the central axis of the rebar, and the deviation of the compression surface from the top of the bulge.

[0058] Furthermore, the selection unit 223 may automatically specify and select a preset model from a plurality of preset models. For example, design data of a structure including a reinforcement structure is stored in the storage unit 25. When an inspector of the reinforcement structure issues an instruction to start inspecting the reinforcement structure using the operation input unit 24, the selection unit 223 automatically identifies the secondary components included in the reinforcement structure from the design data stored in the storage unit 25.

[0059] The design data may be, for example, a three-dimensional model that realizes Building Information Modeling (BIM) for a structure including a reinforcement structure, or design drawing data for a structure including a reinforcement structure. The selection unit 223 creates designation information that specifies a preset model for inferring the identified secondary component, and acquires the preset model searched for by the learning device 3 based on the designation information. This allows the selection unit 223 to accurately select a preset model for inferring the secondary component.

[0060] Furthermore, the selection unit 223 may select all of the preset models stored in the selection candidate DB 331. For example, when an instruction to start an inspection of a reinforcement structure is given using the operation input unit 24, the selection unit 223 selects all of the preset models stored in the selection candidate DB 331. The inference unit 224 infers the secondary components using each of all of the preset models selected by the selection unit 223, and outputs the result obtained from the model with the highest evaluation as the final inference result.

[0061] 3, selection button 23Ac is a selection button for creating a custom model. When selection button 23Ac is pressed using operation input unit 24, selection unit 223 creates designation information indicating the designation of a custom model, and further creates learning data including image information used to create the custom model. Selection unit 223 transmits the custom model designation information and learning data including the image information to learning device 3 via communication unit 21 over a communication line.

[0062] The learning device 3 creates a custom model corresponding to the specified information using the learning data received from the reinforcement arrangement inspection device 2, and transmits the created custom model to the reinforcement arrangement inspection device 2 via a communication line using the communication unit 31. When the communication unit 21 receives information indicating the custom model via the communication line, the selection unit 223 outputs the received information to the inference unit 224 and further stores it in the storage unit 25.

[0063] The training data used to create the custom model is image information taken on-site and in which the positions of the sub-components are identified. Fig. 5 is a screen shot showing an example of image information of a reinforcement structure. The reinforcement structure shown in the image information in Fig. 5 has multiple reinforcing bars 11, which are main reinforcements, arranged in a lattice pattern, and includes shear reinforcement bars 12, spacer blocks 13, lap joints 14, sheath pipes 15, and compression joints 16 as secondary components.

[0064] When the selection button 23Ac is pressed using the operation input unit 24, the selection unit 223 causes the display unit 23 to display the screen shown in Fig. 5. The inspector identifies each of the secondary components on the screen using the operation input unit 24. For example, the inspector uses the operation input unit 24 to surround the area on the screen shown in Fig. 5 where the shear reinforcement 12 is displayed with a bounding box 23B-1, and inputs the color and name (in this case, "shear reinforcement") of the shear reinforcement 12 surrounded by the bounding box 23B-1.

[0065] The inspector also uses operation input unit 24 to surround the area on the screen shown in Fig. 5 where spacer block 13 is displayed with bounding box 23B-2, and inputs the color and name (in this case, "spacer block") of spacer block 13 surrounded by bounding box 23B-2. Furthermore, the inspector uses operation input unit 24 to surround the area on the screen shown in Fig. 5 where lap joint 14 is displayed with bounding box 23B-3, and inputs the color and name (in this case, "lap joint") of lap joint 14 surrounded by bounding box 23B-3.

[0066] Furthermore, the examiner uses the operation input unit 24 to surround the area on the screen shown in Figure 5 where the sheath tube 15 is displayed with a bounding box 23B-4, and inputs the color and name (in this case, "sheath tube") of the sheath tube 15 surrounded by the bounding box 23B-4. Furthermore, the inspector uses the operation input unit 24 to surround the area on the screen shown in Figure 5 where the compression joint 16 is displayed with a bounding box 23B-5, and inputs the color and name (in this case, "compression joint") of the compression joint 16 surrounded by the bounding box 23B-5.

[0067] When the input of the bounding box is complete, the selection unit 223 extracts, for example, the position coordinates of the upper left vertex and the lower right vertex of the rectangular bounding box, and creates position information including the extracted position coordinates. This position information is linked to the color and name of the secondary component within the bounding box. In other words, the position information is information for identifying the bounding box that represents the correct label of the secondary component. Data including image information identifying the secondary component using the bounding box and the above-mentioned position information created by the selection unit 223 is used as training data to create a custom model.

[0068] 5, the shear reinforcement 12 is linked to the bounding box 23B-1, the spacer block 13 is linked to the bounding box 23B-2, the lap joint 14 is linked to the bounding box 23B-3, the sheath pipe 15 is linked to the bounding box 23B-4, and the compression joint 16 is linked to the bounding box 23B-5. The learning device 3 creates a custom model for each of the shear reinforcement 12, the spacer block 13, the lap joint 14, the sheath pipe 15, and the compression joint 16.

[0069] Although the correct labels of the secondary components are shown in the form of bounding boxes, the present invention is not limited to this and they may be shown as lines or mask images. For example, the examiner uses the operation input unit 24 to draw a line along the longitudinal direction of the sheath tube 15 on the screen shown in Fig. 5. The selection unit 223 extracts the position coordinates of the start point and the end point of the line, and creates position information that links the extracted position coordinates with the sheath tube 15. The selection unit 223 creates learning data that includes this position information.

[0070] Furthermore, the examiner uses the operation input unit 24 to mask all areas on the screen shown in Fig. 5 except for the sheath tube 15. The selection unit 223 extracts the position coordinates of the area on the screen shown in Fig. 5 where the unmasked sheath tube 15 is displayed, and creates position information that links the extracted position coordinates with the sheath tube 15. The selection unit 223 creates learning data that includes this position information.

[0071] The selection unit 223 may automatically set a bounding box surrounding an object present on the screen of FIG. 5 by performing image analysis such as pattern matching on the image information or by using a learning model that roughly detects objects shown in the image information. In this case, the inspector uses the operation input unit 24 to confirm whether a secondary component is present within the automatically set bounding box. The selection unit 223 will create the above-mentioned position information for a bounding box that is confirmed to contain a secondary component.

[0072] Furthermore, the reinforcement arrangement inspection device 2 may infer secondary components from image information using at least one of a preset model and a custom model. For example, when the reinforcement arrangement inspection device 2 infers secondary components using only a custom model, the selection unit 223 automatically specifies the custom model when an instruction to start reinforcement arrangement inspection is given using the operation input unit 24, and proceeds to the above-described custom model creation process.

[0073] The selection unit 223 may also select multiple preset models for a common secondary component. In this case, the selection unit 223 selects multiple preset models for the common secondary component by specifying a preset model using the operation input unit 24 or by automatically specifying a preset model. For example, if the type of secondary component is "shear reinforcement," the selection unit 223 selects all preset models whose type is "shear reinforcement." The inference unit 224 infers the secondary component using each of all the preset models selected by the selection unit 223, and outputs the result obtained from the model with the highest evaluation as the final inference result.

[0074] The inference unit 224 infers secondary components from the preprocessed image information using the learning model selected as described above. For example, the inference unit 224 inputs image information preprocessed by the preprocessing unit 222 for the preset model or custom model selected by the selection unit 223. The preset model or custom model infers secondary components appearing in the input image information. For example, these learning models infer the positions and appearance features of secondary components in the image. The preprocessing of the image information may be performed by an external device provided separately from the reinforcement bar arrangement inspection device 2. In this case, the reinforcement bar arrangement inspection device 2 does not need to include the preprocessing unit 222. The acquisition unit 221 acquires the preprocessed image information from the external device, and the inference unit 224 uses the image information acquired by the acquisition unit 221 as is to infer the secondary components shown in the image information. Furthermore, the inference unit 224 may infer the secondary components shown in the image information by directly using image information of a reinforcement structure that has not been subjected to preprocessing. In this case as well, the reinforcement inspection device 2 does not need to be equipped with the preprocessing unit 222. For example, the learning device 3 creates a learning model using image information that has not been subjected to preprocessing as learning data. By using this learning model, the inference unit 224 can infer the secondary components using image information that has not been subjected to preprocessing.

[0075] The inference unit 224 inspects the secondary components based on the inference results, creates display control information for displaying the inspection results, and outputs the created display control information to the display unit 23. For example, the inference unit 224 inspects the number of shear reinforcement bars in a reinforcement structure based on the positions and appearance characteristics of the shear reinforcement bars on the image inferred using the learning model.The inference unit 224 then creates display control information for displaying an electronic whiteboard describing the inspection results superimposed on image information showing the reinforcement structure.Based on the display control information, the display unit 23 superimposes an electronic whiteboard describing the number, color, and shape of the shear reinforcement bars on the image showing the reinforcement structure.The electronic whiteboard is electronic image data on which the inspection results are written.

[0076] Furthermore, the inference unit 224 inspects the number of lap joints in the reinforcement structure and the degree of deviation of their shape from the design value based on the positions and appearance features of the lap joints on the image inferred using the learning model. For example, the inference unit 224 may inspect the deviation of the overlapping length of the ends of two rebars from the design value for each lap joint, and display the inspection results for each lap joint on an electronic whiteboard.

[0077] Furthermore, the inference unit 224 checks the number of compression joints in the reinforcement structure and the degree of deviation of their shapes from the design values ​​based on the positions and appearance features of the compression joints on the image inferred using the learning model. For example, the inference unit 224 may check the deviation from the design values ​​for at least one of the diameter of the bulge in the inferred compression joint, the length of the bulge, the eccentricity of the bulge from the central axis of the rebar, and the deviation of the compression surface from the top of the bulge, and display the result of the inspection on an electronic whiteboard. For example, for each compression joint installed in the reinforced concrete structure being inspected, the inspection results are displayed on an electronic whiteboard, indicating whether the diameter of the bulge at the part where the ends of the rebars are butted together is 1.4 times or more the diameter of the rebar, whether the length of the part where the ends of the rebars are butted together is 1.1 times or more the diameter of the rebar, whether the eccentricity from the central axis of the rebar is less than one-fifth the diameter of the rebar, and whether the deviation of the compression surface from the top of the bulge is less than one-fourth the diameter of the rebar.

[0078] The inference unit 224 may use the measurement results of the secondary components to determine whether the inference result is incorrect. For example, the acquisition unit 221 acquires point cloud data that represents the reinforcement structure as a three-dimensional point cloud. The point cloud data is data that indicates the distance to the reinforcement structure detected by a sensor such as a stereo camera, an infrared camera, or a LIDAR. The inference unit 224 reads out the image information acquired by the acquisition unit 221 and stored in the storage unit 25, and calculates the pixel values ​​(r i ,g i ,b i ) and the distance d between the 3D point on the object and the sensor i As a result, the pixel value in the image area where the object is captured is given as a four-element pixel value (r i ,g i ,b i ,d i ) is obtained.

[0079] The inference unit 224 estimates the size of the image area in which the secondary component appears, calculates the size (1) of the secondary component based on this estimated value, and further calculates the distance d i The inference unit 224 then calculates the size (2) of the secondary component using the above. The inference unit 224 then compares the size (1) and size (2) of the secondary component to determine whether the secondary component has been inferred incorrectly. Here, size (2) corresponds to the actual size of the secondary component. If the value of size (1) exceeds the upper limit of the allowable threshold range relative to the value of size (2), or if it is below the lower limit of the allowable threshold range, the inference unit 224 determines that the inference of the secondary component by the learning model is incorrect.

[0080] The inference unit 224 may also calculate the inference accuracy of the learning model using the determination result of whether the inference result is incorrect. For example, the inference unit 224 causes each of the multiple learning models to perform inference using common image information, and determines whether the inference result is incorrect for each of them, and calculates the ratio of the number of correct inference results to the number of inferences as the inference accuracy. By determining whether the inference result is incorrect in this way, the reinforcement arrangement inspection device 2 can accurately infer the secondary constituent members.

[0081] So far, we have shown the case where the reinforcement inspection device 2 detects secondary components in a reinforcement structure, but it may also automatically detect main reinforcement in addition to secondary components. For example, the preset models and custom models include models related to main reinforcement in addition to models related to secondary components. A model related to main reinforcement is a third learning model that is provided for each external feature including the type, color, and shape of the main reinforcement and is used to infer the main reinforcement reflected in the image information. The third learning model may be a preset model or a custom model.

[0082] The selection unit 223 also displays selection buttons for specifying a preset model for inferring main reinforcement on the operation screen 23A shown in Fig. 3. When the selection button for a preset model related to main reinforcement is operated using the operation input unit 24, the selection unit 223 creates specification information for the preset model related to main reinforcement. When the inference unit 224 acquires the learning model indicated by the specification information from the learning device 3, it uses the acquired learning model to infer main reinforcement from the image information preprocessed by the preprocessing unit 222. This allows the reinforcement inspection device 2 to automatically detect main reinforcement in addition to secondary components in a reinforcement structure.

[0083] The functional components of the learning device 3 will now be described. The data acquisition unit 321 acquires learning data including image information and positional information of objects in the image information. For example, when the data acquisition unit 321 receives designation information of a custom model from the reinforcement arrangement inspection device 2 via the communication line using the communication unit 31, the data acquisition unit 321 acquires learning data including image information of a captured reinforcement structure and positional information of secondary components shown in the image information from the reinforcement arrangement inspection device 2. The data acquisition unit 321 saves the learning data acquired from the reinforcement arrangement inspection device 2 in the learning DB 333.

[0084] The learning unit 322 uses the learning data to create and store a learning model for inferring the secondary component shown in the image information. For example, the learning unit 322 creates a preset model or a custom model using a pre-learning model stored in the pre-learning model DB 332 and learning data stored in the learning DB 333. Furthermore, the learning unit 322 evaluates the created learning model, and determines a learning model whose evaluation value is equal to or greater than an allowable value as the learning model to be used by the reinforcement arrangement inspection device 2. For example, the matching rate with respect to the correct data and the recall rate of the inference result are used as indices for evaluating the learning model.

[0085] The search unit 323 searches for a learning model specified in the reinforcement arrangement inspection device 2 from the learning models created for each appearance feature, including the type, color, and shape, of the secondary component, and outputs the learning model obtained by the search to the reinforcement arrangement inspection device 2. For example, when the search unit 323 acquires specification information for a preset model from the reinforcement arrangement inspection device 2, it searches the selection candidate DB 331 based on information about the secondary component included in the acquired specification information. Then, the search unit 323 transmits information indicating the searched-for preset model to the reinforcement arrangement inspection device 2 via the communication line using the communication unit 31. This allows the reinforcement arrangement inspection device 2 to acquire the specified learning model.

[0086] FIG. 6 is a flowchart showing the reinforcement bar arrangement inspection method according to the first embodiment. The acquiring unit 221 acquires image information (step ST1). For example, the acquiring unit 221 acquires image information of a reinforcement structure photographed by a monocular camera or a stereo camera provided in the reinforcement arrangement inspection device 2. The pre-processing unit 222 pre-processes the image information (step ST2). For example, the pre-processing unit 222 calculates image information by normalizing the color values ​​of the pixels by the maximum color value. In addition, if the acquisition unit 221 acquires preprocessed image information, or if the inference unit 224 infers secondary components using image information that has not been preprocessed, the process may proceed to step ST3 without performing step ST2.

[0087] The selection unit 223 selects a specified learning model from among a plurality of learning models held by the learning device 3 (step ST3). For example, the learning device 3 manages a plurality of preset models provided for each appearance feature including the type, color, and shape of a secondary component, and further creates a custom model specified by the reinforcement arrangement inspection device 2. The selection unit 223 selects a learning model to be used by the inference unit 224 by specifying a preset model or a custom model to the learning device 3.

[0088] The inference unit 224 infers the secondary component from the preprocessed image information using the learning model selected by the selection unit 223 (step ST4). For example, the inference unit 224 causes the display unit 23 to display the inference result of the secondary component and the inspection result of the secondary component using the inference result. By having the reinforcement inspection device 2 execute this method, it is possible to automatically detect secondary components provided other than the main reinforcements in a reinforcement structure.

[0089] Next, the advance creation of a preset model by the learning device 3 will be described. First, the data acquisition unit 321 acquires learning data to be used for creating a preset model. The learning data includes, for example, image information in which an object on an image is identified using the operation input unit 24. The learning data is stored in the learning DB 333 by the data acquisition unit 321. FIG. 7 is a diagram showing an example of the learning data. As shown in FIG. 7, the learning data is data provided for each appearance feature including the type, color, and shape of a secondary component.

[0090] For example, in Fig. 7, the training data No. 1 has bounding box positions A, B, and C set in the image areas in image A that show blue, U-shaped shear reinforcement bars. The training data No. 2 has bounding box positions D and E set in the image areas in image B that show blue, cylindrical lap splices on reinforcing bars. The training data No. 3 has bounding box position F set in the image area in image C that shows a silver, cylindrical sheath tube.

[0091] 8 is a block diagram showing the configuration of the learning unit 322. As shown in FIG. 8, the learning unit 322 includes a data classification unit 3221, a model creation unit 3222, and an evaluation unit 3223. The data classification unit 3221 extracts a plurality of pieces of image information showing secondary components from the image information stored in the learning DB 333 by the data acquisition unit 321, and divides the extracted image information into pieces for learning and pieces for evaluation.

[0092] For example, when creating a preset model for inferring blue, U-shaped shear reinforcement, the data classifying unit 3221 reads out the training data No. 1 shown in Fig. 7 from the training DB 333 and separates this data into training data and evaluation data. As a result, image information relating to the same shear reinforcement is classified into training data and evaluation data.

[0093] The model creation unit 3222 uses the image information for learning to create a learning model for inferring the secondary component shown in the image information. For example, the model creation unit 3222 uses a pre-learning model extracted from the pre-learning model DB 332 to learn the secondary component shown in the image information for learning, thereby creating a learning model for inferring the secondary component. For example, a Siamese network is used as the learning model including the pre-learning model.

[0094] The evaluation unit 3223 evaluates the learning model using the image information for evaluation and saves the learning model that satisfies the evaluation conditions as a preset model. For example, the evaluation unit 3223 evaluates the learning model created by the model creation unit 3222 using the image information for evaluation acquired from the data classification unit 3221.

[0095] Figure 9 shows the evaluation results of the learning model, and is the evaluation result for the learning data No. 1 shown in Figure 7. The learning data and evaluation data in Figure 9 are image information classified into learning and evaluation data by the data classification unit 3221. The evaluation unit 3223 creates a dataset that associates the type of pre-learning model used when the model creation unit 3222 created the learning model with the learning method (e.g., stochastic gradient descent) for the learning data and evaluation data.

[0096] The evaluation unit 3223 uses Model A, which is a pre-training model, to infer shear reinforcement in Data Set No. 1 shown in Figure 9, using Images A, I, and J and the positional information of the shear reinforcement in these images as training data. Next, the evaluation unit 3223 uses the same Model A to infer shear reinforcement, using Images K, L, and M and the positional information of the shear reinforcement in these images as evaluation data. Using these results, the evaluation unit 3223 calculates the precision A and the recall A.

[0097] The evaluation unit 3223 similarly calculates the precision and recall for the datasets after dataset No. 2. Based on the evaluation condition of selecting the learning model with the highest precision and recall, the evaluation unit 3223 compares the precision and recall for all datasets related to the learning data No. 1 shown in FIG. 7 . Based on the comparison results, the evaluation unit 3223 determines the learning model with the highest precision and recall as the preset model and stores this model in the selection candidate DB 331. While the evaluation condition of selecting the learning model with the highest precision and recall is shown, it is not limited thereto. For example, the evaluation condition may be to select a learning model with a precision and recall higher than a certain threshold. Alternatively, the evaluation condition may be to select a learning model with either the highest precision or recall or higher than a threshold.

[0098] Although precision and recall are used as evaluation indices for the learning model, other evaluation indices may be used. For example, the MIOU (Mean Intersection Over Union) may be used, which is the ratio of the area of ​​overlap between the bounding box (correct rectangle) surrounding the sub-component, which is the correct data, and the bounding box (inferred rectangle) surrounding the estimated sub-component.

[0099] FIG. 10 is an explanatory diagram that schematically illustrates matching determination of embedding vectors in a learning model. In FIG. 10, the network to which image information (1) is input and the network to which image information (2) is input are a common network. Image information (1) is image information to which a correct answer label has been assigned using a bounding box or the like. Image information (2) is data that has been classified for learning by the data classification unit 3221 from the image information stored in the learning DB 333. For example, image information (2) is a partial image in which an image area showing one of the secondary components is extracted. Note that, for example, a Siamese network may be used as the machine learning model that is the common network.

[0100] The model creation unit 3222 inputs the correct data, which is image information (1), into the network, and calculates an embedding vector corresponding to the image information (1). For example, in PaDiM, the embedding vector is calculated by combining the outputs from the first to third layers. The model creation unit 3222 stores this embedding vector in the storage unit 33.

[0101] Next, the model creation unit 3222 calculates an embedding vector corresponding to the image information using the image information (2) for learning, and creates a learning model as a preset model if it determines that the calculated embedding vector matches the embedding vector stored in the memory unit 33. By using embedding vectors that can be created before the final inference results are calculated, the time required to create a learning model can be reduced.

[0102] FIG. 11 is a flowchart showing the process of creating a preset model. The data classification unit 3221 extracts multiple pieces of image information showing secondary components from the image information acquired by the data acquisition unit 321 and stored sequentially in the learning DB 333, and divides the extracted image information into pieces for learning and pieces for evaluation (step ST1A). The model creation unit 3222 uses the image information for learning to create a learning model for inferring the secondary component shown in the image information (step ST2A). The evaluation unit 3223 evaluates the learning model using the image information for evaluation, and saves the learning model that satisfies the evaluation conditions as a preset model (step ST3A).

[0103] FIG. 12 is a flowchart showing the custom model creation process. The data classification unit 3221 acquires multiple pieces of image information showing secondary components from the image information stored in the learning DB 333, classifies the acquired multiple pieces of image information into learning data and evaluation data, and outputs the learning image information to the model creation unit 3222 as learning data (step ST1B). Next, the model creation unit 3222 acquires a pre-training model from the pre-training model DB 332 (step ST2B). The model creation unit 3222 uses the pre-learning model and the learning data to create a learning model for inferring the secondary component shown in the image information (step ST3B).

[0104] The evaluation unit 3223 evaluates the learning model using the image information for evaluation from the data classification unit 3221, and transmits the learning model that satisfies the evaluation conditions as a custom model to the reinforcement bar inspection device 2 via the communication line by the communication unit 31 (step ST4B). If the inspector is not satisfied with the inference accuracy of the custom model, he or she may issue a command to recreate the custom model using the operation input unit 24, thereby repeatedly creating learning data, creating the custom model shown in Fig. 12, and evaluating the performance of the custom model in the reinforcement arrangement inspection system 1. Furthermore, the performance of the custom model may be evaluated using precision and recall, or MIOU.

[0105] As described above, the reinforcement bar arrangement inspection device 2 according to the first embodiment includes an acquisition unit 221 that acquires image information, a selection unit 223 that selects a specified learning model from among a plurality of learning models provided for each appearance feature of a secondary component, including the type, color, and shape, for inferring the secondary component shown in the image information, and an inference unit 224 that infers the secondary component from the image information using the selected learning model. This enables the reinforcement bar arrangement inspection device 2 to automatically detect secondary components provided other than the main reinforcement bars in a reinforcement structure.

[0106] In the reinforcement arrangement inspection device 2 according to the first embodiment, the multiple learning models include one or more preset models created in advance before being specified, and a custom model created after being specified. This enables the reinforcement arrangement inspection device 2 to infer secondary components using preset models with high inference accuracy that have been prepared in advance, and to infer secondary components using custom models that suit the situation on site.

[0107] The reinforcement bar arrangement inspection device 2 according to the first embodiment includes a preprocessing unit 222 that preprocesses image information so that the image information is in a form suitable for inference processing performed by the inference unit 224. The inference unit 224 uses a learning model to infer secondary components from the preprocessed image information. This allows the inference unit 224 to smoothly infer secondary components.

[0108] In the bar arrangement inspection device 2 according to the first embodiment, the preprocessing unit 222 normalizes the image information. The normalized image information can be used as input data for a learning model created by DL.

[0109] In the reinforcement bar arrangement inspection device 2 according to the first embodiment, the selection unit 223 outputs display control information for displaying an operation screen for allowing an operation to specify a learning model, and selects the learning model specified by the accepted operation based on the operation screen. This enables the selection unit 223 to accurately select a learning model for inferring a secondary component.

[0110] The bar arrangement inspection device 2 according to the first embodiment includes a display unit 23 that displays an operation screen 23A based on display control information. This allows the bar arrangement inspection device 2 to display an operation screen that allows an operation to specify a learning model.

[0111] The bar arrangement inspection device 2 according to the first embodiment includes an operation input unit 24 that accepts an operation to specify a learning model from a plurality of learning models on an operation screen 23A displayed on the display unit 23. This allows the learning model to be specified by operating the operation input unit 24.

[0112] In the reinforcement bar arrangement inspection device 2 according to the first embodiment, the selection unit 223 automatically specifies and selects a learning model from a plurality of learning models, thereby enabling the selection unit 223 to accurately select a learning model for inferring a secondary component.

[0113] In the reinforcement bar arrangement inspection device 2 according to the first embodiment, the acquisition unit 221 acquires point cloud data that represents a reinforcement structure as a three-dimensional point cloud. The inference unit 224 determines whether or not a secondary component has been erroneously inferred based on the result of comparing the secondary component calculated using the image information with the secondary component calculated using the point cloud data. This allows the reinforcement bar arrangement inspection device 2 to accurately infer the secondary component.

[0114] In the reinforcement bar arrangement inspection device 2 according to the first embodiment, the preset models and custom models include learning models that are provided for each external feature of the main reinforcement bars, including their type, color, and shape, and that are used to infer the main reinforcement bars that appear in the image information. The inference unit 224 uses these learning models to infer the main reinforcement bars from the preprocessed image information. This allows the reinforcement bar arrangement inspection device 2 to automatically detect the main reinforcement bars as well as the secondary components in a reinforcement structure.

[0115] In the reinforcement inspection device 2 according to the first embodiment, the secondary component is at least one of a shear reinforcement, a lap joint, a spacer block, a sheath pipe, and a compression joint. In this way, the reinforcement inspection device 2 can detect various components as secondary components.

[0116] The learning device 3 according to the first embodiment includes a data acquisition unit 321 that acquires learning data including image information and positional information of objects in the image information, a learning unit 322 that uses the learning data to create and store a learning model for inferring a secondary component shown in the image information, and a search unit 323 that searches for a learning model specified in the reinforcement arrangement inspection device 2 from learning models created for each appearance feature including the type, color, and shape of the secondary component, and outputs the learning model obtained by the search to the reinforcement arrangement inspection device 2. In this way, the learning device 3 can create a learning model for inferring a secondary component shown in the image information for each appearance feature including the type, color, and shape of the secondary component.

[0117] In the learning device 3 according to the first embodiment, the multiple learning models include one or more preset models created before the model is specified, and a custom model created after the model is specified. The data acquisition unit 321 acquires and sequentially saves image information. The learning unit 322 includes a data classification unit 3221 that extracts multiple pieces of image information showing secondary components from the saved image information and separates the extracted image information into learning and evaluation models; a model creation unit 3222 that uses the learning image information to create a learning model for inferring the secondary components shown in the image information; and an evaluation unit 3223 that evaluates the learning model using the evaluation image information and saves the learning model that satisfies the evaluation conditions as a preset model. This allows the learning device 3 to create a learning model with high inference accuracy as a preset model.

[0118] In the learning device 3 according to Embodiment 1, the model creation unit 3222 calculates and stores an embedding vector corresponding to correct data, which is image information, calculates an embedding vector corresponding to image information using image information for learning, and creates a learning model in which it is determined that the calculated embedding vector matches the stored embedding vector. This allows the model creation unit 3222 to reduce the time required to create a learning model.

[0119] In the learning device 3 according to the first embodiment, the learning unit 322 uses a pre-learned model that has been trained to infer objects shown in image information to create a custom model specified in the reinforcement arrangement inspection device 2. This allows the learning device 3 to create a custom model with high inference accuracy even when there is a small amount of training data.

[0120] In the learning device 3 according to the first embodiment, the data acquisition unit 321 repeatedly acquires learning data and the learning unit 322 repeatedly creates a custom model until the custom model satisfies the target inference accuracy. This allows the learning device 3 to create a custom model with high inference accuracy.

[0121] The reinforcement inspection system 1 according to the first embodiment includes a reinforcement inspection device 2 and a learning device 3. As a result, the reinforcement inspection system 1 can provide the reinforcement inspection device 2 that can automatically detect secondary components provided in addition to the main reinforcement in a reinforcement structure.

[0122] The reinforcement inspection method according to the first embodiment includes the steps of: acquiring image information by an acquisition unit 221; selecting a designated learning model from a plurality of learning models provided for each appearance feature of a secondary component, including the type, color, and shape, by a selection unit 223 for inferring the secondary component shown in the image information; and inferring the secondary component from the image information by an inference unit 224 using the selected learning model. By having the reinforcement inspection device 2 execute this method, secondary components provided other than the main reinforcement in a reinforcement structure can be automatically detected.

[0123] Any of the components of the embodiments may be modified or omitted. [Industrial Applicability]

[0124] The reinforcement inspection device according to the present disclosure can be used, for example, to inspect a reinforcement structure before concrete is poured. [Explanation of symbols]

[0125] 1 Reinforcement inspection system, 2 Reinforcement inspection device, 3 Learning device, 11 Reinforcement bar, 12 Shear reinforcement bar, 13 Spacer block, 14 Lap splice, 15 Sheath pipe, 16 Compression splice, 21 Communication unit, 22 Calculation unit, 23 Display unit, 23A Operation screen, 23A-1 to 23A-3, 23Ac Selection button, 23Ab Slide bar, 23B-1 to 23B-5 Bounding box, 24 Operation input unit, 25 Memory unit, 31 Communication unit, 32 Calculation unit, 33 Memory unit, 100 Communication interface, 101 Input / output interface, 102 Processor, 103 Memory, 221 Acquisition unit, 222 Preprocessing unit, 223 Selection unit, 224 Inference unit, 321 Data acquisition unit, 322 Learning unit, 323 Search unit, 3221 Data classification unit, 3222 Model Creation Department, 3223 Evaluation Department.

Claims

1. A reinforcing bar inspection device for inspecting a reinforcement structure in which a plurality of reinforcing bars are arranged as main reinforcements and which includes secondary components other than the main reinforcements, an acquisition unit that acquires image information; a selection unit that automatically selects a learning model for inferring the secondary component shown in the image information based on design data of a structure including the reinforcement structure, the learning model being provided for each of the secondary components of the same type based on appearance features including color and shape, and the learning model for inferring the secondary component included in the reinforcement structure from among a plurality of learning models provided for each of the secondary components of the same type based on appearance features including color and shape; an inference unit that infers the secondary component from the image information using the selected learning model. A reinforcement inspection device characterized by the above.

2. The plurality of learning models include one or more first learning models that are created before being designated, and a second learning model that is created after being designated.

2. The reinforcing bar inspection device according to claim 1.

3. a preprocessing unit that preprocesses the image information so that the image information is in a form suitable for inference processing performed by the inference unit; The inference unit infers the sub-component from the preprocessed image information using the learning model.

2. The reinforcing bar inspection device according to claim 1.

4. The preprocessing unit normalizes the image information.

4. The reinforcing bar inspection device according to claim 3.

5. The selection unit outputs display control information for displaying an operation screen for allowing an operation to specify the learning model, and selects the learning model specified by the operation accepted on the operation screen.

2. The reinforcing bar inspection device according to claim 1.

6. a display unit that displays the operation screen based on the display control information; The bar arrangement inspection device according to claim 5 .

7. an operation input unit that accepts an operation to specify the learning model from the plurality of learning models on the operation screen displayed on the display unit; 7. The bar arrangement inspection device according to claim 6.

8. The acquisition unit further acquires point cloud data representing the reinforcement structure as a three-dimensional point cloud, The inference unit determines whether the secondary component has been erroneously inferred based on a result of comparing the secondary component calculated using the image information with the secondary component calculated using the point cloud data.

2. The reinforcing bar inspection device according to claim 1.

9. the first learning model and the second learning model include a third learning model provided for each appearance feature including the type, color, and shape of the main reinforcement bars, for inferring the main reinforcement bars shown in the image information; The inference unit infers the main muscle lines from the preprocessed image information using the third learning model.

3. The bar arrangement inspection device according to claim 2.

10. The secondary component is at least one of a shear reinforcement, a lap joint, a spacer block, a sheath tube, or a compression joint.

2. The reinforcing bar inspection device according to claim 1.

11. A learning device that uses image information of a reinforcement structure in which a plurality of reinforcing bars are arranged as main reinforcements and which includes secondary components other than the main reinforcements to create a learning model for inferring the secondary components shown in the image information, a data acquisition unit that acquires learning data including the image information and position information of an object in the image information; a learning unit that uses the learning data to create and store the learning model for inferring the secondary component shown in the image information; a search unit that searches for the learning model specified in the reinforcement inspection device from the learning models created for each appearance feature including the type, color, and shape of the secondary component, and outputs the learning model obtained by the search to the reinforcement inspection device, the data acquisition unit acquires and sequentially stores the image information; The learning unit a data classification unit that extracts a plurality of pieces of image information showing the secondary component from the stored image information and classifies the extracted image information into pieces of image information for learning and pieces of image information for evaluation; a model creation unit that creates the learning model for inferring the secondary component shown in the image information by using the image information for learning; an evaluation unit that evaluates the learning model using the image information for evaluation and stores the learning model that satisfies an evaluation condition as a first learning model. A learning device characterized by:

12. The model creation unit calculates and stores an embedding vector corresponding to the correct data, which is the image information, calculates an embedding vector corresponding to the image information using the image information for learning, and creates the learning model in which it is determined that the calculated embedding vector matches the stored embedding vector. The learning device according to claim 11 .

13. The plurality of learning models include one or more of the first learning models that are created in advance before being designated, and a second learning model that is created after being designated, The learning unit creates the second learning model specified in the reinforcement bar arrangement inspection device using a pre-learning model that has been trained to infer an object shown in the image information. The learning device according to claim 11 .

14. The acquisition of the learning data by the data acquisition unit and the creation of the second learning model by the learning unit are repeated until the second learning model satisfies a target inference accuracy.

14. The learning device according to claim 13.

15. The bar arrangement inspection device according to any one of claims 1 to 10, A learning device according to any one of claims 11 to 14; Reinforcement inspection system equipped with

16. A reinforcing bar inspection method for a reinforcing bar inspection device that inspects a reinforcing bar structure in which a plurality of reinforcing bars are arranged as main reinforcing bars and which includes secondary components other than the main reinforcing bars, an acquisition unit acquiring image information; a selection unit automatically selecting, based on design data of a structure including the reinforcement structure, from a plurality of learning models for inferring the secondary component shown in the image information, the learning models being provided for each appearance feature including color and shape for the secondary component of the same type; an inference unit inferring the sub-component from the image information using the selected learning model; A reinforcement inspection method characterized by the above.

Images