Information processing apparatus, information processing method, and program

Abstract

To improve a technique for detecting a crack from a photographed image.SOLUTION: The information processing apparatus 10 includes a control unit 11 that acquires an image in which a wall of a structure appears, extracts a wall region representing the wall from the image, detects a crack in the wall region, determines a length of the crack based on the number of pixels of a long side of a rectangle surrounding the detected crack, counts the number of crack pixels that are pixels corresponding to the crack and present on at least one straight line along a direction of a short side of the rectangle, determines a width of the crack based on a value of a result of the counting, and converts the determined length of the crack and width of the crack into actual dimension values.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] Traditionally, inspections have been carried out on infrastructure structures. During inspections, workers check for deterioration or damage, such as cracks in the structure or exposed internal rebar, and measure the extent of this deterioration or damage using a tape measure or similar. Meanwhile, to address issues such as increased inspection time due to aging infrastructure structures and a shortage of workers due to Japan's declining population, the introduction of methods for inspecting structures using images captured by cameras mounted on vehicles or drones is being promoted. For example, Non-Patent Document 1 discloses a method for extracting cracks on concrete surfaces by analyzing images captured by a digital camera. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Terano, Akiyasu, and four others, "Research on methods for extracting and identifying cracks in concrete structures using deep learning," Nagasaki University Graduate School of Engineering Research Report, Vol. 50, No. 95, July 2020. Summary of the Invention [Problem to be solved by the invention]

[0004] The conventional technology only detects the crack itself from the photographed image, but is unable to automatically detect the actual dimensions of the crack, such as its length or width. As such, there is room for improvement in the technology for detecting cracks from photographed images.

[0005] In view of the above circumstances, an object of the present disclosure is to improve the technology for detecting cracks from captured images. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems, an information processing device according to the present disclosure includes: The image capturing the wall of the structure is captured, extracting a wall region representing the wall from the image; Detecting cracks in the wall region; determining a length of the crack based on the number of pixels on the long side of a rectangle surrounding the detected crack; Counting the number of crack pixels that correspond to the crack and exist on at least one straight line along the direction of the short side of the rectangle; determining the width of the crack based on the counted value; A control unit is provided for converting the determined crack length and crack width into actual size values.

[0007] Further, an information processing method according to the present disclosure is an information processing method executed by an information processing device, Acquiring an image in which the wall of the structure is reflected; extracting a wall region representing the wall from the image; detecting cracks in the wall region; Determining the length of the crack based on the number of pixels on the long side of a rectangle surrounding the detected crack; Counting the number of crack pixels that correspond to the crack and exist on at least one straight line along the direction of the short side of the rectangle; determining a width of the crack based on the counted value; and converting the determined crack length and crack width into actual size values.

[0008] Furthermore, the program according to the present disclosure causes a computer to function as an information processing device according to the present disclosure. [Effects of the Invention]

[0009] According to the present disclosure, it is possible to improve the technology for detecting cracks from captured images. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing a configuration of an information processing device. [Figure 2A] 10 is a flowchart illustrating an example of the operation of the information processing device. [Figure 2B] 10 is a flowchart illustrating an example of the operation of the information processing device. [Figure 3] FIG. 10 is a diagram illustrating an example of an image acquired by the information processing device. [Figure 4] FIG. 10 is a diagram for explaining a detected crack. [Figure 5] FIG. 10 is a diagram for explaining the width of a crack. [Figure 6] FIG. 10 is a diagram for explaining the width of a crack. [Figure 7] FIG. 10 is a diagram for explaining a detected seam. [Figure 8] FIG. 10 is a diagram for explaining an extracted overlapping portion. [Figure 9] FIG. 10 is a diagram illustrating an example of an output image. DETAILED DESCRIPTION OF THE INVENTION

[0011] An embodiment will be described below with reference to the drawings. In each drawing, identical or corresponding parts are designated by the same reference numerals. In the description of this embodiment, the description of identical or corresponding parts will be omitted or simplified as appropriate.

[0012] An overview of this embodiment will be described with reference to Fig. 1. An information processing device 10 is installed in a facility such as a data center. The information processing device 10 is, for example, a server belonging to a cloud computing system or other computing system.

[0013] The information processing device 10 acquires an image in which the wall of a structure is reflected, and extracts a wall region representing the wall from the image. The information processing device 10 detects cracks in the wall region and determines the length of the crack based on the number of pixels on the long side of a rectangle surrounding the detected crack. The information processing device 10 counts the number of crack pixels, which are pixels corresponding to the crack and exist on at least one straight line along the direction of the short side of the rectangle, and determines the width of the crack based on the counted value. The information processing device 10 converts the determined crack length and crack width into actual size values.

[0014] The structure includes infrastructure facilities such as roads, tunnels, dams, bridges, and buildings. In this embodiment, the structure is a building, and the concrete walls of the building are reflected in the image acquired by the information processing device 10. The material of the structure's walls is not limited to concrete, and may be cement, mortar, concrete, plaster, plaster, brick, stone, or the like. According to this embodiment, the information processing device 10 can accurately detect cracks in the wall and output the actual dimensions of the crack width and length. Taking the output actual dimensions into consideration, workers can easily plan maintenance and inspection work for the structure. This makes it possible to improve the technology for detecting cracks from captured images.

[0015] The configuration of an information processing device 10 according to this embodiment will be described with reference to Fig. 1. The information processing device 10 includes a control unit 11, a storage unit 12, a communication unit 13, an input unit 14, and an output unit 15.

[0016] The control unit 11 includes at least one processor, at least one programmable circuit, at least one dedicated circuit, or any combination thereof. The processor is a general-purpose processor such as a CPU or GPU, or a dedicated processor specialized for specific processing. "CPU" is an abbreviation for central processing unit. "GPU" is an abbreviation for graphics processing unit. An example of the programmable circuit is an FPGA. "FPGA" is an abbreviation for field-programmable gate array. An example of the dedicated circuit is an ASIC. "ASIC" is an abbreviation for application specific integrated circuit. The control unit 11 controls each unit of the information processing device 10 and executes processing related to the operation of the information processing device 10.

[0017] The storage unit 12 includes at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or any combination thereof. The semiconductor memory is, for example, a RAM, a ROM, or a flash memory. "RAM" is an abbreviation for random access memory. "ROM" is an abbreviation for read only memory. RAM is, for example, an SRAM or a DRAM. "SRAM" is an abbreviation for static random access memory. "DRAM" is an abbreviation for dynamic random access memory. ROM is, for example, an EEPROM. "EEPROM" is an abbreviation for electrically erasable programmable read only memory. Flash memory is, for example, an SSD. "SSD" is an abbreviation for solid-state drive. Magnetic memory is, for example, an HDD. "HDD" is an abbreviation for hard disk drive. The storage unit 12 functions, for example, as a main storage device, an auxiliary storage device, or a cache memory. The storage unit 12 stores information used in the operation of the information processing device 10 and information obtained by the operation of the information processing device 10.

[0018] The communication unit 13 includes at least one communication module. The communication module is, for example, a module compatible with a wired LAN communication standard such as Ethernet (registered trademark), a wireless LAN communication standard such as IEEE802.11, or a mobile communication standard such as LTE, 4G, or 5G. "LAN" is an abbreviation for local area network. "IEEE" is an abbreviation for Institute of Electrical and Electronics Engineers. "LTE" is an abbreviation for Long Term Evolution. "4G" is an abbreviation for 4th generation. "5G" is an abbreviation for 5th generation. The communication unit 13 receives information used in the operation of the information processing device 10 and transmits information obtained by the operation of the information processing device 10. The communication unit 13 enables the information processing device 10 to transmit and receive information to and from other devices via a network.

[0019] The network may include the Internet, at least one WAN, at least one MAN, or a combination thereof. "WAN" is an abbreviation for wide area network. "MAN" is an abbreviation for metropolitan area network. The network may include at least one wireless network, at least one optical network, or a combination thereof. The wireless network may be, for example, an ad hoc network, a cellular network, a wireless LAN, a satellite communication network, or a terrestrial microwave network.

[0020] The input unit 14 includes at least one input interface. The input interface is, for example, a physical key, a capacitance key, a pointing device, a touch screen integrated with a display, or a microphone. The input unit 14 accepts an operation to input information used for the operation of the information processing device 10. The input unit 14 may be connected to the information processing device 10 as an external input device instead of being provided in the information processing device 10. Any connection method may be used, such as USB (Universal Serial Bus), HDMI (High-Definition Multimedia Interface) (registered trademark), or Bluetooth (registered trademark).

[0021] The output unit 15 includes at least one output interface. The output interface is, for example, a display or a speaker. The display is, for example, an LCD or an organic EL display. "LCD" is an abbreviation for liquid crystal display. "EL" is an abbreviation for electroluminescent. The output unit 15 outputs information obtained by the operation of the information processing device 10. The output unit 15 may be connected to the information processing device 10 as an external output device instead of being provided in the information processing device 10. Any connection method can be used, such as USB, HDMI (registered trademark), or Bluetooth (registered trademark).

[0022] The functions of the information processing device 10 are realized by executing a program according to this embodiment on a processor serving as the control unit 11. That is, the functions of the information processing device 10 are realized by software. The program causes a computer to execute the operations of the information processing device 10, thereby causing the computer to function as the information processing device 10. That is, the computer functions as the information processing device 10 by executing the operations of the information processing device 10 in accordance with the program.

[0023] The program can be stored on a non-transitory computer-readable medium. Examples of non-transitory computer-readable media include flash memory, magnetic recording devices, optical disks, magneto-optical recording media, and ROMs. The program can be distributed by selling, transferring, or lending portable media such as SD cards, DVDs, or CD-ROMs that store the program. "SD" is an abbreviation for Secure Digital. "DVD" is an abbreviation for digital versatile disc. "CD-ROM" is an abbreviation for compact disc read only memory. The program can also be distributed by storing it in the storage of a server and transferring it from the server to another computer. The program can also be provided as a program product.

[0024] A computer temporarily stores a program stored on a portable medium or transferred from a server in its main storage device. The computer then reads the program stored in the main storage device using a processor and executes processing in accordance with the read program. The computer may also read the program directly from a portable medium and execute processing in accordance with the program. The computer may also execute processing in accordance with the received program each time a program is transferred from a server to the computer. Processing may also be executed through a so-called ASP-type service that achieves its functions by issuing execution instructions and obtaining results without transferring the program from the server to the computer. "ASP" is an abbreviation for application service provider. A program is information used for processing by a computer and includes something equivalent to a program. For example, data that is not a direct instruction to a computer but has properties that specify computer processing falls under the category of "something equivalent to a program."

[0025] Some or all of the functions of the information processing device 10 may be realized by a programmable circuit or a dedicated circuit as the control unit 11. In other words, some or all of the functions of the information processing device 10 may be realized by hardware.

[0026] Next, the operation of the information processing device 10 according to this embodiment will be described with reference to Figures 2A to 9. The operation of the information processing device 10 corresponds to the information processing method according to this embodiment.

[0027] In step S1 of Fig. 2A, the control unit 11 acquires an image in which the wall of a structure is reflected. The control unit 11 acquires the image by reading the image from the storage unit 12. The control unit 11 may communicate with an unmanned aerial vehicle equipped with a camera used for inspection, for example, via the communication unit 13, and receive the image directly from the unmanned aerial vehicle.

[0028] The control unit 11 may divide the acquired image into a plurality of divided images each having a predetermined size. FIG. 3 shows the result of the control unit 11 dividing the acquired image IM into a plurality of divided images IM1 to IMn. n may be any integer equal to or greater than 2, and is 16 in the example of FIG. 3. The size of the divided images may be freely set. The control unit 11 determines whether or not a wall of a structure is reflected in each divided image using any image analysis technology, and deletes any divided image determined not to reflect a wall of a structure. Without being limited to this, the control unit 11 may delete some of the plurality of divided images in response to a user instruction via the input unit 14. This reduces the time required for crack detection and measurement in the following step S2 and subsequent steps.

[0029] In step S2 of FIG. 2A, the control unit 11 extracts a wall region representing a wall from the image acquired in step S1. Any method may be used to extract the wall region. For example, the control unit 11 reads from the storage unit 12 a first trained model generated by machine learning using, as training data, multiple images labeled with a wall region. The first trained model may be generated by the information processing device 10, or may be received by the control unit 11 from an external server device via the communication unit 13. The control unit 11 applies the first trained model to the image acquired in step S1 to extract the wall region. If the image is segmented in step S1, the control unit 11 may extract the wall region for each segmented image. FIG. 3 shows an enlarged view of one segmented image IMm included in the multiple segmented images IM1 to IMn. The shaded area in the segmented image IMm indicates the wall region W1 extracted by applying the first trained model.

[0030] In step S3 of FIG. 2A, the control unit 11 detects cracks in the extracted wall region. Any method may be used for the detection. For example, the control unit 11 reads from the storage unit 12 a second trained model generated by machine learning using multiple images labeled with cracks as training data. The second trained model may be generated by the information processing device 10, or may be received by the control unit 11 from an external server device via the communication unit 13. The control unit 11 applies the second trained model to the wall region extracted from the image in step S2 to detect cracks. The control unit 11 detects cracks by surrounding them with a rectangle. FIG. 4 shows multiple cracks in the wall region W1 and rectangles SQ1 to SQ17 that surround each of the multiple cracks. The rectangles surrounding the cracks are quadrilaterals with sides horizontal and vertical to the image. However, the rectangle may be a quadrilateral having sides inclined at any angle relative to the horizontal or vertical direction.

[0031] In step S4 of FIG. 2A, the control unit 11 compares the number of pixels on the long side of the rectangle surrounding the detected crack with the number of pixels on the short side, and determines whether the difference between the number of pixels on the long side and the number of pixels on the short side is equal to or greater than a predetermined value. The predetermined value may be set in advance and stored in the memory unit 12. If the difference is equal to or greater than the predetermined value, the control unit 11 proceeds to step S5. If the difference is less than the predetermined value, the control unit 11 proceeds to step S6.

[0032] Instead of determining the difference, the control unit 11 may determine whether the ratio of the number of pixels on the short side to the number of pixels on the long side is equal to or greater than a predetermined ratio. The predetermined ratio is, for example, 2 / 3. If the ratio is equal to or greater than the predetermined ratio, the control unit 11 proceeds to step S5. If the ratio is less than the predetermined ratio, the control unit 11 proceeds to step S6.

[0033] In step S5, the control unit 11 determines the number of pixels on the long side of the rectangle as the length of the crack.

[0034] In step S6, the control unit 11 determines the number of pixels on the diagonal line of the rectangle as the length of the crack.

[0035] In steps S4 to S6, if the rectangle is a square or approximately square, the control unit 11 determines the number of pixels on the diagonal of the rectangle as the length of the crack. This makes it possible to quantitatively measure the length of all cracks that are inclined in any direction.

[0036] In step S7, the control unit 11 performs binarization processing to color the rectangle surrounding the detected crack white or black. Any method may be used for the binarization processing. FIG. 5 shows an enlarged view of the rectangle SQ1 in FIG. 4, and a rectangle SQ1' generated by binarizing the rectangle SQ1. In the rectangle SQ1', pixels corresponding to the crack and the straight lines described below are shown in white, and pixels corresponding to areas other than the crack are shown in black.

[0037] In step S8 of FIG. 2B, the control unit 11 counts the number of crack pixels, which are pixels corresponding to cracks, that exist on at least one straight line along the short side of the rectangle binarized in step S7. The rectangle SQ1′ in FIG. 5 shows multiple straight lines L1 to LN along the short side that the control unit 11 set. N may be any integer greater than or equal to 1. For example, the control unit 11 counts the number of crack pixels, which are white pixels, on line L1 as two pixels, the number of crack pixels on line L2 as three pixels, and the number of crack pixels on line L3 as two pixels. The control unit 11 also counts the number of crack pixels on other straight lines. The number and vertical positions of the straight lines may be freely set. For example, the control unit 11 sets straight lines at at least three locations, including both ends of the long side of the rectangle and a position corresponding to half the length. The control unit 11 may set the number and positions of the straight lines in accordance with a user specification via the input unit 14.

[0038] If the difference between the number of pixels on the long side and the number of pixels on the short side of the rectangle determined by the control unit 11 in step S4 is less than a predetermined value, the control unit 11 may accept a specification of the direction of the straight line in accordance with a user specification via the input unit 14.

[0039] In step S9 of FIG. 2B, the control unit 11 determines the width of the crack based on the counted value. The control unit 11 may determine the average or median of the number of crack pixels on each of the multiple straight lines as the crack width. Specifically, the control unit 11 calculates the average value by dividing the total number of crack pixels on each of the straight lines L1 to LN in FIG. 5 by the total number N of straight lines, and determines this as the crack width. The control unit 11 may also sort the numbers of crack pixels on each of the straight lines L1 to LN in ascending order, extract the number of crack pixels located in the middle, and calculate the median to determine this as the crack width.

[0040] As another example of a method for determining the crack width, the control unit 11 may remove outliers from the number of crack pixels for each of the multiple straight lines, and then determine the median or average of the number of crack pixels for each of the multiple straight lines as the crack width. Specifically, the control unit 11 may remove, as outliers, the number of crack pixels that is equal to or greater than a predetermined threshold. The control unit 11 may remove, as outliers, the number of crack pixels that is different from the median or average of the counted number of crack pixels by more than a predetermined value. The threshold or predetermined value may be set in advance and stored in the memory unit 12.

[0041] FIG. 6 shows an enlarged view of the rectangle SQ2 in FIG. 4 and a rectangle SQ2' generated by binarizing the rectangle SQ2. In the rectangle SQ2', pixels corresponding to the lines and the cracks are shown in white, while pixels corresponding to areas other than the cracks are shown in black. In the rectangle SQ2', the crack branches, and a line L4 along the short side crosses the crack branching point. The number of crack pixels counted by the control unit 11 on the line L4 is, for example, 8 pixels, and the threshold is 5 pixels. In this case, the control unit 11 deletes the values ​​of the 8 crack pixels on the line L4 as outliers and calculates the average value by dividing the total number of crack pixels on the lines other than the line L4 by the total number of lines other than the line L4. The control unit 11 may also calculate the median by sorting the number of crack pixels on each of the lines other than the line L4 in ascending order and extracting the number of crack pixels located in the center. This makes it possible to appropriately determine the width of a crack even if the set straight line spans a crack that partially extends in the width direction.

[0042] As another example of a method for determining the crack width, the control unit 11 may determine the median or average of the number of crack pixels on each of multiple straight lines as the crack width, taking into account the presence of crack pixels that are not continuous on the straight lines. Specifically, the control unit 11 first identifies, among multiple straight lines set on a rectangle, lines on which crack pixels are discontinuous as discontinuous lines. The control unit 11 counts the number of crack pixels on the discontinuous lines, divides the result by the number of discontinuous groups of crack pixels, and determines the value as the number of crack pixels on the discontinuous lines. The control unit 11 further determines the median or average of the number of crack pixels on each of multiple straight lines including the discontinuous line as the crack width.

[0043] A discontinuous line L5 is set in the rectangle SQ2' in FIG. 6. Two discontinuous crack pixel groups DS1 and DS2 exist on the discontinuous line L5. For example, suppose the control unit 11 counts three crack pixels in the first group DS1 and three crack pixels in the second group DS2. In this case, the control unit 11 divides the total number of six pixels counted for the discontinuous line L5 by two, the number of discontinuous groups, to determine the number of crack pixels on the discontinuous line L5 as three. The control unit 11 further calculates the average value by dividing the total number of crack pixels on each of the multiple lines including the discontinuous line L5 by the number of lines including the discontinuous line L5. The control unit 11 may also calculate the median by sorting the numbers of crack pixels on the multiple lines including the discontinuous line L5 in ascending order and extracting the number of crack pixels located in the center. This makes it possible to appropriately determine the width of a crack even when the set straight line spans a crack that has branched off.

[0044] In step S10 of FIG. 2B, the control unit 11 detects seams in the wall region extracted in step S2. Any method may be used for the detection. For example, the control unit 11 reads from the storage unit 12 a third trained model generated by machine learning using, as training data, a plurality of images to which labels indicating seams have been assigned. The third trained model may be generated by the information processing device 10, or may be received by the control unit 11 from an external server device via the communication unit 13. The control unit 11 applies the third trained model to the wall region extracted from the image in step S2 to detect seams. The control unit 11 may extract seams by performing a binarization process to change the wall region to white or black. FIG. 7 shows a wall region W1' generated by binarizing the wall region W1 and the extracted seams. In FIG. 7, pixels corresponding to the seams are shown in white, and pixels corresponding to areas other than the seams are shown in black. Although joints occur during the construction of a structure and are not a sign of deterioration over time, their appearance is similar to that of cracks. By being able to distinguish joints from cracks and detect them, it becomes possible to improve the efficiency of crack inspection work by users, as will be described later.

[0045] In step S11, the control unit 11 extracts overlapping portions between the detected seam pixels and crack pixels. Fig. 8 shows a wall region W1" including overlapping portions CS1 to CS3 between the seam pixels and crack pixels detected from the wall region W1' in Fig. 7.

[0046] In step S12, the control unit 11 converts the crack length and crack width determined in step S5 or step S6 and step S9 into actual size values. For example, the control unit 11 reads from the storage unit 12 a fourth trained model generated by machine learning using, as training data, a plurality of images labeled with a pixel / cm scale. The fourth trained model may be generated by the information processing device 10, or may be received by the control unit 11 from an external server device via the communication unit 13. The control unit 11 applies the fourth trained model to an image in which a crack is reflected to estimate the scale of the image. If the image acquired in S1 was segmented, the control unit 11 may combine the segmented images into a single image and then apply the fourth trained model to estimate the scale of the image. The control unit 11 converts the crack length and crack width determined in step S5 or step S6 and step S9 into actual size values ​​based on the estimated scale.

[0047] The fourth trained model may be generated by machine learning using an image containing an object with known dimensions as training data. In this case, the control unit 11 acquires the image containing the object in step S1. By applying the fourth trained model to the image, the control unit 11 can accurately estimate the scale and convert the crack length and crack width into actual size values.

[0048] The conversion to actual size values ​​is not limited to the above, and any method may be adopted. For example, the control unit 11 may identify the scale of the image based on information indicating the shooting distance from the camera that captured the image to the wall and scale information that associates the shooting distance with the scale. In this case, the control unit 11 converts the crack length and crack width into actual size values ​​using the identified scale. The scale information may be recorded such that the greater the shooting distance, the larger the actual size value for each pixel. The scale information may be set in advance and stored in the memory unit 12. The control unit 11 may accept user input of information indicating the shooting distance via the input unit 14. The information indicating the shooting distance may be included in the image.

[0049] In step S13, the control unit 11 outputs information indicating the detected crack, the presence of the extracted overlapping portion, and the actual length and width of the crack. Specifically, the control unit 11 generates an image on the image acquired in step S1 that shows the detected crack, the position of the extracted overlapping portion, and the actual length and width of the crack, and transmits the image to the user's terminal device via the communication unit 13. The control unit 11 may display the image on a display serving as the output unit 15. If the image acquired in step S1 was divided, the control unit 11 may combine the divided images into a single image and then generate an image on the single image that shows the detected crack, the position of the extracted overlapping portion, and the actual length and width of the crack. The operation of the control unit 11 then ends.

[0050] For example, the control unit 11 may emphasize cracks and overlapping portions in the output image by coloring or the like. For example, the control unit 11 may display a text message such as "Cracks present" for cracks and a text message such as "Possible cracks" for overlapping portions in the output image. By referring to the image, the user can distinguish and recognize the cracks themselves present in the wall from the overlapping portions. This allows the user to flexibly plan inspection work, for example, by determining whether or not to include cracks including overlapping portions indicated as "Possible cracks" in the inspection target.

[0051] FIG. 9 is an example of image IM' output by control unit 11. Image IM' is the result of reintegrating the divided images divided in step S1. In image IM', cracks C1 to C3 in the detected wall region W2 are shown in color, and overlapping portions CS4 between the cracks and the joints are highlighted. Speech bubble B shows the actual length and width values ​​of crack C2. However, the actual length and width values ​​of all cracks in image IM' may be displayed in a list.

[0052] For example, if the actual length or width of a crack converted in step S12 is equal to or greater than a predetermined value, the control unit 11 may emphasize the crack in the output image by changing the color to be colored or by adding a mark such as an arrow symbol, etc. This makes it easier for the user to identify cracks that should be inspected first.

[0053] The following additional notes are provided regarding the above-described embodiments.

[0054] (Additional note 1) The image capturing the wall of the structure is captured, extracting a wall region representing the wall from the image; Detecting cracks in the wall region; determining a length of the crack based on the number of pixels on the long side of a rectangle surrounding the detected crack; Counting the number of crack pixels that correspond to the crack and exist on at least one straight line along the direction of the short side of the rectangle; determining the width of the crack based on the counted value; An information processing device comprising a control unit that converts the determined crack length and crack width into actual size values. (Additional note 2) The control unit comparing the number of pixels on the long side of the rectangle with the number of pixels on the short side; If the difference between the number of pixels on the long side and the number of pixels on the short side is equal to or greater than a predetermined value, the number of pixels on the long side is determined to be the length of the crack; The information processing device described in Appendix 1, wherein if the difference between the number of pixels on the long side and the number of pixels on the short side is less than the predetermined value, the number of pixels on the diagonal of the rectangle is determined to be the length of the crack. (Additional note 3) the at least one straight line is a plurality of straight lines; The control unit Counting the number of crack pixels that exist on each of the plurality of straight lines; An information processing device as described in appendix 1 or 2, wherein outliers are removed from the number of crack pixels for each of the counted multiple straight lines, and the average or median of the number of crack pixels for each of the multiple straight lines is determined as the width of the crack. (Additional note 4) the at least one straight line is a plurality of straight lines; the plurality of straight lines include discontinuous straight lines that are lines along which the crack pixels are discontinuous, The control unit An information processing device according to any one of appendix 1 to 3, wherein the value obtained by counting the number of crack pixels present on the discontinuous straight lines is divided by the number of discontinuous groups of crack pixels to determine the number of crack pixels related to the discontinuous straight lines, and the average or median of the number of crack pixels related to each of the counted multiple straight lines is determined to be the width of the crack. (Additional note 5) The control unit further detects a joint of the wall in the wall region, Extracting an overlapping portion between the detected crack and the joint; 5. The information processing device according to any one of claims 1 to 4, wherein information indicating the presence of the overlapping portion is output. (Additional note 6) The control unit Further extracting a steel region representing a steel member included in the wall from the image; Detecting rust in the steel region; Counting the number of rust pixels that correspond to the detected rust to determine the area of ​​the rust; 6. The information processing device according to any one of appended items 1 to 5, wherein the determined area is converted into an actual size value. (Additional note 7) An information processing method executed by an information processing device, Acquiring an image in which the wall of the structure is reflected; extracting a wall region representing the wall from the image; detecting cracks in the wall region; Determining the length of the crack based on the number of pixels on the long side of a rectangle surrounding the detected crack; Counting the number of crack pixels that correspond to the crack and exist on at least one straight line along the direction of the short side of the rectangle; determining a width of the crack based on the counted value; and converting the determined crack length and crack width into actual size values. (Additional note 8) A program that causes a computer to function as the information processing device according to any one of claims 1 to 6.

[0055] Although the present disclosure has been described based on the drawings and embodiments, it should be noted that those skilled in the art can make various modifications or corrections based on the present disclosure, and therefore, it should be noted that these modifications or corrections are included in the scope of the present disclosure.

[0056] As a modification of the above embodiment, the control unit 11 may extract a steel material region representing the steel material of the wall from the image acquired in step S1, and further detect rust in the extracted steel material region.

[0057] For example, the control unit 11 reads out from the storage unit 12 a fifth trained model generated by machine learning using, as training data, a plurality of images labeled with a label indicating a steel region. The control unit 11 applies the fifth trained model to the image acquired in step S1 to extract the steel region. The control unit 11 further reads out from the storage unit 12 a sixth trained model generated by machine learning using, as training data, a plurality of images labeled with a label indicating rust in the steel region. The control unit 11 applies the sixth trained model to the extracted steel region to detect rust. The fifth trained model and the sixth trained model may be generated by the information processing device 10, or may be received by the control unit 11 from an external server device via the communication unit 13.

[0058] In this modification, the control unit 11 counts the number of rust pixels, which are pixels corresponding to detected rust, and determines the area of ​​the rust. The control unit 11 further calculates the determined area in terms of rust pixels / cm. 2 For example, the control unit 11 converts the value into an actual value according to a predetermined scale such as rust pixels / cm 2 The control unit 11 reads from the storage unit 12 a seventh trained model generated by machine learning using, as training data, a plurality of images labeled with a scale indicating the rust area. The control unit 11 applies the seventh trained model to an image showing rust to estimate the scale of the image. The control unit 11 converts the determined rust area into an actual size value based on the estimated scale. The seventh trained model may be generated by the information processing device 10, or may be received by the control unit 11 from an external server device via the communication unit 13. This is not a limitation. For example, the control unit 11 may identify the scale of the image based on scale information that associates the scale with the shooting distance from the camera that captured the image to the wall, and convert the rust area into an actual size value using the identified scale. In this case, the control unit 11 may accept, via the input unit 14, a user's input of information indicating the shooting distance of the image.

[0059] The control unit 11 outputs information indicating the rust in the detected steel region and the actual size of the converted rust area. For example, in step S13 described above, the control unit 11 may output information indicating the rust and the actual size of the rust area in addition to the cracks, the presence of overlapping portions, and the actual size of the crack length and width. According to this modification, it is possible to measure the rust area from an image of a structure including steel. This improves the efficiency of user inspections, for example by prioritizing the inspection of structures with large rust areas. [Explanation of symbols]

[0060] 10. Information processing equipment 11 Control section 12 Storage section 13 Communications Department 14 Input section 15 Output section

Claims

1. The image capturing the wall of the structure is captured, extracting a wall region representing the wall from the image; Detecting cracks in the wall region; determining a length of the crack based on the number of pixels on the long side of a rectangle surrounding the detected crack; Counting the number of crack pixels that correspond to the crack and exist on at least one straight line along the direction of the short side of the rectangle; determining the width of the crack based on the counted value; An information processing device comprising a control unit that converts the determined crack length and crack width into actual size values.

2. The control unit comparing the number of pixels on the long side of the rectangle with the number of pixels on the short side; If the difference between the number of pixels on the long side and the number of pixels on the short side is equal to or greater than a predetermined value, the number of pixels on the long side is determined to be the length of the crack; The information processing device according to claim 1 , wherein when the difference between the number of pixels on the long side and the number of pixels on the short side is less than the predetermined value, the number of pixels on the diagonal of the rectangle is determined as the length of the crack.

3. the at least one straight line is a plurality of straight lines; The control unit Counting the number of crack pixels that exist on each of the plurality of straight lines; The information processing device described in claim 1, wherein outliers are removed from the number of crack pixels for each of the counted multiple straight lines, and the average or median of the number of crack pixels for each of the multiple straight lines is determined as the width of the crack.

4. the at least one straight line is a plurality of straight lines; the plurality of straight lines include discontinuous straight lines that are lines along which the crack pixels are discontinuous, The control unit The information processing device of claim 1, wherein the number of crack pixels present on the discontinuous straight lines is counted and divided by the number of discontinuous groups of crack pixels to determine the number of crack pixels related to the discontinuous straight lines, and the average or median of the number of crack pixels related to each of the counted multiple straight lines is determined to be the width of the crack.

5. The control unit further detects a joint of the wall in the wall region, Extracting an overlapping portion between the detected crack and the joint; The information processing apparatus according to claim 1 , wherein information indicating the presence of the overlapping portion is output.

6. The control unit Further extracting a steel region representing a steel member included in the wall from the image; Detecting rust in the steel region; Counting the number of rust pixels that correspond to the detected rust to determine the area of ​​the rust; The information processing apparatus according to claim 1 , wherein the determined area is converted into an actual size value.

7. An information processing method executed by an information processing device, Acquiring an image in which the wall of the structure is reflected; extracting a wall region representing the wall from the image; detecting cracks in the wall region; Determining the length of the crack based on the number of pixels on the long side of a rectangle surrounding the detected crack; Counting the number of crack pixels that correspond to the crack and exist on at least one straight line along the direction of the short side of the rectangle; determining a width of the crack based on the counted value; and converting the determined crack length and crack width into actual size values.

8. A program that causes a computer to function as the information processing device according to any one of claims 1 to 6.

Images