Processing apparatus, processing program, processing method, and processing system

Abstract

To provide a processing device, processing program, processing method, and processing system that enable more efficient inspection of objects to be inspected. [Solution] A processing device comprising at least one processor, wherein the at least one processor is configured to receive detection information detected by a sensor from at least a portion of an object to be inspected being constructed at a work site, inspect the construction status of the object to be inspected based on the received detection information and pre-set design information for the object to be inspected, and to perform processing to output the results of the inspection.

Description

Technical Field

[0001] The present disclosure relates to a processing apparatus, a processing program, a processing method, and a processing system configured to execute processing related to inspection of an inspection object constructed at a work site.

Background Art

[0002] Conventionally, a system for inspecting an object constructed at a work site has been known. For example, Patent Document 1 describes a system using "an information processing apparatus including a storage unit that records a captured processing target image, and a control unit that acquires an inspection area, which is an area in the processing target image where a determination target object is captured, using the processing target image recorded in the storage unit, determines the quality of construction related to the determination target object shown in the inspection area, and determines the image quality of the inspection area".

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Therefore, based on the above - mentioned technologies, an object of the present disclosure is to provide a processing apparatus, a processing program, a processing method, and a processing system that can more efficiently inspect an inspection object according to various embodiments.

Means for Solving the Problems

[0005] According to one aspect of the present disclosure, a processing device is provided comprising at least one processor, wherein the at least one processor is configured to receive detection information detected by a sensor from at least a portion of an object to be inspected being constructed at a work site, inspect the construction status of the object to be inspected based on the received detection information and pre-set design information for the object to be inspected, and to perform processing for outputting the results of the inspection.

[0006] According to one aspect of the present disclosure, a processing program is provided which is executed by at least one processor to receive detection information detected by sensors from at least a portion of an object to be inspected being constructed at a work site, inspects the construction status of the object to be inspected based on the received detection information and pre-set design information for the object to be inspected, and causes the at least one processor to function to output the results of the inspection.

[0007] According to one aspect of the present disclosure, a processing method is provided which is performed by at least one processor and includes the steps of: receiving detection information detected by a sensor from at least a portion of an object to be inspected being constructed at a work site; inspecting the construction status of the object to be inspected based on the received detection information and pre-set design information for the object to be inspected; and outputting the results of the inspection.

[0008] According to one aspect of this disclosure, the processing system includes "a sensor configured to detect detection information from at least a portion of an object to be inspected being constructed at a work site, and the processing device described above, which is connected to the sensor by a communication network and configured to receive the detection information from the sensor." [Effects of the Invention]

[0009] This disclosure provides a processing device, a processing program, a processing method, and a processing system that enable more efficient inspection of objects to be inspected.

[0010] The effects described above are merely illustrative for the sake of explanation and are not limiting. In addition to, or in lieu of, any other effects described herein or that would be obvious to those skilled in the art may be achieved. [Brief explanation of the drawing]

[0011] [Figure 1A] Figure 1A is a diagram conceptually illustrating the usage of processing system 1 according to one embodiment of this disclosure. [Figure 1B] Figure 1B is a block diagram showing the configuration of a processing system 1 according to one embodiment of the present disclosure. [Figure 2A] Figure 2A is a block diagram showing the configuration of a processing apparatus 100 according to one embodiment of the present disclosure. [Figure 2B] Figure 2B is a block diagram showing the configuration of a terminal device 200 according to one embodiment of the present disclosure. [Figure 3] Figure 3 is a conceptual diagram showing a field management table stored in a processing device 100 according to one embodiment of the present disclosure. [Figure 4] Figure 4 shows a processing sequence executed by a processing system 1 according to one embodiment of the present disclosure. [Figure 5] Figure 5 is a diagram showing the processing flow performed in the processing apparatus 100 according to one embodiment of the present disclosure. [Figure 6] Figure 6 is a conceptual diagram showing an example of captured image information captured by a terminal device 200 according to one embodiment of this disclosure. [Figure 7] Figure 7 is a conceptual diagram showing an example of point cloud information generated in a processing apparatus 100 according to one embodiment of the present disclosure. [Figure 8] Figure 8 is a conceptual diagram showing an example of point cloud information generated in a processing apparatus 100 according to one embodiment of the present disclosure. [Figure 9A] Figure 9A is a conceptual diagram showing an example of target image information generated in a processing apparatus 100 according to one embodiment of the present disclosure. [Figure 9B] FIG. 9B is a diagram conceptually showing an example of target image information generated in the processing device 100 according to an embodiment of the present disclosure. [Figure 10] FIG. 10 is a diagram showing an example of inspection report information generated by the processing device 100 according to an embodiment of the present disclosure.

Mode for Carrying Out the Invention

[0012] 1. Overview of Processing System 1 Hereinafter, various embodiments of the present invention will be described with reference to the accompanying drawings. In the drawings, the same reference numerals are assigned to common components. Also, note that components shown in one drawing may be omitted in another drawing for convenience of explanation. Furthermore, note that the attached drawings are not necessarily drawn to an exact scale.

[0013] The various systems, methods, and apparatuses described in the present disclosure should not be construed as being limited in any way. In fact, the present disclosure is directed to any novel features and aspects among each of the disclosed various embodiments, combinations of these various embodiments with each other, and combinations of some of these various embodiments with each other. The various systems, methods, and apparatuses described in the present disclosure are not limited to a specific aspect, a specific feature, or a combination of such a specific aspect and a specific feature, and the things and methods described in the present disclosure do not require the existence of one or more specific effects or the solution of problems. Furthermore, various features or aspects of the various embodiments described in the present disclosure, or some of such features or aspects, can be used in combination with each other.

[0014] Although the operations of some of the various methods disclosed in this disclosure are described, for convenience, in a particular order, such a description by such a technique should be understood to include rearranging the order of the above operations, unless a particular order is required by the following specific passage. For example, multiple operations described in sequence may, in some cases, be rearranged or performed simultaneously. Furthermore, for the purpose of simplification, the accompanying drawings do not show various ways in which the various matters and methods described in this disclosure can be used together with other matters and methods.

[0015] The operational theory, scientific principle, or other theoretical descriptions presented in this disclosure in relation to the apparatus or method of this disclosure are provided for better understanding and are not intended to limit the technical scope. The apparatus and method in the appended claims are not limited to the apparatus and method that operate by the method described by such an operational theory.

[0016] Any of the various methods disclosed in this disclosure is implemented using a plurality of computer-executable instructions stored in one or more computer-readable media and can further be executed on a computer. The above one or more media can be, for example, a non-transitory computer-readable storage medium such as at least one optical disk, a plurality of volatile memory components, or a plurality of non-volatile memory components. Here, the above plurality of volatile memory components include, for example, DRAM or SRAM. Also, the above plurality of non-volatile memory components include, for example, hard drives and solid-state drives (SSDs). Furthermore, the above computer includes any computer available in the market, including, for example, smartphones and other mobile devices having hardware for performing calculations.

[0017] Any of the multiple computer-executable instructions for implementing the technology disclosed herein may be stored in one or more computer-readable media (e.g., non-temporary computer-readable storage media) along with any data generated and used during implementations of the various embodiments disclosed herein. Such multiple computer-executable instructions may, for example, be part of a separate software application, or part of a software application accessed or downloaded via a web browser or other software application (such as a remote computing application). Such software may be executed, for example, on a single local computer (as a process run on any suitable computer available on the market), or in a network environment (e.g., the Internet, a wide area network, a local area network, a client-server network (such as a cloud computing network), or other such network) using one or more network computers.

[0018] For clarity, only specific selected aspects of various software-based implementations are described. Other details that are well known in the art are omitted. For example, the technology disclosed in this disclosure is not limited to any particular computer language or program. For example, the technology disclosed in this disclosure may be executed by software written in C, C++, Java®, or any other suitable programming language. Similarly, the technology disclosed in this disclosure is not limited to any particular computer or type of hardware. Specific details of suitable computers and hardware are well known and do not need to be described in detail in this disclosure.

[0019] Furthermore, any of the various embodiments of such software (including, for example, a plurality of computer-executable instructions for causing a computer to perform any of the various methods disclosed herein) may be uploaded, downloaded, or accessed remotely by preferred means of communication. Such preferred means of communication include, for example, the Internet, the World Wide Web, intranets, software applications, cables (including fiber optic cables), magnetic communications, electromagnetic communications (including RF communications, microwave communications, and infrared communications), electronic communications, or other such means of communication.

[0020] The processing system 1 described herein is used to perform processing related to the inspection of an object to be inspected while it is being constructed at a work site. Specifically, the processing system 1 receives detection information detected by sensors from at least a portion of the object to be inspected while it is being constructed at the work site. The processing system 1 also inspects the construction status of the object to be inspected based on the received detection information and pre-set design information for the object to be inspected. The processing system 1 also outputs the inspection results. In this way, the processing system 1 can inspect the object to be inspected instead of a person, such as a user or an inspector, performing the inspection. Furthermore, because the processing system 1 performs the inspection based on detection information detected by sensors, it is possible to perform the inspection more efficiently. Furthermore, because the processing system 1 performs the inspection based on pre-set design information for the object to be inspected, it is possible to perform the inspection more accurately.

[0021] Furthermore, processing system 1 inspects reinforcing bars, such as reinforcement bars used in reinforced concrete structures, as the objects to be inspected. Therefore, processing system 1 can more efficiently inspect such reinforcing bars.

[0022] Furthermore, the processing system 1 utilizes a camera as a sensor and captured image information that includes at least a portion of the object to be inspected as the subject, as detection information. Therefore, the processing system 1 can perform inspections of the object to be inspected with a simpler configuration.

[0023] Furthermore, the processing system 1 inspects the construction status of the object under inspection using point cloud information generated based on the detection information. For example, the processing system 1 generates point cloud information based on marker position information detected by the sensor from marker information positioned to correspond to the object under inspection. The processing system 1 also generates target image information from the point cloud information, extracting at least a portion of the object under inspection, and performs the inspection by comparing the generated target image information with the design information. The processing system 1 also generates the target image information by removing noise components from the point cloud information. Therefore, the processing system 1 can generate point cloud information more accurately and efficiently, enabling more accurate and efficient inspection of the object under inspection.

[0024] Furthermore, the processing system 1 uses multiple captured image information, each containing at least a portion of the object to be inspected, as detection information, and includes at least one selected image information chosen from among the multiple captured image information as a result of the inspection. The processing system 1 also selects the selected image information based on its degree of agreement with pre-specified conditions based on design information. Therefore, the processing system 1 can generate inspection reports and the like more efficiently.

[0025] In this disclosure, "object to be inspected" can refer to any object constructed at the work site. Such objects to be inspected include a variety of things constructed at the work site, such as the ground, foundation, frame, load-bearing wall, floor frame, roof surface, joint, connecting hardware, structural members, underground pipes, interior materials, subflooring, exterior walls, eaves, ventilation structures, drainage pipes, insulation structures, alarm systems, fire-resistant equipment, fire-extinguishing equipment, cleaning ports for pipes, inspection ports for pipes, intake and exhaust ports, rooms, handrails, passageways, bathrooms, toilets, or bedrooms. Among these, foundations are preferred as objects to be inspected, reinforcing bars used in reinforced concrete structures are more preferred, and reinforcement arrangements used in reinforced concrete structures are even more preferred. It should be noted that "object to be inspected" does not mean only individual structures as exemplified above, but can also include combinations of multiple structures as exemplified above, or collections of structures as exemplified above. For example, if the object being inspected is reinforcing bars, it's not just a single bar that's being inspected; a collection of multiple bars (e.g., 12 bars) can also be the object of inspection. Furthermore, for the sake of clarity, the following explanation will focus on the case where the object being inspected is reinforcing bars used in reinforced concrete structures, but naturally, the objects being inspected are not limited to these.

[0026] Furthermore, in this disclosure, "inspection" refers to any method of inspecting the construction information of the object being inspected. Such inspections are performed, for example, by comparing pre-set design information with detection information detected by sensors for the object being inspected, but they may also be performed by other methods, such as comparing the object with pre-set numerical values ​​or standards. In addition, for the sake of explanation, the term "inspection" will be used below, but naturally, various other terms such as evaluation, test, comparison, acceptance, audit, review, examination, inspection, or check can also be included in the definition of inspection.

[0027] Furthermore, in this disclosure, "work site" can refer to any site where the object under inspection is constructed. Such work sites include a variety of sites such as construction, engineering, cleaning, manufacturing, repair / maintenance, or agricultural work. Among these, construction, engineering, or repair / maintenance sites are preferred, with construction sites being more preferred. For the sake of explanation, the following description will focus on the case where the work site is a construction site for a reinforced concrete structure, but of course, the work site is not limited to this.

[0028] Furthermore, in this disclosure, "user" may refer to any person who is able to use the terminal device 200. Such users include persons with various attributes, such as employees, workers, inspectors, contractors, or managers. Moreover, "user" does not refer only to a specific individual, but may also include the organization to which that individual belongs (i.e., in cases where any individual belonging to the organization uses the terminal device 200).

[0029] Figure 1A is a conceptual diagram illustrating the usage of a processing system 1 according to one embodiment of the present disclosure. Specifically, Figure 1A is a conceptual diagram illustrating the case in which the processing system 1 described above is used for inspecting reinforcement. According to Figure 1A, a work site 10 is equipped with a reinforced concrete structure 30 including reinforcement, other structures 31-33 (e.g., tools, columns, or exterior panels before construction), and marker information 20. For example, in such a work site 10, the construction status of the reinforcement installed in the reinforced concrete structure 30 has traditionally been inspected visually by a user or inspector. In Figure 1A, instead of or in addition to this, a sensor (e.g., a camera) provided in the terminal device 200 detects detection information (e.g., captured image information) from the reinforced concrete structure 30 on which the reinforcement is installed, which is the object to be inspected. Then, a processing device (not shown in Figure 1A) processes the detected information to inspect the construction status of the reinforcement, which is the object to be inspected. Furthermore, the object to be inspected can be identified from among multiple objects by detecting marker information 20 that is positioned to correspond to the object to be inspected. In this way, instead of a person such as a user or inspector inspecting the object to be inspected, the processing unit performs the inspection based on detection information detected by the sensor, making it possible to inspect the object to be inspected more efficiently.

[0030] In the following description, we will explain the use of processing system 1 when inspecting reinforcement at the work site shown in Figure 1A, but the use of processing system 1 is not limited to this work site.

[0031] 2. Configuration of Processing System 1 Figure 1B is a block diagram showing the configuration of a processing system 1 according to one embodiment of the present disclosure. According to Figure 1B, the processing system 1 includes at least a processing unit 100 and a terminal device 200-1, each device being connected to communicate via a wired or wireless communication network. The processing unit 100 is used to inspect an object based on detection information received from the terminal device 200. The terminal device 200 also includes a sensor, which is used to acquire detection information used for inspecting the object and to transmit the acquired detection information to the processing unit 100.

[0032] In this disclosure, the processing unit 100 may be any device capable of performing the processing that the processing unit 100 performs. That is, various devices such as on-premise server devices, cloud server devices, smartphones, tablet devices, laptop PCs, and desktop PCs can be used as processing units. Furthermore, a terminal device 200 can also function as the processing unit 100. Moreover, in this disclosure, the storage and processing performed by the processing unit 100 may be distributed to other terminal devices, including the terminal device 200, or other server devices. In other words, the processing unit 100 is not limited to being composed of a single enclosure, and the processing unit 100 also includes various combinations of devices as exemplified above.

[0033] Furthermore, in this disclosure, the terminal device 200 may be any device equipped with a sensor and capable of detecting detection information. In other words, various devices such as camera devices, radar devices, smartphones, tablet devices, laptop PCs, and desktop PCs can be used as terminal devices. Note that the terminal device 200 does not need to be manufactured specifically for the processing system 1; a general-purpose terminal device can be used, for example, by installing other applications and using it for other purposes.

[0034] (A) Configuration of the processing unit 100 Figure 2A is a block diagram showing the configuration of a processing unit 100 according to one embodiment of the present disclosure. According to Figure 2A, the processing unit 100 includes a processor 111, a memory 112, and a communication interface 113. Each of these components is electrically connected to the others via control lines and data lines. The processing unit 100 does not need to have all of the components shown in Figure 2A; it is possible to omit some components or add other components. For example, it is possible to use a communicationally connected external memory, a database device, or a server device as memory. It is also possible to distribute and execute some processing with other processing units, including other server devices. In other words, the processing unit 100 is not limited to a single device, but also includes cases where it is distributed across multiple devices depending on the handling of information and the processing load.

[0035] The processor 111 functions as a control unit that controls other components of the processing system 1 based on processing programs stored in the memory 112. The processor 111 is mainly composed of one or more CPUs, but may be combined with a GPU or FPGA as appropriate. The processor 111 executes processes related to the inspection of the object to be inspected based on processing programs stored in the memory 112. Specifically, the processor 111 executes processes such as "receiving detection information detected by sensors from at least a part of the object to be inspected being constructed at the work site," "inspecting the construction status of the object to be inspected based on the received detection information and pre-set design information for the object to be inspected," "outputting the inspection results," "generating point cloud information based on the received detection information and marker position information detected from marker information arranged to correspond to the object to be inspected," "removing noise components from the generated point cloud information and generating target image information from which at least a part of the object to be inspected has been extracted," "selecting at least one selected image information from a plurality of captured image information including at least a part of the object to be inspected as the subject, based on the design information," and "generating an inspection report based on the inspection results and the selected image information," based on processing programs stored in the memory 112.

[0036] Memory 112 consists of RAM, ROM, non-volatile memory, HDD, SSD, etc., and functions as a storage unit. In addition to the program, memory 112 stores various information stored in field management tables, etc. This information does not need to be constantly stored in memory 112 within the processing unit 100; it may be stored in a database device installed remotely. In that case, the database device is also included in memory 112.

[0037] Furthermore, the memory 112 stores instruction commands for various controls of the processing system 1 according to this embodiment as processing programs. Specifically, the memory 112 stores programs for the processor 111 to execute processes such as: "receiving detection information detected by sensors from at least a portion of the object to be inspected being constructed at the work site"; "inspecting the construction status of the object to be inspected based on the received detection information and pre-set design information for the object to be inspected"; "outputting the inspection results"; "generating point cloud information based on the received detection information and marker position information detected from marker information arranged to correspond to the object to be inspected"; "removing noise components from the generated point cloud information and generating target image information from which at least a portion of the object to be inspected has been extracted"; "selecting at least one selected image information from a plurality of captured image information including at least a portion of the object to be inspected as the subject, based on the design information"; and "generating an inspection report based on the inspection results and the selected image information".

[0038] The communication interface 113 functions as a notification unit for sending and receiving various information between the terminal device 200 and other processing devices connected via a wired or wireless communication network. Examples of the communication interface 113 include wired communication connectors such as USB and SCSI, wireless communication transceivers such as wireless LAN, Bluetooth®, LTE, and infrared, and various connection terminals for printed circuit boards and flexible circuit boards.

[0039] (B) Configuration of terminal device 200 Figure 2B is a block diagram showing the configuration of a terminal device 200 according to one embodiment of the present disclosure. Specifically, Figure 2B is a block diagram showing the configuration of a terminal device 200. Although one terminal device 200 is shown in Figure 1B, of course, multiple terminal devices may be included in the processing system 1.

[0040] The terminal device 200 includes a processor 211, memory 212, input interface 213, output interface 214, communication interface 215, and sensor 216. Each of these components is electrically connected to the others via control lines and data lines. Note that the terminal device 200 does not need to have all of the components shown in Figure 2B; it is possible to omit some components or add other components.

[0041] The processor 211 functions as a control unit that controls other components of the terminal device 200 based on a program stored in the memory 212. The processor 211 is mainly composed of one or more CPUs, but may be combined with a GPU, FPGA, etc. as appropriate.

[0042] Based on the program stored in the memory 212, the processor 211 executes the following processes: "a process that accepts user operation input via the input interface 213 and starts an application that can be used for inspecting the object to be inspected," "a process that accepts user operation input via the input interface 213 and starts a sensor provided in the terminal device 200," "a process that uses the sensor to detect detection information from marker information placed at the work site corresponding to the object to be inspected," "a process that uses the sensor to detect detection information from the object to be inspected," and "a process that uses the communication interface 215 to transmit the detection information detected by the sensor to the processing unit 100."

[0043] Memory 212 consists of RAM, ROM, non-volatile memory, HDD, etc., and functions as a storage unit. Memory 212 stores instruction commands for various controls of the processing system 1 according to this embodiment as programs.

[0044] Specifically, the memory 212 stores programs for the processor 211 to execute, such as "a process that accepts user input via the input interface 213 and starts an application that can be used for inspecting the object to be inspected," "a process that accepts user input via the input interface 213 and starts a sensor provided in the terminal device 200," "a process that uses the sensor to detect detection information from marker information placed at the work site corresponding to the object to be inspected," "a process that uses the sensor to detect detection information from the object to be inspected," and "a process that uses the communication interface 215 to transmit the detection information detected by the sensor to the processing unit 100."

[0045] The input interface 213 functions as an input unit that receives user input to the terminal device 200. Examples of the input interface 213 include physical key buttons and a touch panel having an input coordinate system corresponding to the display coordinate system of the display. In the case of a touch panel, icons are displayed on the display, and the user makes a selection for each icon by making an input via the touch panel. The method for detecting user input via the touch panel can be any method, such as capacitive or resistive. The input interface 213 does not always need to be physically provided on the terminal device 200 and may be connected as needed via a wired or wireless network. Therefore, in addition to the above, a mouse, keyboard, microphone for voice input, camera for gesture input, etc., can also be used as the input interface 213.

[0046] The output interface 214 functions as an output unit for outputting various information received from the processing unit 100, etc. An example of the output interface 214 is a display, but it is not limited to this and may consist of other liquid crystal panels, organic EL displays, plasma displays, speakers for audio output, light-emitting devices such as LEDs, vibration devices, printers, etc. Furthermore, it is not necessary for a display to be provided; for example, an interface for connecting to a display, etc. that can be connected to the processing unit 100 via a wired or wireless network can also function as the output interface 214 for outputting display data to the display, etc.

[0047] The communication interface 215 functions as a communication unit for sending and receiving information with the processing unit 100, other terminal devices 200, and storage device 300. Examples of the communication interface 215 include wired communication connectors such as USB and SCSI, transceiver devices for wideband wireless communication such as wireless LAN, Bluetooth®, and LTE, short-range or contactless wireless communication such as infrared and NFC, and various connection terminals for printed circuit boards and flexible circuit boards.

[0048] Sensor 216 functions as a detection unit for detecting detection information from the object to be inspected. Examples of sensors 216 include various sensors such as cameras (e.g., stereo cameras), distance measuring sensors, ultrasonic sensors, infrared sensors, ultraviolet sensors, proximity sensors, various radar sensors, and photoelectric sensors, but a sensor capable of detecting the distance and positional relationship to the object to be inspected, as well as the shape of the object to be inspected, is preferred. Note that sensor 216 does not need to be constantly connected to terminal device 200, and may be connected to terminal device 200 or processing device 100 via a wired or wireless communication network. This configuration increases the flexibility of where sensor 216 can be used.

[0049] 3. Various information used in processing in processing system 1 Figure 3 is a conceptual diagram showing a site management table stored in a processing unit 100 according to one embodiment of this disclosure. Such a site management table is generated for each site ID information used to identify each work site with information unique to each work site. That is, although Figure 3 shows a site management table for managing the work site with site ID information = X1, a similar site management table is generated for a different work site (for example, the work site with site ID information = X2).

[0050] As shown in Figure 3, the site management table stores design information, marker information, captured image information, point cloud information, target image information, result information, and selected image information, all associated with the inspection object ID information of the inspection objects constructed at each work site. The "inspection object ID information" is unique to each inspection object constructed at the work site and is used to identify each inspection object. For example, the inspection object ID information is generated each time a user registers any structure included in the design drawing information as an inspection object in the site management table via the terminal device 200.

[0051] "Design information" is information that serves as a design drawing referenced in the construction of the object to be inspected, which is identified by the object ID information. Therefore, design information includes, for example, at least one of the following: the arrangement, dimensions, number, and materials of the structures that make up the object to be inspected. Such design information is used, for example, as a comparison point for inspection information when inspecting the object to be inspected based on detection information. Furthermore, such design information is generated from, for example, digitized information such as design drawings of the entire work site or so-called miniature drawings that supplement the design drawings, design drawing data of the work site or the object to be inspected (e.g., CAD data), three-dimensional models showing the completed work site or the object to be inspected, image information of other work sites or other objects to be inspected, or specification information showing the work content at the work site or the construction content of the object to be inspected.

[0052] "Marker information" is placed at the work site in a manner corresponding to the object being inspected in order to identify the object being inspected. Furthermore, marker information is detected by a sensor and used to determine the positional information from the sensor (the positional relationship between the sensor and the marker information). The positional information of the marker information is used to determine the positional relationship between the marker information and the object being inspected, based on the positional information of the object being inspected detected by the sensor (the positional relationship between the sensor and the object being inspected), and to determine the coordinate position (positional information) of the object being inspected in a three-dimensional space with the marker information as the origin. Note that various types of marker information can be used, such as barcodes, two-dimensional codes (e.g., ArUco markers), arbitrary strings of characters, arbitrary shapes, arbitrary symbols, or combinations thereof. Also, as mentioned above, marker information is used for both the identification of the object being inspected and the determination of its position, but a single marker information does not need to fulfill both roles. For example, marker information for object identification and marker information for location determination could be placed separately at the work site. Furthermore, by configuring the three-dimensional space with the marker information as the origin to correspond to the coordinate axis system indicated by the design information (for example, the coordinate axis system of the two-dimensional space shown in the design drawing or the three-dimensional space representing the three-dimensional model), it becomes possible to easily compare the point cloud of the object to be inspected, which is placed in the three-dimensional space with the marker information as the origin, with the design information. Therefore, the marker information is positioned to correspond to the origin position of the coordinate axis system indicated by the design information.

[0053] "Captured image information" is information used as one of the detection information detected by a sensor. In other words, when a camera is used as the sensor, it refers to the image information detected, i.e., captured, by the camera. Such captured image information includes image data detected by the camera, depth data of the subject, shooting attribute data (for example, shooting position information of the terminal device 200 in a three-dimensional space with the marker image as the origin, angle of view (the orientation of the terminal device 200 at the time of shooting), focal length, sensitivity, aperture value or shutter speed, etc.), or a combination of these. The image data is not limited to a single still image, but may be in any form, such as a burst of images obtained by continuously capturing multiple still images, or a video composed of multiple frames. Furthermore, the image data may be any image detected by the sensor, and may be of any type, such as a color image, a black and white image, an infrared image, an ultraviolet image, or an X-ray image.

[0054] As mentioned above, captured image information is one type of detection information. However, such detection information may include various types of information detected by distance sensors, ultrasonic sensors, infrared sensors, ultraviolet sensors, proximity sensors, various radar sensors, photoelectric sensors, or combinations thereof, in addition to captured image information. Furthermore, for the sake of clarity, captured image information will be described below as a specific example of detection information, but detection information is not limited to this alone.

[0055] "Point cloud information" is information that represents the shape of a subject contained in captured image information as a collection of points. As an example, such point cloud information can represent the distance (depth) from the shooting position to any point on the surface or inside of the subject as a point in a virtual three-dimensional space with an arbitrary position (for example, the position of marker information or the position of the sensor) as the origin. In addition to the position information (coordinates) of each point in three-dimensional space as described above, point cloud information can also include color information of each point.

[0056] "Target image information" is image information that includes at least a portion of the object to be inspected, extracted from various subjects included in the point cloud information. Such target image information is constructed, for example, based on the positional information in three-dimensional space of each point in the point cloud, i.e., at least a portion of the object to be inspected, and color information as needed. The extraction of target image information that includes at least a portion of the object to be inspected is performed, for example, by using a bounding box set in three-dimensional space with marker information as the origin, by removing various noise components from the point cloud information, or by a combination of these methods.

[0057] "Result information" refers to information indicating the results of the inspection of the construction status of the object being inspected. Such result information may be expressed in various ways, such as information indicating classifications like pass or fail, good or poor, information indicating scores such as arbitrary numbers, information indicating degree, or information indicated by arbitrary strings of characters such as comments. Furthermore, result information may also include various formats such as inspection reports, documents, emails, audio, images, or combinations thereof.

[0058] "Selected image information" is one or more image pieces selected from captured image information, point cloud information, or target image information as one of the result information to show the construction status of the object being inspected. For example, by including one or more selected image pieces in a document such as an inspection report, it becomes possible to visually confirm the current construction status while comparing it with the inspection results. Such selected image information may be one or more image pieces selected from multiple captured image pieces based on arbitrary conditions. Various arbitrary conditions can be set, such as a field of view that allows all reinforcement to be seen based on design information (for example, a three-dimensional model showing the completed object being inspected), the number of reinforcements included in each captured image piece, or a combination of these.

[0059] The various pieces of information shown in Figure 3 are just examples of information stored in association with the site ID information, and may naturally include other types of information as well.

[0060] 4. Processing sequence executed by processing system 1 Figure 4 is a diagram showing a processing sequence performed by a processing system 1 according to one embodiment of the present disclosure. Specifically, Figure 4 is a diagram showing a series of processing sequences from when the object to be inspected is photographed by a sensor (e.g., a camera) until inspection report information is output. In the following description, we will refer to Figure 1A as appropriate.

[0061] As shown in Figure 4, the processor 211 of the terminal device 200 accepts user input via the input interface 213 as needed and starts an application program that can be used for inspecting the object to be inspected by the processing system 1 (S11). It is assumed that the application program is pre-installed on the terminal device 200. When the application program is started, the processor 211 of the terminal device 200 accepts user input via the input interface 213 and selects various functions and menus that can be executed by the application program. Then, when the processor 211 of the terminal device 200 accepts user input via the input interface 213 and selects a function or menu for performing the inspection of the object to be inspected, it activates the sensor 216 (e.g., camera) to detect detection information from the object to be inspected (S12).

[0062] Next, the user, while holding the terminal device 200 at the work site 10, photographs the marker information 20 that is positioned to correspond to the object to be inspected (S13). Figure 1A describes the case where a two-dimensional code is positioned as the marker information 20. The processor 211 of the terminal device 200 acquires the object ID information assigned to identify the object to be inspected by photographing the marker information 20 with the sensor 216. The processor 211 of the terminal device 200 also determines the coordinates (position information) of the terminal device 200 in a three-dimensional space with the marker information 20 as the origin by photographing the marker information 20 with the sensor 216.

[0063] Next, the user, holding the terminal device 200 at the work site 10 with the sensor 216 pointed toward the object to be inspected, moves around the object to be inspected and takes photographs that include at least a part of the object to be inspected. That is, the processor 211 of the terminal device 200 controls the sensor 216 (i.e., the camera) to detect detection information (i.e., captured image information) from the object to be inspected (i.e., captures an image) (S14). As a result, the processor 211 of the terminal device 200 acquires captured image information, which includes detection information, specifically image data, depth data of the subject, and various shooting attribute data. The processor 211 of the terminal device 200 then transmits the captured image information (T11) along with the object ID information identified from the marker information 20 to the processing unit 100 via the communication interface 215.

[0064] The processes in S13 and S14 will be explained in detail in Figure 7.

[0065] When the processor 111 of the processing unit 100 receives inspection object ID information and captured image information from the terminal device 200 via the communication interface 113, it refers to the field management table and stores the captured image information in association with the inspection object ID information. Then, the processor 111 of the processing unit 100 generates point cloud information based on the stored captured image information (S15). As an example of this process, the processor 111 of the processing unit 100 identifies the location information (coordinates in a three-dimensional space with marker information 20 as the origin) of each point of the subject in the image data, based on the location information (coordinates in a three-dimensional space with marker information 20 as the origin) captured by the terminal device 200 contained in the captured image information. Then, the processor 111 of the processing unit 100 generates point cloud information from the captured image information by placing each point in the three-dimensional space based on the identified location information of each point. When the processor 111 of the processing unit 100 generates point cloud information, it refers to the field management table and stores the point cloud information in association with the inspection object ID information. This process will be explained in detail in Figure 5.

[0066] Next, the processor 111 of the processing unit 100 extracts at least a portion of the object to be inspected from the stored point cloud information and generates target image information (S16). Then, the processor 111 of the processing unit 100 refers to the field management table and stores the generated target image information in association with the object to be inspected ID information. This process will be explained in detail in Figure 5.

[0067] Next, the processor 111 of the processing unit 100 reads the design information associated with the inspection target ID information from the site management table and inspects the construction status of the inspection target by comparing the design information with the target image information (S17). As an example of this process, the processor 111 of the processing unit 100 calculates the degree of agreement between the three-dimensional model of the inspection target stored as design information and the generated target image information. As another example of this process, the processor 111 of the processing unit 100 identifies, for example, the number of reinforced concrete bars installed from the target image information and compares it with the number of reinforced concrete bars stored as design information to determine whether they match. It should be noted that such comparison methods are not limited to these methods, and any method may be used.

[0068] Next, the processor 111 of the processing unit 100 obtains result information indicating the results of the inspection of the construction status of the object to be inspected by comparing the target image information with the design information (S18), and stores the result information in association with the object to be inspected ID information by referring to the site management table.

[0069] Next, the processor 111 of the processing unit 100 selects one or more image information from among the captured image information, point cloud information, or target image information, and stores the selected image information in association with the inspection target ID information by referring to the field management table. Then, the processor 111 of the processing unit 100 generates inspection report information in document format, for example (S19), based on the result information and the selected image information. The processor 111 of the processing unit 100 transmits the generated inspection report information (T12) via the communication interface 113 to the terminal device 200 that transmitted the captured image information, etc., or to other terminal devices. This process will be explained in detail in Figures 5 and 10.

[0070] When the processor 211 of the terminal device 200 receives inspection report information in document format from the processing unit 100 via the communication interface 215, it outputs the received inspection report information via the output interface 214 (S20). This completes the processing sequence.

[0071] Thus, the processing unit 100 performs inspection of the object to be inspected based on detection information (e.g., captured image information) detected by the sensor 216 of the terminal device 200, making inspection more efficient. Furthermore, the processing unit 100 performs inspection of the object to be inspected based on pre-set design information, making inspection more accurate.

[0072] 5. Processing flow executed by the processing unit 100 Figure 5 is a diagram showing the processing flow performed in a processing apparatus 100 according to one embodiment of the present disclosure. Specifically, Figure 5 is a diagram showing the specific processing flow performed by the processing apparatus 100 in S15 to S19 of the processing sequence shown in Figure 4. This processing flow is mainly performed by the processor 111 of the processing apparatus 100 reading and executing a program stored in the memory 112.

[0073] According to Figure 5, the processor 111 receives inspection object ID information and captured image information from the terminal device 200 via the communication interface 113 (S111). The captured image information includes image data, depth data of the subject, and shooting attribute data. The processor 111 refers to each field management table to identify the field management table in which the received inspection object ID information is stored, and stores the received captured image information in the identified field management table in association with the inspection object ID information (S112).

[0074] Next, the processor 111 reads the stored captured image information and generates point cloud information based on the depth data of the subject, etc. (S113). Once the processor 111 has generated the point cloud information, it refers to the field management table and stores the point cloud information in association with the inspection target ID information.

[0075] Here, Figure 6 is a conceptual diagram showing an example of captured image information captured by a terminal device 200 according to one embodiment of the present disclosure. Specifically, Figure 6 is a diagram for explaining the mechanism by which captured image information used when generating point cloud information is acquired. According to Figure 6, the work site 10 is equipped with a reinforced concrete structure 30 including the object to be inspected for reinforcement, other structures 31-33 (for example, tools, columns, or exterior panels before construction), and marker information 20. In such a work site 10, the user, while holding the terminal device 200, first photographs the marker information 20 that is positioned to correspond to the object to be inspected. As a result, the processor 211 of the terminal device 200 acquires the object ID information assigned to identify the object to be inspected, and also identifies the coordinates (position information) of the terminal device 200 in a three-dimensional space with the marker information 20 as the origin, based on the depth data up to the marker information 20. In other words, the processor 211 of the terminal device 200 identifies the coordinates P(x1,y1,z1) of the terminal device 200 as the position information of the terminal device 200 in a three-dimensional space where the position of the marker information 20 is the origin C(x0,y0,z0).

[0076] Next, the user, holding the terminal device 200 at the work site 10, moves around the reinforced concrete structure 30, which contains the object to be inspected, with the sensor 216 facing the reinforced concrete structure 30, and takes photographs that include at least a part of the object to be inspected. In Figure 6, the captured image information is detected (captured) as detection information, but more specifically, the case in which it is captured as a video will be explained. Therefore, Figure 6 shows that a video composed of multiple captured image information, designated as frames D1-1, D1-2, D1-3, D1-4 to D1-n, has been captured.

[0077] Each frame of the captured image information preferably includes image data in which at least a portion of the reinforced concrete structure 30, including the object to be inspected, is the subject of the photograph.

[0078] Furthermore, each captured image contains depth data to each point on the surface or inside the object being inspected. This depth data may be calculated using any of the following methods: for example, by using a stereo camera and calculating based on the distance between each camera, the focal length of each camera, and the parallax of each camera; by irradiating the subject with infrared light and detecting the reflection; by irradiating the subject with light from a projector and detecting the state of the light; or a combination of these methods. The depth data calculated for each point on the surface or inside the subject included in each captured image is used to identify the positional information of each point in a three-dimensional space with the marker information 20 as the origin (for example, coordinate O(x2, y2, z2)).

[0079] Furthermore, each captured image information includes the position information of the terminal device 200 (coordinates P1-1, P1-2, P1-3, P1-4 to P1-n) at the time each captured image information, which is composed of each frame, was taken. The position information of each part of the terminal device 200 (coordinates P1-1, P1-2, P1-3, P1-4 to P1-n) is determined by calculating the amount of movement from the predetermined coordinates P(x1, y1, z1) of the terminal device 200 using a position sensor (not shown), such as a gyro sensor, included in the terminal device 200.

[0080] The processor 111 of the processing unit 100 identifies the location information (coordinates) of each point constituting the subject included in each captured image information, based on the captured image information (image data, depth data, and location information of the captured terminal device 200) which is composed of frames D1-1, D1-2, D1-3, D1-4~D1-n generated as described above. Specifically, the processor 111 of the processing unit 100 calculates the coordinates (x2, y2, z2) as the location information of point O from the location information (coordinates P1-1) of the terminal device 200 from which the image data composed of frame D1-1 was captured and the depth data of an arbitrary point (for example, point O) of the subject of the reinforced concrete structure 30 that was captured as the subject. Similarly, the processor 111 of the processing unit 100 calculates the location information for other points of the subject in frame D1-1. Based on the calculated location information (coordinates) of each point, the processor 111 of the processing unit 100 places the points constituting the point cloud in a three-dimensional space with the marker information 20 as the origin. By repeating this process, point cloud information for frame D1-1 is generated from the captured image information that constitutes frame D1-1. Similarly, the processor 111 of the processing unit 100 generates point cloud information for frames D1-2, D1-3, D1-4 to D1-n from the captured image information of frames D1-2, D1-3, and D1-4 to D1-n. In this way, the point cloud information shown in S113 of Figure 5 is generated. Note that the point cloud information for frame D1-1 is generated from the captured image information that constitutes frame D1-1 based on a method that generates single point cloud information from multi-view images, such as the Structure From Motion (SfM) method.

[0081] Returning to Figure 5, the processor 111 reads the stored point cloud information and extracts at least a portion of the object to be inspected from the point cloud information to generate target image information (S114). Then, the processor 111 refers to the field management table and stores the generated target image information in association with the object to be inspected ID information.

[0082] The process of extracting the object to be inspected from point cloud information can be performed in various ways, but as an example, the processor 111 removes noise components that are outside the inspection area in which the object to be inspected is included as a subject from the read point cloud information (S114a). Next, the processor 111 removes noise components that are inside the inspection area from the point cloud information from which the noise components outside the inspection area have been removed (S114b). Then, the processor 111 generates target image information by recognizing the reinforcement bar arrangement from the remaining point cloud information and extracting only that area (S114c).

[0083] Here, Figures 7 and 8 conceptually show an example of point cloud information generated by the processing apparatus 100 according to one embodiment of the present disclosure. Also, Figures 9A and 9B conceptually show an example of target image information generated by the processing apparatus 100 according to one embodiment of the present disclosure. Specifically, Figure 7 shows point cloud information 40 generated by the processing of S113 in Figure 5. Also, Figure 8 shows point cloud information 50 after noise components have been removed by processing S114a in S114 of Figure 5. Also, Figures 9A and 9B show target image information 60a or 60b generated by removing noise components by processing S114b in S114 of Figure 5 and further extracting the reinforcement area by processing S114c.

[0084] As shown in Figure 7, the point cloud information 40 generated from the captured image information includes not only the reinforced concrete structure 30 containing the object to be inspected, but also surrounding structures 31-33 as subjects, as shown in Figure 6. Therefore, in addition to the region 41 which includes at least a portion of the reinforcement that is the object to be inspected as a subject, the point cloud information 40 also includes regions 42 and 43, etc., which include structures other than the object to be inspected as subjects. In other words, the point clouds that include regions 42 and 43, etc., are not the object to be inspected and therefore can be considered noise components.

[0085] Therefore, as shown in S114a of Figure 5, the processor 111 of the processing unit 100 performs a process to remove noise components that are outside the inspection target area in which the object to be inspected is included as the subject from the read point cloud information. As an example, the processor 111 of the processing unit 100 uses a three-dimensional model, which is pre-design information, to identify the location where the object to be inspected will be constructed from marker information 20 placed at predetermined locations. Then, the processor 111 of the processing unit 100 sets a bounding box in a three-dimensional space with the marker information 20 as the origin, so as to surround the object to be inspected at the identified location. The processor 111 of the processing unit 100 considers the area enclosed by the bounding box set in the three-dimensional space as the inspection target area, and erases the point cloud of the area not enclosed by this bounding box. That is, in the example of Figure 7, the point cloud of the area 41 enclosed by the bounding box 44 is maintained as is, while the point cloud of the areas 42 and 43 not enclosed by the bounding box 44 is erased. In this way, noise components outside the inspection target area are removed from the point cloud information.

[0086] Next, Figure 8 shows the point cloud information 50 after noise components outside the inspection target area have been removed from the point cloud information in Figure 7. That is, according to Figure 8, the point cloud information 50 contains only the point cloud within the inspection target area enclosed by the bounding box. Here, in the point cloud information 50, as shown by zooming in on area 51, for example, there are various noise components even within the inspection target area other than the reinforcement (e.g., the point cloud shown in area 52), which is the object being inspected. For example, the reinforced concrete structure 30 includes point clouds of various objects other than the reinforcement, such as thin wires used to fix them (e.g., the point cloud shown in area 53) or concrete structures, but these are not the objects being inspected and therefore can become noise components.

[0087] Therefore, as shown in S114b of Figure 5, the processor 111 of the processing unit 100 performs a process to remove noise components within the inspection target area from the point cloud information from which noise components outside the inspection target area have been removed. As an example, the processor 111 of the processing unit 100 obtains information on the width of the cross-section of the reinforcing bars used as reinforcement from the specifications, which are design information, and sets conditions for removing noise components based on the obtained information. Such conditions include, for example, if the width of the cross-section of the reinforcement is T mm, whether or not it is included in a predetermined range (for example, T1 mm to T2 mm) that includes T mm. Accordingly, the processor 111 of the processing unit 100 determines that the areas of each recognized object that are included in the above range correspond to the reinforcement, which is the object to be inspected, while determining that the areas not included in the above range are noise components that are not the object to be inspected, and deletes the point cloud of those areas. That is, in the example of Figure 8, the point cloud of areas such as area 52 that are included in the above range is maintained as is, while the point cloud of areas such as area 53 that are not included in the above range is deleted. In this way, noise components within the inspection area are removed from the point cloud information.

[0088] Next, Figure 9A shows the target image information 60a generated by removing noise components within the inspection target area from the point cloud information in Figure 8 and then extracting only the point clouds arranged vertically. In other words, Figure 9A shows the target image information 60a which includes point cloud information indicating a total of 11 reinforcing bars, including the reinforcing bar (main bar) 61 which is the object to be inspected. Furthermore, Figure 9B shows the target image information 60b generated by removing noise components within the inspection target area from the point cloud information in Figure 8 and then extracting the point clouds arranged horizontally (i.e., point clouds other than those arranged vertically). In other words, Figure 9B shows the reinforcing bar (hoop bar) 62 which is the object to be inspected. In this way, target image information is generated in which at least a portion of the reinforcing bar, which is the object to be inspected, is extracted from the point cloud information.

[0089] Returning to Figure 5, the processor 111 reads the design information associated with the inspection target ID information from the site management table and compares the target image information with the design information to inspect the construction status of the object to be inspected (S115). As an example of this process, the processor 111 calculates the degree of agreement between the three-dimensional model of the object to be inspected stored as design information and the generated target image information. As another example of this process, the processor 111 identifies, for example, the number of reinforced concrete bars installed from the target image information and compares it with the number of reinforced concrete bars stored as design information to determine whether they match. It should be noted that such comparison methods are not limited to these methods, and any method may be used. Furthermore, each point cloud of the object to be inspected is placed in a three-dimensional space with the marker information as the origin, and it is desirable that this three-dimensional space be configured to correspond to the coordinate axis system indicated by the design information (for example, the coordinate axis system of the two-dimensional space shown in the design drawing or the three-dimensional space showing the three-dimensional model). This method makes it easy to compare the object under inspection, represented by a point cloud placed in a three-dimensional space with the marker information as the origin, with the three-dimensional model of the object under inspection, which is the design information.

[0090] Next, the processor 111 obtains result information indicating the results of the inspection of the construction status of the object to be inspected by comparing the target image information with the design information (S116). Then, it refers to the site management table and stores the result information in association with the object to be inspected ID information.

[0091] Furthermore, at this time, the processor 111 can identify and store various information about the reinforcement arrangement identified based on the target image information, in addition to the inspection results obtained by comparing the target image information with the design information. For example, the processor 111 can determine the number of reinforcement bars from the number of point clouds identified as reinforcement bars in the target image information. The processor 111 can also determine the diameter of the reinforcement bars by detecting the width of the point clouds identified as reinforcement bars in the target image information. The processor 111 can also determine the direction of the reinforcement bars by detecting the angle of the line segments formed by the point clouds identified as reinforcement bars in the target image information. The processor 111 can also determine the spacing between each reinforcement bar by detecting the width between the point clouds identified as reinforcement bars in the target image information. The processor 111 stores this various identified information in addition to the result information.

[0092] Next, the processor 111 selects one or more image information from among the captured image information, point cloud information, or target image information, and stores the selected image information in association with the inspection target ID information by referring to the field management table (S117). As an example of this process, the case in which one or more selected image information is selected from multiple captured image information (multiple frames D1-1, D1-2, D1-3, D1-4~D1-n that constitute a video) as shown in Figure 6 will be described. The processor 111 sets conditions in advance for selecting selected image information from the multiple captured image information, and selects one or more captured image information that meets those conditions as selected image information.

[0093] As an example of such processing, the processor 111 pre-sets a field of view range in which all of the reinforcement bars of the object to be inspected can be seen, based on a three-dimensional model of the object to be inspected, which is design information. The processor 111 then reads the field of view at the time each image was taken from the shooting attribute data contained in the captured (detected) image information, and determines whether the field of view falls within the range set as a condition. The processor 111 then selects the captured image information that is determined to fall within the above range as selected image information.

[0094] Another example of such processing is when the processor 111 pre-sets a condition, for example, the number of reinforcing bars to be installed based on the specifications, which are design information. The processor 111 then identifies the number of reinforcing bars specified by the target image information and selects the captured image information associated with the target image information that contains the number of bars that matches or is closest to the above condition as selected image information.

[0095] In this way, by automatically selecting image information based on design information, it becomes possible to efficiently select appropriate image information from multiple captured image data.

[0096] Furthermore, the selection process described above may be performed using a combination of methods. Additionally, if multiple image selections are possible, the selection may be further narrowed down by random selection from among them.

[0097] Next, the processor 111 generates, for example, inspection report information in document format (S118) based on the inspection result information acquired in S116 and the selected image information selected in S117.

[0098] Here, Figure 10 shows an example of inspection report information generated by a processing apparatus 100 according to one embodiment of the present disclosure. Specifically, Figure 10 shows inspection report information 70 generated in S118 of Figure 6. According to Figure 10, the inspection report information 70 includes information indicating the site ID information of the work site, which includes inspection target ID information A1 and inspection target ID information A2, as well as selected image information, design information and result information associated with inspection target ID information A1, and selected image information, design information and result information associated with inspection target ID information A2, respectively.

[0099] Specifically, the inspection report information 70 includes, along with the classification "Pass" as result information obtained by S115 in Figure 5 for the object to be inspected with the object ID information A1, various information such as the number of reinforcing bars, the type of reinforcing bars, the diameter of the reinforcing bars, the direction of the reinforcing bars, or the spacing between each reinforcing bar, as identified from the target image information. Furthermore, the inspection report information 70 also includes, in parallel, the electronically generated miniature drawing of the object to be inspected from the design information associated with the object to be inspected with the object ID information A1, and the selection information 71a associated with the object to be inspected with the object ID information A1, obtained by S117 in Figure 5. Therefore, it becomes possible for the user to easily visually confirm whether the object to be inspected, identified by the object to be inspected with the object ID information A1, has been constructed correctly in accordance with the design information.

[0100] Furthermore, the inspection report information 70 includes, along with the classification "Pass" as result information obtained by S115 in Figure 5 for the object to be inspected with the object ID information A2, various information such as the number of reinforcing bars, the type of reinforcing bars, the diameter of the reinforcing bars, the direction of the reinforcing bars, or the spacing between each reinforcing bar, identified from the target image information. In addition, the inspection report information 70 includes, in parallel, the electronic information of a small drawing of the object to be inspected, which is part of the design information associated with the object to be inspected with the object ID information A2, and the selection information 71b associated with the object to be inspected with the object ID information A2, which was obtained by S117 in Figure 5.Therefore, it becomes possible for the user to easily visually confirm whether or not the object to be inspected, identified by the object to be inspected with the object ID information A2, has been constructed correctly in accordance with the design information.

[0101] Returning to Figure 5, the processor 111 generates inspection report information as illustrated in Figure 10. It then stores the generated inspection report information in association with the inspection object ID information and transmits it to a predetermined destination via the communication interface 113. This completes the processing flow.

[0102] Thus, the processing unit 100 performs inspection of the object to be inspected based on detection information (e.g., captured image information) detected by the sensor 216 of the terminal device 200, making inspection more efficient. Furthermore, the processing unit 100 performs inspection of the object to be inspected based on pre-set design information, making inspection more accurate.

[0103] In other words, according to this embodiment, there is a processing device comprising at least one processor, wherein the at least one processor is configured to receive detection information detected by a sensor from at least a portion of an object to be inspected being constructed at a work site, inspect the construction status of the object to be inspected based on the received detection information and pre-set design information for the object to be inspected, and to perform processing to output the results of the inspection.

[0104] Furthermore, according to this embodiment, there is a processing device comprising at least one processor, wherein the at least one processor forms a bounding box in a three-dimensional space based on marker information arranged to correspond to an object to be inspected at a work site so as to surround the object to be inspected, extracts at least a portion of the object to be inspected based on the bounding box from captured image information taken by a camera and including at least a portion of the object to be inspected as a subject, and performs processing to inspect the construction status of the object to be inspected based on the extracted at least a portion of the object to be inspected.

[0105] Furthermore, according to this embodiment, there is a processing device comprising at least one processor, wherein the at least one processor is configured to receive a plurality of captured image information taken by a camera so as to include at least a part of an object to be inspected being constructed at a work site as the subject, inspect the construction status of the object to be inspected based on the received captured image information and pre-set design information for the object to be inspected, and perform processing to generate inspection report information based on the inspection results and at least one selected piece of information selected from the plurality of captured image information.

[0106] In this embodiment, it is possible to provide a processing apparatus, a processing program, a processing method, and a processing system that enable more efficient inspection of objects to be inspected.

[0107] 6. Variations As described above, Figures 1A to 10 illustrate the case where processing system 1 is primarily used for inspecting reinforcement arrangements. While main reinforcement and hoop reinforcement are shown as examples of such reinforcement arrangements in Figures 9A and 9B, processing system 1 can also be used to inspect various types of reinforcement, such as distribution reinforcement, top reinforcement, bottom reinforcement, longitudinal reinforcement, transverse reinforcement, diagonal reinforcement, stirrups, tie reinforcement, abdominal reinforcement, width-retaining reinforcement, core reinforcement, or combinations thereof, used in columns, floors, or beams constructed at construction sites. In addition to the reinforcing bars exemplified above, the same inspection system 1 can also be used for various structures constructed at the work site, such as the ground, foundation, frame, load-bearing walls, floor framing, roof surfaces, joints, connecting hardware, structural members, underground pipes, interior materials, subflooring, exterior walls, eaves, ventilation structures, drainage pipes, insulation structures, alarm systems, fire-resistant equipment, fire-fighting equipment, cleaning ports for pipes, inspection ports for pipes, intake and exhaust ports, rooms, handrails, passageways, bathrooms, toilets, or bedrooms.

[0108] As described above, Figure 6 and other diagrams illustrate the case where multiple images are acquired as frames D1-1, D1-2, D1-3, D1-4 to D1-n. However, when multiple images are acquired in this way, it is not necessary to generate point cloud information or target image information for all of them. For example, the processor 111 of the terminal device 200 may pre-set the required field of view and shooting interval based on a three-dimensional model of the object to be inspected, which is design information, and then screen the acquired shooting attribute data to select only the images that match the specified field of view and shooting interval, and transmit only the screened images to the processing unit 100. Alternatively, the processor 111 of the processing unit 100 may pre-set the required field of view and shooting interval based on a three-dimensional model of the object to be inspected, which is design information, and then screen the received shooting attribute data to select only the images that match the specified field of view and shooting interval, and generate point cloud information only for the screened images. This reduces the burden on the terminal device 200 for transmission processing and the burden on the processing unit 100 for generating point cloud information.

[0109] In other words, according to the above modified example, a processing device is provided which comprises at least one processor, wherein the at least one processor is configured to inspect the construction status of an object to be inspected based on design information of the object to be inspected and design information set in advance for the object to be inspected, from among a plurality of captured image information taken by a camera so as to include at least a part of the object to be inspected as the subject, and to perform processing for outputting the results of the inspection.

[0110] As described above, Figures 1A to 10 illustrate the use of a camera as an example of the sensor 216 used in the processing system 1. However, the system is not limited to this; various other sensors such as distance measuring sensors, ultrasonic sensors, infrared sensors, ultraviolet sensors, proximity sensors, various radar sensors, photoelectric sensors, or combinations thereof can also be used in the processing system 1.

[0111] As described above, Figures 1A to 10 illustrate the case where the processing related to the generation and inspection of point cloud information is mainly performed by the processing unit 100. However, the system is not limited to this, and some or all of these processes may be performed by the terminal device 200. Also, Figures 1A to 10 illustrate the case where the processing to identify the location information of the captured image is performed by the terminal device 200. However, the system is not limited to this, and the terminal device 200 may only store information indicating the amount of movement of the terminal device 200, and the processing unit 100 may perform the processing to identify the location information.

[0112] As described above, Figures 4, 5, and 7 illustrate the case where point cloud information is generated based on depth data. However, the processor 111 may also generate point cloud information by inputting captured image information into a pre-trained transformation model generated by machine learning or deep learning, for example, and obtaining point cloud information as output from the pre-trained transformation model. Furthermore, in Figures 4, 5, 7 to 9B, etc., inspection targets such as rebar are identified by generating point cloud information from captured image information and then generating target image information from the point cloud information. However, the processor 111 may also input captured image information into a pre-trained object recognition model generated by machine learning or deep learning, for example, and obtain placement position information of inspection targets such as rebar as output from the pre-trained object recognition model.

[0113] The processes and procedures described herein can be implemented not only by those explicitly described in the embodiments, but also by software, hardware, or a combination thereof. Specifically, the processes and procedures described herein can be implemented by implementing the logic corresponding to the process on a medium such as an integrated circuit, volatile memory, non-volatile memory, magnetic disk, or optical storage. Furthermore, the processes and procedures described herein can be implemented as computer programs and executed by various computers, including processing units and server devices.

[0114] Even if it is stated that the processes and procedures described herein are performed by a single device, software, component, or module, such processes or procedures may be performed by multiple devices, multiple software programs, multiple components, and / or multiple modules. Similarly, even if it is stated that the various types of information described herein are stored in a single memory or storage unit, such information may be distributed and stored in multiple memories within a single device or in multiple memories distributed across multiple devices. Furthermore, the software and hardware elements described herein may be implemented by integrating them into fewer components or by decomposing them into more components. [Explanation of Symbols]

[0115] 1. Processing System 10 Work site 20 Marker Information 30. Reinforced concrete structures including the object to be inspected 31 Structures 32 Structures 33 Structures 100 Processing Units 200 terminal devices

Claims

1. A processing unit comprising at least one processor, The at least one processor, The system receives detection information from sensors located on at least a portion of the object being inspected during construction at the work site. Based on the received detection information and the design information pre-set for the object to be inspected, the construction status of the object to be inspected is inspected. Output the results of the above inspection. A processing unit configured to perform a process for that purpose.

2. The apparatus according to claim 1, wherein the object to be inspected is reinforcing steel used in reinforced concrete structures.

3. The apparatus according to claim 1, wherein the object to be inspected is reinforcing bars used in a reinforced concrete structure.

4. The aforementioned sensor is a camera, The detection information is captured image information that includes at least a portion of the object to be inspected, as the subject, captured by the camera. The apparatus according to claim 1.

5. The processing apparatus according to claim 1, wherein the construction status of the object to be inspected is inspected by generating point cloud information based on the detection information.

6. The sensor detects marker position information from marker information arranged to correspond to the object to be inspected, in addition to at least a portion of the object to be inspected. The point cloud information is generated based on the marker position information. The apparatus according to claim 5.

7. The processing apparatus according to claim 5, wherein the construction status of the object to be inspected is inspected by generating target image information from which at least a part of the object to be inspected is extracted from the point cloud information, and comparing the generated target image information with the design information.

8. The processing apparatus according to claim 7, wherein the target image information is generated by removing noise components from the point cloud information.

9. The detection information is a plurality of captured image pieces that include at least a part of the object to be inspected as the subject, The results of the above inspection include at least one selected image information chosen from the plurality of captured image information, The apparatus according to claim 1.

10. The apparatus according to claim 9, wherein the selected image information is selected based on conditions specified in advance based on the design information.

11. By being executed by at least one processor, The system receives detection information from sensors located on at least a portion of the object being inspected during construction at the work site. Based on the received detection information and the design information pre-set for the object to be inspected, the construction status of the object to be inspected is inspected. Output the results of the above inspection. A processing program that causes the aforementioned at least one processor to function in this manner.

12. A processing method that is performed by at least one processor, The steps include receiving detection information detected by sensors from at least a portion of the object to be inspected during construction at the work site, A step of inspecting the construction status of the object to be inspected based on the received detection information and the design information set in advance for the object to be inspected, The step of outputting the results of the aforementioned inspection, A processing method that includes this.

13. A sensor configured to detect detection information from at least a portion of the object to be inspected during construction at a work site, A processing device according to claim 1, which is connected to the sensor by a communication network and configured to receive the detection information from the sensor, A processing system that includes this.

Images