Program, execution determination device, execution determination system, and execution determination method

Abstract

To provide a program, a construction determination device, a construction determination system, and a construction determination method capable of determining that construction satisfying a standard is performed without taking labor and time as compared with a conventional manner.SOLUTION: A program according to an embodiment causes a processor included in an execution determination device to function as an input control unit and a notification control unit. The input control unit performs control to input an image showing a portion where construction has been performed to a learned model capable of evaluating that the construction satisfies a standard. The notification control unit performs control to notify of information indicating whether or not the construction of the portion satisfies a standard based on an output of the learned model.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a program, a construction assessment device, a construction assessment system, and a construction assessment method. [Background technology]

[0002] To ensure that construction meets the standards, photographs of the construction site are uploaded to a server, and personnel randomly select from the photographs to inspect that construction is being carried out as required.

[0003] However, this inspection requires an experienced person to perform the inspection, and is time-consuming, making it difficult to inspect all products.

[0004] Inspections can also take time. Therefore, even if an inspection reveals a problem with the construction, depending on the location and progress of the work, it may be difficult to correct or redo the work or take new photographs. This is because the construction area may be hidden by other work, for example. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2018-045355 Summary of the Invention [Problem to be solved by the invention]

[0006] The problem that the embodiments of the present invention aim to solve is to provide a program, a construction assessment device, a construction assessment system, and a construction assessment method that can determine whether construction that meets standards is being carried out with less effort and time than conventional methods. [Means for solving the problem]

[0007] The program of the embodiment causes a processor included in the construction assessment device to function as an input control unit and a notification control unit. The input control unit controls inputting an image showing a construction site into a trained model that can evaluate whether the construction meets standards. The notification control unit controls to report information indicating whether the construction at the site meets standards based on the output of the trained model. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram showing an example of a construction management system according to an embodiment and a main configuration of components included in the construction management system; [Figure 2] 2 is a flowchart showing an example of processing by a processor of the server device in FIG. 1; [Figure 3] 2 is a flowchart showing an example of processing by a processor of the terminal device in FIG. 1; [Figure 4] 2 is a flowchart showing an example of processing by a processor of the terminal device in FIG. 1; [Figure 5] FIG. 2 is a diagram showing an example of a selection screen displayed on the display device in FIG. [Figure 6] FIG. 2 is a diagram showing an example of a list screen displayed on the display device in FIG. 1. [Figure 7] FIG. 2 is a diagram showing an example of a shooting screen displayed on the display device in FIG. [Figure 8] FIG. 2 is a diagram showing an example of a result screen displayed on the display device in FIG. 1. [Figure 9] FIG. 2 is a diagram showing an example of a selection screen displayed on the display device in FIG. DETAILED DESCRIPTION OF THE INVENTION

[0009] The construction management system according to the embodiment will be described below with reference to the drawings. Note that the scale of each part in each drawing used in the following description of the embodiment may be changed as appropriate. Also, for the purpose of explanation, each drawing used in the following description of the embodiment may omit the configuration. Also, in each drawing and in this specification, the same reference numerals indicate similar elements.

[0010] FIG. 1 is a block diagram showing an example of the configuration of a construction management system 1 according to an embodiment and the main components of the components included in the construction management system 1. Note that each component of the device may be built-in or external. The construction management system 1 is a system that provides a construction management service. The construction management service in this embodiment uses images of the construction site to determine whether construction is being carried out that meets standards using AI (artificial intelligence). This determination will be referred to hereinafter as "construction determination." As an example, the construction management system 1 includes a server device 100 and a terminal device 200. Note that the construction management system 1 may include some of these. Note that the construction management system 1 is an example of a construction determination system.

[0011] The server device 100 and the terminal device 200 are connected to a network NW. The network NW is typically a communication network including the Internet. The network NW is typically a communication network including a WAN (wide area network). The network NW may be a communication network including a private network such as an intranet. The network NW may be a communication network including a LAN (local area network). The network NW may be a wireless line or a wired line, or may be a mixture of wireless and wired lines. The network NW may also be a communication network including a dedicated line or a public mobile phone network.

[0012] The server device 100 provides a construction management service. The server device 100 includes, for example, a processor 110, a ROM (read-only memory) 120, a RAM (random-access memory) 130, an auxiliary storage device 140, and a communication interface 150. A bus 160 and the like connect these components. The server device 100 is an example of a server device and an example of a construction assessment device.

[0013] The processor 110 is the central part of a computer that performs various calculations and processes, such as calculations and controls, necessary for the operation of the server device 100. The processor 110 may be, for example, a central processing unit (CPU), a microprocessing unit (MPU), a system on a chip (SoC), a digital signal processor (DSP), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a field-programmable gate array (FPGA). Alternatively, the processor 110 may be a combination of these. The processor 110 may also be a combination of these with a hardware accelerator or the like. The processor 110 controls each component to realize various functions of the server device 100 based on programs such as firmware, system software, and application software stored in the ROM 120 or the auxiliary storage device 140. The processor 110 also executes the processes described below based on the programs. Note that some or all of the programs may be incorporated into the circuitry of the processor 110.

[0014] The ROM 120 and RAM 130 are the main storage devices of the computer centered around the processor 110. The ROM 120 is a non-volatile memory used exclusively for reading data. The ROM 120 stores, for example, firmware among the above programs. The ROM 120 also stores data used by the processor 110 when it performs various processes.

[0015] The RAM 130 is a memory used for reading and writing data. The RAM 130 is used as a work area for storing data that is temporarily used when the processor 110 performs various processes. The RAM 130 is typically a volatile memory.

[0016] The auxiliary storage device 140 is an auxiliary storage device of a computer centered around the processor 110. The auxiliary storage device 140 is, for example, an EEPROM (electric erasable programmable read-only memory), an HDD (hard disk drive), or flash memory. The auxiliary storage device 140 stores, among the above programs, for example, system software and application software. The auxiliary storage device 140 also stores data used by the processor 110 when performing various processes, data generated by the processes in the processor 110, and various setting values. Examples of data stored in the auxiliary storage device 140 include a user DB (database) 141, an image DB 142, and a construction judgment AI 143.

[0017] User DB141 is a database that stores and manages information about each user (hereinafter simply referred to as "user") who uses the construction management service. Users include, for example, construction workers and construction managers. User DB141 stores construction schedule information for each user, associated with a user ID (identifier). The construction schedule information includes information about construction that is scheduled to be carried out by the user identified by the associated user ID. The construction schedule information may also include information about construction that has already been carried out. The user ID is identification information that is unique to each user.

[0018] Image DB 142 is a database that stores and manages images showing construction sites. Image DB 142 stores an image ID and one or more user IDs for each image, in association with each other. The user ID associated with an image ID indicates the user who can view the image identified by the image ID. Note that the image ID is unique identification information for each image. By accessing server device 100 via terminal device 200, a user can view and download images associated with the user ID of the user.

[0019] The construction judgment AI 143 is a trained AI model using machine learning. The construction judgment AI 143 receives input of an image showing a construction location and information specifying the construction location. The construction judgment AI 143 outputs output information in response to the input. The output information includes information indicating whether the construction location shown in the input image meets the construction standards for the specified construction location. The information may include a score indicating the degree to which the construction meets the standards. Furthermore, if the construction does not meet the standards, the output information may include the reason. Furthermore, if the construction does not meet the standards, the output information may include a method for completing the construction in accordance with the standards. Note that the construction judgment AI 143 may be an AI model dedicated to construction judgment or a general-purpose AI model. The construction judgment AI 143 is an example of a trained model that can evaluate whether construction meets the standards.

[0020] The construction determination AI 143 may receive input of information indicating the construction standard for the construction location, instead of or in addition to the information specifying the construction location.

[0021] The processor 110 may train the construction judgment AI 143 using, for example, a plurality of images that meet the construction standards. The processor 110 may train the construction judgment AI 143 using a plurality of images that do not meet the construction standards, in addition to a plurality of images that meet the construction standards. Instead of the processor 110, a device other than the server device 100 may train the construction judgment AI 143.

[0022] The communication interface 150 is an interface for the server device 100 to communicate via a network NW or the like.

[0023] The bus 160 includes a control bus, an address bus, a data bus, etc., and transmits signals exchanged among the various components of the server device 100 .

[0024] The terminal device 200 is a device used by a user. The terminal device 200 is, for example, a general-purpose device such as a smartphone, a tablet terminal, or a PC (personal computer). The terminal device 200 may also be a device dedicated to construction management services. The terminal device 200 includes, for example, a processor 210, a ROM 220, a RAM 230, an auxiliary storage device 240, a display device 250, an input device 260, a communication interface 270, and an imaging device 280. A bus 290 and the like connect these components. The terminal device 200 is an example of a terminal device and an example of a construction assessment device.

[0025] The processor 210 is the central part of a computer that performs various calculations and processes, such as calculations and controls, necessary for the operation of the terminal device 200. The processor 210 is, for example, a CPU, MPU, SoC, DSP, GPU, ASIC, PLD, or FPGA. Alternatively, the processor 210 may be a combination of several of these. The processor 210 may also be a combination of these with a hardware accelerator or the like. The processor 210 controls each component to realize various functions of the terminal device 200 based on programs such as firmware, system software, and application software stored in the ROM 220 or the auxiliary storage device 240. The processor 210 also executes the processes described below based on the programs. Note that some or all of the programs may be incorporated into the circuitry of the processor 210.

[0026] The ROM 220 and RAM 230 are main storage devices of a computer centered around the processor 210. The ROM 220 is a non-volatile memory used exclusively for reading data. The ROM 220 stores, for example, firmware among the above programs. The ROM 220 also stores data used by the processor 210 when performing various processes.

[0027] The RAM 230 is a memory used for reading and writing data. The RAM 230 is used as a work area for storing data that is temporarily used when the processor 210 performs various processes. The RAM 230 is typically a volatile memory.

[0028] The auxiliary storage device 240 is an auxiliary storage device of a computer with the processor 210 as its central device. The auxiliary storage device 240 is, for example, an EEPROM, a HDD, or a flash memory. The auxiliary storage device 240 stores, for example, system software and application software among the above programs. The auxiliary storage device 240 also stores data used by the processor 210 when performing various processes, data generated by the processes in the processor 210, various setting values, and the like.

[0029] The application software stored in the auxiliary storage device 240 includes, for example, application software for using a construction management service (hereinafter referred to as a "construction management app"). The construction management app may be software dedicated to the construction management service, or may be general-purpose software such as a web browser.

[0030] The display device 250 displays a screen for notifying various pieces of information to an operator (hereinafter simply referred to as "operator") of the terminal device 200. The display device 250 is, for example, a display such as a liquid crystal display or an organic EL (electro-luminescence) display.

[0031] The input device 260 accepts operations by an operator. The input device 260 is, for example, a keyboard, a keypad, a touchpad, a mouse, or a controller. The input device 260 may also be a device for voice input. A touch panel may also be used as the display device 250 and the input device 260. In this case, the display panel included in the touch panel functions as the display device 250. And, a pointing device included in the touch panel that is used for touch input functions as the input device 260.

[0032] The communication interface 270 is an interface for the terminal device 200 to communicate via a network NW or the like.

[0033] The imaging device 280 is also called a camera. The imaging device 280 includes an image sensor. The imaging device 280 captures images such as still images and moving images. That is, the imaging device 280 converts light incident on the image sensor into an image. The imaging device 280 also outputs captured image data.

[0034] The bus 290 includes a control bus, an address bus, a data bus, etc., and transmits signals exchanged between each section of the terminal device 200 .

[0035] The operation of the construction management system 1 according to the embodiment will be described below with reference to FIGS. 2 to 4. Note that the processing details in the following operation description are merely examples, and various processing steps that can achieve similar results can be used as appropriate. FIG. 2 is a flowchart showing an example of processing by the processor 110 of the server device 100. The processor 110 executes the processing of FIG. 2 based on a program stored in, for example, the ROM 120 or the auxiliary storage device 140. FIGS. 3 and 4 are flowcharts showing an example of processing by the processor 210 of the terminal device 200. The processor 210 executes the processing of FIGS. 3 and 4 based on a program stored in, for example, the ROM 220 or the auxiliary storage device 240. The processor 210 starts the processing of FIGS. 3 and 4, for example, when a construction management app is launched. Alternatively, the processor 210 starts the processing of FIGS. 3 and 4 when a web page that provides construction management services is accessed using the construction management app.

[0036] The processor 210 starts the processing shown in FIGS. 3 and 4, for example, when the construction management app is launched. Alternatively, the processor 210 starts the processing shown in FIGS. 3 and 4 when a web page that provides the construction management service is accessed using the construction management app. Note that, in order to use some or all of the functions of the construction management service, it is necessary to log in to the construction management service. The terminal device 200 logs in to the construction management service using, for example, a user ID. This login may be performed automatically or may require an operator's operation. When the terminal device 200 is currently logged in to the construction management service, it stores at least one of the user ID used for the login and the session ID assigned during the login. Hereinafter, the user ID or the session ID will be referred to as a "login ID." Also, a user identified by the login ID will be referred to as a "login user." For simplicity, the following description will be given assuming that the login ID is a user ID. Note that the user ID is identification information unique to each user of the construction management service.

[0037] In step ST31, the processor 210 generates a screen request. The screen request includes, for example, a login ID. The screen request is information requesting the transmission of information necessary to display various screens. The various screens include, for example, a selection screen, a list screen, a shooting screen, and a confirmation screen, which will be described later. After generating the screen request, the processor 210 instructs the communication interface 270 to transmit the screen request to the server device 100. Upon receiving this transmission instruction, the communication interface 270 transmits the screen request to the server device 100. The transmitted screen request is received by the communication interface 150 of the server device 100.

[0038] 2, the processor 110 of the server device 100 determines whether or not a screen request has been received by the communication interface 150. If a screen request has not been received, the processor 110 determines No in step ST11 and proceeds to step ST12.

[0039] In step ST12, the processor 110 determines whether or not a determination request has been received by the communication interface 150. If a determination request has not been received, the processor 110 determines No in step ST12 and proceeds to step ST13.

[0040] In step ST13, processor 110 determines whether or not a storage request has been received by communication interface 150. If a storage request has not been received, processor 110 determines No in step ST13 and returns to step ST11. Thus, processor 110 enters a standby state in which steps ST11 to ST13 are repeated until a display request, a determination request, or a storage request is received.

[0041] If a screen request is received while processor 110 is in a standby state where steps ST11 to ST13 are repeated, processor 110 determines Yes in step ST11 and proceeds to step ST14.

[0042] In step ST14, processor 110 refers to user DB 141 to acquire construction schedule information associated with the user ID included in the screen request received in step ST11. Processor 110 also acquires other information necessary for displaying various screens.

[0043] In step ST15, the processor 110 generates screen information using the construction plan information acquired in step ST14. The screen information includes information necessary for displaying various screens. The screen information includes some or all of the construction plan information and other information acquired in step ST14. After generating the screen information, the processor 110 instructs the communication interface 150 to transmit the screen information to the terminal device 200. Upon receiving this transmission instruction, the communication interface 150 transmits the screen information to the terminal device 200. The transmitted screen information is received by the communication interface 270 of the terminal device 200.

[0044] 3, the processor 210 of the terminal device 200 waits for the screen information to be received by the communication interface 270. If the screen information is received, the processor 210 determines Yes in step ST32 and proceeds to step ST33.

[0045] In step ST33, processor 210 generates an image corresponding to selection screen SC1 as shown in Fig. 5. Then, processor 210 instructs display device 250 to display this generated image. Upon receiving the display instruction, display device 250 displays the selection screen.

[0046] FIG. 5 is a diagram showing an example of the selection screen SC1 displayed on the display device 250. The selection screen SC1 is a screen that displays a list of construction types (work types) that the logged-in user plans to perform. The selection screen SC1 is also a screen that allows the operator to select a desired work type from the list. Here, the desired work type refers to the work type for which a construction judgment is desired. The criteria for classifying work types are not limited. As an example, the terminal device 200 displays constructions with the same process type as the same work type. As an example, the terminal device 200 displays constructions with the same planned completion date as the same work type. The selection screen SC1 includes, as an example, areas AR11 to AR13.

[0047] The selection screen SC1 includes one or more areas AR11. Each area AR11 corresponds one-to-one with a respective work type. Area AR11 includes an image showing the corresponding work type. Area AR11 includes the name of the corresponding work type. Area AR11 displays the number of images that need to be taken for the corresponding work type to assess the construction. Note that area AR11 may also display the number of construction locations that need to be photographed for the corresponding work type to assess the construction. Note that the selection screen SC1 is an example of a second list that shows a list of construction types.

[0048] In addition, the area AR11 is a button that the user operates to select a desired type of work. When the area AR11 is operated, it indicates that the type of work corresponding to the area AR11 has been selected.

[0049] When a user inputs a character string into the area AR12, the processor 110 displays the area AR11 corresponding to the work type whose name includes the character string on the selection screen SC1.

[0050] Area AR13 is an area that displays, for example, the name of the construction site and the name of the company to which the logged-in user belongs.

[0051] In step ST34, processor 210 waits for an operation to select a type of work. That is, processor 210 waits for a predetermined operation, such as operating area AR11 corresponding to the type of work. If an operation to select a type of work is performed, processor 210 determines Yes in step ST34 and proceeds to step ST35. The type of work selected here will be referred to as the "selected type of work" hereinafter.

[0052] In step ST35, the processor 210 generates an image corresponding to the list screen SC2 as shown in Fig. 6. Then, the processor 210 instructs the display device 250 to display the generated image. In response to the display instruction, the display device 250 displays the list screen SC2.

[0053] 6 is a diagram showing an example of a list screen SC2 displayed on the display device 250. The list screen SC2 is a screen showing a list of construction locations that need to be photographed for the selected work type. The list screen SC2 is also a screen for the operator to select the construction location that he or she wishes to photograph from the list. The list screen SC2 includes, as an example, an area AR21, an area AR22, a button B21, and a complete button B22.

[0054] The list screen SC2 includes the same number of areas AR21 as the number of construction locations that need to be photographed for the selected work type. Each area AR21 corresponds one-to-one to each construction location that needs to be photographed. As an example, the area AR21 includes an area AR211 and an area AR212.

[0055] The area AR211 is an area for displaying a character string indicating the corresponding construction location. The character string includes, for example, the name of the construction location. The character string includes, for example, an explanation of the construction location.

[0056] The area AR212 is an image showing the corresponding construction site. The image is, for example, an image showing a photograph of the construction site.

[0057] Area AR22 displays the number of images that need to be taken for the selected work type. Area AR22 also displays the number of construction locations that have passed for the selected work type. Area AR22 also displays the remaining number of images that need to be taken for the selected work type. The remaining number of images that need to be taken is the number of images that need to be taken minus the number of construction locations that have passed or the number of passed images.

[0058] The button B21 is a button that the operator operates when instructing the terminal device 200 to display a manual for the selected work type. When the button B21 is operated, the processor 110 causes the manual to be displayed on the display device 250. The contents of the manual include the construction method and construction standards for the selected work type.

[0059] The Complete button B22 is a button that the operator operates when the image capturing of each image of the selected work type is completed. The Complete button B22 is also a button that the operator operates when instructing the terminal device 200 to upload the captured images to the server device 100. The Complete button B22 is also a button that the operator operates when instructing the terminal device 200 to return to the list screen SC1.

[0060] The list screen SC2 is an example of a first list showing a list of locations that require photographing. Therefore, by performing the process of step ST35, processor 210 functions as an example of a notification control unit that controls to display the first list showing a list of locations that require photographing.

[0061] As described above, processor 210 performs processing of steps ST33 to ST35 to display a second list, and when a type is selected in the second list, functions as an example of an alarm control unit that controls to display a first list showing a list of the images that need to be taken for construction work included in the selected type.

[0062] In step ST36, processor 210 determines whether an operation to select a construction location to be photographed has been performed. That is, processor 210 determines whether a predetermined operation, such as operating area AR21, has been performed. If an operation to select a construction location to be photographed has not been performed, processor 210 determines No in step ST36 and proceeds to step ST37.

[0063] In step ST37, processor 210 determines whether or not the complete button B22 has been operated. If the complete button B22 has not been operated, processor 210 determines No in step ST37 and returns to step ST36. Thus, processor 210 enters a standby state in which steps ST36 and ST37 are repeated until an operation to select the construction location to be photographed is performed or the complete button B22 is operated.

[0064] If an operation to select a construction location to be photographed is performed while processor 210 is in a standby state in which steps ST36 and ST37 are repeated, processor 210 determines Yes in step ST36 and proceeds to step ST38 in Fig. 4. The construction location selected here will be referred to as the "selected location" hereinafter.

[0065] In step ST38, processor 210 generates an image corresponding to shooting screen SC3 as shown in Fig. 7. Then, processor 210 instructs display device 250 to display this generated image. Upon receiving the display instruction, display device 250 displays shooting screen SC3.

[0066] 7 is a diagram showing an example of a shooting screen SC3 displayed on the display device 250. The shooting screen SC3 is a screen for shooting a selected location. The shooting screen SC3 includes, as an example, an area AR31, an area AR32, a shooting button B31, and a send button B32.

[0067] Area AR31 displays a real-time image captured by imaging device 280. That is, area AR31 functions as a finder for imaging device 280. Area AR31 also includes a guide G31.

[0068] The guide G31 is an image that serves as a guide to assist the operator in photographing the selected location. The processor 210 displays the guide G31 superimposed on the real-time image captured by the imaging device 280. That is, the processor 210 displays the guide G31 superimposed on the viewfinder. The operator photographs the construction location in the image displayed in the area AR31 so that the guide matches the construction location. This makes it easier for the operator to photograph the selected location in a predetermined direction, within a predetermined shooting range, and in a predetermined composition. For example, the guide G31 includes a contour line of the outline of the selected location when photographed in a predetermined direction, within a predetermined shooting range, and in a predetermined composition. For example, the guide G31 includes a center line of the selected location when photographed in a predetermined direction, within a predetermined shooting range, and in a predetermined composition. For example, the guide G31 includes a center point of the selected location when photographed in a predetermined direction, within a predetermined shooting range, and in a predetermined composition. The guide G31 may also include a frame. The frame indicates that the selected location or a specific portion of the selected location is to be photographed within the frame. The shape of the frame may be a rectangle or other quadrangle, an ellipse, a circle, or any other shape. The guide G31 may also be color-coded using multiple colors. The guide G31 is not limited to lines, and may include filled shapes. The guide G31 may be partially or entirely translucent.

[0069] The processor 210 may be able to hide part or all of the guide G31 based on an operation by the operator. The processor 210 may be able to change the transparency of the guide G31 based on an operation by the operator. The processor 210 may be able to change at least one of the thickness and color of the lines in the guide G31 based on an operation by the operator.

[0070] Area AR32 is an area that displays an explanation of the selected location. For example, when there are similar construction locations, the explanation of the selected location identifies which construction location is the selected location. Area AR32 may also display an explanation of how to photograph the selected location.

[0071] The photographing button B31 is a button that the operator operates when instructing the terminal device 200 to photograph an image.

[0072] The send button B32 is a button that the operator operates when instructing the terminal device 200 to send a captured image to the server device 100. The send button B32 is also a button that the operator operates when instructing the terminal device 200 to perform construction assessment using the image.

[0073] The shooting screen SC3 may also include controls (widgets) for changing various settings related to image capture, such as switching between shooting modes, shutter speeds, focal lengths, sensitivity, F-stops, and switching between focus and flash modes.

[0074] As described above, by performing the processing of step ST38, processor 210 functions as an example of a notification control unit that controls the display of an image that assists in photographing the area where construction has been performed, superimposed on the viewfinder.

[0075] In step ST39, processor 210 determines whether an operation to instruct capturing an image has been performed. That is, processor 210 determines whether a predetermined operation, such as operating the capture button B31, has been performed. If an operation to instruct capturing an image has not been performed, processor 210 determines No in step ST39 and proceeds to step ST40.

[0076] In step ST40, processor 210 determines whether an operation to instruct transmission of an image has been performed. That is, processor 210 determines whether a predetermined operation, such as operating send button B32, has been performed. If an operation to instruct transmission of an image has not been performed, processor 210 determines No in step ST40 and returns to step ST39. Thus, processor 210 enters a standby state in which steps ST39 and ST40 are repeated until an operation to instruct capture of an image or an operation to instruct transmission of an image is performed.

[0077] If an operation to instruct image capture is performed while in a standby state in which steps ST39 and ST40 are repeated, processor 210 determines Yes in step ST39 and proceeds to step ST41.

[0078] In step ST41, the processor 210 controls the imaging device 280 to instruct it to capture an image. Based on this instruction, the imaging device 280 outputs light incident on the image sensor as an image. The processor 210 then stores the image in the RAM 230 or the auxiliary storage device 240 either directly or after performing predetermined image processing. After processing step ST41, the processor 210 returns to step ST39.

[0079] If an operation to instruct image transmission is performed while in a standby state in which steps ST39 and ST40 are repeated, processor 210 determines Yes in step ST40 and proceeds to step ST42. However, processor 210 will not determine Yes in step ST40 unless the shooting button B31 has been operated.

[0080] In step ST42, the processor 210 generates a judgment request. The judgment request includes the login ID, the image captured in step ST41, and the construction location ID of the selected location. The judgment request is information requesting that a construction judgment be made for the selected location using the image. The construction location ID is identification information unique to each construction location. After generating the judgment request, the processor 210 instructs the communication interface 270 to send the judgment request to the server device 100. Upon receiving this instruction to send, the communication interface 270 sends the judgment request to the server device 100. The sent judgment request is received by the communication interface 150 of the server device 100.

[0081] Upon receiving the judgment request, the server device 100 inputs the image included in the judgment request to the construction judgment AI 143 based on the request indicated by the judgment request. Therefore, by performing the processing in step ST42, the processor 210 functions as an example of an input control unit that controls the input of an image showing the construction site to a trained model that can evaluate whether the construction meets the standards.

[0082] Furthermore, the processor 210 cooperates with the communication interface 270 to perform the process of step ST42, thereby functioning as an example of a transmission unit that transmits an image showing the area where construction has been performed to the server device.

[0083] On the other hand, if processor 110 of server device 100 receives a determination request while in a standby state where steps ST11 to ST13 in FIG. 2 are repeated, it determines Yes in step ST12 and proceeds to step ST16.

[0084] In step ST16, the processor 110 inputs the image included in the determination request received in step ST12 and information specifying the construction location to the construction determination AI 143. The construction location is the construction location identified by the construction location ID included in the determination request.

[0085] The construction judgment AI 143 outputs output information based on the input image and information. The processor 110 acquires the output information.

[0086] When the output information indicates that the construction standard is met, processor 110 determines that the construction standard is met. When the output information includes a score indicating the degree to which the construction meets the standard, processor 110 determines that the construction standard is met when the score is equal to or greater than a predetermined value.

[0087] As described above, the processor 110 functions as an example of an input control unit that inputs images into the construction judgment AI 143, thereby controlling the input of images showing the areas where construction has been performed into a trained model that can evaluate whether the construction meets the standards.

[0088] In addition, the processor 110 functions as an example of an input unit that inputs an image into the construction judgment AI 143, thereby inputting the image into a trained model that can evaluate whether the construction meets the standards.

[0089] In addition, by performing the processing of step ST16, processor 110 functions as an example of a judgment unit that judges whether the construction work at the location where the construction work was performed meets the standards based on the output of the trained model.

[0090] In step ST17, the processor 110 generates judgment information. The judgment information is information that notifies the result of the construction judgment. The judgment information is also information that instructs the result to be displayed. The judgment information includes the judgment result of step ST16. The judgment information may include some or all of the output information acquired in step ST16. After generating the judgment information, the processor 110 instructs the communication interface 150 to transmit the judgment information to the terminal device 200. Upon receiving this transmission instruction, the communication interface 150 transmits the judgment information to the terminal device 200. The transmitted judgment information is received by the communication interface 270 of the terminal device 200. After processing step ST17, the processor 110 returns to step ST11.

[0091] The construction assessment result is an example of information indicating whether the construction at the location where construction was performed meets the criteria, based on the output of the trained model. By performing the processing of step ST17, the processor 110 causes the terminal device 200 to display the information. Therefore, by performing the processing of step ST17, the processor 110 functions as an example of a notification control unit that controls to notify the information. Furthermore, by performing the processing of step ST17 in cooperation with the display device 250, the processor 110 functions as an example of a notification unit that notifies the result of the assessment.

[0092] 4, the processor 210 of the terminal device 200 waits for the determination information to be received by the communication interface 270. If the determination information is received, the processor 210 determines Yes in step ST43 and proceeds to step ST44.

[0093] In step ST44, the processor 210 generates an image corresponding to the result screen SC4 as shown in Fig. 8. Then, the processor 210 instructs the display device 250 to display this generated image. Upon receiving the display instruction, the display device 250 displays the result screen.

[0094] FIG. 8 is a diagram showing an example of a result screen SC4 displayed on the display device 250. The result screen SC4 is a screen that displays the result of the construction assessment. The result is based on the assessment information received in step ST43. The result includes, for example, information indicating whether the construction at the selected location passed or failed. Here, "pass" indicates that the construction at the selected location met the standards. "fail" indicates that the construction at the selected location did not meet the standards. Furthermore, if the result is "fail," the result screen SC4 may include the reason why the construction did not meet the standards and a method for completing the construction in accordance with the standards. The result screen SC4 may include not only the result of the most recent construction assessment, but also the results of previous construction assessments. The result screen SC4 may also include the date and time when the image was sent to the server device 100 and the date and time when the construction assessment was made.

[0095] As described above, by performing the processing of step ST44, processor 210 functions as an example of a notification control unit that controls the notification of information indicating whether the construction work at the location where the work was performed meets the standards, based on the output of the trained model.

[0096] In step ST45, processor 210 determines whether or not to end the display of result screen SC4. For example, processor 210 determines to end the display of result screen SC4 when an operation is performed to instruct to end the display of result screen SC4. If processor 210 does not determine to end the display of result screen SC4, it determines No in step ST45 and repeats the processing of step ST45. On the other hand, if processor 210 determines to end the display of result screen SC4, it determines Yes in step ST45 and returns to step ST35 of FIG. 3.

[0097] After performing the processes of steps ST42 to ST44 one or more times, processor 210 causes list screen SC2 as shown in Fig. 9 to be displayed on display device 250. Therefore, when returning from step ST45 to step ST35, processor 210 also causes list screen SC2 as shown in Fig. 9 to be displayed on display device 250.

[0098] Fig. 9 is a diagram showing an example of a list screen SC2 displayed on the display device 250. The list screen SC2 in Fig. 9 displays the results of construction assessment for each area AR21 for which construction assessment has been carried out. As an example, the list screen SC2 displays the results of construction assessment by superimposing an image IM21 or an image IM22 showing the results of construction assessment on each area AR21.

[0099] Image IM21 is a symbol or the like indicating that the result of the construction assessment is pass, and image IM22 is a symbol or the like indicating that the result of the construction assessment is fail.

[0100] If the complete button B22 is operated while processor 210 is in a standby state in which it repeats steps ST36 and ST37, processor 210 determines Yes in step ST37 and proceeds to step ST49. However, processor 210 does not determine Yes in step ST37 unless one or more construction locations for the selected construction type have passed the construction judgment.

[0101] In step ST49, the processor 210 generates a storage request. The storage request includes a login ID and one or more images. The images are images that satisfy the following conditions: the image is stored in step ST41, the image is a photograph of the construction location of the selected work type, and the image passes the construction assessment. After generating the storage request, the processor 210 instructs the communication interface 270 to send the storage request to the server device 100. Upon receiving this instruction to send, the communication interface 270 sends the storage request to the server device 100. The sent storage request is received by the communication interface 150 of the server device 100.

[0102] As described above, by processing step ST49, processor 210 functions as an example of a transmission control unit that transmits an image in which the construction work in the area shown has been determined to meet the standards, and controls the transmission destination to request that the image be saved.

[0103] On the other hand, if a storage request is received while processor 110 of server device 100 is in a standby state where steps ST11 to ST13 in FIG. 2 are repeated, it determines "Yes" in step ST13 and proceeds to step ST18.

[0104] In step ST18, processor 110 associates the image included in the storage request received in step ST13 with the login ID and a predetermined user ID and stores the image in image DB 142. After processing step ST18, processor 110 returns to step ST11.

[0105] According to the construction management system 1 of the embodiment, the server device 100 and the terminal device 200 input images of the completed area into a trained AI model to determine whether the construction meets the standards. Furthermore, the server device 100 and the terminal device 200 of the embodiment control the notification of the determination result. As described above, the server device 100 and the terminal device 200 of the embodiment use an AI model, so an expert is not required to determine whether the construction meets the standards. This eliminates the need for an expert and reduces labor costs. Furthermore, because the server device 100 and the terminal device 200 of the embodiment use an AI model, the determination can be made immediately, taking less time than conventional methods. As a result, the operator can know the result of the determination immediately after the construction. Therefore, if the determination result is unsuccessful, the operator can immediately correct or redo the construction. By immediately correcting or redoing the construction, the correction or redoing of the construction and re-photographing the construction area can be performed more easily than conventional methods.

[0106] Furthermore, according to the construction management system 1 of the embodiment, the terminal device 200 displays a list screen SC2 that is a list of construction locations that need to be photographed. By looking at the list, the operator can easily understand which construction locations need to be photographed. This also helps the operator to prevent forgetting to photograph.

[0107] Furthermore, according to the construction management system 1 of the embodiment, the terminal device 200 displays the results of construction assessment for each construction location displayed on the list screen SC2. This makes it easy for the operator to understand which construction locations need to be photographed again. It also makes it easy for the operator to understand which construction locations have not yet been photographed. It also makes it easy for the operator to understand which construction locations have passed the construction assessment.

[0108] Furthermore, according to the construction management system 1 of the embodiment, the terminal device 200 displays a guide G21 superimposed on the viewfinder. This makes it easier for the operator to photograph the selected location in a predetermined direction, within a predetermined shooting range, and in a predetermined composition. Furthermore, by using an image of the selected location photographed from a predetermined direction within a predetermined shooting range, the accuracy of construction assessment is improved.

[0109] Furthermore, according to the construction management system 1 of the embodiment, the terminal device 200 displays a selection screen SC1 that is a list of construction types. By using this list, the operator can easily grasp the items required for each construction type. If the construction types are divided by schedule, the operator can easily grasp the items required for construction on that day.

[0110] Furthermore, according to the construction management system 1 of the embodiment, the terminal device 200 displays at least one of the number of images that need to be photographed for each type of work and the number of locations that need to be photographed. This makes it easier for the operator to grasp the number of images and locations that need to be photographed for each type of work. This also helps the operator to prevent forgetting to photograph.

[0111] Furthermore, according to the construction management system 1 of the embodiment, the terminal device 200 notifies the remaining locations that require photographing. This makes it easier for the operator to understand which locations need to be photographed. This also helps the operator to prevent forgetting to photograph.

[0112] Furthermore, according to the construction management system 1 of the embodiment, the terminal device 200 notifies the operator of at least one of the number of remaining images that need to be photographed and the number of remaining locations that need to be photographed. This makes it easier for the operator to grasp the number of remaining images and locations that need to be photographed. This also helps the operator to prevent forgetting to photograph.

[0113] Furthermore, according to the construction management system 1 of the embodiment, the terminal device 200 notifies the operator of the number of construction locations that have not passed the construction assessment. This makes it easier for the operator to grasp the locations that require photography. This also helps the operator to prevent forgetting to take photos.

[0114] Furthermore, according to the construction management system 1 of the embodiment, the terminal device 200 notifies the operator of images to be taken of construction locations that have not passed the construction assessment. This makes it easier for the operator to grasp the number of images that need to be taken. This also helps the operator to prevent forgetting to take a photo.

[0115] Furthermore, according to the construction management system 1 of the embodiment, the terminal device 200 transmits images that have passed the construction judgment to the server device 100. The terminal device 200 of the embodiment also requests the server device 100 to save the images. This allows the terminal device 200 of the embodiment to prevent images that have failed the construction judgment from being saved in the server device 100, and to save only images that have passed the construction judgment in the server device 100. Furthermore, by preventing images that have failed the construction judgment from being saved in the server device 100, the terminal device 200 of the embodiment can prevent problems during quality assurance using the images. The quality assurance relates to, for example, subsidy applications and the issuance of warranty certificates.

[0116] The above embodiment can be modified as follows.

[0117] In the above embodiment, the captured image is a still image, but the captured image may be a moving image.

[0118] The construction management system 1 of the embodiment may be required to capture a plurality of images for one construction site, and each of the plurality of images may be captured from a different direction of the construction site.

[0119] The construction judgment AI 143 may be capable of inputting multiple images of one construction site. In this case, the construction judgment AI receives the multiple images as input and outputs output information.

[0120] In the above embodiment, the terminal device 200 acquires information necessary for displaying various screens from the server device 100 at the start of the processes in Figures 3 and 4. However, the terminal device 200 may acquire information necessary for displaying each screen when displaying each of the various screens.

[0121] The server device 100 may instruct the terminal device 200 to display various screens. The terminal device 200 displays various screens based on the instructions.

[0122] It may not be necessary to operate the photographing button B31 and the sending button B32. In this case, the processor 210 of the terminal device 200 automatically photographs an image and automatically transmits the image to the server device 100. For example, the processor 210 automatically photographs an image when it is determined that the construction location in the real-time video captured by the imaging device 280 matches the guide G31 to a predetermined degree or more.

[0123] In the above embodiment, the terminal device 200 displays a selection screen SC1 showing a list of construction types. Then, the terminal device 200 displays a list screen SC2 showing a list of construction locations that need to be photographed and are included in the construction type selected on the selection screen SC1. However, the terminal device 200 may not classify the construction types, but may instead display a screen showing a list of construction locations that need to be photographed for all construction projects, for example.

[0124] The terminal device 200 may use an image captured by a device other than the terminal device 200 as an image showing the construction site. In this case, the terminal device 200 acquires the image from the device. The terminal device 200 acquires the image from the device by communication or via a removable storage medium.

[0125] Area AR11 may display the same information as area AR22.

[0126] 2, the processor 110 of the server device 100 may store the images in a device external to the server device 100 instead of in the auxiliary storage device 140. In this case, the image DB 142 may be stored in the server device 100 or in the external device. By accessing the server device 100 or the external device via the terminal device 200, a user can view and download images associated with the user ID of the user.

[0127] In step ST49 of FIG. 3, the terminal device 200 may transmit the storage request to a device external to the server device 100 instead of the server device 100. The external device is, for example, a server device having a function of storing images. The external device is, for example, a cloud server. The external device stores the image DB 142. In response to receiving the storage request, the external device executes the same process as step ST18 of FIG. 2. By accessing the external device via the terminal device 200, the user can view and download images associated with the user ID of the user.

[0128] The server device 100 may execute part of the processing performed by the terminal device 200 in the above embodiment. The terminal device 200 may execute part or all of the processing performed by the server device 100 in the above embodiment.

[0129] In the above embodiment, the terminal device 200 notifies the operator of various pieces of information by displaying the pieces of information on the display device 250. However, instead of displaying the pieces of information on the display device 250, the terminal device 200 may notify the operator of the pieces of information by a method other than displaying the information on the display device 250, such as audio output.

[0130] The processor 110 may perform additional learning of the construction judgment AI 143 using the image included in the judgment request.

[0131] Each device in the embodiment may be composed of a plurality of devices, and each device in the embodiment may be realized using cloud computing.

[0132] The processor 110 and the processor 210 may implement part or all of the processing that is implemented by a program in the above-described embodiments by a hardware circuit configuration.

[0133] A program for implementing the processes of the embodiments may be transferred in a state where it is stored in a non-transitory computer-readable storage medium within the device. However, the device may also be transferred without the program stored therein. The program may then be transferred separately and written to the device. In this case, the program may be transferred by, for example, recording it on a removable non-transitory computer-readable storage medium or by downloading it via a network such as the Internet or a LAN.

[0134] Although the embodiments of the present invention have been described above, they are merely examples and are not intended to limit the scope of the present invention. The embodiments of the present invention can be implemented in various forms without departing from the spirit of the present invention. [Explanation of symbols]

[0135] 1 Construction management system 100 Server Devices 110,210 processors 120,220 ROM 130,230 RAM 140,240 Auxiliary storage 141 User DB 142 Image DB 143 Construction judgment AI 150,270 communication interface 160,290 bus 200 Terminal Device 250 display devices 260 Input Devices 280 Imaging Device

Claims

1. The processor in the construction judgment device is an input control unit that controls input of images showing the construction site into a trained model that can evaluate whether the construction meets standards; and A program that functions as a notification control unit that controls to notify information indicating whether the construction at the location meets the standards based on the output of the trained model.

2. The program according to claim 1 , wherein the notification control unit controls to display a first list showing a list of the locations that require photographing.

3. The program according to claim 2 , wherein the notification control unit notifies, based on the output, information indicating whether the construction meets a standard for each of the locations in the first list.

4. The program according to claim 1 , wherein the notification control unit controls to display an image that assists in photographing the location in a superimposed manner on a viewfinder.

5. The program described in claim 2 or claim 3, wherein the notification control unit displays a second list showing a list of construction types, and when a type is selected in the second list, controls to display the first list showing a list of the images that need to be taken for the construction included in the selected type.

6. The program according to claim 5 , wherein the notification control unit controls to display at least one of the required number of images for each type and the number of construction locations for which images for each type need to be taken.

7. The program according to claim 1 , wherein the notification control unit notifies at least one of information indicating the remaining locations that need to be photographed, the number of remaining images that need to be photographed, and the number of remaining locations that need to be photographed.

8. The remaining areas that require photography are the areas where the construction has not been determined to meet the standards, The program according to claim 7 , wherein the remaining images that need to be captured are images of the areas where the construction has not been determined to satisfy the criteria.

9. The processor, The program described in any one of claims 1 to 3 further functions as a transmission control unit that controls the transmission of the image in which the construction of the area shown is determined to meet the standards, and requests the destination to save the image.

10. an input control unit that controls input of images showing the construction site into a trained model that can evaluate whether the construction meets standards; and A construction assessment device comprising: a notification control unit that controls to notify information indicating whether the construction at the location meets a standard based on the output of the trained model.

11. Including a terminal device and a server device, A transmission unit included in the terminal device that transmits an image showing the location where construction has been performed to the server device; An input unit included in the server device that inputs the image into a trained model that can evaluate whether the construction meets the standards; A determination unit that determines whether the construction of the location satisfies a standard based on the output of the trained model; a notification unit provided in the terminal device that notifies the result of the determination.

12. Images showing the areas where work has been done are input into a trained model that can evaluate whether the work meets standards. A construction assessment method that reports information indicating whether the construction at the location meets standards based on the output of the trained model.

Images