system

Abstract

Provide a system. 【Solution means】 Data preprocessing means for receiving image data acquired at the construction site and converting it into a processable format, Artificial intelligence means for generating an analysis model for evaluating the quality of the construction location based on the image data, Report generation means for identifying abnormal locations and generating improvement proposals for the corresponding locations based on the results of the quality evaluation by the analysis model, Interface means for presenting the generated report to the user and enabling corrections and additional inputs, Output means for outputting the report in a specified format and transmitting it to the relevant parties, Information presentation means for enabling the real-time confirmation of the evaluation results of the construction quality, Data transmission means for quickly analyzing the images of the construction site and enabling immediate guidance, A system including.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of the chatbot's character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Quality inspection at a construction site tends to rely on visual inspection and experience by skilled workers. As a result, there is a problem that the inspection requires a great deal of time and human costs. Also, when there is a shortage of skilled workers, quality variations are likely to occur, and it is difficult to maintain constant quality control. Therefore, an efficient and standardized method for evaluating construction quality is required.

Means for Solving the Problems

[0005] This invention solves the above problem by performing quality evaluation using image data acquired at construction sites. Specifically, a data preprocessing means converts the image data into an analyzable format, and an artificial intelligence means generates an analysis model for evaluating construction sites. The analysis model identifies abnormal areas in the construction sites, and based on the results, a report generation means makes appropriate improvement suggestions. Furthermore, the system is equipped with interface means and output means to realize these functions, allowing the user to easily review the report and make necessary corrections. In addition, a training data usage means allows the analysis model to be trained from past construction data, enabling quality checks equivalent to those performed by skilled personnel.

[0006] A "data preprocessing means" is an element that has the function of converting image data acquired at a construction site into a format that can be analyzed.

[0007] "Artificial intelligence means" refers to an element that generates an analytical model for evaluating the quality of construction sites based on image data and has the function of identifying abnormal areas.

[0008] A "report generation means" is an element that has the function of creating a report, including improvement suggestions for abnormal areas, based on the results of quality evaluation by artificial intelligence.

[0009] An "interface means" is an element that provides functionality for users to review generated reports and make corrections or additional inputs.

[0010] "Output mechanism" refers to an element that has the function of outputting the final report in a specified format and sending it to the relevant parties.

[0011] A "training data usage method" refers to an element that has the function of using past construction data to train an analysis model based on the knowledge of experts.

[0012] An "analysis model" is a mathematical or algorithmic framework generated by artificial intelligence to evaluate the quality of a construction site. [Brief explanation of the drawing]

[0013] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0014] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0017] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0018] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0019] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0021] [First Embodiment]

[0022] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0026] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0033] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0034] The automated construction completion report generation system in this invention evaluates construction quality based on image data acquired at the construction site and automatically generates a report. This system consists of a server, terminals, and a user interface, and functions as follows.

[0035] The server first receives image data uploaded from the terminal. This data, captured under various field conditions, cannot be used directly for quality evaluation; therefore, it is converted into an analyzable format using data preprocessing. This includes processes such as standardizing image resolution and removing noise.

[0036] Next, the server analyzes the data using artificial intelligence. Specifically, it inputs images into a trained analysis model to evaluate the quality of the construction site. This model is trained on past construction data and reflects the knowledge of skilled workers. The AI ​​analyzes the images and detects anomalies such as uneven paint application or cracks.

[0037] Based on the detection results, the server uses a report generation mechanism to create a construction completion report. The report includes a detailed explanation of the abnormal areas and proposed corrective measures. This allows the user to quickly understand specific improvement measures to be taken on-site.

[0038] Users can view the reports generated on their devices. Through the interface, users can add corrections or additional information to the reports as needed. This interface is designed to allow for easy editing of each section of the report.

[0039] Finally, the terminal outputs the revised report in the specified format and sends it to the relevant parties via email or other means. This output method ensures that the report reaches the necessary individuals quickly and reliably.

[0040] As a concrete example, consider a wall painting inspection at a construction site. The user takes a photo of the wall with their device and uploads it to a server. The server uses AI to analyze the photo and detects results such as "insufficient uniformity of paint." The report includes suggestions for additional painting to correct the unevenness, which the user can review and communicate to the site as correction instructions. This process significantly streamlines construction management.

[0041] The following describes the processing flow.

[0042] Step 1:

[0043] After the work is completed, the user takes pictures of the work area with their device. The captured images are saved to a designated folder on the device and prepared for upload to the server.

[0044] Step 2:

[0045] The device sends a request to the server to send the stored image data. The request includes image metadata (such as the date and time the image was taken, and location information), which is organized by project.

[0046] Step 3:

[0047] The server processes the received image data using data preprocessing means. By standardizing the image resolution and removing noise, it converts the image into a state suitable for analysis.

[0048] Step 4:

[0049] The server analyzes the pre-processed image data using artificial intelligence. A trained analysis model evaluates the image content and identifies abnormalities related to construction quality.

[0050] Step 5:

[0051] The server uses a report generation mechanism to create a construction completion report based on the results of the quality assessment. The report includes any detected abnormalities and recommended corrective actions.

[0052] Step 6:

[0053] Users can review reports generated on their own devices. Through the interface, they can add new notes or modify the report to a different format.

[0054] Step 7:

[0055] The terminal outputs the finalized report in a format such as PDF and distributes it to designated stakeholders via email or other means. This ensures that information reaches stakeholders quickly.

[0056] (Example 1)

[0057] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0058] Traditional construction quality control relies primarily on visual inspections by skilled personnel, consuming significant human resources and time, and making it difficult to ensure consistent quality. Furthermore, the manual process of report creation made it challenging to efficiently propose improvement measures for the construction site. To address these issues, a more automated and precise quality evaluation system is needed.

[0059] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0060] In this invention, the server includes: information preprocessing means for receiving image information acquired by a camera and converting it into a processable format; machine learning means for creating an analysis model for evaluating the quality of work areas based on the image information; and document generation means for identifying abnormal areas and generating improvement suggestions for those areas based on the quality evaluation results from the analysis model. This enables the automatic and consistent creation of construction quality evaluation and improvement reports.

[0061] "Photography equipment" refers to devices used to acquire image information from a construction site, and includes, for example, cameras and smartphones.

[0062] "Image information" refers to digital visual data acquired by a photographic device.

[0063] "Information preprocessing means" refers to a series of processes performed to convert received image information into a format that can be processed by the analysis model.

[0064] "Machine learning methods" refer to techniques for generating analytical models to evaluate the quality of work areas based on image information.

[0065] "Document generation means" refers to a technology that automatically generates reports based on the results of quality evaluations using an analysis model.

[0066] "Display means" refers to user interface technology that presents a generated document to the user and allows for modification and additional input.

[0067] "Transmission method" refers to the technology for outputting the generated report in a specified format and distributing it to the relevant parties.

[0068] "Training data usage method" refers to a technique in which a model is trained using previously acquired work data, enabling quality checks equivalent to past evaluations.

[0069] "Reference means" refers to a technology that, when identifying abnormal areas in image information, provides more specific correction suggestions by referring to a database of similar past cases.

[0070] This system is designed to automatically evaluate construction quality and generate construction completion reports. Its main components include a server, terminals, and a user interface. The following describes the role of each component and the specific hardware and software usage.

[0071] The server plays a central role in processing image information transmitted from the terminal. First, it receives image information acquired by the terminal using an imaging device (e.g., a smartphone or tablet with a camera). The server uses an image analysis library (e.g., OpenCV) to preprocess the information, performing resolution conversion and noise reduction. This enables uniform analysis.

[0072] Next, the server analyzes the pre-processed image information using a machine learning platform (e.g., TENSORFLOW® or PyTorch). It utilizes a pre-trained generative AI model to evaluate the quality of the work area. The evaluated data is updated as an analysis model, and abnormal areas are detected. This process allows for specific judgments, such as "the painted surface is uneven."

[0073] Based on the analysis results, the server automatically generates a construction completion report through a document generation system. This report includes details of the detected anomalies and proposed improvements. The report is automatically generated using LaTeX or Microsoft® Word templates.

[0074] Users review reports generated through display methods on their devices, and modify the content or enter additional information as needed. This user interface is designed for easy operation, whether browser-based or app-based.

[0075] Finally, the terminal uses a transmission method to output the revised report in the specified format (PDF or Word). The report is then sent to the relevant parties via email or cloud service.

[0076] As a concrete example, consider inspecting the paint condition of a wall on site. The user takes a photograph of the wall and uploads it to the server. The server analyzes the photograph, evaluates the uniformity of the paint, and notes any inconsistencies in the report. Based on this proposal, the user can instruct the on-site workers to apply additional paint.

[0077] As an example of a prompt for the generating AI model, it would be: "Please describe a system that evaluates quality from image information at a construction site and automatically generates a report." This ensures that the system is implemented and operated correctly, and construction quality management becomes significantly more efficient.

[0078] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0079] Step 1:

[0080] The user acquires image information using a camera at the construction site. They take photos of walls and equipment using a terminal and upload the image information to the server. In this process, images are transferred from a smartphone or tablet via Wi-Fi or mobile data communication. Image information from the site is acquired as input and ready to be transferred to the server as output.

[0081] Step 2:

[0082] The server performs preprocessing on the received image information. Using image analysis libraries such as OpenCV, it standardizes resolution, removes noise, and converts the image into a format that the analysis model can process. It takes user-uploaded image information as input and generates standardized image format data as output.

[0083] Step 3:

[0084] The server analyzes pre-processed image data using machine learning techniques. It inputs images into a generative AI model trained on a platform such as TensorFlow to evaluate the quality of the work area. Pre-processed image data is used as input, and the output provides quality assessment results and information on abnormal areas.

[0085] Step 4:

[0086] The server automatically generates a construction completion report using a document generation system based on the analysis results. The report is created using LaTeX or Word templates, and includes quality evaluation results and improvement suggestions. Quality evaluation data is provided as input, and the completed report is generated as output.

[0087] Step 5:

[0088] Users view the generated report on their device through a display device and input corrections or additional information as needed. They can view and edit each section of the report using a browser or application. The generated report data is used as input, and the corrected report is obtained as output.

[0089] Step 6:

[0090] The terminal outputs the revised report in the specified format and sends it to the relevant parties using the transmission method. It exports in PDF or Word format and distributes the report via email or cloud storage. The input is the revised report, and the output is the completed report after distribution.

[0091] (Application Example 1)

[0092] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0093] In on-site construction, there is a need to quickly and accurately evaluate construction quality, efficiently identify defects, and propose improvements. However, conventional methods require evaluation by skilled personnel, which is time-consuming and costly. Furthermore, because proposed corrections for defects are not immediately communicated to the site, it can delay the efficiency of construction. There is a need to solve these problems and achieve highly accurate and rapid quality evaluation and efficient decision-making.

[0094] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0095] In this invention, the server includes: data preprocessing means for receiving image data acquired at a construction site and converting it into a format that can be processed; artificial intelligence means for generating an analysis model for evaluating the quality of construction locations based on the image data; report generation means for identifying abnormal locations and generating improvement suggestions for those locations based on the quality evaluation results by the analysis model; interface means for presenting the generated report to the user and enabling modifications and additional inputs; output means for outputting the report in a specified format and sending it to relevant parties; information presentation means for allowing real-time confirmation of the construction quality evaluation results; and data transmission means for rapidly analyzing images of the construction site and enabling immediate guidance. This enables rapid evaluation of construction quality and immediate guidance.

[0096] "Data preprocessing means" refers to means for receiving image data acquired at a construction site and performing processing to convert it into an analyzable format.

[0097] "Artificial intelligence means" refers to a means of generating an analysis model based on image data to evaluate the quality of a construction site, and then using that model to perform quality analysis.

[0098] The "report generation means" is a means that identifies abnormal areas based on the results of quality evaluation by artificial intelligence and automatically generates a report that includes suggestions for improvement.

[0099] "Interface means" refers to means that present the generated report to the user and provide a user interface that allows the user to make corrections to the report content or input additional information.

[0100] "Output method" refers to a means of outputting a report in a specified format and sending it to the relevant parties.

[0101] An "information presentation method" is a means of presenting information to users in a way that allows them to check the evaluation results of construction quality in real time.

[0102] A "data transmission method" is a means for quickly analyzing images from a construction site and immediately transmitting the analysis results and instructions.

[0103] This invention provides a system for performing rapid and accurate quality evaluations at construction sites. This system consists of three elements: a server, a terminal, and a user.

[0104] The server receives image data acquired at the construction site from the terminal and converts it into a format that can be processed. Specifically, it uses image processing libraries such as OpenCV as a data preprocessing measure to unify the image resolution and remove noise. Then, it uses TensorFlow as an artificial intelligence measure and inputs the images into a trained analysis model to evaluate the quality of the construction area. Based on this evaluation, a report generation measure automatically generates a report that includes a detailed explanation of the abnormal areas and suggestions for improvement.

[0105] The terminal serves to present the generated report to the user. Through the interface, the user can review the report and enter corrections or additional information as needed. This interface is designed to be user-friendly, allowing for intuitive operation of each section of the report.

[0106] Users take pictures at construction sites using mobile devices such as smartphones and tablets and upload them to the server. The generated reports can be viewed on the device, allowing for real-time evaluation of construction quality. Furthermore, the evaluation results are displayed immediately and clearly through the information presentation system, enabling users to provide prompt guidance on-site.

[0107] As a concrete example, when checking the paint condition of a wall at a construction site, the user takes a photo of the wall with their smartphone and sends it to the server. The server uses an AI model to detect uneven paint and cracks, and generates a report that includes information on areas that need correction. This report can be viewed on the device, and the user can then communicate specific countermeasures to the on-site workers based on it.

[0108] As a concrete example of a prompt message for the generating AI model, it can be written in the format of, "Based on this image, please evaluate the concrete construction quality and automatically generate a report. Please also include any abnormalities and suggestions for appropriate correction methods."

[0109] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0110] Step 1:

[0111] The user takes pictures of the construction site using a smartphone or tablet. The captured image data is saved on the device and uploaded to the server. The input for this step is the captured image data, and the output is the transmission of the image data to the server.

[0112] Step 2:

[0113] The server processes the received image data using data preprocessing. Specifically, it uses image processing libraries such as OpenCV to standardize the image resolution, remove noise, and convert the image into an analyzable format. The input for this step is the uploaded raw image data, and the output is the preprocessed image data.

[0114] Step 3:

[0115] The server analyzes the pre-processed image data using artificial intelligence. The images are input to a TensorFlow-trained AI model to evaluate the quality of the construction area. The AI ​​model detects anomalies such as uneven paint application and cracks within the image. The input for this step is the pre-processed image data, and the output is the detection result of the anomalies.

[0116] Step 4:

[0117] Based on the detection results of the anomalies, the server automatically generates a report using a report generation mechanism. The report includes details of the anomalies and suggested corrections. The input for this step is the quality assessment results by AI, and the output is the automatically generated report.

[0118] Step 5:

[0119] The server sends the generated report to the terminal. The terminal presents this report to the user, who can then make corrections or input additional information as needed through the interface. The input in this step is the automatically generated report, and the output is the report content presented to the user.

[0120] Step 6:

[0121] The user reviews the report on their terminal and enters correction instructions as needed. The final report is then generated. The input in this step is the report sent from the server, and the output is the final report reflecting the user's review and corrections.

[0122] Step 7:

[0123] The terminal outputs the revised final report in the specified format and sends it to the relevant parties. This allows the parties to quickly review the report and take the necessary actions. The input for this step is the revised report, and the output is the final report sent to the relevant parties.

[0124] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0125] In this invention, a system that performs quality evaluation based on image data of a construction site and generates a report is combined with a function that recognizes the user's emotions to achieve more effective interaction.

[0126] This system primarily consists of a server, terminals, a user interface, and an emotion engine. The server processes image data of construction sites received from terminals using data preprocessing means and converts it into an analyzable format. Subsequently, artificial intelligence means are used to analyze the image data and evaluate the construction quality. The analysis model is trained using past construction data through training data utilization means, enabling quality checks equivalent to those performed by experts.

[0127] Based on the analysis results, the server automatically generates a construction completion report through the report generation mechanism. This report includes details of the detected abnormalities and suggestions for improvement. Furthermore, by referencing a database of similar past cases using the reference mechanism, more specific correction proposals are also included.

[0128] A newly incorporated feature is an emotion engine that detects user emotions and adapts the system's response accordingly. When a user reviews a report, the emotion engine analyzes the user's voice and facial expression data to recognize their emotions. Based on this, it adjusts the report content and interface display to ensure communication that is sensitive to the user's feelings.

[0129] For example, if a user expresses dissatisfaction with a report, the sentiment engine detects this emotion and instructs the system to either propose a revised report or add further explanations. Furthermore, by accumulating historical sentiment data, the system learns user preferences and tendencies, optimizing future interactions.

[0130] As the final stage of the system, the terminal outputs the final report in the specified format and sends it to the relevant parties. The report is then quickly shared via email or a cloud platform.

[0131] This system not only improves the efficiency of report creation in construction management, but also provides psychological support to users, enabling more consistent quality control.

[0132] The following describes the processing flow.

[0133] Step 1:

[0134] The user takes pictures of the construction site with their device and saves them to the project folder. These images are in preparation for later uploading to the server.

[0135] Step 2:

[0136] The device sends the image data specified by the user to the server. At the same time, the image metadata (e.g., date and time of capture, location information) is also sent.

[0137] Step 3:

[0138] The server processes the received image data using data preprocessing. Specifically, it unifies the image resolution and performs noise reduction to convert it into a state optimized for analysis.

[0139] Step 4:

[0140] The server uses artificial intelligence to analyze pre-processed image data. The analysis model evaluates construction quality based on past training data and identifies abnormal areas.

[0141] Step 5:

[0142] Based on the analysis results, the server uses a report generation mechanism to create a construction completion report. The report includes identification of abnormal areas and improvement suggestions, and, if necessary, refers to a database of similar cases to add specific correction proposals.

[0143] Step 6:

[0144] When a user views a report on their device, the emotion engine activates and analyzes the user's facial expressions and voice. The emotion engine understands the user's emotional state, adjusts the interface accordingly, and re-evaluates the report content if necessary.

[0145] Step 7:

[0146] Users can add corrections and comments through the interface based on the reported content. The sentiment engine's feedback allows users to receive advice that aligns with their intentions.

[0147] Step 8:

[0148] The terminal outputs the final, verified report in a predefined format and sends it to relevant parties via email or cloud services. As a result, reports are shared quickly and accurately.

[0149] (Example 2)

[0150] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0151] Traditionally, quality control at construction sites has heavily relied on the experience and judgment of skilled workers, often resulting in challenges regarding the consistency and speed of evaluations. Furthermore, the system was not designed to adjust reports based on user sentiment, making it difficult to consistently improve user satisfaction. In addition, there was a need to improve the specificity of corrective suggestions based on past similar cases.

[0152] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0153] In this invention, the server includes data preparation means for receiving image information acquired at a construction site and converting it into a processable format; intelligent technology means for generating an analysis structure for evaluating the quality of construction areas based on the image information; and information generation means for identifying abnormal areas and generating improvement suggestions for those areas based on the quality evaluation results from the analysis structure. This makes it possible to efficiently and consistently manage quality at construction sites, and further enables flexible responses that respond to the user's feelings and concrete correction suggestions based on past cases.

[0154] "Data preparation means" refers to a function that receives image information acquired at the construction site and processes it to convert it into an analyzable format.

[0155] "Intelligent technology means" refers to a function that generates an analysis structure for evaluating the quality of the construction site based on the received image information.

[0156] The "information generation means" is a function that automatically generates improvement suggestions and reports on identified abnormal areas based on the results of quality evaluation by intelligent technology means.

[0157] "Communication means" refers to a function that presents generated information to the user and provides an interface that allows for modification or additional input as needed.

[0158] "Communication means" refers to a function that allows the final generated report to be output in a specified format and easily sent to the relevant parties.

[0159] "Emotional analysis means" refers to a function that recognizes emotions from the user's voice, facial expressions, etc., and adaptively adjusts the system's responses and reports based on the results.

[0160] The "training data usage method" is a function that uses previously acquired construction information to train the analysis structure and obtain results equivalent to those of a quality check by a skilled professional.

[0161] The "reference means" refers to a function that searches a database of past similar cases to identify abnormal areas and proposes specific corrective measures.

[0162] This invention relates to a system for efficiently and highly automating quality control at construction sites. This system mainly consists of a server, terminals, and a user interface.

[0163] The server receives image information acquired from terminals at the construction site. Specifically, terminals such as smartphones and tablets send images taken at the site to the server via a dedicated application. The server preprocesses this image information using data preparation tools and converts it into an analyzable format. This preprocessing includes adjusting the image resolution and removing noise.

[0164] Next, the server uses intelligent technology to generate an analysis structure for evaluating the quality of the construction site based on the pre-processed image information. Here, a generative AI model trained on past construction data is utilized to identify defects and abnormalities in the site photographs. This achieves an accuracy equivalent to that of quality checks by skilled personnel.

[0165] Subsequently, the server uses information generation means to automatically generate information, including suggestions for improving the abnormal areas, based on the analysis results. This generation process includes using reference means to search for specific correction suggestions from a database of past similar cases.

[0166] Users receive reports through a communication method that displays the generated information, and can make additional inputs such as corrections and comments. Furthermore, sentiment analysis tools analyze user reactions at an emotional level, allowing the system's responses and report content to be adapted accordingly. For example, if a user expresses dissatisfaction with the report content, the system will automatically adjust to offer new suggestions.

[0167] Finally, the terminal outputs the generated report in PDF or document format via a communication method and quickly sends it to a designated email address or cloud platform. This mechanism allows relevant stakeholders to access the information in a timely manner.

[0168] As a concrete example, an example of a prompt message would be: "Evaluate the quality based on the photos of the construction site, and if there are any abnormalities, create a report that includes details and suggestions for improvement."

[0169] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0170] Step 1:

[0171] Users take pictures at construction sites using smartphones or tablets and upload these images to their devices via a dedicated application. The input is the captured image on the user's device, and the output is the original image file sent to the server. The application formats and encrypts the captured image before sending it to the server.

[0172] Step 2:

[0173] The server receives image information transmitted from the terminal and processes it using data preparation tools. The input is the encrypted original image file, and the output is image data converted into an analyzable format. Specifically, the server applies a noise reduction filter and performs adjustments to optimize the resolution.

[0174] Step 3:

[0175] The server evaluates the quality of the construction site using intelligent technology based on pre-processed image data. The input is image data converted into an analyzable format, and the output is the quality evaluation result, including identified anomalies. A generative AI model is used to perform pattern recognition within the image and execute computational processing to detect defects and problems.

[0176] Step 4:

[0177] The server automatically generates the report content using information generation means based on the quality evaluation results. The input is the quality evaluation results, and the output is a draft report. In this process, the server utilizes reference means to search a database of similar past cases and add specific correction suggestions.

[0178] Step 5:

[0179] Users review the reports generated via communication methods in writing and add corrections and comments as needed. The input is a draft report, and the output is the final, reviewed report. Users enter comments while referring to indicated sections on the interface, and real-time feedback is provided through sentiment analysis.

[0180] Step 6:

[0181] The terminal outputs the final completed report in the specified format via a designated means and sends it to the relevant parties. The input is the completed report reviewed by the user, and the output is a report file uploaded to an email or cloud platform. The terminal formats the output to PDF or another format and sends it quickly based on a pre-configured list.

[0182] (Application Example 2)

[0183] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0184] In construction site management, quality assessment and report creation are crucial, but they are time-consuming and labor-intensive. Furthermore, it can be difficult to immediately reflect user emotions and feedback, sometimes hindering improvements in satisfaction. In this context, there is a need for efficient and effective report generation and methods to enhance user interaction.

[0185] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0186] In this invention, the server includes data preprocessing means for receiving image data acquired at a construction site and converting it into a processable format; artificial intelligence means for generating an analysis model for evaluating the quality of the construction site based on the image data; and sentiment analysis means for presenting the generated report to the user and recognizing the user's emotions to adaptively change the report content and interface. This enables efficient quality evaluation at construction sites and the provision of flexible reports that respond to the user's emotions.

[0187] "Data preprocessing means" refers to a device or program that receives image data acquired at a construction site and converts it into a format that can be processed by the analysis model.

[0188] An "analysis model" is a model that uses artificial intelligence technology to quantitatively or qualitatively evaluate the quality of construction sites based on various data from the construction site.

[0189] "Artificial intelligence means" refers to trained models and algorithms for analyzing image data and evaluating construction quality, and is a technology that enables automated evaluation of construction quality.

[0190] A "report generation means" is a device or software equipped with the function of automatically creating a report that includes the identification of abnormal areas and improvement suggestions based on the results of quality evaluation by an analysis model.

[0191] "Interface means" refers to an operation screen or means that presents the generated report to the user, allows the user to make corrections or additional inputs, and adaptively changes the display to reflect information based on sentiment analysis.

[0192] "Emotion analysis means" refers to a technology or process that analyzes a user's facial expressions and voice when they review a report to recognize their emotions, and then adjusts the system's response and report content based on that.

[0193] "Output means" refers to a device or program that has the function of outputting the final report in a specified format and sending it to the relevant parties.

[0194] The system of the present invention combines data preprocessing means, artificial intelligence means, sentiment analysis means, and various output means to more efficiently perform quality evaluation and report creation at construction sites.

[0195] The server has a data preprocessing function that receives image data from the construction site from the terminal and converts it into an analyzable format. During this process, image processing libraries such as Python and OpenCV are used to perform noise reduction and resolution adjustment. After that, the image data is passed to an artificial intelligence system on the server, and its quality is evaluated using deep learning frameworks such as TensorFlow and PyTorch.

[0196] Based on the quality assessment results, if any abnormalities are identified, the server automatically generates a report containing improvement suggestions. Natural language generation technology is used in this report generation process, presenting the information in a user-friendly format.

[0197] Furthermore, emotion analysis is used to analyze the user's facial expressions and voice as they review the report, recognizing their emotions. Based on this, the system adaptively changes the report content and interface display to improve user satisfaction. Emotion recognition software such as the Emotion AI SDK is used for emotion analysis.

[0198] The final report is generated in the specified format and quickly sent to relevant parties via email or cloud services. This enables rapid decision-making on-site.

[0199] As a concrete example, consider a scenario where a construction site supervisor takes photos of the construction progress with their smartphone and uploads them to the system. The system immediately analyzes the images, evaluates their quality, and generates a report. Furthermore, by using emotional information gleaned from the supervisor's facial expressions, the system can adjust the explanations and emphasis points, providing a more valuable report to the user.

[0200] Examples of prompts include: "Please tell me how to evaluate the quality of concrete placement at the site and improve the report based on sentiment data."

[0201] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0202] Step 1:

[0203] The server receives image data uploaded from the construction site via a terminal. The received image data is temporarily stored to be sent directly to the next processing step. During the receiving process, the server verifies that the image format and size are appropriate, and corrects or filters out any inappropriate formats.

[0204] Step 2:

[0205] The server performs data preprocessing on the acquired image data. By using OpenCV to denoise and adjust the resolution of the received image data, it obtains output in a format suitable for analysis. This processing improves image quality and makes analysis by the AI ​​model more accurate.

[0206] Step 3:

[0207] The server inputs pre-processed image data into an artificial intelligence system and uses an analysis model to evaluate construction quality. The input is pre-processed image data, and the output is the quality evaluation result of the construction site. In this evaluation, a deep learning framework such as TensorFlow is used, and the AI ​​model identifies abnormal areas in the image.

[0208] Step 4:

[0209] The server generates a report using a report generation mechanism based on the quality evaluation results. Using natural language generation technology, it automatically generates details of abnormalities and improvement suggestions from the evaluation results received as input. The output report is formatted to be easily understood by the user.

[0210] Step 5:

[0211] The server presents the generated report to the user and collects and analyzes the user's facial expressions and voice using emotion analysis tools. It identifies the user's emotions from the acquired facial and voice data and provides optimized output for the user, including system responses and report content. The Emotion AI SDK is used for this analysis.

[0212] Step 6:

[0213] The server adjusts the content of reports and interface displays based on the user's emotions. Based on the entered emotion data, it outputs reports with additional explanations and emphasis where necessary. This adjustment improves user satisfaction and enables more effective communication.

[0214] Step 7:

[0215] The terminal outputs the final report in the specified format and sends it to relevant parties via email or cloud services. Receiving the adjusted final report and uploading it to the cloud service enables rapid sharing. Output means the report arrives at the relevant parties in the specified format.

[0216] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0217] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0218] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0219] [Second Embodiment]

[0220] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0221] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0222] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0223] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0224] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0225] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0226] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0227] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0228] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0229] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0230] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0231] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0232] The automated construction completion report generation system in this invention evaluates construction quality based on image data acquired at the construction site and automatically generates a report. This system consists of a server, terminals, and a user interface, and functions as follows.

[0233] The server first receives image data uploaded from the terminal. This data, captured under various field conditions, cannot be used directly for quality evaluation; therefore, it is converted into an analyzable format using data preprocessing. This includes processes such as standardizing image resolution and removing noise.

[0234] Next, the server analyzes the data using artificial intelligence. Specifically, it inputs images into a trained analysis model to evaluate the quality of the construction site. This model is trained on past construction data and reflects the knowledge of skilled workers. The AI ​​analyzes the images and detects anomalies such as uneven paint application or cracks.

[0235] Based on the detection results, the server uses a report generation mechanism to create a construction completion report. The report includes a detailed explanation of the abnormal areas and proposed corrective measures. This allows the user to quickly understand specific improvement measures to be taken on-site.

[0236] Users can view the reports generated on their devices. Through the interface, users can add corrections or additional information to the reports as needed. This interface is designed to allow for easy editing of each section of the report.

[0237] Finally, the terminal outputs the revised report in the specified format and sends it to the relevant parties via email or other means. This output method ensures that the report reaches the necessary individuals quickly and reliably.

[0238] As a concrete example, consider a wall painting inspection at a construction site. The user takes a photo of the wall with their device and uploads it to a server. The server uses AI to analyze the photo and detects results such as "insufficient uniformity of paint." The report includes suggestions for additional painting to correct the unevenness, which the user can review and communicate to the site as correction instructions. This process significantly streamlines construction management.

[0239] The following describes the processing flow.

[0240] Step 1:

[0241] After the work is completed, the user takes pictures of the work area with their device. The captured images are saved to a designated folder on the device and prepared for upload to the server.

[0242] Step 2:

[0243] The device sends a request to the server to send the stored image data. The request includes image metadata (such as the date and time the image was taken, and location information), which is organized by project.

[0244] Step 3:

[0245] The server processes the received image data using data preprocessing means. By standardizing the image resolution and removing noise, it converts the image into a state suitable for analysis.

[0246] Step 4:

[0247] The server analyzes the pre-processed image data using artificial intelligence. A trained analysis model evaluates the image content and identifies abnormalities related to construction quality.

[0248] Step 5:

[0249] The server uses a report generation mechanism to create a construction completion report based on the results of the quality assessment. The report includes any detected abnormalities and recommended corrective actions.

[0250] Step 6:

[0251] Users can review reports generated on their own devices. Through the interface, they can add new notes or modify the report to a different format.

[0252] Step 7:

[0253] The terminal outputs the finalized report in a format such as PDF and distributes it to designated stakeholders via email or other means. This ensures that information reaches stakeholders quickly.

[0254] (Example 1)

[0255] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0256] Traditional construction quality control relies primarily on visual inspections by skilled personnel, consuming significant human resources and time, and making it difficult to ensure consistent quality. Furthermore, the manual process of report creation made it challenging to efficiently propose improvement measures for the construction site. To address these issues, a more automated and precise quality evaluation system is needed.

[0257] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0258] In this invention, the server includes: information preprocessing means for receiving image information acquired by a camera and converting it into a processable format; machine learning means for creating an analysis model for evaluating the quality of work areas based on the image information; and document generation means for identifying abnormal areas and generating improvement suggestions for those areas based on the quality evaluation results from the analysis model. This enables the automatic and consistent creation of construction quality evaluation and improvement reports.

[0259] "Photography equipment" refers to devices used to acquire image information from a construction site, and includes, for example, cameras and smartphones.

[0260] "Image information" refers to digital visual data acquired by a photographic device.

[0261] "Information preprocessing means" refers to a series of processes performed to convert received image information into a format that can be processed by the analysis model.

[0262] "Machine learning methods" refer to techniques for generating analytical models to evaluate the quality of work areas based on image information.

[0263] "Document generation means" refers to a technology that automatically generates reports based on the results of quality evaluations using an analysis model.

[0264] "Display means" refers to user interface technology that presents a generated document to the user and allows for modification and additional input.

[0265] "Transmission method" refers to the technology for outputting the generated report in a specified format and distributing it to the relevant parties.

[0266] "Training data usage method" refers to a technique in which a model is trained using previously acquired work data, enabling quality checks equivalent to past evaluations.

[0267] "Reference means" refers to a technology that, when identifying abnormal areas in image information, provides more specific correction suggestions by referring to a database of similar past cases.

[0268] This system is designed to automatically evaluate construction quality and generate construction completion reports. Its main components include a server, terminals, and a user interface. The following describes the role of each component and the specific hardware and software usage.

[0269] The server plays a central role in processing image information transmitted from the terminal. First, it receives image information acquired by the terminal using an imaging device (e.g., a smartphone or tablet with a camera). The server uses an image analysis library (e.g., OpenCV) to preprocess the information, performing resolution conversion and noise reduction. This enables uniform analysis.

[0270] Next, the server analyzes the pre-processed image information using a machine learning platform (e.g., TensorFlow or PyTorch). It utilizes a pre-trained generative AI model to evaluate the quality of the construction area. The evaluated data is updated as an analysis model, and abnormal areas are detected. This process allows for specific judgments, such as "the painted surface is uneven."

[0271] Based on the analysis results, the server automatically generates a construction completion report through a document generation system. This report includes details of the detected anomalies and proposed improvements. The report is automatically generated using LaTeX or Microsoft Word templates.

[0272] Users review reports generated through display methods on their devices, and modify the content or enter additional information as needed. This user interface is designed for easy operation, whether browser-based or app-based.

[0273] Finally, the terminal uses a transmission method to output the revised report in the specified format (PDF or Word). The report is then sent to the relevant parties via email or cloud service.

[0274] As a concrete example, consider inspecting the paint condition of a wall on site. The user takes a photograph of the wall and uploads it to the server. The server analyzes the photograph, evaluates the uniformity of the paint, and notes any inconsistencies in the report. Based on this proposal, the user can instruct the on-site workers to apply additional paint.

[0275] As an example of a prompt for the generating AI model, it would be: "Please describe a system that evaluates quality from image information at a construction site and automatically generates a report." This ensures that the system is implemented and operated correctly, and construction quality management becomes significantly more efficient.

[0276] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0277] Step 1:

[0278] The user acquires image information using a camera at the construction site. They take photos of walls and equipment using a terminal and upload the image information to the server. In this process, images are transferred from a smartphone or tablet via Wi-Fi or mobile data communication. Image information from the site is acquired as input and ready to be transferred to the server as output.

[0279] Step 2:

[0280] The server performs preprocessing on the received image information. Using image analysis libraries such as OpenCV, it standardizes resolution, removes noise, and converts the image into a format that the analysis model can process. It takes user-uploaded image information as input and generates standardized image format data as output.

[0281] Step 3:

[0282] The server analyzes pre-processed image data using machine learning techniques. It inputs images into a generative AI model trained on a platform such as TensorFlow to evaluate the quality of the work area. Pre-processed image data is used as input, and the output provides quality assessment results and information on abnormal areas.

[0283] Step 4:

[0284] The server automatically generates a construction completion report using the document generation means based on the analysis results. The report is created using LaTeX or Word templates and includes the results of quality evaluations and improvement proposals. Quality evaluation data is provided as input, and a completed report is generated as output.

[0285] Step 5:

[0286] The user checks the report generated on the terminal through the display means and inputs corrections or additional information as needed. The user browses and edits each section of the report using a browser or application. The generated report data is used as input, and a corrected report is obtained as output.

[0287] Step 6:

[0288] The terminal outputs the corrected report in a specified format and transmits it to the relevant parties using the transmission means. The report is exported in PDF or Word format and distributed via email or cloud storage. The corrected report is provided as input, and a report for which distribution has been completed is obtained as output.

[0289] (Application Example 1)

[0290] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0291] In on-site construction, it is required to quickly and accurately evaluate the construction quality and efficiently identify and propose improvements for abnormal locations. However, in the conventional method, evaluation by skilled workers is required, which has problems of taking time and cost. In addition, since the proposal for correcting abnormal locations is not immediately transmitted to the site, there may be a delay in improving the construction efficiency. Means for solving these problems and realizing high-precision and rapid quality evaluation and efficient decision-making are required.

[0292] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0293] In this invention, the server includes: data preprocessing means for receiving image data acquired at a construction site and converting it into a format that can be processed; artificial intelligence means for generating an analysis model for evaluating the quality of construction locations based on the image data; report generation means for identifying abnormal locations and generating improvement suggestions for those locations based on the quality evaluation results by the analysis model; interface means for presenting the generated report to the user and enabling modifications and additional inputs; output means for outputting the report in a specified format and sending it to relevant parties; information presentation means for allowing real-time confirmation of the construction quality evaluation results; and data transmission means for rapidly analyzing images of the construction site and enabling immediate guidance. This enables rapid evaluation of construction quality and immediate guidance.

[0294] "Data preprocessing means" refers to means for receiving image data acquired at a construction site and performing processing to convert it into an analyzable format.

[0295] "Artificial intelligence means" refers to a means of generating an analysis model based on image data to evaluate the quality of a construction site, and then using that model to perform quality analysis.

[0296] The "report generation means" is a means that identifies abnormal areas based on the results of quality evaluation by artificial intelligence and automatically generates a report that includes suggestions for improvement.

[0297] "Interface means" refers to means that present the generated report to the user and provide a user interface that allows the user to make corrections to the report content or input additional information.

[0298] "Output method" refers to a means of outputting a report in a specified format and sending it to the relevant parties.

[0299] An "information presentation method" is a means of presenting information to users in a way that allows them to check the evaluation results of construction quality in real time.

[0300] A "data transmission method" is a means for quickly analyzing images from a construction site and immediately transmitting the analysis results and instructions.

[0301] This invention provides a system for performing rapid and accurate quality evaluations at construction sites. This system consists of three elements: a server, a terminal, and a user.

[0302] The server receives image data acquired at the construction site from the terminal and converts it into a format that can be processed. Specifically, it uses image processing libraries such as OpenCV as a data preprocessing measure to unify the image resolution and remove noise. Then, it uses TensorFlow as an artificial intelligence measure and inputs the images into a trained analysis model to evaluate the quality of the construction area. Based on this evaluation, a report generation measure automatically generates a report that includes a detailed explanation of the abnormal areas and suggestions for improvement.

[0303] The terminal serves to present the generated report to the user. Through the interface, the user can review the report and enter corrections or additional information as needed. This interface is designed to be user-friendly, allowing for intuitive operation of each section of the report.

[0304] Users take pictures at construction sites using mobile devices such as smartphones and tablets and upload them to the server. The generated reports can be viewed on the device, allowing for real-time evaluation of construction quality. Furthermore, the evaluation results are displayed immediately and clearly through the information presentation system, enabling users to provide prompt guidance on-site.

[0305] As a specific example, when checking the painting condition of a wall at a construction site, the user takes a photo of the wall with a smartphone and sends it to the server. The server uses an AI model to detect uneven painting and cracks, and generates a report containing information on the locations that need correction. This report can be viewed on the terminal, and the user can convey specific countermeasures to the on-site workers based on it.

[0306] As a specific example of the prompt text for the AI model generation, it can be described in the form of "Based on this image, evaluate the construction quality of the concrete and automatically generate a report. Also include the proposed abnormal locations and appropriate correction methods."

[0307] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0308] Step 1:

[0309] The user uses a smartphone or tablet at the construction site to take a photo of the construction location. The captured image data is saved on the terminal and uploaded to the server. The input of this step is the captured image data, and the output is the transmission of the image data to the server.

[0310] Step 2:

[0311] The server processes the received image data by data preprocessing means. Specifically, an image processing library such as OpenCV is used to unify the image resolution, remove noise, and convert it into an analyzable format. The input of this step is the uploaded raw image data, and the output is the preprocessed image data.

[0312] Step 3:

[0313] The server analyzes the pre-processed image data using artificial intelligence. The images are input to a TensorFlow-trained AI model to evaluate the quality of the construction area. The AI ​​model detects anomalies such as uneven paint application and cracks within the image. The input for this step is the pre-processed image data, and the output is the detection result of the anomalies.

[0314] Step 4:

[0315] Based on the detection results of the anomalies, the server automatically generates a report using a report generation mechanism. The report includes details of the anomalies and suggested corrections. The input for this step is the quality assessment results by AI, and the output is the automatically generated report.

[0316] Step 5:

[0317] The server sends the generated report to the terminal. The terminal presents this report to the user, who can then make corrections or input additional information as needed through the interface. The input in this step is the automatically generated report, and the output is the report content presented to the user.

[0318] Step 6:

[0319] The user reviews the report on their terminal and enters correction instructions as needed. The final report is then generated. The input in this step is the report sent from the server, and the output is the final report reflecting the user's review and corrections.

[0320] Step 7:

[0321] The terminal outputs the revised final report in the specified format and sends it to the relevant parties. This allows the parties to quickly review the report and take the necessary actions. The input for this step is the revised report, and the output is the final report sent to the relevant parties.

[0322] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0323] In this invention, a system that performs quality evaluation based on image data of a construction site and generates a report is combined with a function that recognizes the user's emotions to achieve more effective interaction.

[0324] This system primarily consists of a server, terminals, a user interface, and an emotion engine. The server processes image data of construction sites received from terminals using data preprocessing means and converts it into an analyzable format. Subsequently, artificial intelligence means are used to analyze the image data and evaluate the construction quality. The analysis model is trained using past construction data through training data utilization means, enabling quality checks equivalent to those performed by experts.

[0325] Based on the analysis results, the server automatically generates a construction completion report through the report generation mechanism. This report includes details of the detected abnormalities and suggestions for improvement. Furthermore, by referencing a database of similar past cases using the reference mechanism, more specific correction proposals are also included.

[0326] A newly incorporated feature is an emotion engine that detects user emotions and adapts the system's response accordingly. When a user reviews a report, the emotion engine analyzes the user's voice and facial expression data to recognize their emotions. Based on this, it adjusts the report content and interface display to ensure communication that is sensitive to the user's feelings.

[0327] For example, if a user expresses dissatisfaction with a report, the sentiment engine detects this emotion and instructs the system to either propose a revised report or add further explanations. Furthermore, by accumulating historical sentiment data, the system learns user preferences and tendencies, optimizing future interactions.

[0328] As the final stage of the system, the terminal outputs the final report in the specified format and sends it to the relevant parties. The report is then quickly shared via email or a cloud platform.

[0329] This system not only improves the efficiency of report creation in construction management, but also provides psychological support to users, enabling more consistent quality control.

[0330] The following describes the processing flow.

[0331] Step 1:

[0332] The user takes pictures of the construction site with their device and saves them to the project folder. These images are in preparation for later uploading to the server.

[0333] Step 2:

[0334] The device sends the image data specified by the user to the server. At the same time, the image metadata (e.g., date and time of capture, location information) is also sent.

[0335] Step 3:

[0336] The server processes the received image data using data preprocessing. Specifically, it unifies the image resolution and performs noise reduction to convert it into a state optimized for analysis.

[0337] Step 4:

[0338] The server uses artificial intelligence to analyze pre-processed image data. The analysis model evaluates construction quality based on past training data and identifies abnormal areas.

[0339] Step 5:

[0340] Based on the analysis results, the server uses a report generation mechanism to create a construction completion report. The report includes identification of abnormal areas and improvement suggestions, and, if necessary, refers to a database of similar cases to add specific correction proposals.

[0341] Step 6:

[0342] When a user views a report on their device, the emotion engine activates and analyzes the user's facial expressions and voice. The emotion engine understands the user's emotional state, adjusts the interface accordingly, and re-evaluates the report content if necessary.

[0343] Step 7:

[0344] Users can add corrections and comments through the interface based on the reported content. The sentiment engine's feedback allows users to receive advice that aligns with their intentions.

[0345] Step 8:

[0346] The terminal outputs the final, verified report in a predefined format and sends it to relevant parties via email or cloud services. As a result, reports are shared quickly and accurately.

[0347] (Example 2)

[0348] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0349] Traditionally, quality control at construction sites has heavily relied on the experience and judgment of skilled workers, often resulting in challenges regarding the consistency and speed of evaluations. Furthermore, the system was not designed to adjust reports based on user sentiment, making it difficult to consistently improve user satisfaction. In addition, there was a need to improve the specificity of corrective suggestions based on past similar cases.

[0350] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0351] In this invention, the server includes data preparation means for receiving image information acquired at a construction site and converting it into a processable format; intelligent technology means for generating an analysis structure for evaluating the quality of construction areas based on the image information; and information generation means for identifying abnormal areas and generating improvement suggestions for those areas based on the quality evaluation results from the analysis structure. This makes it possible to efficiently and consistently manage quality at construction sites, and further enables flexible responses that respond to the user's feelings and concrete correction suggestions based on past cases.

[0352] "Data preparation means" refers to a function that receives image information acquired at the construction site and processes it to convert it into an analyzable format.

[0353] "Intelligent technology means" refers to a function that generates an analysis structure for evaluating the quality of the construction site based on the received image information.

[0354] The "information generation means" is a function that automatically generates improvement suggestions and reports on identified abnormal areas based on the results of quality evaluation by intelligent technology means.

[0355] "Communication means" refers to a function that presents generated information to the user and provides an interface that allows for modification or additional input as needed.

[0356] "Communication means" refers to a function that allows the final generated report to be output in a specified format and easily sent to the relevant parties.

[0357] "Emotional analysis means" refers to a function that recognizes emotions from the user's voice, facial expressions, etc., and adaptively adjusts the system's responses and reports based on the results.

[0358] The "training data usage method" is a function that uses previously acquired construction information to train the analysis structure and obtain results equivalent to those of a quality check by a skilled professional.

[0359] The "reference means" refers to a function that searches a database of past similar cases to identify abnormal areas and proposes specific corrective measures.

[0360] This invention relates to a system for efficiently and highly automating quality control at construction sites. This system mainly consists of a server, terminals, and a user interface.

[0361] The server receives image information acquired from terminals at the construction site. Specifically, terminals such as smartphones and tablets send images taken at the site to the server via a dedicated application. The server preprocesses this image information using data preparation tools and converts it into an analyzable format. This preprocessing includes adjusting the image resolution and removing noise.

[0362] Next, the server uses intelligent technology to generate an analysis structure for evaluating the quality of the construction site based on the pre-processed image information. Here, a generative AI model trained on past construction data is utilized to identify defects and abnormalities in the site photographs. This achieves an accuracy equivalent to that of quality checks by skilled personnel.

[0363] Subsequently, the server uses information generation means to automatically generate information, including suggestions for improving the abnormal areas, based on the analysis results. This generation process includes using reference means to search for specific correction suggestions from a database of past similar cases.

[0364] Users receive reports through a communication method that displays the generated information, and can make additional inputs such as corrections and comments. Furthermore, sentiment analysis tools analyze user reactions at an emotional level, allowing the system's responses and report content to be adapted accordingly. For example, if a user expresses dissatisfaction with the report content, the system will automatically adjust to offer new suggestions.

[0365] Finally, the terminal outputs the generated report in PDF or document format via a communication method and quickly sends it to a designated email address or cloud platform. This mechanism allows relevant stakeholders to access the information in a timely manner.

[0366] As a concrete example, an example of a prompt message would be: "Evaluate the quality based on the photos of the construction site, and if there are any abnormalities, create a report that includes details and suggestions for improvement."

[0367] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0368] Step 1:

[0369] Users take pictures at construction sites using smartphones or tablets and upload these images to their devices via a dedicated application. The input is the captured image on the user's device, and the output is the original image file sent to the server. The application formats and encrypts the captured image before sending it to the server.

[0370] Step 2:

[0371] The server receives image information transmitted from the terminal and processes it using data preparation tools. The input is the encrypted original image file, and the output is image data converted into an analyzable format. Specifically, the server applies a noise reduction filter and performs adjustments to optimize the resolution.

[0372] Step 3:

[0373] The server evaluates the quality of the construction site using intelligent technology based on pre-processed image data. The input is image data converted into an analyzable format, and the output is the quality evaluation result, including identified anomalies. A generative AI model is used to perform pattern recognition within the image and execute computational processing to detect defects and problems.

[0374] Step 4:

[0375] The server automatically generates the report content using information generation means based on the quality evaluation results. The input is the quality evaluation results, and the output is a draft report. In this process, the server utilizes reference means to search a database of similar past cases and add specific correction suggestions.

[0376] Step 5:

[0377] Users review the reports generated via communication methods in writing and add corrections and comments as needed. The input is a draft report, and the output is the final, reviewed report. Users enter comments while referring to indicated sections on the interface, and real-time feedback is provided through sentiment analysis.

[0378] Step 6:

[0379] The terminal outputs the final completed report in the specified format via a designated means and sends it to the relevant parties. The input is the completed report reviewed by the user, and the output is a report file uploaded to an email or cloud platform. The terminal formats the output to PDF or another format and sends it quickly based on a pre-configured list.

[0380] (Application Example 2)

[0381] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0382] In construction site management, quality assessment and report creation are crucial, but they are time-consuming and labor-intensive. Furthermore, it can be difficult to immediately reflect user emotions and feedback, sometimes hindering improvements in satisfaction. In this context, there is a need for efficient and effective report generation and methods to enhance user interaction.

[0383] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0384] In this invention, the server includes data preprocessing means for receiving image data acquired at a construction site and converting it into a processable format; artificial intelligence means for generating an analysis model for evaluating the quality of the construction site based on the image data; and sentiment analysis means for presenting the generated report to the user and recognizing the user's emotions to adaptively change the report content and interface. This enables efficient quality evaluation at construction sites and the provision of flexible reports that respond to the user's emotions.

[0385] "Data preprocessing means" refers to a device or program that receives image data acquired at a construction site and converts it into a format that can be processed by the analysis model.

[0386] An "analysis model" is a model that uses artificial intelligence technology to quantitatively or qualitatively evaluate the quality of construction sites based on various data from the construction site.

[0387] "Artificial intelligence means" refers to trained models and algorithms for analyzing image data and evaluating construction quality, and is a technology that enables automated evaluation of construction quality.

[0388] A "report generation means" is a device or software equipped with the function of automatically creating a report that includes the identification of abnormal areas and improvement suggestions based on the results of quality evaluation by an analysis model.

[0389] "Interface means" refers to an operation screen or means that presents the generated report to the user, allows the user to make corrections or additional inputs, and adaptively changes the display to reflect information based on sentiment analysis.

[0390] "Emotion analysis means" refers to a technology or process that analyzes a user's facial expressions and voice when they review a report to recognize their emotions, and then adjusts the system's response and report content based on that.

[0391] "Output means" refers to a device or program that has the function of outputting the final report in a specified format and sending it to the relevant parties.

[0392] The system of the present invention combines data preprocessing means, artificial intelligence means, sentiment analysis means, and various output means to more efficiently perform quality evaluation and report creation at construction sites.

[0393] The server has a data preprocessing function that receives image data from the construction site from the terminal and converts it into an analyzable format. During this process, image processing libraries such as Python and OpenCV are used to perform noise reduction and resolution adjustment. After that, the image data is passed to an artificial intelligence system on the server, and its quality is evaluated using deep learning frameworks such as TensorFlow and PyTorch.

[0394] Based on the quality assessment results, if any abnormalities are identified, the server automatically generates a report containing improvement suggestions. Natural language generation technology is used in this report generation process, presenting the information in a user-friendly format.

[0395] Furthermore, emotion analysis is used to analyze the user's facial expressions and voice as they review the report, recognizing their emotions. Based on this, the system adaptively changes the report content and interface display to improve user satisfaction. Emotion recognition software such as the Emotion AI SDK is used for emotion analysis.

[0396] The final report is generated in the specified format and quickly sent to relevant parties via email or cloud services. This enables rapid decision-making on-site.

[0397] As a concrete example, consider a scenario where a construction site supervisor takes photos of the construction progress with their smartphone and uploads them to the system. The system immediately analyzes the images, evaluates their quality, and generates a report. Furthermore, by using emotional information gleaned from the supervisor's facial expressions, the system can adjust the explanations and emphasis points, providing a more valuable report to the user.

[0398] Examples of prompts include: "Please tell me how to evaluate the quality of concrete placement at the site and improve the report based on sentiment data."

[0399] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0400] Step 1:

[0401] The server receives image data uploaded from the construction site via a terminal. The received image data is temporarily stored to be sent directly to the next processing step. During the receiving process, the server verifies that the image format and size are appropriate, and corrects or filters out any inappropriate formats.

[0402] Step 2:

[0403] The server performs data preprocessing on the acquired image data. By using OpenCV to denoise and adjust the resolution of the received image data, it obtains output in a format suitable for analysis. This processing improves image quality and makes analysis by the AI ​​model more accurate.

[0404] Step 3:

[0405] The server inputs pre-processed image data into an artificial intelligence system and uses an analysis model to evaluate construction quality. The input is pre-processed image data, and the output is the quality evaluation result of the construction site. In this evaluation, a deep learning framework such as TensorFlow is used, and the AI ​​model identifies abnormal areas in the image.

[0406] Step 4:

[0407] The server generates a report using a report generation mechanism based on the quality evaluation results. Using natural language generation technology, it automatically generates details of abnormalities and improvement suggestions from the evaluation results received as input. The output report is formatted to be easily understood by the user.

[0408] Step 5:

[0409] The server presents the generated report to the user and collects and analyzes the user's facial expressions and voice using emotion analysis tools. It identifies the user's emotions from the acquired facial and voice data and provides optimized output for the user, including system responses and report content. The Emotion AI SDK is used for this analysis.

[0410] Step 6:

[0411] The server adjusts the content of reports and interface displays based on the user's emotions. Based on the entered emotion data, it outputs reports with additional explanations and emphasis where necessary. This adjustment improves user satisfaction and enables more effective communication.

[0412] Step 7:

[0413] The terminal outputs the final report in the specified format and sends it to relevant parties via email or cloud services. Receiving the adjusted final report and uploading it to the cloud service enables rapid sharing. Output means the report arrives at the relevant parties in the specified format.

[0414] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0415] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0416] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0417] [Third Embodiment]

[0418] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0419] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0420] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0421] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0422] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0423] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0424] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0425] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0426] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0427] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0428] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0429] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0430] The automated construction completion report generation system in this invention evaluates construction quality based on image data acquired at the construction site and automatically generates a report. This system consists of a server, terminals, and a user interface, and functions as follows.

[0431] The server first receives image data uploaded from the terminal. This data, captured under various field conditions, cannot be used directly for quality evaluation; therefore, it is converted into an analyzable format using data preprocessing. This includes processes such as standardizing image resolution and removing noise.

[0432] Next, the server analyzes the data using artificial intelligence. Specifically, it inputs images into a trained analysis model to evaluate the quality of the construction site. This model is trained on past construction data and reflects the knowledge of skilled workers. The AI ​​analyzes the images and detects anomalies such as uneven paint application or cracks.

[0433] Based on the detection results, the server uses a report generation mechanism to create a construction completion report. The report includes a detailed explanation of the abnormal areas and proposed corrective measures. This allows the user to quickly understand specific improvement measures to be taken on-site.

[0434] Users can view the reports generated on their devices. Through the interface, users can add corrections or additional information to the reports as needed. This interface is designed to allow for easy editing of each section of the report.

[0435] Finally, the terminal outputs the revised report in the specified format and sends it to the relevant parties via email or other means. This output method ensures that the report reaches the necessary individuals quickly and reliably.

[0436] As a concrete example, consider a wall painting inspection at a construction site. The user takes a photo of the wall with their device and uploads it to a server. The server uses AI to analyze the photo and detects results such as "insufficient uniformity of paint." The report includes suggestions for additional painting to correct the unevenness, which the user can review and communicate to the site as correction instructions. This process significantly streamlines construction management.

[0437] The following describes the processing flow.

[0438] Step 1:

[0439] After the work is completed, the user takes pictures of the work area with their device. The captured images are saved to a designated folder on the device and prepared for upload to the server.

[0440] Step 2:

[0441] The device sends a request to the server to send the stored image data. The request includes image metadata (such as the date and time the image was taken, and location information), which is organized by project.

[0442] Step 3:

[0443] The server processes the received image data using data preprocessing means. By standardizing the image resolution and removing noise, it converts the image into a state suitable for analysis.

[0444] Step 4:

[0445] The server analyzes the pre-processed image data using artificial intelligence. A trained analysis model evaluates the image content and identifies abnormalities related to construction quality.

[0446] Step 5:

[0447] The server uses a report generation mechanism to create a construction completion report based on the results of the quality assessment. The report includes any detected abnormalities and recommended corrective actions.

[0448] Step 6:

[0449] Users can review reports generated on their own devices. Through the interface, they can add new notes or modify the report to a different format.

[0450] Step 7:

[0451] The terminal outputs the finalized report in a format such as PDF and distributes it to designated stakeholders via email or other means. This ensures that information reaches stakeholders quickly.

[0452] (Example 1)

[0453] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0454] Traditional construction quality control relies primarily on visual inspections by skilled personnel, consuming significant human resources and time, and making it difficult to ensure consistent quality. Furthermore, the manual process of report creation made it challenging to efficiently propose improvement measures for the construction site. To address these issues, a more automated and precise quality evaluation system is needed.

[0455] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0456] In this invention, the server includes: information preprocessing means for receiving image information acquired by a camera and converting it into a processable format; machine learning means for creating an analysis model for evaluating the quality of work areas based on the image information; and document generation means for identifying abnormal areas and generating improvement suggestions for those areas based on the quality evaluation results from the analysis model. This enables the automatic and consistent creation of construction quality evaluation and improvement reports.

[0457] "Photography equipment" refers to devices used to acquire image information from a construction site, and includes, for example, cameras and smartphones.

[0458] "Image information" refers to digital visual data acquired by a photographic device.

[0459] "Information preprocessing means" refers to a series of processes performed to convert received image information into a format that can be processed by the analysis model.

[0460] "Machine learning methods" refer to techniques for generating analytical models to evaluate the quality of work areas based on image information.

[0461] "Document generation means" refers to a technology that automatically generates reports based on the results of quality evaluations using an analysis model.

[0462] "Display means" refers to user interface technology that presents a generated document to the user and allows for modification and additional input.

[0463] "Transmission method" refers to the technology for outputting the generated report in a specified format and distributing it to the relevant parties.

[0464] "Training data usage method" refers to a technique in which a model is trained using previously acquired work data, enabling quality checks equivalent to past evaluations.

[0465] "Reference means" refers to a technology that, when identifying abnormal areas in image information, provides more specific correction suggestions by referring to a database of similar past cases.

[0466] This system is designed to automatically evaluate construction quality and generate construction completion reports. Its main components include a server, terminals, and a user interface. The following describes the role of each component and the specific hardware and software usage.

[0467] The server plays a central role in processing image information transmitted from the terminal. First, it receives image information acquired by the terminal using an imaging device (e.g., a smartphone or tablet with a camera). The server uses an image analysis library (e.g., OpenCV) to preprocess the information, performing resolution conversion and noise reduction. This enables uniform analysis.

[0468] Next, the server analyzes the pre-processed image information using a machine learning platform (e.g., TensorFlow or PyTorch). It utilizes a pre-trained generative AI model to evaluate the quality of the construction area. The evaluated data is updated as an analysis model, and abnormal areas are detected. This process allows for specific judgments, such as "the painted surface is uneven."

[0469] Based on the analysis results, the server automatically generates a construction completion report through a document generation system. This report includes details of the detected anomalies and proposed improvements. The report is automatically generated using LaTeX or Microsoft Word templates.

[0470] Users review reports generated through display methods on their devices, and modify the content or enter additional information as needed. This user interface is designed for easy operation, whether browser-based or app-based.

[0471] Finally, the terminal uses a transmission method to output the revised report in the specified format (PDF or Word). The report is then sent to the relevant parties via email or cloud service.

[0472] As a concrete example, consider inspecting the paint condition of a wall on site. The user takes a photograph of the wall and uploads it to the server. The server analyzes the photograph, evaluates the uniformity of the paint, and notes any inconsistencies in the report. Based on this proposal, the user can instruct the on-site workers to apply additional paint.

[0473] As an example of a prompt for the generating AI model, it would be: "Please describe a system that evaluates quality from image information at a construction site and automatically generates a report." This ensures that the system is implemented and operated correctly, and construction quality management becomes significantly more efficient.

[0474] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0475] Step 1:

[0476] The user acquires image information using a camera at the construction site. They take photos of walls and equipment using a terminal and upload the image information to the server. In this process, images are transferred from a smartphone or tablet via Wi-Fi or mobile data communication. Image information from the site is acquired as input and ready to be transferred to the server as output.

[0477] Step 2:

[0478] The server performs preprocessing on the received image information. Using image analysis libraries such as OpenCV, it standardizes resolution, removes noise, and converts the image into a format that the analysis model can process. It takes user-uploaded image information as input and generates standardized image format data as output.

[0479] Step 3:

[0480] The server analyzes pre-processed image data using machine learning techniques. It inputs images into a generative AI model trained on a platform such as TensorFlow to evaluate the quality of the work area. Pre-processed image data is used as input, and the output provides quality assessment results and information on abnormal areas.

[0481] Step 4:

[0482] The server automatically generates a construction completion report using a document generation system based on the analysis results. The report is created using LaTeX or Word templates, and includes quality evaluation results and improvement suggestions. Quality evaluation data is provided as input, and the completed report is generated as output.

[0483] Step 5:

[0484] Users view the generated report on their device through a display device and input corrections or additional information as needed. They can view and edit each section of the report using a browser or application. The generated report data is used as input, and the corrected report is obtained as output.

[0485] Step 6:

[0486] The terminal outputs the revised report in the specified format and sends it to the relevant parties using the transmission method. It exports in PDF or Word format and distributes the report via email or cloud storage. The input is the revised report, and the output is the completed report after distribution.

[0487] (Application Example 1)

[0488] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0489] In on-site construction, there is a need to quickly and accurately evaluate construction quality, efficiently identify defects, and propose improvements. However, conventional methods require evaluation by skilled personnel, which is time-consuming and costly. Furthermore, because proposed corrections for defects are not immediately communicated to the site, it can delay the efficiency of construction. There is a need to solve these problems and achieve highly accurate and rapid quality evaluation and efficient decision-making.

[0490] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0491] In this invention, the server includes: data preprocessing means for receiving image data acquired at a construction site and converting it into a format that can be processed; artificial intelligence means for generating an analysis model for evaluating the quality of construction locations based on the image data; report generation means for identifying abnormal locations and generating improvement suggestions for those locations based on the quality evaluation results by the analysis model; interface means for presenting the generated report to the user and enabling modifications and additional inputs; output means for outputting the report in a specified format and sending it to relevant parties; information presentation means for allowing real-time confirmation of the construction quality evaluation results; and data transmission means for rapidly analyzing images of the construction site and enabling immediate guidance. This enables rapid evaluation of construction quality and immediate guidance.

[0492] "Data preprocessing means" refers to means for receiving image data acquired at a construction site and performing processing to convert it into an analyzable format.

[0493] "Artificial intelligence means" refers to a means of generating an analysis model based on image data to evaluate the quality of a construction site, and then using that model to perform quality analysis.

[0494] The "report generation means" is a means that identifies abnormal areas based on the results of quality evaluation by artificial intelligence and automatically generates a report that includes suggestions for improvement.

[0495] "Interface means" refers to means that present the generated report to the user and provide a user interface that allows the user to make corrections to the report content or input additional information.

[0496] "Output method" refers to a means of outputting a report in a specified format and sending it to the relevant parties.

[0497] An "information presentation method" is a means of presenting information to users in a way that allows them to check the evaluation results of construction quality in real time.

[0498] A "data transmission method" is a means for quickly analyzing images from a construction site and immediately transmitting the analysis results and instructions.

[0499] This invention provides a system for performing rapid and accurate quality evaluations at construction sites. This system consists of three elements: a server, a terminal, and a user.

[0500] The server receives image data acquired at the construction site from the terminal and converts it into a format that can be processed. Specifically, it uses image processing libraries such as OpenCV as a data preprocessing measure to unify the image resolution and remove noise. Then, it uses TensorFlow as an artificial intelligence measure and inputs the images into a trained analysis model to evaluate the quality of the construction area. Based on this evaluation, a report generation measure automatically generates a report that includes a detailed explanation of the abnormal areas and suggestions for improvement.

[0501] The terminal serves to present the generated report to the user. Through the interface, the user can review the report and enter corrections or additional information as needed. This interface is designed to be user-friendly, allowing for intuitive operation of each section of the report.

[0502] Users take pictures at construction sites using mobile devices such as smartphones and tablets and upload them to the server. The generated reports can be viewed on the device, allowing for real-time evaluation of construction quality. Furthermore, the evaluation results are displayed immediately and clearly through the information presentation system, enabling users to provide prompt guidance on-site.

[0503] As a concrete example, when checking the paint condition of a wall at a construction site, the user takes a photo of the wall with their smartphone and sends it to the server. The server uses an AI model to detect uneven paint and cracks, and generates a report that includes information on areas that need correction. This report can be viewed on the device, and the user can then communicate specific countermeasures to the on-site workers based on it.

[0504] As a concrete example of a prompt message for the generating AI model, it can be written in the format of, "Based on this image, please evaluate the concrete construction quality and automatically generate a report. Please also include any abnormalities and suggestions for appropriate correction methods."

[0505] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0506] Step 1:

[0507] The user takes pictures of the construction site using a smartphone or tablet. The captured image data is saved on the device and uploaded to the server. The input for this step is the captured image data, and the output is the transmission of the image data to the server.

[0508] Step 2:

[0509] The server processes the received image data using data preprocessing. Specifically, it uses image processing libraries such as OpenCV to standardize the image resolution, remove noise, and convert the image into an analyzable format. The input for this step is the uploaded raw image data, and the output is the preprocessed image data.

[0510] Step 3:

[0511] The server analyzes the pre-processed image data using artificial intelligence. The images are input to a TensorFlow-trained AI model to evaluate the quality of the construction area. The AI ​​model detects anomalies such as uneven paint application and cracks within the image. The input for this step is the pre-processed image data, and the output is the detection result of the anomalies.

[0512] Step 4:

[0513] Based on the detection results of the anomalies, the server automatically generates a report using a report generation mechanism. The report includes details of the anomalies and suggested corrections. The input for this step is the quality assessment results by AI, and the output is the automatically generated report.

[0514] Step 5:

[0515] The server sends the generated report to the terminal. The terminal presents this report to the user, who can then make corrections or input additional information as needed through the interface. The input in this step is the automatically generated report, and the output is the report content presented to the user.

[0516] Step 6:

[0517] The user reviews the report on their terminal and enters correction instructions as needed. The final report is then generated. The input in this step is the report sent from the server, and the output is the final report reflecting the user's review and corrections.

[0518] Step 7:

[0519] The terminal outputs the revised final report in the specified format and sends it to the relevant parties. This allows the parties to quickly review the report and take the necessary actions. The input for this step is the revised report, and the output is the final report sent to the relevant parties.

[0520] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0521] In this invention, a system that performs quality evaluation based on image data of a construction site and generates a report is combined with a function that recognizes the user's emotions to achieve more effective interaction.

[0522] This system primarily consists of a server, terminals, a user interface, and an emotion engine. The server processes image data of construction sites received from terminals using data preprocessing means and converts it into an analyzable format. Subsequently, artificial intelligence means are used to analyze the image data and evaluate the construction quality. The analysis model is trained using past construction data through training data utilization means, enabling quality checks equivalent to those performed by experts.

[0523] Based on the analysis results, the server automatically generates a construction completion report through the report generation mechanism. This report includes details of the detected abnormalities and suggestions for improvement. Furthermore, by referencing a database of similar past cases using the reference mechanism, more specific correction proposals are also included.

[0524] A newly incorporated feature is an emotion engine that detects user emotions and adapts the system's response accordingly. When a user reviews a report, the emotion engine analyzes the user's voice and facial expression data to recognize their emotions. Based on this, it adjusts the report content and interface display to ensure communication that is sensitive to the user's feelings.

[0525] For example, if a user expresses dissatisfaction with a report, the sentiment engine detects this emotion and instructs the system to either propose a revised report or add further explanations. Furthermore, by accumulating historical sentiment data, the system learns user preferences and tendencies, optimizing future interactions.

[0526] As the final stage of the system, the terminal outputs the final report in the specified format and sends it to the relevant parties. The report is then quickly shared via email or a cloud platform.

[0527] This system not only improves the efficiency of report creation in construction management, but also provides psychological support to users, enabling more consistent quality control.

[0528] The following describes the processing flow.

[0529] Step 1:

[0530] The user takes pictures of the construction site with their device and saves them to the project folder. These images are in preparation for later uploading to the server.

[0531] Step 2:

[0532] The device sends the image data specified by the user to the server. At the same time, the image metadata (e.g., date and time of capture, location information) is also sent.

[0533] Step 3:

[0534] The server processes the received image data using data preprocessing. Specifically, it unifies the image resolution and performs noise reduction to convert it into a state optimized for analysis.

[0535] Step 4:

[0536] The server uses artificial intelligence to analyze pre-processed image data. The analysis model evaluates construction quality based on past training data and identifies abnormal areas.

[0537] Step 5:

[0538] Based on the analysis results, the server uses a report generation mechanism to create a construction completion report. The report includes identification of abnormal areas and improvement suggestions, and, if necessary, refers to a database of similar cases to add specific correction proposals.

[0539] Step 6:

[0540] When a user views a report on their device, the emotion engine activates and analyzes the user's facial expressions and voice. The emotion engine understands the user's emotional state, adjusts the interface accordingly, and re-evaluates the report content if necessary.

[0541] Step 7:

[0542] Users can add corrections and comments through the interface based on the reported content. The sentiment engine's feedback allows users to receive advice that aligns with their intentions.

[0543] Step 8:

[0544] The terminal outputs the final, verified report in a predefined format and sends it to relevant parties via email or cloud services. As a result, reports are shared quickly and accurately.

[0545] (Example 2)

[0546] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0547] Traditionally, quality control at construction sites has heavily relied on the experience and judgment of skilled workers, often resulting in challenges regarding the consistency and speed of evaluations. Furthermore, the system was not designed to adjust reports based on user sentiment, making it difficult to consistently improve user satisfaction. In addition, there was a need to improve the specificity of corrective suggestions based on past similar cases.

[0548] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0549] In this invention, the server includes data preparation means for receiving image information acquired at a construction site and converting it into a processable format; intelligent technology means for generating an analysis structure for evaluating the quality of construction areas based on the image information; and information generation means for identifying abnormal areas and generating improvement suggestions for those areas based on the quality evaluation results from the analysis structure. This makes it possible to efficiently and consistently manage quality at construction sites, and further enables flexible responses that respond to the user's feelings and concrete correction suggestions based on past cases.

[0550] "Data preparation means" refers to a function that receives image information acquired at the construction site and processes it to convert it into an analyzable format.

[0551] "Intelligent technology means" refers to a function that generates an analysis structure for evaluating the quality of the construction site based on the received image information.

[0552] The "information generation means" is a function that automatically generates improvement suggestions and reports on identified abnormal areas based on the results of quality evaluation by intelligent technology means.

[0553] "Communication means" refers to a function that presents generated information to the user and provides an interface that allows for modification or additional input as needed.

[0554] "Communication means" refers to a function that allows the final generated report to be output in a specified format and easily sent to the relevant parties.

[0555] "Emotional analysis means" refers to a function that recognizes emotions from the user's voice, facial expressions, etc., and adaptively adjusts the system's responses and reports based on the results.

[0556] The "training data usage method" is a function that uses previously acquired construction information to train the analysis structure and obtain results equivalent to those of a quality check by a skilled professional.

[0557] The "reference means" refers to a function that searches a database of past similar cases to identify abnormal areas and proposes specific corrective measures.

[0558] This invention relates to a system for efficiently and highly automating quality control at construction sites. This system mainly consists of a server, terminals, and a user interface.

[0559] The server receives image information acquired from terminals at the construction site. Specifically, terminals such as smartphones and tablets send images taken at the site to the server via a dedicated application. The server preprocesses this image information using data preparation tools and converts it into an analyzable format. This preprocessing includes adjusting the image resolution and removing noise.

[0560] Next, the server uses intelligent technology to generate an analysis structure for evaluating the quality of the construction site based on the pre-processed image information. Here, a generative AI model trained on past construction data is utilized to identify defects and abnormalities in the site photographs. This achieves an accuracy equivalent to that of quality checks by skilled personnel.

[0561] Subsequently, the server uses information generation means to automatically generate information, including suggestions for improving the abnormal areas, based on the analysis results. This generation process includes using reference means to search for specific correction suggestions from a database of past similar cases.

[0562] Users receive reports through a communication method that displays the generated information, and can make additional inputs such as corrections and comments. Furthermore, sentiment analysis tools analyze user reactions at an emotional level, allowing the system's responses and report content to be adapted accordingly. For example, if a user expresses dissatisfaction with the report content, the system will automatically adjust to offer new suggestions.

[0563] Finally, the terminal outputs the generated report in PDF or document format via a communication method and quickly sends it to a designated email address or cloud platform. This mechanism allows relevant stakeholders to access the information in a timely manner.

[0564] As a concrete example, an example of a prompt message would be: "Evaluate the quality based on the photos of the construction site, and if there are any abnormalities, create a report that includes details and suggestions for improvement."

[0565] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0566] Step 1:

[0567] Users take pictures at construction sites using smartphones or tablets and upload these images to their devices via a dedicated application. The input is the captured image on the user's device, and the output is the original image file sent to the server. The application formats and encrypts the captured image before sending it to the server.

[0568] Step 2:

[0569] The server receives image information transmitted from the terminal and processes it using data preparation tools. The input is the encrypted original image file, and the output is image data converted into an analyzable format. Specifically, the server applies a noise reduction filter and performs adjustments to optimize the resolution.

[0570] Step 3:

[0571] The server evaluates the quality of the construction site using intelligent technology based on pre-processed image data. The input is image data converted into an analyzable format, and the output is the quality evaluation result, including identified anomalies. A generative AI model is used to perform pattern recognition within the image and execute computational processing to detect defects and problems.

[0572] Step 4:

[0573] The server automatically generates the report content using information generation means based on the quality evaluation results. The input is the quality evaluation results, and the output is a draft report. In this process, the server utilizes reference means to search a database of similar past cases and add specific correction suggestions.

[0574] Step 5:

[0575] Users review the reports generated via communication methods in writing and add corrections and comments as needed. The input is a draft report, and the output is the final, reviewed report. Users enter comments while referring to indicated sections on the interface, and real-time feedback is provided through sentiment analysis.

[0576] Step 6:

[0577] The terminal outputs the final completed report in the specified format via a designated means and sends it to the relevant parties. The input is the completed report reviewed by the user, and the output is a report file uploaded to an email or cloud platform. The terminal formats the output to PDF or another format and sends it quickly based on a pre-configured list.

[0578] (Application Example 2)

[0579] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0580] In construction site management, quality assessment and report creation are crucial, but they are time-consuming and labor-intensive. Furthermore, it can be difficult to immediately reflect user emotions and feedback, sometimes hindering improvements in satisfaction. In this context, there is a need for efficient and effective report generation and methods to enhance user interaction.

[0581] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0582] In this invention, the server includes data preprocessing means for receiving image data acquired at a construction site and converting it into a processable format; artificial intelligence means for generating an analysis model for evaluating the quality of the construction site based on the image data; and sentiment analysis means for presenting the generated report to the user and recognizing the user's emotions to adaptively change the report content and interface. This enables efficient quality evaluation at construction sites and the provision of flexible reports that respond to the user's emotions.

[0583] "Data preprocessing means" refers to a device or program that receives image data acquired at a construction site and converts it into a format that can be processed by the analysis model.

[0584] An "analysis model" is a model that uses artificial intelligence technology to quantitatively or qualitatively evaluate the quality of construction sites based on various data from the construction site.

[0585] "Artificial intelligence means" refers to trained models and algorithms for analyzing image data and evaluating construction quality, and is a technology that enables automated evaluation of construction quality.

[0586] A "report generation means" is a device or software equipped with the function of automatically creating a report that includes the identification of abnormal areas and improvement suggestions based on the results of quality evaluation by an analysis model.

[0587] "Interface means" refers to an operation screen or means that presents the generated report to the user, allows the user to make corrections or additional inputs, and adaptively changes the display to reflect information based on sentiment analysis.

[0588] "Emotion analysis means" refers to a technology or process that analyzes a user's facial expressions and voice when they review a report to recognize their emotions, and then adjusts the system's response and report content based on that.

[0589] "Output means" refers to a device or program that has the function of outputting the final report in a specified format and sending it to the relevant parties.

[0590] The system of the present invention combines data preprocessing means, artificial intelligence means, sentiment analysis means, and various output means to more efficiently perform quality evaluation and report creation at construction sites.

[0591] The server has a data preprocessing function that receives image data from the construction site from the terminal and converts it into an analyzable format. During this process, image processing libraries such as Python and OpenCV are used to perform noise reduction and resolution adjustment. After that, the image data is passed to an artificial intelligence system on the server, and its quality is evaluated using deep learning frameworks such as TensorFlow and PyTorch.

[0592] Based on the quality assessment results, if any abnormalities are identified, the server automatically generates a report containing improvement suggestions. Natural language generation technology is used in this report generation process, presenting the information in a user-friendly format.

[0593] Furthermore, emotion analysis is used to analyze the user's facial expressions and voice as they review the report, recognizing their emotions. Based on this, the system adaptively changes the report content and interface display to improve user satisfaction. Emotion recognition software such as the Emotion AI SDK is used for emotion analysis.

[0594] The final report is generated in the specified format and quickly sent to relevant parties via email or cloud services. This enables rapid decision-making on-site.

[0595] As a concrete example, consider a scenario where a construction site supervisor takes photos of the construction progress with their smartphone and uploads them to the system. The system immediately analyzes the images, evaluates their quality, and generates a report. Furthermore, by using emotional information gleaned from the supervisor's facial expressions, the system can adjust the explanations and emphasis points, providing a more valuable report to the user.

[0596] Examples of prompts include: "Please tell me how to evaluate the quality of concrete placement at the site and improve the report based on sentiment data."

[0597] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0598] Step 1:

[0599] The server receives image data uploaded from the construction site via a terminal. The received image data is temporarily stored to be sent directly to the next processing step. During the receiving process, the server verifies that the image format and size are appropriate, and corrects or filters out any inappropriate formats.

[0600] Step 2:

[0601] The server performs data preprocessing on the acquired image data. By using OpenCV to denoise and adjust the resolution of the received image data, it obtains output in a format suitable for analysis. This processing improves image quality and makes analysis by the AI ​​model more accurate.

[0602] Step 3:

[0603] The server inputs pre-processed image data into an artificial intelligence system and uses an analysis model to evaluate construction quality. The input is pre-processed image data, and the output is the quality evaluation result of the construction site. In this evaluation, a deep learning framework such as TensorFlow is used, and the AI ​​model identifies abnormal areas in the image.

[0604] Step 4:

[0605] The server generates a report using a report generation mechanism based on the quality evaluation results. Using natural language generation technology, it automatically generates details of abnormalities and improvement suggestions from the evaluation results received as input. The output report is formatted to be easily understood by the user.

[0606] Step 5:

[0607] The server presents the generated report to the user and collects and analyzes the user's facial expressions and voice using emotion analysis tools. It identifies the user's emotions from the acquired facial and voice data and provides optimized output for the user, including system responses and report content. The Emotion AI SDK is used for this analysis.

[0608] Step 6:

[0609] The server adjusts the content of reports and interface displays based on the user's emotions. Based on the entered emotion data, it outputs reports with additional explanations and emphasis where necessary. This adjustment improves user satisfaction and enables more effective communication.

[0610] Step 7:

[0611] The terminal outputs the final report in the specified format and sends it to relevant parties via email or cloud services. Receiving the adjusted final report and uploading it to the cloud service enables rapid sharing. Output means the report arrives at the relevant parties in the specified format.

[0612] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0613] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0614] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0615] [Fourth Embodiment]

[0616] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0617] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0618] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0619] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0620] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0621] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0622] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0623] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0624] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0625] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0626] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0627] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0628] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0629] The automated construction completion report generation system in this invention evaluates construction quality based on image data acquired at the construction site and automatically generates a report. This system consists of a server, terminals, and a user interface, and functions as follows.

[0630] The server first receives image data uploaded from the terminal. This data, captured under various field conditions, cannot be used directly for quality evaluation; therefore, it is converted into an analyzable format using data preprocessing. This includes processes such as standardizing image resolution and removing noise.

[0631] Next, the server analyzes the data using artificial intelligence. Specifically, it inputs images into a trained analysis model to evaluate the quality of the construction site. This model is trained on past construction data and reflects the knowledge of skilled workers. The AI ​​analyzes the images and detects anomalies such as uneven paint application or cracks.

[0632] Based on the detection results, the server uses a report generation mechanism to create a construction completion report. The report includes a detailed explanation of the abnormal areas and proposed corrective measures. This allows the user to quickly understand specific improvement measures to be taken on-site.

[0633] Users can view the reports generated on their devices. Through the interface, users can add corrections or additional information to the reports as needed. This interface is designed to allow for easy editing of each section of the report.

[0634] Finally, the terminal outputs the revised report in the specified format and sends it to the relevant parties via email or other means. This output method ensures that the report reaches the necessary individuals quickly and reliably.

[0635] As a concrete example, consider a wall painting inspection at a construction site. The user takes a photo of the wall with their device and uploads it to a server. The server uses AI to analyze the photo and detects results such as "insufficient uniformity of paint." The report includes suggestions for additional painting to correct the unevenness, which the user can review and communicate to the site as correction instructions. This process significantly streamlines construction management.

[0636] The following describes the processing flow.

[0637] Step 1:

[0638] After the work is completed, the user takes pictures of the work area with their device. The captured images are saved to a designated folder on the device and prepared for upload to the server.

[0639] Step 2:

[0640] The device sends a request to the server to send the stored image data. The request includes image metadata (such as the date and time the image was taken, and location information), which is organized by project.

[0641] Step 3:

[0642] The server processes the received image data using data preprocessing means. By standardizing the image resolution and removing noise, it converts the image into a state suitable for analysis.

[0643] Step 4:

[0644] The server analyzes the pre-processed image data using artificial intelligence. A trained analysis model evaluates the image content and identifies abnormalities related to construction quality.

[0645] Step 5:

[0646] The server uses a report generation mechanism to create a construction completion report based on the results of the quality assessment. The report includes any detected abnormalities and recommended corrective actions.

[0647] Step 6:

[0648] Users can review reports generated on their own devices. Through the interface, they can add new notes or modify the report to a different format.

[0649] Step 7:

[0650] The terminal outputs the finalized report in a format such as PDF and distributes it to designated stakeholders via email or other means. This ensures that information reaches stakeholders quickly.

[0651] (Example 1)

[0652] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0653] Traditional construction quality control relies primarily on visual inspections by skilled personnel, consuming significant human resources and time, and making it difficult to ensure consistent quality. Furthermore, the manual process of report creation made it challenging to efficiently propose improvement measures for the construction site. To address these issues, a more automated and precise quality evaluation system is needed.

[0654] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0655] In this invention, the server includes: information preprocessing means for receiving image information acquired by a camera and converting it into a processable format; machine learning means for creating an analysis model for evaluating the quality of work areas based on the image information; and document generation means for identifying abnormal areas and generating improvement suggestions for those areas based on the quality evaluation results from the analysis model. This enables the automatic and consistent creation of construction quality evaluation and improvement reports.

[0656] "Photography equipment" refers to devices used to acquire image information from a construction site, and includes, for example, cameras and smartphones.

[0657] "Image information" refers to digital visual data acquired by a photographic device.

[0658] "Information preprocessing means" refers to a series of processes performed to convert received image information into a format that can be processed by the analysis model.

[0659] "Machine learning methods" refer to techniques for generating analytical models to evaluate the quality of work areas based on image information.

[0660] "Document generation means" refers to a technology that automatically generates reports based on the results of quality evaluations using an analysis model.

[0661] "Display means" refers to user interface technology that presents a generated document to the user and allows for modification and additional input.

[0662] "Transmission method" refers to the technology for outputting the generated report in a specified format and distributing it to the relevant parties.

[0663] "Training data usage method" refers to a technique in which a model is trained using previously acquired work data, enabling quality checks equivalent to past evaluations.

[0664] "Reference means" refers to a technology that, when identifying abnormal areas in image information, provides more specific correction suggestions by referring to a database of similar past cases.

[0665] This system is designed to automatically evaluate construction quality and generate construction completion reports. Its main components include a server, terminals, and a user interface. The following describes the role of each component and the specific hardware and software usage.

[0666] The server plays a central role in processing image information transmitted from the terminal. First, it receives image information acquired by the terminal using an imaging device (e.g., a smartphone or tablet with a camera). The server uses an image analysis library (e.g., OpenCV) to preprocess the information, performing resolution conversion and noise reduction. This enables uniform analysis.

[0667] Next, the server analyzes the pre-processed image information using a machine learning platform (e.g., TensorFlow or PyTorch). It utilizes a pre-trained generative AI model to evaluate the quality of the construction area. The evaluated data is updated as an analysis model, and abnormal areas are detected. This process allows for specific judgments, such as "the painted surface is uneven."

[0668] Based on the analysis results, the server automatically generates a construction completion report through a document generation system. This report includes details of the detected anomalies and proposed improvements. The report is automatically generated using LaTeX or Microsoft Word templates.

[0669] Users review reports generated through display methods on their devices, and modify the content or enter additional information as needed. This user interface is designed for easy operation, whether browser-based or app-based.

[0670] Finally, the terminal uses a transmission method to output the revised report in the specified format (PDF or Word). The report is then sent to the relevant parties via email or cloud service.

[0671] As a concrete example, consider inspecting the paint condition of a wall on site. The user takes a photograph of the wall and uploads it to the server. The server analyzes the photograph, evaluates the uniformity of the paint, and notes any inconsistencies in the report. Based on this proposal, the user can instruct the on-site workers to apply additional paint.

[0672] As an example of a prompt for the generating AI model, it would be: "Please describe a system that evaluates quality from image information at a construction site and automatically generates a report." This ensures that the system is implemented and operated correctly, and construction quality management becomes significantly more efficient.

[0673] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0674] Step 1:

[0675] The user acquires image information using a camera at the construction site. They take photos of walls and equipment using a terminal and upload the image information to the server. In this process, images are transferred from a smartphone or tablet via Wi-Fi or mobile data communication. Image information from the site is acquired as input and ready to be transferred to the server as output.

[0676] Step 2:

[0677] The server performs preprocessing on the received image information. Using image analysis libraries such as OpenCV, it standardizes resolution, removes noise, and converts the image into a format that the analysis model can process. It takes user-uploaded image information as input and generates standardized image format data as output.

[0678] Step 3:

[0679] The server analyzes pre-processed image data using machine learning techniques. It inputs images into a generative AI model trained on a platform such as TensorFlow to evaluate the quality of the work area. Pre-processed image data is used as input, and the output provides quality assessment results and information on abnormal areas.

[0680] Step 4:

[0681] The server automatically generates a construction completion report using a document generation system based on the analysis results. The report is created using LaTeX or Word templates, and includes quality evaluation results and improvement suggestions. Quality evaluation data is provided as input, and the completed report is generated as output.

[0682] Step 5:

[0683] Users view the generated report on their device through a display device and input corrections or additional information as needed. They can view and edit each section of the report using a browser or application. The generated report data is used as input, and the corrected report is obtained as output.

[0684] Step 6:

[0685] The terminal outputs the revised report in the specified format and sends it to the relevant parties using the transmission method. It exports in PDF or Word format and distributes the report via email or cloud storage. The input is the revised report, and the output is the completed report after distribution.

[0686] (Application Example 1)

[0687] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0688] In on-site construction, there is a need to quickly and accurately evaluate construction quality, efficiently identify defects, and propose improvements. However, conventional methods require evaluation by skilled personnel, which is time-consuming and costly. Furthermore, because proposed corrections for defects are not immediately communicated to the site, it can delay the efficiency of construction. There is a need to solve these problems and achieve highly accurate and rapid quality evaluation and efficient decision-making.

[0689] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0690] In this invention, the server includes: data preprocessing means for receiving image data acquired at a construction site and converting it into a format that can be processed; artificial intelligence means for generating an analysis model for evaluating the quality of construction locations based on the image data; report generation means for identifying abnormal locations and generating improvement suggestions for those locations based on the quality evaluation results by the analysis model; interface means for presenting the generated report to the user and enabling modifications and additional inputs; output means for outputting the report in a specified format and sending it to relevant parties; information presentation means for allowing real-time confirmation of the construction quality evaluation results; and data transmission means for rapidly analyzing images of the construction site and enabling immediate guidance. This enables rapid evaluation of construction quality and immediate guidance.

[0691] "Data preprocessing means" refers to means for receiving image data acquired at a construction site and performing processing to convert it into an analyzable format.

[0692] "Artificial intelligence means" refers to a means of generating an analysis model based on image data to evaluate the quality of a construction site, and then using that model to perform quality analysis.

[0693] The "report generation means" is a means that identifies abnormal areas based on the results of quality evaluation by artificial intelligence and automatically generates a report that includes suggestions for improvement.

[0694] "Interface means" refers to means that present the generated report to the user and provide a user interface that allows the user to make corrections to the report content or input additional information.

[0695] "Output method" refers to a means of outputting a report in a specified format and sending it to the relevant parties.

[0696] An "information presentation method" is a means of presenting information to users in a way that allows them to check the evaluation results of construction quality in real time.

[0697] A "data transmission method" is a means for quickly analyzing images from a construction site and immediately transmitting the analysis results and instructions.

[0698] This invention provides a system for performing rapid and accurate quality evaluations at construction sites. This system consists of three elements: a server, a terminal, and a user.

[0699] The server receives image data acquired at the construction site from the terminal and converts it into a format that can be processed. Specifically, it uses image processing libraries such as OpenCV as a data preprocessing measure to unify the image resolution and remove noise. Then, it uses TensorFlow as an artificial intelligence measure and inputs the images into a trained analysis model to evaluate the quality of the construction area. Based on this evaluation, a report generation measure automatically generates a report that includes a detailed explanation of the abnormal areas and suggestions for improvement.

[0700] The terminal serves to present the generated report to the user. Through the interface, the user can review the report and enter corrections or additional information as needed. This interface is designed to be user-friendly, allowing for intuitive operation of each section of the report.

[0701] Users take pictures at construction sites using mobile devices such as smartphones and tablets and upload them to the server. The generated reports can be viewed on the device, allowing for real-time evaluation of construction quality. Furthermore, the evaluation results are displayed immediately and clearly through the information presentation system, enabling users to provide prompt guidance on-site.

[0702] As a concrete example, when checking the paint condition of a wall at a construction site, the user takes a photo of the wall with their smartphone and sends it to the server. The server uses an AI model to detect uneven paint and cracks, and generates a report that includes information on areas that need correction. This report can be viewed on the device, and the user can then communicate specific countermeasures to the on-site workers based on it.

[0703] As a concrete example of a prompt message for the generating AI model, it can be written in the format of, "Based on this image, please evaluate the concrete construction quality and automatically generate a report. Please also include any abnormalities and suggestions for appropriate correction methods."

[0704] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0705] Step 1:

[0706] The user takes pictures of the construction site using a smartphone or tablet. The captured image data is saved on the device and uploaded to the server. The input for this step is the captured image data, and the output is the transmission of the image data to the server.

[0707] Step 2:

[0708] The server processes the received image data using data preprocessing. Specifically, it uses image processing libraries such as OpenCV to standardize the image resolution, remove noise, and convert the image into an analyzable format. The input for this step is the uploaded raw image data, and the output is the preprocessed image data.

[0709] Step 3:

[0710] The server analyzes the pre-processed image data using artificial intelligence. The images are input to a TensorFlow-trained AI model to evaluate the quality of the construction area. The AI ​​model detects anomalies such as uneven paint application and cracks within the image. The input for this step is the pre-processed image data, and the output is the detection result of the anomalies.

[0711] Step 4:

[0712] Based on the detection results of the anomalies, the server automatically generates a report using a report generation mechanism. The report includes details of the anomalies and suggested corrections. The input for this step is the quality assessment results by AI, and the output is the automatically generated report.

[0713] Step 5:

[0714] The server sends the generated report to the terminal. The terminal presents this report to the user, who can then make corrections or input additional information as needed through the interface. The input in this step is the automatically generated report, and the output is the report content presented to the user.

[0715] Step 6:

[0716] The user reviews the report on their terminal and enters correction instructions as needed. The final report is then generated. The input in this step is the report sent from the server, and the output is the final report reflecting the user's review and corrections.

[0717] Step 7:

[0718] The terminal outputs the revised final report in the specified format and sends it to the relevant parties. This allows the parties to quickly review the report and take the necessary actions. The input for this step is the revised report, and the output is the final report sent to the relevant parties.

[0719] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0720] In this invention, a system that performs quality evaluation based on image data of a construction site and generates a report is combined with a function that recognizes the user's emotions to achieve more effective interaction.

[0721] This system primarily consists of a server, terminals, a user interface, and an emotion engine. The server processes image data of construction sites received from terminals using data preprocessing means and converts it into an analyzable format. Subsequently, artificial intelligence means are used to analyze the image data and evaluate the construction quality. The analysis model is trained using past construction data through training data utilization means, enabling quality checks equivalent to those performed by experts.

[0722] Based on the analysis results, the server automatically generates a construction completion report through the report generation mechanism. This report includes details of the detected abnormalities and suggestions for improvement. Furthermore, by referencing a database of similar past cases using the reference mechanism, more specific correction proposals are also included.

[0723] A newly incorporated feature is an emotion engine that detects user emotions and adapts the system's response accordingly. When a user reviews a report, the emotion engine analyzes the user's voice and facial expression data to recognize their emotions. Based on this, it adjusts the report content and interface display to ensure communication that is sensitive to the user's feelings.

[0724] For example, if a user expresses dissatisfaction with a report, the sentiment engine detects this emotion and instructs the system to either propose a revised report or add further explanations. Furthermore, by accumulating historical sentiment data, the system learns user preferences and tendencies, optimizing future interactions.

[0725] As the final stage of the system, the terminal outputs the final report in the specified format and sends it to the relevant parties. The report is then quickly shared via email or a cloud platform.

[0726] This system not only improves the efficiency of report creation in construction management, but also provides psychological support to users, enabling more consistent quality control.

[0727] The following describes the processing flow.

[0728] Step 1:

[0729] The user takes pictures of the construction site with their device and saves them to the project folder. These images are in preparation for later uploading to the server.

[0730] Step 2:

[0731] The device sends the image data specified by the user to the server. At the same time, the image metadata (e.g., date and time of capture, location information) is also sent.

[0732] Step 3:

[0733] The server processes the received image data using data preprocessing. Specifically, it unifies the image resolution and performs noise reduction to convert it into a state optimized for analysis.

[0734] Step 4:

[0735] The server uses artificial intelligence to analyze pre-processed image data. The analysis model evaluates construction quality based on past training data and identifies abnormal areas.

[0736] Step 5:

[0737] Based on the analysis results, the server uses a report generation mechanism to create a construction completion report. The report includes identification of abnormal areas and improvement suggestions, and, if necessary, refers to a database of similar cases to add specific correction proposals.

[0738] Step 6:

[0739] When a user views a report on their device, the emotion engine activates and analyzes the user's facial expressions and voice. The emotion engine understands the user's emotional state, adjusts the interface accordingly, and re-evaluates the report content if necessary.

[0740] Step 7:

[0741] Users can add corrections and comments through the interface based on the reported content. The sentiment engine's feedback allows users to receive advice that aligns with their intentions.

[0742] Step 8:

[0743] The terminal outputs the final, verified report in a predefined format and sends it to relevant parties via email or cloud services. As a result, reports are shared quickly and accurately.

[0744] (Example 2)

[0745] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0746] Traditionally, quality control at construction sites has heavily relied on the experience and judgment of skilled workers, often resulting in challenges regarding the consistency and speed of evaluations. Furthermore, the system was not designed to adjust reports based on user sentiment, making it difficult to consistently improve user satisfaction. In addition, there was a need to improve the specificity of corrective suggestions based on past similar cases.

[0747] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0748] In this invention, the server includes data preparation means for receiving image information acquired at a construction site and converting it into a processable format; intelligent technology means for generating an analysis structure for evaluating the quality of construction areas based on the image information; and information generation means for identifying abnormal areas and generating improvement suggestions for those areas based on the quality evaluation results from the analysis structure. This makes it possible to efficiently and consistently manage quality at construction sites, and further enables flexible responses that respond to the user's feelings and concrete correction suggestions based on past cases.

[0749] "Data preparation means" refers to a function that receives image information acquired at the construction site and processes it to convert it into an analyzable format.

[0750] "Intelligent technology means" refers to a function that generates an analysis structure for evaluating the quality of the construction site based on the received image information.

[0751] The "information generation means" is a function that automatically generates improvement suggestions and reports on identified abnormal areas based on the results of quality evaluation by intelligent technology means.

[0752] "Communication means" refers to a function that presents generated information to the user and provides an interface that allows for modification or additional input as needed.

[0753] "Communication means" refers to a function that allows the final generated report to be output in a specified format and easily sent to the relevant parties.

[0754] "Emotional analysis means" refers to a function that recognizes emotions from the user's voice, facial expressions, etc., and adaptively adjusts the system's responses and reports based on the results.

[0755] The "training data usage method" is a function that uses previously acquired construction information to train the analysis structure and obtain results equivalent to those of a quality check by a skilled professional.

[0756] The "reference means" refers to a function that searches a database of past similar cases to identify abnormal areas and proposes specific corrective measures.

[0757] This invention relates to a system for efficiently and highly automating quality control at construction sites. This system mainly consists of a server, terminals, and a user interface.

[0758] The server receives image information acquired from terminals at the construction site. Specifically, terminals such as smartphones and tablets send images taken at the site to the server via a dedicated application. The server preprocesses this image information using data preparation tools and converts it into an analyzable format. This preprocessing includes adjusting the image resolution and removing noise.

[0759] Next, the server uses intelligent technology to generate an analysis structure for evaluating the quality of the construction site based on the pre-processed image information. Here, a generative AI model trained on past construction data is utilized to identify defects and abnormalities in the site photographs. This achieves an accuracy equivalent to that of quality checks by skilled personnel.

[0760] Subsequently, the server uses information generation means to automatically generate information, including suggestions for improving the abnormal areas, based on the analysis results. This generation process includes using reference means to search for specific correction suggestions from a database of past similar cases.

[0761] Users receive reports through a communication method that displays the generated information, and can make additional inputs such as corrections and comments. Furthermore, sentiment analysis tools analyze user reactions at an emotional level, allowing the system's responses and report content to be adapted accordingly. For example, if a user expresses dissatisfaction with the report content, the system will automatically adjust to offer new suggestions.

[0762] Finally, the terminal outputs the generated report in PDF or document format via a communication method and quickly sends it to a designated email address or cloud platform. This mechanism allows relevant stakeholders to access the information in a timely manner.

[0763] As a concrete example, an example of a prompt message would be: "Evaluate the quality based on the photos of the construction site, and if there are any abnormalities, create a report that includes details and suggestions for improvement."

[0764] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0765] Step 1:

[0766] Users take pictures at construction sites using smartphones or tablets and upload these images to their devices via a dedicated application. The input is the captured image on the user's device, and the output is the original image file sent to the server. The application formats and encrypts the captured image before sending it to the server.

[0767] Step 2:

[0768] The server receives image information transmitted from the terminal and processes it using data preparation tools. The input is the encrypted original image file, and the output is image data converted into an analyzable format. Specifically, the server applies a noise reduction filter and performs adjustments to optimize the resolution.

[0769] Step 3:

[0770] The server evaluates the quality of the construction site using intelligent technology based on pre-processed image data. The input is image data converted into an analyzable format, and the output is the quality evaluation result, including identified anomalies. A generative AI model is used to perform pattern recognition within the image and execute computational processing to detect defects and problems.

[0771] Step 4:

[0772] The server automatically generates the report content using information generation means based on the quality evaluation results. The input is the quality evaluation results, and the output is a draft report. In this process, the server utilizes reference means to search a database of similar past cases and add specific correction suggestions.

[0773] Step 5:

[0774] Users review the reports generated via communication methods in writing and add corrections and comments as needed. The input is a draft report, and the output is the final, reviewed report. Users enter comments while referring to indicated sections on the interface, and real-time feedback is provided through sentiment analysis.

[0775] Step 6:

[0776] The terminal outputs the final completed report in the specified format via a designated means and sends it to the relevant parties. The input is the completed report reviewed by the user, and the output is a report file uploaded to an email or cloud platform. The terminal formats the output to PDF or another format and sends it quickly based on a pre-configured list.

[0777] (Application Example 2)

[0778] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0779] In construction site management, quality assessment and report creation are crucial, but they are time-consuming and labor-intensive. Furthermore, it can be difficult to immediately reflect user emotions and feedback, sometimes hindering improvements in satisfaction. In this context, there is a need for efficient and effective report generation and methods to enhance user interaction.

[0780] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0781] In this invention, the server includes data preprocessing means for receiving image data acquired at a construction site and converting it into a processable format; artificial intelligence means for generating an analysis model for evaluating the quality of the construction site based on the image data; and sentiment analysis means for presenting the generated report to the user and recognizing the user's emotions to adaptively change the report content and interface. This enables efficient quality evaluation at construction sites and the provision of flexible reports that respond to the user's emotions.

[0782] "Data preprocessing means" refers to a device or program that receives image data acquired at a construction site and converts it into a format that can be processed by the analysis model.

[0783] An "analysis model" is a model that uses artificial intelligence technology to quantitatively or qualitatively evaluate the quality of construction sites based on various data from the construction site.

[0784] "Artificial intelligence means" refers to trained models and algorithms for analyzing image data and evaluating construction quality, and is a technology that enables automated evaluation of construction quality.

[0785] A "report generation means" is a device or software equipped with the function of automatically creating a report that includes the identification of abnormal areas and improvement suggestions based on the results of quality evaluation by an analysis model.

[0786] "Interface means" refers to an operation screen or means that presents the generated report to the user, allows the user to make corrections or additional inputs, and adaptively changes the display to reflect information based on sentiment analysis.

[0787] "Emotion analysis means" refers to a technology or process that analyzes a user's facial expressions and voice when they review a report to recognize their emotions, and then adjusts the system's response and report content based on that.

[0788] "Output means" refers to a device or program that has the function of outputting the final report in a specified format and sending it to the relevant parties.

[0789] The system of the present invention combines data preprocessing means, artificial intelligence means, sentiment analysis means, and various output means to more efficiently perform quality evaluation and report creation at construction sites.

[0790] The server has a data preprocessing function that receives image data from the construction site from the terminal and converts it into an analyzable format. During this process, image processing libraries such as Python and OpenCV are used to perform noise reduction and resolution adjustment. After that, the image data is passed to an artificial intelligence system on the server, and its quality is evaluated using deep learning frameworks such as TensorFlow and PyTorch.

[0791] Based on the quality assessment results, if any abnormalities are identified, the server automatically generates a report containing improvement suggestions. Natural language generation technology is used in this report generation process, presenting the information in a user-friendly format.

[0792] Furthermore, emotion analysis is used to analyze the user's facial expressions and voice as they review the report, recognizing their emotions. Based on this, the system adaptively changes the report content and interface display to improve user satisfaction. Emotion recognition software such as the Emotion AI SDK is used for emotion analysis.

[0793] The final report is generated in the specified format and quickly sent to relevant parties via email or cloud services. This enables rapid decision-making on-site.

[0794] As a concrete example, consider a scenario where a construction site supervisor takes photos of the construction progress with their smartphone and uploads them to the system. The system immediately analyzes the images, evaluates their quality, and generates a report. Furthermore, by using emotional information gleaned from the supervisor's facial expressions, the system can adjust the explanations and emphasis points, providing a more valuable report to the user.

[0795] Examples of prompts include: "Please tell me how to evaluate the quality of concrete placement at the site and improve the report based on sentiment data."

[0796] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0797] Step 1:

[0798] The server receives image data uploaded from the construction site via a terminal. The received image data is temporarily stored to be sent directly to the next processing step. During the receiving process, the server verifies that the image format and size are appropriate, and corrects or filters out any inappropriate formats.

[0799] Step 2:

[0800] The server performs data preprocessing on the acquired image data. By using OpenCV to denoise and adjust the resolution of the received image data, it obtains output in a format suitable for analysis. This processing improves image quality and makes analysis by the AI ​​model more accurate.

[0801] Step 3:

[0802] The server inputs pre-processed image data into an artificial intelligence system and uses an analysis model to evaluate construction quality. The input is pre-processed image data, and the output is the quality evaluation result of the construction site. In this evaluation, a deep learning framework such as TensorFlow is used, and the AI ​​model identifies abnormal areas in the image.

[0803] Step 4:

[0804] The server generates a report using a report generation mechanism based on the quality evaluation results. Using natural language generation technology, it automatically generates details of abnormalities and improvement suggestions from the evaluation results received as input. The output report is formatted to be easily understood by the user.

[0805] Step 5:

[0806] The server presents the generated report to the user and collects and analyzes the user's facial expressions and voice using emotion analysis tools. It identifies the user's emotions from the acquired facial and voice data and provides optimized output for the user, including system responses and report content. The Emotion AI SDK is used for this analysis.

[0807] Step 6:

[0808] The server adjusts the content of reports and interface displays based on the user's emotions. Based on the entered emotion data, it outputs reports with additional explanations and emphasis where necessary. This adjustment improves user satisfaction and enables more effective communication.

[0809] Step 7:

[0810] The terminal outputs the final report in the specified format and sends it to relevant parties via email or cloud services. Receiving the adjusted final report and uploading it to the cloud service enables rapid sharing. Output means the report arrives at the relevant parties in the specified format.

[0811] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0812] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0813] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0814] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0815] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. In the upper and lower directions of the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. Also, the upper side of the concentric circles is where "pleasant" emotions are located, and the lower side is where "unpleasant" emotions are located. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0816] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0817] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0818] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0819] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0820] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0821] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0822] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0823] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0824] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0825] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0826] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0827] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0828] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0829] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0830] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0831] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0832] The following is further disclosed regarding the embodiments described above.

[0833] (Claim 1)

[0834] A data preprocessing means that receives image data acquired at a construction site and converts it into a format that can be processed,

[0835] An artificial intelligence means for generating an analysis model to evaluate the quality of the construction site based on the aforementioned image data,

[0836] A report generation means that identifies abnormal areas and generates improvement proposals for those areas based on the results of the quality evaluation by the aforementioned analysis model,

[0837] An interface that presents the generated report to the user and allows for corrections and additional input,

[0838] An output method that outputs the report in a specified format and sends it to the relevant parties,

[0839] A system that includes this.

[0840] (Claim 2)

[0841] The system according to claim 1, comprising means for using training data, in which an analysis model is trained using construction data acquired in advance, enabling quality checks equivalent to those performed by skilled personnel in the past.

[0842] (Claim 3)

[0843] The system according to claim 1, which includes a reference means for referring to a database of similar past cases and suggesting more specific correction methods when identifying abnormal parts of image data.

[0844] "Example 1"

[0845] (Claim 1)

[0846] Information preprocessing means that receives image information acquired by a shooting device and converts it into a format that can be processed,

[0847] A machine learning means for creating an analysis model to evaluate the quality of a work area based on the aforementioned image information,

[0848] A document generation means that identifies abnormal areas and generates improvement suggestions for those areas based on the results of quality evaluation using the aforementioned analysis model,

[0849] A display means that presents the generated document to the user and allows for modification and additional input,

[0850] A means of sending documents that output them in a specified format and send them to the relevant parties,

[0851] A system that includes this.

[0852] (Claim 2)

[0853] The system according to claim 1, comprising means for using training data, in which an analytical model is trained using previously obtained work data, enabling quality inspection equivalent to past evaluations by experts.

[0854] (Claim 3)

[0855] The system according to claim 1, which includes a reference means for referring to a database of similar past cases and suggesting more specific correction methods when identifying abnormal areas in image information.

[0856] "Application Example 1"

[0857] (Claim 1)

[0858] A data preprocessing means that receives image data acquired at a construction site and converts it into a format that can be processed,

[0859] An artificial intelligence means for generating an analysis model to evaluate the quality of the construction site based on the aforementioned image data,

[0860] A report generation means that identifies abnormal areas and generates improvement proposals for those areas based on the results of the quality evaluation by the aforementioned analysis model,

[0861] An interface that presents the generated report to the user and allows for corrections and additional input,

[0862] An output method that outputs the report in a specified format and sends it to the relevant parties,

[0863] An information presentation method that allows real-time confirmation of construction quality evaluation results,

[0864] A data transmission method that enables rapid analysis of images from construction sites and immediate guidance,

[0865] A system that includes this.

[0866] (Claim 2)

[0867] The system according to claim 1, comprising means for using training data, in which an analysis model is trained using construction data acquired in advance, enabling quality checks equivalent to those performed by skilled personnel in the past.

[0868] (Claim 3)

[0869] The system according to claim 1, which includes a reference means for referring to a database of similar past cases and suggesting more specific correction methods when identifying abnormal parts of image data.

[0870] "Example 2 of combining an emotion engine"

[0871] (Claim 1)

[0872] A data preparation means that receives image information acquired at a construction site and converts it into a format that can be processed,

[0873] An intelligent technology means for generating an analysis structure to evaluate the quality of the construction site based on the aforementioned image information,

[0874] Information generation means for identifying abnormal areas and generating improvement suggestions for those areas based on the results of quality evaluation using the aforementioned analysis structure,

[0875] A communication means that presents the generated information to the user and allows for modification and additional input,

[0876] A means of communication that outputs the report in a specified format and sends it to the relevant parties,

[0877] A means of sentiment analysis that recognizes the user's emotions and adapts the system's response accordingly,

[0878] A system that includes this.

[0879] (Claim 2)

[0880] The system according to claim 1, comprising means for using training data, which enables the analysis structure to be trained using construction information acquired in advance, and to enable quality checks equivalent to those performed by skilled personnel in the past.

[0881] (Claim 3)

[0882] The system according to claim 1, which includes a reference means for referring to a database of past similar cases to suggest more specific correction options when identifying abnormal parts of image information.

[0883] "Application example 2 when combining with an emotional engine"

[0884] (Claim 1)

[0885] A data preprocessing means that receives image data acquired at a construction site and converts it into a format that can be processed,

[0886] An artificial intelligence means for generating an analysis model to evaluate the quality of the construction site based on the aforementioned image data,

[0887] A report generation means that identifies abnormal areas and generates improvement proposals for those areas based on the results of the quality evaluation by the aforementioned analysis model,

[0888] A sentiment analysis tool that presents the generated report to the user, recognizes the user's emotions, and adaptively changes the report content and interface,

[0889] An output method that outputs the report in a specified format and sends it to the relevant parties,

[0890] A system that includes this.

[0891] (Claim 2)

[0892] The system according to claim 1, comprising means for using training data, in which an analysis model is trained using construction data acquired in advance, enabling quality checks equivalent to those performed by skilled personnel in the past.

[0893] (Claim 3)

[0894] The system according to claim 1, which includes a reference means for referring to a database of similar past cases and suggesting more specific correction methods when identifying abnormal parts of image data. [Explanation of Symbols]

[0895] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A data preprocessing means that receives image data acquired at a construction site and converts it into a format that can be processed, An artificial intelligence means for generating an analysis model to evaluate the quality of the construction site based on the aforementioned image data, A report generation means that identifies abnormal areas and generates improvement proposals for those areas based on the results of the quality evaluation by the aforementioned analysis model, An interface that presents the generated report to the user and allows for corrections and additional input, An output method that outputs the report in a specified format and sends it to the relevant parties, An information presentation method that allows real-time confirmation of construction quality evaluation results, A data transmission method that enables rapid analysis of images from construction sites and immediate guidance, A system that includes this.

2. The system according to claim 1, which includes means for using training data, enabling an analysis model to be trained using construction data acquired in advance, and enabling quality checks equivalent to those performed by skilled personnel in the past.

3. The system according to claim 1, which includes a reference means for referring to a database of similar past cases and suggesting more specific correction methods when identifying abnormal parts of image data.

Images